# Modeling early phonetic acquisition from child-centered
audio data

Marvin Lavechin, Maureen de Seyssel, Marianne Métais, Florian Metze,
Abdelrahman Mohamed, Hervé Bredin, Emmanuel Dupoux, Alejandrina
Cristia

► To cite this version:

Marvin Lavechin, Maureen de Seyssel, Marianne Métais, Florian Metze, Abdelrahman Mohamed, et al.. Modeling early phonetic acquisition from child-centered audio data. Cognition, 2024, 245, 245, pp.21. 10.1016/j.cognition.2024.105734. hal-04876433

HAL Id: hal-04876433

https://hal.science/hal-04876433v1

Submitted on 9 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Modeling early phonetic acquisition from child-centered audio data

Marvin Lavechin<sup>a,b,c,*</sup>, Maureen de Seyssel<sup>a,b,d</sup>, Marianne Métais<sup>a,b</sup>, Florian Metze<sup>c</sup>, Abdelrahman Mohamed<sup>e</sup>, Hervé Bredin<sup>f,1</sup>, Emmanuel Dupoux<sup>a,b,c,1</sup> and Alejandrina Cristia<sup>a,b,1</sup>

<sup>a</sup>Laboratoire de Sciences Cognitives et Psycholinguistique, Département d'Etudes Cognitives, ENS, EHESS, CNRS, PSL University, Paris, France

<sup>b</sup>Cognitive Machine Learning Team, INRIA, Paris, France

<sup>c</sup>Meta AI Research, Paris, France

<sup>d</sup>Laboratoire de linguistique formelle, Université de Paris, CNRS, Paris, France

<sup>e</sup>Rembrand, Palo Alto, California, United States

<sup>f</sup>Institut de Recherche en Informatique de Toulouse, Université de Toulouse, CNRS, Toulouse, France

## ARTICLE INFO

**Keywords:**

language acquisition
computational modeling
statistical learning
predictive coding
phonetic learning
perceptual attunement
child-centered long-forms

## ABSTRACT

Infants learn their native language(s) at an amazing speed. Before they even talk, their perception adapts to the language(s) they hear. However, the mechanisms responsible for this *perceptual attune-ment* and the circumstances in which it takes place remain unclear. This paper presents the first attempt to study perceptual attunement using ecological child-centered audio data. We show that a simple prediction algorithm exhibits perceptual attunement when applied on unrealistic clean audio-book data, but fails to do so when applied on ecologically-valid child-centered data. In the latter scenario, perceptual attunement only emerges when the prediction mechanism is supplemented with inductive biases that force the algorithm to focus exclusively on speech segments while learning speaker-, pitch-, and room-invariant representations. We argue these biases are plausible given previous research on infants and non-human animals. More generally, we show that *what* our model learns and *how* it develops through exposure to speech depends exquisitely on the details of the input signal. By doing so, we illustrate the importance of considering ecologically valid input data when modeling language acquisition.

# 1. Introduction

## 1.1. Perceptual attunement to speech sounds

The way adults perceive speech sounds is shaped by the language(s) they have been exposed to. For instance, while most French speakers have no difficulties distinguishing between [u] and [y] (as in ‘dessous’, *below* versus ‘dessus’, *above*), American English speakers struggle hearing the difference (Levy and Strange, 2008). Similarly, Japanese native speakers often confuse [ɪ] and [l] (as in ‘right’ versus ‘light’) whereas American English speakers do not (Miyawaki, Jenkins, Strange, Liberman, Verbrugge and Fujimura, 1975).

Meta-analytic evidence suggests that the process of tuning in to certain sounds, known as *perceptual attunement*, begins in early childhood and gives rise to one of the earliest language-specific effects in infant speech development (Tsuji and Cristia, 2014). According to the mainstream description of this process, young infants' ability to distinguish between different sounds does not initially depend on the language they are exposed to. As they age, infants' ability to distinguish between sounds that are relevant to their native language improves while their ability to distinguish non-native sounds deteriorates (Kuhl, 2004). As an example, at 6-8 months, American English and Japanese infants show similar discrimination scores for the [ɪ] versus [l] pair. However, by 10-12 months, American English infants' discrimination score for the [ɪ] versus [l] pair increases, while Japanese infants' discrimination score deteriorates (Kuhl, Stevens,

Hayashi, Deguchi, Kiritani and Iverson, 2006), resulting in a *native advantage* (higher scores for the infants for whom the contrast is native). According to some researchers, this process involves the emergence of discrete phonemic categories (Werker and Tees, 1992), though others question this view (Feldman, Goldwater, Dupoux and Schatz, 2022), or even strongly contest it (McMurray, 2022). Moreover, a recent systematic review cautions against overly strong generalization from a literature that has not accurately represented phonetic contrasts and infant demography (Singh, Rajendra and Mazuka, 2022). The studies presented here cannot contribute to resolving these debates, and thus we do not take a position in them. We merely build on the observation that, at least for some contrasts, infants exhibit improvements in native discrimination concomitant with deterioration in non-native discrimination. Our proposed models attempt to shed light on the processes and conditions that may be required for such perceptual attunement, without assuming underlying categories nor that this process occurs for each and every infant group and sound contrast in a similar fashion.

## 1.2. How does early phonetic learning proceed?

Within the cognitive sciences, computational modeling can play a crucial role in helping us understand how cognitive processes may unfold. The present work builds on a rich literature that has shed light on different aspects of early phonetic learning. A full review is not possible given space limitations, but we provide a few entry points into this vast literature, by clarifying the scope of what our modeling efforts will cover. Building on terminology proposed in the recent SCALa framework (Tsuji, Cristia and Dupoux,

<footnote>*Corresponding author at Département d'Etudes Cognitives, Ecole Normale Supérieure, Paris, France
*E-mail address: marvinlavechin@gmail.com (M. Lavechin)
¹H.B., E.D., and A.C. contributed equally to this work.</footnote>
2021), our study focuses on just one of three essential aspects of language, namely "language as structure". This means that we do not explicitly consider lexical effects on phonetic learning, which would entail considering "language as a description of the world" (but see Feldman, Griffiths, Goldwater and Morgan, 2013); nor "language as a social construct", despite evidence that early phonetic learning is affected by social factors (Lytle, Garcia-Sierra and Kuhl, 2018).

Inspecting how far one can go in language learning from the "language as structure" point of view, a recurrent symposium invites machine learners and speech technologists to develop self-supervised learning algorithms in several languages (Versteegh, Thiollière, Schatz, Cao, Miró, Jansen and Dupoux, 2017; van den Oord, Li and Vinyals, 2019; Schneider, Baevski, Collobert and Auli, 2019). On the surface, one may think that these models are solving a similar task to that of children, because they come to distinguish sounds and even recognize words following exposure to running speech, without using any implausible annotations, such as orthographic transcriptions or even markers of where phones or words start and end. For example, Schatz, Feldman, Goldwater, Cao and Dupoux (2021) modeled learning through an algorithm based on a mixture of Gaussians, a mechanism that is conceptually related to a hypothesis proposing that infants track the modes in distributions of acoustic properties (Maye, Werker and Gerken, 2002). Other self-supervised models employ prediction-based learning algorithms to better capture the temporal dynamics of speech (Lavechin, De Seyssel, Titeux, Bredin, Wisniewski, Cristia and Dupoux, 2022a), which are conceptually related to the predictive coding hypothesis (Huang and Rao, 2011), in that they attempt to predict upcoming signal from the previous spoken context. In recent years, we have observed the rise in language capabilities by machines that employed such prediction-based learning mechanism: With sufficient textual input, large language models come to exhibit surprising linguistic and reasoning abilities (Liang, Bommasani, Lee, Tsipras, Soylu, Yasunaga, Zhang, Narayanan, Wu, Kumar et al., 2022; Trott, Jones, Chang, Michaelov and Bergen, 2023). This begs the question: Can one obtain phonetic attunement simply by exposure to speech using prediction-based learning mechanisms?

In this paper, following recent recommendations (Dupoux, 2018), we expand the scope of models of perceptual attunement by considering the whole process, from input to outcomes. To this end, we employ terminology from Cristia (2020), which distinguishes filters, which "allow certain aspects of the environment to enter the child's processing space, and direct this information to a given procedure", from procedures which "take into account the input (...) to lead to some learning outputs". By considering the whole process, we can better understand the filters and procedures that may be necessary for perceptual attunement to occur. This shift in scope has led us to unveil an important limitation of computational models of perceptual attunement,

arising from the fact that they have employed highly curated inputs, with even the most realistic computational approaches using studio recordings of conversations or audiobooks (De Boer and Kuhl, 2003; Coen, 2006; Vallabha, McClelland, Pons, Werker and Amano, 2007; Miyazawa, Kikuchi and Mazuka, 2010; Schatz et al., 2021; Millet, Chitoran and Dunbar, 2021). This differs in important ways from children's input, as laid out in the next section. Through a series of 6 experiments and additional manipulations reported on in supplementary materials, this paper shows that differences in the ecological validity of input data can result in divergent results and conclusions from modeling studies. Furthermore, we show that these differences can be partly alleviated with inductive biases. Consequently, future models of perceptual attunement should consider the ecological validity of their simulations and take into account our proposed biases, and perhaps others, if they hope to make their results relevant to child language development by employing child-centered ecological audio as input.

## 1.3. What infants truly hear

Neither studio recordings of isolated syllables nor audiobooks reflect what infants truly hear, constituting crucial limitations of previous modeling studies. First, the large majority of what children hear during their wake time is not speech at all, but various ambient sounds, noise, music, or non-linguistic vocalizations. Second, the speech sections are not isolated syllables but multi-words utterances. Additionally, speech signals are far from studio-quality; instead they may be distorted as they are spoken far from the child and are multiply reverberated and absorbed by the surrounding obstacles in the environment. Speech signals may also be covered by a variety of background noises or crowded by other concurrent speech sounds. Finally, in real life, people do not speak in full and well-articulated sentences as in audiobooks, but may speak in ways that distort the clear articulation of phonemes: People may produce short turns that sometimes overlap across speakers, and they may underarticulate, mumble, shout, whisper, sing, or laugh while speaking.

To demonstrate the extent of some of these differences, we compare, in Table 1, the statistics of open-source audiobooks, self-recordings of books by volunteers using home equipment (Kearns, 2014) and long-forms that are captured using lightweight recorders worn by the infant, which continuously collect audio over long periods of time (typically 8+ hours of the infant's waking time). Whereas audiobooks contain nearly 100% of speech, long-forms contain only about 20% of speech, the rest being various environmental noises (vacuum cleaner, music, traffic noise, silence, etc.). Genders are also more balanced in audiobooks than in long-forms, and while audiobooks contain long stretches of speech read by the same speaker (median turn duration of 17mn), long-forms contain relatively short turns spoken by different speakers (median turn duration of 2.4s).

Figure 1 displays signal characteristics of the two sources of audio, notably: 1) the Speech-to-Noise Ratio (SNR) which
<table><caption>**Table 1**</caption><thead><tr><th>Corpus</th><th>Audio dur.</th><th>Speech prop.</th><th>Women ratio</th><th>Median turn dur.</th></tr></thead><tbody><tr><td></td><td></td><td><strong>Audiobooks</strong></td><td></td><td></td></tr><tr><td>R-Eng.</td><td>1071h</td><td>94.9%</td><td>51.0%</td><td>14.5mn</td></tr><tr><td>R-Fr.</td><td>1089h</td><td>94.0%</td><td>57.3%</td><td>20.4mn</td></tr><tr><td></td><td></td><td><strong>Long-forms</strong></td><td></td><td></td></tr><tr><td>E-Eng.</td><td>1054h</td><td>17.6%</td><td>67.6%</td><td>2.3s</td></tr><tr><td>E-Fr.</td><td>1008h</td><td>19.0%</td><td>69.9%</td><td>2.5s</td></tr></tbody></table>

Audio duration, proportion of adult speech, proportion of adult speech pronounced by women, and median turn duration for our 4 corpora. R-<language> indicates read-speech corpora from audiobooks. E-<language> indicates ecological audio corpora from child-centered long-form recordings. Proportions of speech are estimated using a pretrained model (Lavechin, Bousbib, Bredin, Dupoux and Cristia, 2020). The median turn duration is estimated as 1) the median duration of chapters for audiobooks (each chapter is read by a single speaker); and 2) the median cumulated duration of successive sequences produced by the same voice type for long-forms. Data are taken from (Kahn, Rivière, Zheng, Kharitonov, Xu, Mazar'e, Karadayi, Liptchinsky, Collobert, Fuegen, Likhomanenko, Synnaeve, Joulin, rahman Mohamed and Dupoux, 2020; Bergelson, 2017; Canault, Normand, Foudil, Loundon and Thai-Van, 2016a; Cristia, 2021; Lavechin and Cristia, 2021).

measures the strength of the speech signal relative to the strength of background noise (the higher the SNR the lower the amount of noise); and 2) the $C_{50}$ measure which quantifies the level of reverberation (a higher $C_{50}$ indicates less reverberation). Both measures are computed at the utterance level (i.e., speech only) on either English long-forms or audiobooks using the pretrained model proposed in (Lavechin, Métais, Titeux, Boissonnet, Copet, Rivière, Bergelson, Cristia, Dupoux and Bredin, 2022b). Our analysis reveals that, on average, speech utterances extracted from audiobooks have an SNR of 46 dB, while those from long-forms have an SNR of 10 dB, indicating, therefore, a higher level of background noise in long-forms. Similarly, audiobooks contain utterances with a high $C_{50}$ indicating a low reverberation level. On the contrary, utterances extracted from long-forms span a $C_{50}$ between -5 and 57 dB indicating a wider variety of reverberant conditions. In total, audiobooks that are recorded in a quiet environment, close to the microphone, tend to be of higher quality than long-form recordings, that are recorded in uncontrolled environment, with the source of speech being often far away from the listener child. Note that similar results for both the SNR and the $C_{50}$ were found in our French corpora.

## 1.4. Inductive biases to guide early phonetic learning

The difference between clean and well-articulated speech used in previous studies and the signal available to infants is so large that one might rightfully wonder: Given what infants truly hear, can we assume that the algorithms proposed in previous modeling research will work in real life? Can we

<figure><img src="image_2.png" /><figcaption>**Figure 1:** The quantity of noise and reverberation in child-centered long-forms and audiobooks. Speech-to-Noise Ratio (SNR) and $C_{50}$ distributions on 16 hours of speech utterances in English extracted from long-forms (in purple) or audiobooks (in blue). Both measures are automatically extracted using the pretrained model proposed in Lavechin et al. (2022b).</figcaption></figure>

be certain that they parallel the filters and procedures that infants probably possess?

Previous experimental research in infants provides ample evidence for potential filters. Among other astonishing capacities, human infants have a preference for listening to speech from birth (Cooper and Aslin, 1990; Vouloumanos and Werker, 2007), come pre-equipped with an auditory system capable of source separation (Bregman, 1994), and display an early ability to discriminate human voices (Decasper and Prescott, 1984; Flocchia, Nazzi and Bertoncini, 2000). Could these abilities shape or guide learning in order to improve the acquisition of language-relevant dimensions?

In machine learning, mechanisms that constrain or guide the learning process by forcing the algorithm to learn on specific parts of the signal, or to extract specific information are called inductive biases (Hüllermeier, Fober and Mernberger, 2013). In this study, we equip a computational model with inductive biases, and study how they affect the learning of our model from ecological signals. We propose inductive biases that have been designed with two considerations in mind. First, they must be designed to address the signal degradations found in long-forms, as documented in Section 1.3. Second, they should align with documented or plausible behaviour found in infants (more details to be found in Experiment 1).

## 1.5. Key questions

The aim of this paper is to study the effect of input realism (audiobooks versus long-forms) and inductive biases (absence or presence) in a computer simulation of early phonetic learning. This is relevant for both theoretical and practical reasons. On the theory side it helps us assess the kind of biases infants' perceptual attunement likely benefits from. On the practical side, it indicates the level of evidence we can obtain from a given modeling experiment. That is, if results from audiobooks can not be replicated with truly naturalistic data (long-forms), then this casts important doubts on any work relying on audiobooks.

In this study, we examine one of the earliest language-specific phenomenon, perceptual attunement, and ask:
1. Can a prediction-based learning mechanism lead to perceptual attunement when given such sparse, variable and noisy signals as found in ecological data?

2. Can additional inductive biases help overcome the difficult conditions found in long-forms?

3. What characteristics of ecological long-forms impact the phonetic learning outcomes of our model learner?

In Experiment 1, we document that our base learner (endowed with a prediction-based mechanism but no additional inductive biases) can reproduce perceptual attunement when exposed to audiobooks but not when exposed to long-forms, whereas our biased learner can. One may wonder whether our base learner can catch up simply by exposure to even more data. We show this is not the case in Experiment 2, which examines the impact of varying the quantity of total audio. In Experiment 3, we conduct a complete ablation study examining the individual contribution of each of the three proposed biases. Although each makes a unique contribution, a bias that filters out non-speech (Voice Activity Detection, presented in detail below) appears crucial for the emergence of native advantage after exposure to long-forms. This bias entails an important exposure difference: since the biased learner focuses on speech data, it accumulates more relevant experience when overall input quantities are matched. In Experiment 4, we show that it is not merely input quantity, but also input composition, that explains reduced native advantage, with the presence of non-speech in the training set reducing native advantage much faster than native discrimination. We find that input quality also matters in Experiment 5, where we document that our biased learner remains sensitive to noise and reverberation, two salient characteristics of long-forms. Finally, in Experiment 6, we explore the possibility to simulate ecological long-forms with audiobooks artificially contaminated with noise and reverberation. While such a manipulation suffices to reproduce native discrimination patterns, it does not for native advantage. In sum, over 6 experiments, we document the effect of the proposed inductive biases on enabling phonetic attunement as well as the importance of employing not just realistic but actually ecological child-centered signals when modeling language development.

## 2. Experiment 1: the impact of inductive biases

In this first Experiment, we ask how native discrimination and native advantage differ as a function of the learner and the input. We first train our base learner on audiobooks, which constitutes a conceptual replication of past studies (Schatz et al., 2021) with a more powerful algorithm and on a different pair of languages. We separately train a learner with the same powerful algorithm on realistic child-centered long-forms: If our prediction-based learning mechanism suffices to extract language-relevant patterns, then we should observe perceptual attunement in this condition as well. In addition, we introduce a modified learner incorporating three inductive biases aiming at

guiding the learning process in our algorithm. Based on newborns' preference for speech (Cooper and Aslin, 1990; Vouloumanos and Werker, 2007), we propose an inductive bias that helps our learner deal with the high quantity of non-speech found in long-forms (Table 1) by restricting learning to speech segments. Based on infants early ability to discriminate human voices (Mehler, Bertoncini, Barriere and Jassik-Gerschenfeld, 1978; Decasper and Fifer, 1980), we propose an inductive bias that leverages the speaker information to allow our learner to deal with the frequent speaker change and the gender imbalance (Table 1); this is done by restricting learning to segments that plausibly come from the same speaker. Our third inductive bias helps our learner in achieving better 'perceptual constancy', i.e., the ability to identify speech sound signals showing some degree of acoustic variability as belonging to the same phonetic category (Kuhl, 1979; Beeston, Brown and Watkins, 2014). This is done by nudging the learning algorithm to be invariant with respect to changes in pitch and to audio modifications induced by varying listening conditions. By comparing the learning outcomes obtained by our base learner with those of our biased learner, also trained on long-forms, we measure the impact of the inductive biases with which our biased learner is endowed. In this first experiment, our learners' experience is matched in duration. Matching in duration, audiobooks have more speech than long-forms. Moreover, thanks to its focus on speech, our biased learner will be exposed to more speech than our base learner. We return to the issue of input quantity and composition in subsequent experiments.

### 2.1. Methods

Our simulation approach consists in implementing three key components of language learning (Dupoux, 2018): the environment, the learner and the outcome measure (see Fig. 2). Regarding the environment, we build on 2,000 hours of child-centered, ecologically valid audio recordings of infants learning either American English or Metropolitan French, which represents an essential step forward as compared to previous modeling studies. Regarding the learner, we employ one of the best self-supervised learning algorithms as a key component of our learner (Dunbar, Bernard, Hamilakis, Nguyen, de Seyssel, Rozé, Rivière, Kharitonov and Dupoux, 2021). The learner's model we built for this paper comes in two flavors: the *base learner* implements a predictive coding mechanism (introduced below), whereas the *biased learner* filters and processes its input before applying the predictive coding mechanism. Finally, regarding the outcome, we employ the same outcomes as previous work (Schatz et al., 2021). In a nutshell, we will study *native discrimination* (to what extent the learner discriminates between native contrasts) and *native advantage* (to what extent it does so better than a non-native learner).

#### 2.1.1. Environment: training datasets

**Audiobooks.** The English and French read-speech corpora were built from audiobooks using the LibriVox platform (Kearns, 2014; Kahn et al., 2020).
<figure><img src="image_3.png" /><figcaption>**Figure 2: Phonetic learning simulation.** (a) Simulated language environments in American English and Metropolitan French built from (top) audiobooks, or (bottom) child-centered long-forms. (b) Simulated learners built from (top) a prediction-based learning mechanism (contrastive predictive coding applied on nearby audio sequences), (bottom) with the addition of inductive biases (filtering of non-speech, speaker-invariant and pitch-/room-resistant training). (c) Outcome measures obtained via an ABX auditory discrimination task in English and French. *Native discrimination* measures the ability of the learner to discriminate sounds in its native language. *Native advantage*, measures perceptual attunement, i.e., the extent to which the native learner is better at discriminating sounds of its native language than the non-native learner.</figcaption></figure>

**Long-forms.** The long-form training set was built from 156 recordings from 80 children (40 female) aged 2-48 months from four studies: the American English SEEDLingS study (Bergelson, Casillas, Soderstrom, Seidl, Warlaumont and Amatuni, 2019; Bergelson, 2017), and three Metropolitan French studies (Canault, Normand, Foudil, Loundon and Thai-Van, 2016b; Canault et al., 2016a; Cristia, 2021; Lavechin and Cristia, 2021). All recordings were collected using a LENA audio recorder (single channel, 16 kHz; see Ford, Baer, Xu, Yapanel and Gray (2008) for full specification). Half of the recordings came from English infants and the other half from French infants. The French and English sets of recordings were matched as much as possible for the age of the child wearing the recording device, within the limitations of independently collected corpora. This resulted in a set of Metropolitan French recordings acquired from 40 children (21 female) whose age varies from 2 to 41 months ($\mu_{FR} = 17.3$ mths, $\sigma_{FR} = 10.9$ mths); and a set of American English recordings acquired from 40 children (19 female) whose age varies from 6 to 17 months ($\mu_{EN} = 12.5$ mths, $\sigma_{EN} = 3.3$ mths).

**Creation of training sets.** For each language (English or French) and each audio source (audiobooks or long-forms), we built mutually exclusive training sets of 128 hours so as to measure the robustness of the learning outcomes across different language exposures. As our base learner receives

both speech and non-speech, while our biased learner receives exclusively speech segments, the number of training sets depends both on the source of the data (audiobooks or long-forms) and the type of learner considered (base or biased). We created 8 training sets from audiobooks and 7 from long-forms for our base learner (containing both speech and non-speech). We were only able to create one training set of 128 hours of speech from long-forms, as that was the amount of speech present in our 1,000 hours of long-forms, reflecting the scarcity of speech in these naturalistic data. (see Appendix A for more details about the creation of the training set).

### 2.1.2. Learners and baseline

**Learning from raw audio.** In this section, we present the mechanism at the heart of both our base and biased learners.

We chose *Contrastive Predictive Coding* (CPC) (van den Oord et al., 2019) as the heart of our learners. In machine learning, CPC has been shown to be powerful in a wide variety of modalities ranging from audio and images to natural language and reinforcement learning (van den Oord et al., 2019). In the ZeroSpeech 2021 international challenge on unsupervised representation learning, CPC was the best system to learn representations that accurately discriminate speech sounds (Dunbar et al., 2021). The key idea behind CPC is to predict the future states of a sequence given its
<figure><img src="image_4.png" /><figcaption>**Figure 3: Base learner versus biased learner.** Schematic structure of (a) our base learner (trained without inductive biases); and (b) our speech-biased learner (trained with inductive biases). *Contrastive Predictive Coding (CPC)* learns audio representations from predicting the near future; *Voice Activity Detection* filters out non-speech segments of the audio. *Pseudo Speaker Separation* is used to sort the different speech segments according to by whom they have been pronounced. *Data Augmentation* leads to both channel and pitch mismatch between past and future samples.</figcaption></figure>

past context. The learner is given an example that is drawn from the near future up to 120 ms (called positive example), and multiple examples that are not drawn from the near future (called negative examples). Given the past context of a sequence, the learner has to come to reliably choose the positive sample over the negative ones (see Contrastive Predictive Coding panel, top right of Fig. 3). In layman's terms, this is done by encoding the input, selecting one sample among those provided as choices as the correct one, then observing whether it was indeed the right one. If it was not, the neural network's weights are shifted to encode the input slightly differently. This allows us to then process any kind of audio samples, including those used in our ABX test.

In technical terms, we used a contrastive loss which forces the latent space to retain information useful to predict future samples, as originally proposed in (van den Oord et al., 2019). Precisely, the input sequence of observations $x_t$ is mapped to a sequence of latent representations through an encoder $g_{enc}$, such that $z_t = g_{enc}(x_t)$. Then, all $z_{\le t}$ are aggregated with an auto-regressive model that produces a context-dependent latent representation $c_t = g_{ar}(z_{\le t})$. Given the past context $c_t$, a predictor $g_{pred}$ is asked to predict future representations $z_{t+k}$ for $k \in \{1, ..., K\}$. Given a set $X =$

$\{x_1, ..., x_n\}$ of $N$ random samples containing one positive sample from the true positive distribution $p(x_{t+k} | c_t)$ and $N - 1$ negative samples from the proposal negative distribution $p(x_{t+k})$, we optimize the categorical cross-entropy loss of classifying the positive sample correctly:

$$ \mathcal{L} = - \mathbb{E}_{X} \left[ \log \frac{\exp (g_{pred}(c_t)^T z_{t+k})}{\sum_{x_j \in X} \exp (g_{pred}(c_j)^T z_j)} \right] $$

A key difference compared to previous implementations (Rivière, Joulin, Mazaré and Dupoux, 2020; Kharitonov, Rivière, Synnaeve, Wolf, Mazaré, Douze and Dupoux, 2021) lies in the fact that we sample negatives from temporally close sequences from the time series of interest. This strategy ensures that the positive/negative examples used in the contrastive task takes place in a short time span, therefore reducing mismatch between both types of examples in terms of their local environment. Implementation details can be found in Appendix B.

**Inductive biases** In this section, we present inductive biases designed to help our biased learner to overcome difficulties found in realistic audio (see Section 1.3). For the present
study, these mechanisms are fixed, but other work could assess whether they may be learned through exposure or whether they may have been selected throughout evolution. **Voice Activity Detection.** First, based on newborns' preference for speech (Cooper and Aslin, 1990; Vouloumanos and Werker, 2007), we propose an inductive bias that discards non-speech segments and induces the learner to learn only from the speech signal, alleviating the issue of the high proportion of non-speech found in long-forms (see Table 1). This is achieved via a pretrained Voice Activity Detection (VAD) model Lavechin et al. (2020).

**Pseudo Speaker Separation.** Second, a Pseudo Speaker Separation<fnref n="1" /> inductive bias groups the speech segments according to by whom they have been produced. By way of illustration, without this bias, positive and negative examples could be spoken by different speakers, allowing the learner to focus on easier cues, namely the low-level acoustic differences between speakers. Pseudo Speaker Separation decreases the probability of having a speaker mismatch between the positive and the negative examples, forcing the learner to learn speaker-invariant representations (van den Oord et al., 2019). This bias incorporates the idea that infants have an early ability to discriminate human voices (Mehler et al., 1978; Decasper and Fifer, 1980), including unfamiliar ones (Decasper and Prescott, 1984; Floccia et al., 2000), but that they also progressively learn representations that are invariant with respect to a change in speaker (see Seidl, Onishi and Cristia (2014); Bergmann, Cristia and Dupoux (2016) for behavioral evidence found in infants, and Choi and Shukla (2021) for a review). This bias aims at alleviating the issues of the frequent speaker change and the gender imbalance found in long-forms (see Table 1).

**Data Augmentation.** Third, the Data Augmentation inductive bias consists in applying acoustic transformations to the past context of each segment, but not to the segments from which the positive and the negative examples are drawn (as proposed in (Kharitonov et al., 2021)), therefore inducing CPC to learn representations that are invariant with respect to these transformations and achieve better perceptual constancy (Kuhl, 1979; Beeston et al., 2014). Here we use two such transformations: artificial reverberation and pitch modification. Artificial reverberation induces invariance with respect to the distance of the speaker to the 'ears' of the artificial learner and to the presence of various sound-reflective objects in the environment. Pitch modification, on the other hand, induces invariance with respect to voice fundamental frequency, which varies across speakers as a function of their vocal cord anatomy. Pitch modification aims at alleviating the short turn duration and

the gender imbalance found in long-forms (Table 1) while artificial reverberation aims at alleviating the wide variety of reverberant environments (Fig. 1).

**Auditory baseline** Following recommendations to include a control to check for the effects of learning, we compare the performance of our learners against an auditory baseline describing the spectral shape of the input signal. We first slice the speech signal into 25ms-long frames sampled every 10ms. Descriptors of each frame consist of mel-frequency cepstral coefficients (MFCCs) with the first and second derivatives, resulting in a 39-dimensional feature vector for each 10ms-frame. This baseline allows us to compare the performance of our model against a general and non-language-specific descriptor of the input signal.

### 2.1.3. Outcomes: the machine ABX discrimination test

The ABX discrimination test is commonly used to study human perceptual system (Schatz, Peddinti, Bach, Jansen, Hermansky and Dupoux, 2013). In this test, the machine is given three triphones: A, B, and X, with A and X two different occurrences of the same triphone (e.g., /bup/) and B another triphone differing only in its center phone (e.g., /bœp/). Representations of each stimulus ($R_A$, $R_B$ and $R_X$) are extracted and pairwise distances $d(R_A, R_X)$ and $d(R_B, R_X)$ are computed. As representations can have different lengths (depending on the duration of the input stimuli), the distance between two representations is computed along their dynamic time-warped alignment. Following (Versteegh et al., 2017), we used the cosine distance function to measure similarity between individual frames (Supplementary Fig. 9). The machine is considered to be right if $d(R_A, R_X) < d(R_B, R_X)$ as A and X represent the same triphone. Context, as defined by the preceding and the following sound, is controlled as the same sound can be pronounced differently in different contexts. Thus, for each contrast (e. g., /u/ vs /œ/) and each context (e.g., [b_p]), a discrimination accuracy is computed as the amount of times the learner is right. If we note, $S(C_1)$ the set of sounds from category $C_1$ (e.g., all /bup/ triphones of the evaluation set) and $S(C_2)$ the set of sounds from category $C_2$ (e.g., all /bœp/ triphones of the evaluation set), the non-symmetric discrimination accuracy between category $C_1$ and $C_2$ can be computed as:

$$ \theta(C_1, C_2) := \frac{1}{m(m-1)n} \sum_{A \in S(C_1)} \sum_{B \in S(C_2)} \sum_{X \neq A} \left( \mathbb{1}_{d(R_A, R_X) < d(R_B, R_X)} + \frac{1}{2} \mathbb{1}_{d(R_A, R_X) = d(R_B, R_X)} \right) $$

with $m$ and $n$ the cardinal of $S(C_1)$ and $S(C_2)$ respectively, $\mathbb{1}$ the indicator function, and $R_A$, $R_B$ and $R_X$ the representations of sounds A, B and X respectively. A symmetric discrimination accuracy is obtained by averaging $\theta(C_1, C_2)$ and $\theta(C_2, C_1)$.

Representations are extracted from the last layer of the auto-regressive model, which results in a 256-dimensional feature vector for each 10ms frame.

<footnote n="1">¹The prefix 'pseudo' refers to the fact that we approximate speaker identity by combining the automatically-determined gender information with the recording unique identifier. This pre-supposes that all speech by females in a given recording was spoken by a single person, and the same for all male speech. In long-forms, we cannot tell how many talkers there are, nor attribute specific vocalizations to specific talkers. Analyses of the corpus our English long-form data comes from suggests that infants heard between 12 and 46 different talkers in a day (Bulgarelli, private communication, 2023-09-05), with about 64% of speech coming from a single talker, often the mother (Bulgarelli, Mielke and Bergelson, 2021).</footnote>
The ABX discrimination test was built from 10 hours of speech downloaded from Common Voice (Ardila, Branson, Davis, Henretty, Kohler, Meyer, Morais, Saunders, Tyers and Weber, 2020) for each of the two target languages, English and French. More information about the test data can be found in Appendix C. The evaluated phonetic inventory for each of the two target languages can be found in Supplementary Table 4.

Based on this ABX discrimination test, we consider two outcome measures, which we call native discrimination and native advantage. *Native discrimination* is the average ABX accuracy of both learners when tested on all contrasts of the language they have been exposed to, measuring how good the learner is at discriminating sounds of their native language. *Native advantage* is the relative difference between native and non-native learners averaged across the two languages. It measures to which extent our learners develop a language-specific perceptual space. A positive native advantage indicates that the native learner is better at discriminating the sounds of its native language than the non-native learner is on the same sounds, similarly to what we observe in infants towards the end of the first year of life Kuhl et al. (2006). A negative native advantage indicates the opposite, and a native advantage of 0 indicates that the native and the non-native learners are equally good at discriminating the same set of sounds.

## 2.2. Results and discussion

Panel (a) of Fig. 4 shows the *native discrimination* of our base learner (in blue) and our biased learner (in orange). In all conditions, learners outperformed the baseline (the dashed black line), indicating performance improvements with language exposure over generic auditory processing. That said, performance was not equivalent throughout.

A comparison between the two blue bars in this panel indicates that our base learner is better at discriminating sounds of its native language when trained on highly curated audio from audiobooks (result that conceptually replicates Schatz et al., 2021) than when trained on ecological audio from long-forms, with a drop in performance of 5%.

Our biased learner (in orange), endowed with inductive biases, performs much better when exposed to long-forms, reaching the same level of native discrimination as the base learner trained on curated data. These results suggest that inductive biases help overcome difficult learning conditions found in long-forms. Of note, the Voice Activity Detection bias allows this learner to accumulate more speech experience when matching for total duration, as we did in the present experiment. We address the impact of input quantity in Experiment 2.

Panel (b) of Fig. 4 shows the *native advantage* obtained by both types of learners. First, let us consider our base learner. When trained on audiobooks, our base learner is 9.7% better at discriminating native sounds than the same non-native learner (first blue bar; a conceptual replication of Schatz et al., 2021). That is, the American English model has learned to discriminate American English sounds better

<figure><img src="image_5.png" /><figcaption>**Figure 4: Results of Exp. 1: The impact of inductive biases.**
(a) Native discrimination and (b) native advantage obtained by our base learner (trained without inductive biases, in blue) and our biased learner (trained with inductive biases, in orange). Both learners are trained on 128 hours of audio in each of the two target languages (American English and Metropolitan French) as a function of data type (audiobooks, long-forms). The dashed black line corresponds to the score obtained by a spectral representation typically used for speech recognition (39 mel-frequency cepstral coefficients). Each dot represents an independent replication on a separate training set of 128 hours. The number of independent replications, i.e., number of points, for each condition depends on the amount of data available.</figcaption></figure>

than the Metropolitan French model (and vice-versa). However, our base learner trained on ecological long-forms fails to show any native advantage (second blue bar). In other words, when exposed to realistic data, the American English learner discriminates American English sounds as well as the Metropolitan French learner (and vice versa). Therefore, our base learner trained on long-forms has failed to develop a language-specific perceptual space.

Turning to our biased learner in orange, we observe that the inductive biases allow the learner to exhibit a native advantage when faced with long-forms. However, unlike native discrimination results, performance is not matched to that with curated data, which suggests that native advantage is relatively more fragile to noisy environments than native discrimination. This result underlines the importance of considering biases in the context of ecological data, an issue to which we return in subsequent experiments.

## 3. Experiment 2: the effect of data quantity

Experiment 1 suggested that inductive biases helped overcome difficulties found in long-forms; in particular, inductive biases allowed our learner trained on 128 hours of audio to develop a language-specific perceptual space. Before concluding that inductive biases are truly *necessary* for
our learner to develop a language-specific perceptual space,
we must ask: could difficulties found in long-forms disappear
when learning on *more* audio? After all, in machine learning,
one can often replace specially engineered mechanisms with
generic ones fed with enough data.<fnref n="2" /> Moreover, since in
Experiment 1 our biased learner experienced relatively more
speech, it may seem like the comparison is not appropriate.
In this Experiment 2, we vary the total duration of exposure
(see also Experiment 4, which keeps the speech exposure to
study the effects of non-speech exposure).

**3.1. Methods**

In Experiment 1, learners were provided with 128 hours
of audio. In the present Experiment, learners were provided
between 8 and 1,024 hours, depending on the condition.
We generated mutually exclusive training sets of varying
lengths. Given limitations in data availability, we could
provide our base learner up to 1,024h from audiobooks or
512h from long-forms (containing both speech and non-
speech). As for our biased learner, it receives up to 1,024h
from audiobooks, or up to 128h from long-forms (contain-
ing exclusively speech). We vary the quantity of data by
splitting larger training sets into two mutually exclusive
subsets. Hence, our base learner exposed to audiobooks is
trained separately on 2 training sets of duration 512h, 4 of
duration 256h, etc. Our biased learner exposed to long-forms
is trained separately on 2 training sets of duration 64h, 4 of
duration 32h, etc (see Appendix A for more details).

Apart from the training data quantity, we employ the
same auditory baseline, learners, and ABX discrimination
test as in Experiment 1.

**3.2. Results**

Let us first focus on the top panels (a) and (b) of Fig.
5 showing the native discrimination and native advantage
obtained by learners trained on audiobooks. Panel (a) shows
that the native discrimination obtained by learners supple-
mented with inductive biases plateaus at 89.4% after 128
hours of data, resulting in native learners that are 16.6% bet-
ter at discriminating native sounds than non-native learners
(solid orange lines of panels (a) and (b)). Without inductive
biases (blue curve), the base learner exhibits significantly
worse performance, both in terms of native discrimination
and native advantage. However, on the 1024-hour-long train-
ing set, both types of learners eventually catch up. This
can be explained by the highly curated nature of the data
our learners are trained on, as we will see in subsequent
Experiments.

The bottom panels (c) and (d) illustrate radically dif-
ferent trends when learners are exposed to ecological long-
forms. Starting with the base learner (blue dot-dot-dashed
lines), its native discrimination plateaus at 81.3% after 64
hours of data, with no benefit from additional exposure; and
its native advantage remains relatively flat and close to 0%
 panel (d)). In contrast, our biased learner exhibits a higher

<figure><img src="image_6.png" /></figure>

**Figure 5: Results of Exp. 2: Effect of the quantity of data.**
Comparison of our base learner (trained without inductive biases, in blue) and our biased learner (trained with inductive biases, in orange). Top panels (a) and (b) show the native discrimination and the native advantage obtained by learners trained on audiobooks. Bottom panels (c) and (d) correspond to learners trained on long-forms. Each learner is trained on either American English or Metropolitan French. Error bars represent standard errors computed across mutually exclusive training sets whose number depends on the quantity of data.

discrimination performance and develops a positive native
advantage with increasing exposure. The contrast across
learners is even more striking when we consider quantity
of speech, rather than quantity of audio. Indeed, one of the
concerns above was that the comparison was unfair because
the biased learner experienced relatively more speech when
matching for total duration. We can now compare the base
learner trained on 512 hours of long-forms who receives
approximately 100 hours of speech (Table 1) against the bi-
ased learner receiving exposures of lower duration. In terms
of native advantage, the base learner trained on 512 hours
of long-forms (roughly 100 hours of speech) does not even
reach performance comparable to that of the biased learner
trained on 32 hours of long-form speech. Thus, quantity
of speech alone is not sufficient to explain the difference
in behavior we observe between our learners trained with
or without inductive biases. Instead, it suggests that the
inductive biases themselves are responsible for the radically
different behavior we observe.

**3.3. Discussion**

This Experiment shows that it does not take a great
deal of highly curated audiobooks for our base learner to
develop a native advantage. However, on long-forms, our
base learner plateaus after 64 hours of audio and fails to de-
velop a language-specific perceptual space, unlike what we
observe with our biased learner. Interestingly, a systematic

<footnote n="2">²The Bitter Lesson of Rich Sutton: http://incompleteideas.net/IncIdeas/BitterLesson.html</footnote>
review suggests infants accumulate between 60 and 1,000h of infant-directed speech in the first year of life (Cristia, 2019), and there is about 128h of speech in our 1,024h-long long-form dataset. Although the present manipulation covers only the low end of this range, the plateau observed by 64h of audio (roughly 12h of speech) in the long-form condition suggests that further increasing the training set size would not result in the emergence of perceptual attunement. Thus, from this work, it does appear to be the case that, when faced with *ecological long-forms*, inductive biases are *necessary* for our learner to develop a language-specific perceptual space and to reproduce perceptual attunement.

One may wonder if a more powerful learning algorithm with no inductive biases would succeed in reproducing perceptual attunement from child-centered long-form recordings – perhaps with even more data. In our view, a more powerful pure learning algorithm may pick up on more of the structure of noise and distortions, leading to a more general and non-language-specific perceptual space. Nonetheless, we may be proven wrong by further work.

# 4. Experiment 3: ablation study

Experiments 1 and 2 strongly suggest that our three inductive biases are crucial for perceptual attunement to emerge from long-form recordings. But what is their individual contribution? One way to assess this is via an *ablation study*, i.e., one in which each component is turned off. This is what we do in the present Experiment.

## 4.1. Methods

In addition to using the same base and biased learner as in Experiment 1, we also create other learners in which each of the inductive biases (Voice Activity Detection, Pseudo Speaker Separation, Data Augmentation) can be turned on or off. Note that Pseudo Speaker Separation cannot be activated without Voice Activity Detection also being activated. This results in six learners. We do not include a baseline since it is the same as in Experiment 1. The same ABX discrimination test is also used as in Experiment 1.

## 4.2. Results and Discussion

Inspection of Table 2 suggests that for the audiobooks, the Voice Activity Detection and the Pseudo Speaker Separation mechanisms have little effect on the performance obtained by our base learner (lines 1, 2, and 5). This is most certainly due to the high proportion of speech and the long speaker turns found in audiobooks (see Table 1). Indeed, the base learner trained without the Voice Activity Detection mechanism already receives 95% of speech, and our results show that removing the 5% non-speech remaining does not have much of an impact. Similarly, as audiobooks are organized in long stretches of speech read by the same speaker, the base learner already receives single-speaker utterances, and including the Pseudo Speaker Separation mechanism has little to no impact on the learning outcomes.

It appears that Data Augmentation mechanism is the most impactful mechanism, with a drop of up to 5% absolute

native advantage when it is not included (lines 1 and 3). However, note that the results presented in Table 2 only include the performance of learners trained on 128 hours of audio. Fig. 5, presented earlier, suggests that the Data Augmentation mechanism has a beneficial impact only for low data quantities as the base and the biased learner eventually catch up when trained on 1,024 hours of audio. This is typical of data augmentation methods, which help increase the training data's diversity and quantity but are less helpful when applied to a larger training set.

The results obtained by the learner trained on long-forms suggest that, among all three inductive biases, Voice Activity Detection plays the most crucial role, with a drop of up to 3.2% absolute native discrimination when it is not included (lines 7 and 8). The learner trained with both the Voice Activity Detection and the Pseudo Speaker Separation mechanisms reaches a similar performance than the learner trained with the Voice Activity Detection mechanism alone (lines 8 and 11). However, the best native discrimination and the best native advantage are obtained by our biased learner, i.e., trained with all three inductive biases, which suggests that all three biases contribute to the emergence of a native advantage (line 12).

# 5. Experiment 4: the effect of the speech/non-speech ratio

In this Experiment, we focus on the largest difference between long-forms and audiobooks: the proportion of speech as opposed to non-speech in the recordings (20% of speech and 80% of non-speech in long-forms versus 95% of speech and 5% of non-speech in audiobooks; Table 1). This difference is the primary reason why we introduced a filtering mechanism (the voice activity detection, VAD, Fig. 3) to shield our learner from the large quantity of non-speech present in long-forms. In the current Experiment, we ask: What is the real impact of this particular inductive bias? We study this by providing our biased learner with input containing varying amounts of non-speech, as if their VAD mechanism was dysfunctional.

## 5.1. Methods

To better control for audio quality and quantity, we employ the same 128h audiobooks training set as in Experiment 1. We simulate a dysfunctional VAD mechanism that allows non-speech to leak in by adding to the training set varying amounts of non-speech sections extracted from long-forms. Thus, the learner always receives 128h of speech, but the total quantity of input audio data varies, so that the proportion that is speech varies across 4 conditions, from 100% speech to an eighth of speech (i.e., 128h of speech in 1024h of audio). We employ the same biased learner as in Experiment 1, except that the learner used in the present Experiment comes with the dysfunctional VAD mechanism described above. The auditory baseline and machine ABX discrimination test are the exact same as those used in Experiment 1.
<table><caption>**Table 2**</caption><thead><tr><th>Corpora</th><th>#</th><th>Inductive biases</th><th>Native discrimination (%)</th><th>Native advantage (%)</th></tr></thead><tbody><tr><td rowspan="6"> Audiobooks </td><td> 1 </td><td> none </td><td> 87.37 </td><td> 11.01 </td></tr><tr><td> 2 </td><td> VAD </td><td> 87.73 </td><td> 12.95 </td></tr><tr><td> 3 </td><td> DA </td><td> 88.98 </td><td> 16.00 </td></tr><tr><td> 4 </td><td> VAD + DA </td><td> 89.37 </td><td> 14.87 </td></tr><tr><td> 5 </td><td> VAD + PSS </td><td> 87.37 </td><td> 10.70 </td></tr><tr><td> 6 </td><td> VAD + PSS + DA </td><td> 89.51 </td><td> 16.53 </td></tr><tr><td rowspan="6"> Long-forms </td><td> 7 </td><td> none </td><td> 81.41 </td><td> -0.15 </td></tr><tr><td> 8 </td><td> VAD </td><td> 84.64 </td><td> 1.32 </td></tr><tr><td> 9 </td><td> DA </td><td> 83.96 </td><td> 0.63 </td></tr><tr><td> 10 </td><td> VAD + DA </td><td> 84.64 </td><td> 1.50 </td></tr><tr><td> 11 </td><td> VAD + PSS </td><td> 84.53 </td><td> 1.32 </td></tr><tr><td> 12 </td><td> VAD + PSS + DA </td><td> 85.09 </td><td> 2.45 </td></tr></tbody></table>

**Results of Exp. 3: Ablation study.** All models are trained on 128 hours of audio taken from audiobooks (top panel) or long-forms (bottom panel). VAD stands for Voice Activity Detection. PSS stands for Pseudo Speaker Separation. DA stands for Data Augmentation. The PSS mechanism can not be activated without the VAD mechanism. Lines numbered 1 and 7, where none of the inductive bias is activated, correspond to the base learner. Lines numbered 6 and 12, where all of the inductive biases are activated, correspond to the biased learner.

<figure><img src="image_7.png" /><figcaption>**Figure 6: Results of Exp. 4: Simulating a dysfunctional voice activity detection mechanism.** Native discrimination (panel (a)) and native advantage (panel (b)) obtained by our learner supplemented with inductive biases, depending on the proportion of speech (in green) and non-speech (in pink) in the training set. Learners are trained on 128 hours of English and French audiobooks (first column) to which we added a fixed quantity of non-speech segments drawn from long-forms: 128 hours (second column), 384 hours (third column), and 896 hours (fourth column). The dashed black line corresponds to the score obtained by a spectral representation typically used for speech recognition (39 mel-frequency cepstral coefficients).</figcaption></figure>

## 5.2. Results and discussion

Focusing first on panel (a) of Fig. 6, results indicate that the lower the proportion of speech, the lower the native discrimination. Regarding native discrimination, the biased learner trained exclusively on speech (128 hours) scores

4.2% higher than the same learner trained on only an eighth of speech, with the rest being non-speech (corresponding to 128 hours of speech and 896 hours of non-speech).

Importantly, results show that our biased learner is robust to a certain amount of non-speech, as shown by the native discrimination of 88.4% developed by learners trained on half of speech, half of non-speech (128 hours of each). More surprisingly, the learner trained on an eighth of speech still exhibits a higher native discrimination than the auditory baseline (dashed black line). This shows that despite the important quantity of non-speech segments in the training set, the learner still develops a perceptual space in which sounds are accurately discriminated.

Turning to panel (b), the effect of the proportion of non-speech segments over speech segments is even greater on the native advantage. Indeed, learners trained exclusively on speech exhibit a native advantage of 16.5%, while this goes down to 2.5% for learners trained on an eighth of speech.

Together, these results suggest that the proportion of non-speech segments in the input harms both the native discrimination and the native advantage. Although it is difficult to provide precise evidence on this, we hypothesize that the perceptual space the learner develops during training becomes less relevant to discriminate between speech sounds as the subspace allocated to discriminate between non-speech sounds grows.

# 6. Experiment 5: sensitivity to noise and reverberation

In this Experiment, we turn to another important difference between audiobooks and long-forms: the *quality* of the speech signal itself. More specifically, how do the different degradations that occur in ecological data affect the perceptual space developed by our learner? To answer this question,
we trained our learner on audiobooks that we corrupted by manipulating two major factors: additive background noise and reverberation, which are prevalent in long-forms. In this Experiment, we examine performance obtained by our biased learner because: 1) simulating naturalistic non-speech sounds found in naturalistic data (required by the base learner that must be trained on 80% of non-speech to match the long-form condition) is not trivial as audiobooks contain almost exclusively speech; 2) if any detrimental effect is observed on the performance obtained by our biased learner, then it is very likely that an even bigger effect will be observed on our base learner, as the latter does not include mechanisms aimed at overcoming difficulties found in naturalistic data.

**6.1. Methods**

To study the effects of audio degradations, we created a new dataset by contaminating the audiobook data used in Experiment 2 with additive noise and reverberation. For the additive noise, we extracted various domestic noises from long-forms, as detailed in Appendix D. For the reverberation, we used two sets of impulse responses, the MIT Acoustical Reverberation set (Traer and McDermott, 2016) and the EchoThief impulse response library (Warren, 2013), whose combination resulted in 385 impulse responses acquired in a wide variety of places. We then applied these two sources of contamination as follows.

We started with speech segments extracted from the English and French audiobooks. For each speech segment, we built a noise segment of the same duration by crossfading successive noise sequences with a crossfade duration of 50 ms. To simulate a realistic acoustical scene, we randomly chose two impulse responses: the first was convoluted with the speech segment, and the second with the noise segment. It is only when the speech and the noise segments have been convoluted that they are normalized and added together. This effect makes the speech segment seem to exist in one location and the noise segment in another. Using this pipeline, we examine the audiobook corpora corrupted with: 1) reverberation only; 2) additive noise only for a signal-to-noise ratio uniformly sampled between 0 and 15 dB; and 3) both reverberation and additive noise (which we called simulated long-forms condition, and for which a comparative analysis in terms of SNR and $C_{50}$ can be found in Appendix E). We employ the same biased learner and the same machine ABX discrimination test as in previous experiments.

**6.2. Results**

In general terms, Fig. 7 shows that challenging acoustic conditions negatively impact the learning outcomes of our biased learner, yielding lower native discrimination and lower native advantage for all conditions compared to the clean one.

Applying additive noise to the training set causes only a slight decrease in the performance obtained by our biased learner. The drop in performance is more important when reverberation is applied. And the worst performance

<figure><img src="image_8.png" /></figure>

**Figure 7: Results of Exp. 5: Sensitivity to noise and reverberation.** Panel (a) and (b) indicate the native discrimination and the native advantage, respectively, obtained by our biased learner (trained with inductive biases) as a function of both the quantity of speech, and varying acoustic conditions. Learners are trained on clean read-speech segments extracted from audiobooks (solid line). These clean read-speech segments are corrupted with 1) additive noise for a signal-to-noise ratio (SNR) randomly chosen between 0 and 15 dB (dashed line); 2) reverberation (dotted line); 3) both reverberation and additive noise (dash-dotted line). The latter corresponds to the simulated long-form condition. Error bars represent standard errors computed across mutually exclusive training sets whose number depends on the quantity of data.

is observed when both additive noise and reverberation are applied to the training set.

May this negative effect be overcome by adding more data? Inspection of the curves as a function of data quantity suggests that our learner's native discrimination and native advantage both plateau after 128 hours of speech, regardless of the acoustic condition, and thus more data is unlikely to solve the problem. Therefore, it appears that additive noise and reverberation strongly degrade representations learned during training, leading to a less language-specific perceptual space in situations where signal and noise compete with each other than in cleaner conditions.

**6.3. Discussion**

Results in this Experiment replicate those in Experiment 2: An increase in the quantity of data yields a higher native discrimination and a higher native advantage, regardless of the acoustic condition, although performance eventually plateaus. The present Experiment adds an important caveat: the perceptual space developed by our learner becomes less language-specific (i.e., lower native advantage) when it is exposed to more challenging acoustic conditions, with reverberation having a massive negative effect, and additive noise a substantially smaller one. Thus, our biased learner is sensitive to the lower acoustic quality found in ecological recordings, and the ensuing negative effect cannot be solved by increasing the quantity of data.
# 7. Experiment 6: using audiobooks to simulate long-form recordings

In Experiments 4-5, we covered the impact of some of the most important differences between audiobooks and long-forms, namely the proportion of speech, and their quality in terms of presence of additive noise and reverberation. In this final Experiment 6, we ask a slightly different question connected to the importance of understanding the differences between audiobooks and long-forms for the purposes of modeling. Audiobooks are easier to collect and share than long-forms, and simulated long-forms could be a useful tool to calibrate the amount of data collection needed to obtain enough statistical power in a computational modeling study. So, can we use *simulated long-forms* as a proxy for real long-forms? To answer this, we carry out a direct comparison between our learner trained with inductive biases on long-forms, and the same learner trained on simulated long-forms.

## 7.1. Methods

We focus on the simulated long-forms that are most challenging, those with reverberation and additive noise for a random SNR between 0 and 15 dB (dash-dotted orange line of Fig. 7). As in Experiments 2 and 5, we vary the quantity of data. In this Experiment, we employ only the biased learner and the same ABX discrimination test as in Experiment 1.

## 7.2. Results and discussion

Fig. 8 shows the native discrimination and native advantage obtained by our biased learners when trained on varying quantity of speech extracted from long-forms, or simulated long-forms. Learners have essentially identical native discrimination when trained on long-forms or simulated long-forms (panel (a)). Overall, the close match between learners trained on real and simulated data indicates that noise and reverberation capture the essential components of ecological recordings and can be used to simulate the performance obtained by our learner.

One could expect a similar match in terms of native advantage. Surprisingly, however, learners trained on long-forms exhibit a lower native advantage than learners trained on simulated long-forms (relative difference of 37% for models trained on 128 hours of speech, panel (b)). This mismatch in terms of native advantage must then be due to other characteristics of ecological speech that are not captured in our simulated long-forms. Beyond acoustic characteristics, audiobooks likely have lower speech rate, greater hyperarticulation, and are primarily composed of complex utterances with diverse lexical items, features that in the language acquisition literature have been argued to be beneficial for infant learning (Rowe, 2012; Weisleder and Fernald, 2013; Anderson, Graham, Prime, Jenkins and Madi gan, 2021). In contrast, ecological data contains infant-directed speech which has been claimed to facilitate learning (Eaves Jr, Feldman, Griffiths and Shafto; Adriaans and Swingley, 2016; 2017, but see McMurray, Kovack-Lesh, Goodwin and McEchron, 2013 for a different view). Further investigation is needed to verify the contributions of these

<figure><img src="image_9.png" /><figcaption>**Figure 8:** *Results of Exp. 6: Comparing real against simulated long-forms* Comparison of native discrimination (panel (a)) and native advantage (panel (b)) of learners trained with inductive biases on long-forms (dot-dot-dashed line) or simulated long-forms (dash-dotted line). Simulated long-forms are generated using read-speech retrieved from audiobooks whose speech segments are corrupted with additive noise and reverberation. Error bars represent standard errors computed across mutually exclusive training sets whose number depends on the quantity of data.</figcaption></figure>

additional characteristics on artificial learners. In the meantime, we conclude that simulated long-forms cannot replace realistic long-forms when modeling perceptual attunement.

# 8. General discussion

We carried out six experiments investigating how the design of a learner, with a prediction-based learning mechanism at heart, as well as the type of data it receives affect perceptual attunement. These are summarized in Table 3. We discuss below limitations of our study and ways in which future modeling work can improve upon it. Indeed, we believe our study’s most salient contributions are in the field of modeling. However, before doing so, we point out three predictions for human infant data drawn from the present work, as we hope to see increased interactions between model- and human-based research.

**Predictions for human infant studies and future experimental work.** The first prediction is that quantity of input is beneficial to infants’ ability to discriminate native-language phonetic contrasts (Fig. 5 and 7, and Li, Schatz, Matusevych, Goldwater and Feldman; Schatz et al., 2020a; 2021 for relevant previous results). Our results suggest that the beneficial effect of input quantity still holds in challenging acoustic environments, although with different slopes (Fig. 7), for both types of learners (Fig. 5). This prediction is largely compatible with current views of language acquisition and empirical evidence correlating input quantity with vocabulary and standardized language tests (Weisleder and Fernald; Gilkerson, Richards, Warren, Oller, Russo and Vohr, 2013; 2018, but see Sperry, Sperry and Miller, 2019 for a different view). The evidence for input quantity effects on native advantage specifically is scarce and equivocal (Cristia, 2020), but at least one study has found compatible results (Marklund, Schwarz and Lacerda, 2019).
<table><tr><td>Experiment 1</td><td>Our base learner trained on audiobooks reproduces perceptual attunement in English and French<br/>The same learner fails to do so when trained on long-forms<br/>Inductive biases allow for full recovery of native discrimination, and partial recovery of native advantage</td></tr><tr><td>Experiment 2</td><td>Our base learner never reproduces perceptual attunement on long-forms, even with more data as performance plateaus</td></tr><tr><td>Experiment 3</td><td>Inductive biases work in concert to achieve perceptual attunement, showing a supra-additive effect<br/>The higher performance gain is brought by the voice activity detection mechanism</td></tr><tr><td>Experiment 4</td><td>When non-speech is added, native discrimination diminishes and native advantage vanishes</td></tr><tr><td>Experiment 5</td><td>When background noise and reverberation are applied, native discrimination diminishes and native advantage vanishes, despite the data augmentation mechanism, and regardless of data quantity</td></tr><tr><td>Experiment 6</td><td>Long-forms and simulated long-forms yield similar native discrimination, but different native advantage, i.e., there remain other differences between long-forms and simulated long-forms (e.g., hyperarticulation found in read speech)</td></tr></table>

**Table 3**

Summary of key results

The second prediction is that both acoustic and speech quality positively impact infants' ability to discriminate native-language phonetic contrasts. We have shown, in Fig. 7, that poor signal quality of speech segments yields lower discrimination accuracy and poorer language attunement. We found a similar result when biased learners were partially trained on non-speech segments (Fig. 6). To the best of our knowledge, input quality thus defined has not been linked to infants' discrimination abilities of phonetic contrasts yet.

We also recommend additional studies to check whether human infants have filters like the ones with which we endow our biased learner, namely processing only speech, streaming different voices, and processes like our data augmentation. The evidence for a voice activity detection mechanism is very strong: Infants display a stable preference for speech over other types of sounds (Issard, Tsuji and Cristia, 2023), demonstrating, therefore, an early ability to discriminate between speech and non-speech segments. As for the pseudo speaker separation and the pitch augmentation mechanisms, evidence suggests that newborns can discriminate even among unfamiliar voices (Decasper and Prescott, 1984; Floccia et al., 2000), and that this may engage a different brain network than distinguishing between speech sounds (Paquette, Dionne-Dostie, Lassonde and Gallagher, 2018). Previous evidence also shows that this does not prevent speech processing across diverse voices: Infants as young as 4 months have been found to compute speech representations that are invariant with respect to a change in speaker (Seidl et al., 2014; Bergmann et al., 2016; Choi and Shukla, 2021). For example, Kuhl (1983) demonstrated that infants can perceive distinct occurrences of the same vowel category as equivalent, despite these occurrences being generated by a speech synthesizer simulating productions of men, women, and children. Regarding our last imputed bias, invariance with respect to a change in reverberation, there is, to our knowledge, no experimental evidence documenting a similar phenomenon in infants – but see Beeston et al. (2014) for evidence in adults.

Further work is needed to shed light on how infants may come to have such biases. We suspect that these inductive biases are, to a certain extent, inherited from our evolutionary past, and to another extent, learned. This is similar to theoretical proposals for face perception. Inspired by work on chicks (and other social vertebrates), Morton and Johnson (1991) proposed two mechanisms that work in concert to explain human infants' preference for faces from birth, together with strong changes over the course of development, CONSPEC and CONLERN. CONSPEC is an inductive bias that guides human newborns' preference for human face-like patterns, and CONLERN allows them to learn facial features of specific individuals. Extrapolating to speech, we suspect that a better understanding of how these (or other) biases come to be active in human infants will minimally require a concerted theoretical and empirical effort by modelers, developmental scientists, and ethologists.

Finally, we firmly believe that considering input data that do not reflect characteristics of children's language environments can lead us to underestimate the complexity of a given learning problem (that is, we may believe a problem has been solved when it has only been solved on unrealistic data), or overestimate the power of a proposed mechanism (that is, we may believe a mechanism is sufficient to explain the phenomenon that is being modeled when it is not). This is not to say that simplification of the input data in computational modeling studies is an insightless enterprise. In fact, checking computational models against data, which may not necessarily reflect real-world properties can help us gain a better understanding of the phenomena being modeled. However, we argue here that a computational model of language acquisition should not only be evaluated based on its ability to reproduce infants' developmental trajectories, but also on its ability to learn from input that is as close as possible to the sensory signals infants actually experience.

**Limitations and further modeling work.** Our study is, to our knowledge, the first fully-implemented proof of a
model for early phonetic learning that exhibits perceptual
attunement after exposure to realistic child-centered audio
data. But we are certain it will not be the last one. Despite the
benefits of the proposed inductive biases in improving native
discrimination and enabling the emergence of perceptual
attunement, our biased learner trained on long-forms still
did not achieve the same level of native advantage as the
base learner trained on audiobooks (as shown in Figures 4
and 5). If we want to tackle the complexity inherent in long-
forms, it is very likely that we should aim at finding what
other inductive biases might guide phonetic learning in our
algorithms.

Indeed, as shown in Experiment 5, our biased learner
is still greatly sensitive to additive background noise and
reverberation that are ubiquitous in children learning envi-
ronments. This background noise issue could be alleviated
with an additional mechanism filtering out segments with
a low speech-to-noise ratio. Such a filter remains plausible
as evidence shows that 4.5-month-olds prefer listening to
speech in a quiet rather than in a noisy environment, sug-
gesting that they are aware of the presence of noise (Newman
and Hussain, 2006). Similarly, overlapping speech, found in
long-forms but not in audiobooks, may necessitate an ad-
ditional separation mechanism. Indeed, overlapping speech
represents a challenging source of data for phonetic learning
as sounds produced by different speakers blend together.
Such a source separation mechanism may take place in
infants as evidence for the so-called 'cocktail party effect'
(our ability to focus our auditory attention on a particular
source while filtering out other competing sources) has been
found at 7.5 months (Newman, 2005). Finally, long-forms
contain both hyperarticulated infant-directed speech and
spontaneous speech that may sometimes be hypoarticulated.
It is possible that a mechanism forcing the algorithm to
learn preferentially on better articulated speech segments
may help improve both native discrimination and native
advantage. These three types of biases could meaningfully
adopt by and large the same approach we had here, including
minimally audiobooks and long-forms as input, and utilizing
an ablation approach to assess the contribution of individual
components.

In this study, we examined if our learners could re-
produce perceptual attunement such as measured by the
native advantage. However, most research on perceptual
attunement in humans only covers a few contrasts (Singh
et al., 2022) and it is unclear to which extent human infants
or adults exhibit a native advantage on other contrasts that
remain to be studied. In the absence of systematic data
across contrasts and populations, we do not know the size
of infants’ native advantage and are limited to a quantitative
comparison between infants and artificial language learners.
However, further work could aim at comparing the effect size
of the native advantage obtained by our learners with those
of infants such as provided by meta-analytic studies (see
methodology proposed in Blandón, Cristia and Räsänen,
2021).

Other aspects not addressed here will require substan-
tially different modeling decisions. Easiest to model are
the potential effects of restricting input to that which is
high quality, by attending more (or exclusively) to vocaliza-
tions that have a high signal-to-noise ratio, are not reverber-
ated, or are in prosodic focus. For instance, Adriaans and
Swingley (2017) showed that, in one corpus of American
English infant-directed speech, separability between vowel
categories was greater for sounds that were prosodically
exaggerated (having higher pitch, greater duration of some
sounds, etc.) That said, the models in Ludusan, Mazuka and
Dupoux (2021) performed less well on infant- than adult-
directed Japanese, although the former is cued with precisely
those modifications. Some experimental evidence suggests
that infants’ first productions is promoted by caregivers’
pitch variability (including in the very infants whose input
serves as our English corpus; Bulgarelli and Bergelson,
2023). Bridging the gap across these diverse results, on
different languages, model or human learners, will require a
consideration of the diverse ways in which biases could oper-
ate, particularly in infants. For instance, infants’ first produc-
tions may favor words spoken with higher pitch not because
higher pitch causes better learning but rather because higher
pitch is associated with moments of more positive valence. A
model that is endowed with a bias towards high pitch would
be right for the wrong reasons.

Another dimension that could be explored relates to talker and language variability. By employing long-forms, we were able to capture children’s realistic input, but we did not have any control over aspects of exposure that could be relevant, including the number and diversity of speakers’ voices (likely more relevant to machines than human infants; see Li, Schatz, Matusevych, Goldwater and Feldman; Bergmann et al., 2020b; 2016). It is possible that advances in automated speaker diarization may enable description of the number and frequency of speakers in long-form data, although this kind of data has been dropped from speech technology efforts due to its challenging nature. Another dimension that could be studied involves exposure to multiple languages, which is again hard to simulate, but could be fruitfully analyzed using long-form recordings, although it will be hard to control precise exposure to each language.

Beyond inductive biases that apply to the auditory
stream, infants may use more than just audio to learn about
the phonemes of their native language. Visual cues, such as
lip movements, may complement the audio channel and help
infants achieve better and possibly faster learning. Modeling
research combining other modalities typically uses even
more manicured input than speech-only work (with the
exception of Alishahi, Chrupała, Cristià, Dupoux, Higy,
Lavechin, Räsänen and Yu, 2021).

At the time of writing, large scale naturalistic audio-
visual long-form data does not exist. It would be necessary
for these data to be similarly ego-centered as our long-
forms, since modeling development from a third- versus a
first-person perspective implies distinct challenges. Recent
discussions also highlight the potential role of supervisory
signals McMurray (2022), considering the overwhelming
evidence that perception continues to develop beyond the
first year. While promising, incorporating supervisory feed-
back in our learner would require substantially different
modeling approaches than employed here, both in terms of
input and of mechanisms.

Additional dimensions will be relevant to address per-
ceptual attunement in the context of broader theories of
language, including "language as a description of the world"
and "language as a social construct". As to the latter, fu-
ture work could investigate the effect of additional filters
relying on social cues. For instance, one could examine
the effect of training only when an adult speaker directly
interacts with the child, or when there are a high number
of conversational turns between the child and an adult – see
Kuhl, Tsao and Liu (2003); Lytle et al. (2018) for evidence
of improved perceptual learning in the presence of peers.
That said, For instance, Kuhl et al. (2003) showed that
9-month-old American English-learning infants who spent
time in laboratory sessions with native Mandarin speakers
can discriminate between phonemes that occur in Mandarin,
whereas infants exposed to the same input via audiovisual
or audio-only recordings cannot. Social interactions may
typically be the kind of situations where the input signal is
of better quality (e.g., close proximity between the child and
their interlocutor, less distracting background noises, etc.).
In return, this high-quality signal could facilitate speech
sound discrimination and language attunement, as has been
shown for our learner in Experiment 6.

9. Conclusion

Recent advances in machine learning algorithms that
learn from raw speech, together with ongoing efforts to
acquire large-scale datasets of infants' language environ-
ments, make building ecologically valid models of language
acquisition feasible. Through six experiments, we argued
that prediction-based learning leads to language-specific
perception provided it is focused on clean speech signal,
either as input or thanks to processing mechanisms; and that
such biases are required given the uniquely difficult input
data afforded to human infants. Evidently, our work does
not constitute a proof of infeasibility. Perceptual attunement
to sounds may be feasible with other algorithms that are
yet to be discovered. However, we strongly believe that
for researchers to discover such mechanisms, computational
models of early language learning must address the problem
in its full complexity, learning from the same data than that
available to infants. In that regard, we believe that our work
paves the way for a deeper understanding of mechanisms
involved in early language acquisition.

Data and code availability

We used audiobooks – publicly available from LibriVox (Kearns, 2014) – and child-centered long-forms<fnref n="3" /> to train our learners. Long-forms are available on HomeBank and via ChildProject repositories (Bergelson, 2017; Canault et al., 2016a; Cristia, 2021; Lavechin and Cristia, 2021). Access to long-forms requires ethical training and full approval by the principal investigator(s) in charge. Code to reproduce the experiments are available in https://github.com/MarvinLvn/CPC2.

Acknowledgements

We are grateful to DARCLE, LAAC, and CoML mem-
bers for helpful discussion. All errors remain our own.
A.C. gratefully acknowledges financial and institutional
support from Agence Nationale de la Recherche (ANR-
16-DATA-0004 ACLEW, ANR-17-EURE-0017); the J. S.
McDonnell Foundation (Understanding Human Cognition
Scholar Award); and the European Research Council (ERC)
under the European Union's Horizon 2020 research and
innovation programme (ExELang, Grant agreement No.
101001095). E.D., in his academic role (EHESS), acknowl-
edges funding from Agence Nationale de la Recherche
(ANR-19-P3IA-0001 PRAIRIE 3IA Institute), a grant from
CIFAR (Learning in Machines and Brains), and the HPC
resources from GENCI-IDRIS (Grant 2020-AD011011829).
M.S. acknowledges PhD funding from Agence de l'Innovation
de Défense.

A. Training sets

In this section, we first describe our data sets, and the
differences between long-forms and audiobooks. Next, we
describe how the ABX evaluation set has been built to assess
phonetic discrimination of our models, and in particular,
which phonemes have been retained for each of the two
target languages.

For each language (English or French), and each au-
dio source (audiobooks or long-forms), we built separate
training sets by randomly splitting the whole set of audio
segments into mutually exclusive training sets of 8 hours.
8-hours training sets were then merged two by two to build
the 16-hours training sets. This procedure was repeated until
a single training set covering the entirety of the dataset is
obtained. Therefore, the number of subsets depends on the
total amount of data available.

On audiobooks, 128, 64, 32, 16, 8, 4, 2 and 1 training
sets were built whose duration was 8, 16, 32, 64, 128, 256,
512 and 1,024 hours respectively, which contain both speech
and non-speech segments in the case of our base learner; and
only speech as detected by the VAD pretrained model in the
case of our speech-biased learner.

For long-forms, the procedure resulted in 16, 8, 4, 2 and
1 training sets whose duration was 8, 16, 32, 64 and 128

<footnote n="3">³Child-centered long-forms were accessed and stored only by au-
thors outside of Meta AI Research using the GENCI-IDRIS (Grant 2020-
AD011011829) infrastructure.</footnote>
hours respectively containing only speech as detected by the pretrained VAD model. Similarly 32, 16, 8, 4, 2, 1 training sets were built whose duration was 8, 16, 32, 64, 128, 256, and 512 hours respectively (containing both speech and non-speech). In Experiment 1, only the 128 h sets were used. The other sets were created for use in Experiments 2 and 4.

# B. Learners: implementation details

## B.1. Contrastive predictive coding

As proposed in (Kharitonov et al., 2021), the encoder $g_{enc}$ consists of a 5-layer convolutional neural network with kernel sizes [10, 8, 4, 4, 4] and strides [5, 4, 2, 2, 2] that returns a 256-dimensional vector every 10 milliseconds. The auto-regressive model $g_{ar}$ is a 2-layer long short-term memory network of dimension 256. The model is asked to predict up to $K = 12$ time steps in the future (which is equivalent to 120 ms). The predictor $g_{pred}$ is a single multi-head transformer layer with $K = 12$ heads, each predicting at time step $k \in \{1, ..., 12\}$. Negative samples are drawn from sequences that are temporally close to the sequence the positive sample are drawn from. More precisely, creating a batch consists of selecting 64 successive sequences in the case of the domain-general learner (or 64 successive sequences that have been pronounced by the same speaker for the domain-specific learner). For a current sequence $seq_i$, negative samples are taken from all other sequences $seq_j$, with $j \neq i$. All models have been trained on 8 GPUs with batches of 64 sequences, and each sequence has a duration of 1.28 seconds. All models are trained until convergence, and the best epoch is selected according to validation loss (5% of the original training set).

## B.2. Inductive biases

**Voice Activity Detection.** Speech segments were identified using a voice type classification model (Lavechin et al., 2020) that has been specifically trained on a multilingual corpus of long-forms. This pretrained model partitions the audio stream into segments belonging to one or more of the following classes: 1) KCHI, for vocalizations produced by the child wearing the recording device; 2) OCH, for vocalizations produced by any other children; 3) FEM, for female adult speech; 4) MAL, for male adult speech and 5) SPEECH, for when there is speech. When none of the 5 classes were activated, the segment was considered to be non-speech. Only segments belonging to the adult (MAL or FEM categories) were selected; thus, segments with speech by children or containing only non-speech were filtered out.

For long-forms, a total of about 200 hours of speech was extracted this way. Approximately 1000 hours of speech were extracted from the audiobooks.

**Pseudo Speaker Separation.** For long-forms, the category information (MAL, FEM) was combined with the long-form unique identifier to approximate speaker identity. We would have liked to use a speaker diarization system but no such system performs reasonably well on long-forms (García, Villalba, Bredin, Du, Castán, Cristia, Bullock, Guo,

Okabe, Nidadavolu, Kataria, Chen, Galmant, Lavechin, Sun, Gill, Ben-Yair, Abdoli, Wang, Bouaziz, Titeux, Dupoux, Lee and Dehak, 2020). During the CPC training procedure, this results in positive and negative examples that have been pronounced by: 1) the same category (MAL or FEM); and 2) within the same long-form. Note that the second constraint is further reinforced by the Temporal Proximity Sampling mechanism which ensures positive and negative examples are temporally close to one another. For audiobooks, and contrary to long-forms, the speaker identity was retrieved from the metadata, because it is constant throughout each recording. We used this information as-is in the Pseudo Speaker Separation mechanism. Consequently, positive and negative examples during the CPC training procedure were drawn exactly within speakers (and not within pseudo speakers as in the case of long-forms).

**Data Augmentation.** A distinction needs to be made about the way we 1) simulated long-forms by adding additive noise and reverberation on clean-read speech taken from audiobooks (section 6.1); and 2) apply data augmentation methods during training. When simulating long-forms, the training set is contaminated and then stored on disk. In other words, a clean segment of speech will always be associated to the same transformation (additive noise, and reverberation using impulse responses), which has for effect to make the learning process more challenging. In contrast, in the Data Augmentation mechanism, the modifications applied to the audio sequences are performed in an online fashion. This has for effect that a given audio sequence can be associated to multiple different transformations (chosen randomly), therefore forcing the model to extract features that are invariant with respect to the applied transformations.

Furthermore, following (Kharitonov et al., 2021), only past sequences were augmented with pitch modification and artificial reverberation. Modifications applied to the past sequences are: pitch shifting by a random integer between 300 and -300 (measured by 1/100 of a tone), and artificial reverberation using a random room size.

# C. The machine ABX discrimination test

The logic of the ABX is represented in Figure 9. To build the ABX evaluation set, 10 hours of read-speech in American English and Metropolitan French were downloaded from Common Voice (Ardila et al., 2020). Segments were pronounced by 24 speakers whose gender was balanced in both languages. The phone-level alignment was obtained by aligning the audio stream with its transcript using Kaldi recipes (Povey, Ghoshal, Boulianne, Burget, Glembek, Goel, Hannemann, Motlicek, Qian, Schwarz et al., 2011). The ensuing phonetic inventory in International Phonetic Alphabet (IPA) standard for both languages is shown in Table 4.

# D. A database of noises to use as additive noise

We extracted all non-speech segments from SEEDlingS' long-forms. For each non-speech segment, the pretrained voice type classification model (Lavechin et al., 2020)
returns a probability that can be interpreted as how confident
the model is that the segment is indeed non-speech (com-
puted as 1-p_s where p_s is the probability of the segment being
speech). All non-speech segments whose probability of be-
ing non-speech was lower than the median non-speech prob-
ability were discarded. We did so to minimize the amount of
segments that have been wrongly identified as being non-
speech when in fact they do contain speech. Finally, 1,024
hours of non-speech segments were kept among those having
the highest energy to ensure that segments containing only
white noise (which are closer to silence) were discarded.
Based on manual inspection of a small subset, selected non-
speech segments contained various noises including running
water, animal sounds, vacuum cleaner, heartbeats of the key
child, etc.

E. Noise and reverberation in long-forms and simulated long-forms

We used the same pretrained model Lavechin et al. (2022b) as in Fig. 1 to study the distributions of the SNR and the C₅₀ of utterances extracted from English long-forms or simulated long-forms (i.e. audiobooks contaminated with noise and reverberation). While Fig. 1 revealed an important mismatch in terms of SNR and C₅₀ between long-forms and audiobooks, we observe in Fig. 10 that this difference has been reduced between long-forms and simulated long-forms.

On average, utterances extracted from long-forms have
an SNR of 10.4 dB while those from simulated long-forms
have an SNR of 10.6 dB. The mismatch in terms of C50 has
also been reduced with an average of 29.4 dB for long-form
utterances and 15 dB for simulated long-form utterances (as
opposed to 54 dB for audiobooks). The lower C50 obtained
on simulated long-forms indicate those contain more rever-
berant environments than real long-forms. Similar results
were found on French.
<figure><img src="image_10.png" /><figcaption>**Figure 9:** (The machine ABX discrimination test) a) The machine is given 3 stimuli A, B and X whose representations $R_A$, $R_B$ and $R_X$ are extracted by the composition $g_{enc} \circ g_{ar}$ of the encoder and the autoregressive model of CPC; b) Distances between $R_A$ and $R_X$ and $R_B$ and $R_X$ is computed with dynamic time warping (DTW). In the example, DTW cost matrix and shortest path (in white) is shown for A = [bup] and X = [bœp]; c) If A and X are different occurrences of the same triphone (e.g. [bup]), and B an other triphone differing only in its center phone (e.g. [bœp]), the machine is right if $R_A$ is closer to $R_X$ than $R_B$, wrong otherwise.</figcaption></figure>

<table><thead><tr><th>Manner of articulation</th><th>Metropolitan French</th><th>American English</th></tr></thead><tbody><tr><td></td><td colspan="2"><strong>Consonants</strong></td></tr><tr><td>Stops:</td><td>p,b,t,d,k,g</td><td>p,b,t,d,k,g</td></tr><tr><td>Nasals:</td><td>m,n,n</td><td>m,n,n</td></tr><tr><td>Fricatives:</td><td>f,v,s,z,ʃ,ʒ,μ</td><td>f,v,θ,ð,s,z,ʃ,ʒ,h</td></tr><tr><td>Approximants:</td><td>j,w,l</td><td>j,u,w,l</td></tr><tr><td>Affricates:</td><td>ess</td><td>ess,tf</td></tr><tr><td></td><td colspan="2"><strong>Vowels</strong></td></tr><tr><td>Orals:</td><td>i,y,e,ø,œ,ε,a,ə,ɔ,u</td><td>i,u,ε,æ,ə,ʌ,e,u,u,ɔ,u</td></tr><tr><td>Nasals:</td><td>ä, ë, õ,ð</td><td>ä, ë, õ,ð</td></tr><tr><td>Diphthongs:</td><td>ai,ɔi,au,ei,ou</td><td>ai,ɔi,au,ei,ou</td></tr></tbody></table>

**Table 4**

Evaluated phonetic inventory in Metropolitan French and American English in International Phonetic Alphabet (IPA) standard.

<figure><img src="image_11.png" /><figcaption>**Figure 10:** (The quantity of noise and reverberation in long-forms and simulated long-forms) Speech-to-Noise Ratio (SNR) and $C_{50}$ distributions on 16 hours of speech utterances extracted from long-forms (slanting hatches) or simulated long-forms (vertical hatches). Both measures are automatically extracted using the pretrained model proposed in Lavechin et al. (2022b)</figcaption></figure>
## References

Adriaans, F., Swingley, D., 2017. Prosodic exaggeration within infant-directed speech: Consequences for vowel learnability. The Journal of the Acoustical Society of America 141 5, 3070.

Alishahi, A., Chrupala, G., Cristià, A., Dupoux, E., Higy, B., Lavechin, M., Räsänen, O., Yu, C., 2021. ZR-2021VG: Zero-resource speech challenge, visually-grounded language modelling track. arXiv preprint arXiv:2107.06546.

Anderson, N.J., Graham, S.A., Prime, H., Jenkins, J.M., Madigan, S., 2021. Linking quality and quantity of parental linguistic input to child language skills: A meta-analysis. Child Development 92, 484–501.

Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F.M., Weber, G., 2020. Common voice: A massively-multilingual speech corpus, in: Language Resources and Evaluation Conference (LREC).

Beeston, A.V., Brown, G.J., Watkins, A.J., 2014. Perceptual compensation for the effects of reverberation on consonant identification: evidence from studies with monaural stimuli. The Journal of the Acoustical Society of America 136 6, 3072.

Bergelson, E., 2017. SEEDLingS HomeBank corpus. https://homebank.talkbank.org/access/Password/Bergelson.html. doi:10.21415/T5PK6D.

Bergelson, E., Casillas, M., Soderstrom, M., Seidl, A., Warlaumont, A.S., Amatuni, A., 2019. What do North American babies hear? A large-scale cross-corpus analysis. Developmental science 22 1, e12724.

Bergmann, C., Cristia, A., Dupoux, E., 2016. Discriminability of sound contrasts in the face of speaker variation quantified, in: 38th Annual Conference of the Cognitive Science Society, Cognitive Science Society. pp. 1331–1336.

Blandón, M.A.C., Cristia, A., Räsänen, O., 2021. Evaluation of computational models of infant language development against robust empirical data from meta-analyses: what, why, and how?.

Bregman, A.S., 1994. Auditory scene analysis: The perceptual organization of sound. MIT press.

Bulgarelli, F., Bergelson, E., 2023. Linking acoustic variability in the infants' input to their early word production URL: https://psyarxiv.com/su768/.

Bulgarelli, F., Mielke, J., Bergelson, E., 2021. Quantifying talker variability in north-american infants' daily input. Cognitive science 46, e13075.

Canault, M., Normand, M.T., Foudil, S., Loundon, N., Thai-Van, H., 2016a. LYON HomeBank corpus. https://homebank.talkbank.org/access/Password/Lyon.html. doi:21415/T58P6Q.

Canault, M., Normand, M.T.L., Foudil, S., Loundon, N., Thai-Van, H., 2016b. Reliability of the Language Environment Analysis system (LENA™) in European French. Behavior Research Methods 48, 1109–1124.

Choi, M., Shukla, M., 2021. A new proposal for phoneme acquisition: Computing speaker-specific distribution. Brain Sciences 11, 177.

Coen, M., 2006. Self-supervised acquisition of vowels in American English, in: Association for the Advancement of Artificial Intelligence (AAAI).

Cooper, R.P., Aslin, R.N., 1990. Preference for infant-directed speech in the first month after birth. Child development 61 5, 1584–95.

Cristia, A., 2019. A systematic review suggests marked differences in the prevalence of infant-directed vocalization across groups of populations. https://osf.io/c86ew/?view_only=f9af0cf7d2574234a8517c38151e4210.

Cristia, A., 2020. Language input and outcome variation as a test of theory plausibility: The case of early phonological acquisition. Developmental Review 57, 100914.

Cristia, A., 2021. PhonSES: socioeconomic status effects on infants' word and sound processing. https://gin.g-node.org/LAAC-LSCP/phonSES-public.

De Boer, B., Kuhl, P.K., 2003. Investigating the role of infant-directed speech with a computer model. Acoustics Research Letters Online 4, 129–134.

Decasper, A.J., Fifer, W.P., 1980. Of human bonding: newborns prefer their mothers' voices. Science 208 4448, 1174–6.

Decasper, A.J., Prescott, P., 1984. Human newborns' perception of male voices: preference, discrimination, and reinforcing value. Developmental psychobiology 17 5, 481–91.

Dunbar, E., Bernard, M., Hamilakis, N., Nguyen, T., de Seyssel, M., Rozé, P., Rivière, M., Kharitonov, E., Dupoux, E., 2021. The zero resource speech challenge 2021: Spoken language modelling, in: Interspeech.

Dupoux, E., 2018. Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner. Cognition 173, 43–59.

Eaves Jr, B.S., Feldman, N.H., Griffiths, T.L., Shafto, P., 2016. Infant-directed speech is consistent with teaching. Psychological review 123, 758.

Feldman, N.H., Goldwater, S., Dupoux, E., Schatz, T., 2022. Do infants really learn phonetic categories? Open Mind 5, 113–131.

Feldman, N.H., Griffiths, T.L., Goldwater, S., Morgan, J.L., 2013. A role for the developing lexicon in phonetic category acquisition. Psychological review 120, 751.

Floccia, C., Nazzi, T., Bertoncini, J., 2000. Unfamiliar voice discrimination for short stimuli in newborns. Developmental Science 3, 333–343.

Ford, M., Baer, C.T., Xu, D., Yapanel, U., Gray, S., 2008. The LENA™ language environment analysis system.

García, P., Villalba, J., Bredin, H., Du, J., Castán, D., Cristia, A., Bullock, L., Guo, L., Okabe, K., Nidadavolu, P.S., Kataria, S., Chen, S., Galmant, L., Lavechin, M., Sun, L., Gill, M.P., Ben-Yair, B., Abdoli, S., Wang, X., Bouaziz, W., Titeux, H., Dupoux, E., Lee, K.A., Dehak, N., 2020. Speaker detection in the wild: Lessons learned from JSALT 2019, in: Odyssey.

Gilkerson, J., Richards, J.A., Warren, S.F., Oller, D.K., Russo, R., Vohr, B.R., 2018. Language experience in the second year of life and language outcomes in late childhood. Pediatrics 142.

Huang, Y., Rao, R.P., 2011. Predictive coding. Wiley Interdisciplinary Reviews: Cognitive Science 2, 580–593.

Hüllermeier, E., Fober, T., Mernberger, M., 2013. Inductive bias. Encyclopaedia of systems biology. Springer, New York.

Issard, C., Tsuji, S., Cristia, A., 2023. Infants' preference for speech is stable across the first year of life: Meta-analytic evidence. Infancy 28, 550–568.

Kahn, J., Rivière, M., Zheng, W., Kharitonov, E., Xu, Q., Mazar'e, P.E., Karadayi, J., Liptchinsky, V., Collobert, R., Fuegen, C., Likhomanenko, T., Synnaeve, G., Joulin, A., rahman Mohamed, A., Dupoux, E., 2020. Libri-light: A benchmark for ASR with limited or no supervision, in: International Conference on Acoustics, Speech and Signal Processing (ICASSP).

Kearns, J., 2014. Librivox: Free public domain audiobooks, in: Reference Reviews, Emerald Group Publishing Limited.

Kharitonov, E., Rivière, M., Synnaeve, G., Wolf, L., Mazaré, P.E., Douze, M., Dupoux, E., 2021. Data augmenting contrastive learning of speech representations in the time domain, in: Spoken Language Technology Workshop (SLT).

Kuhl, P.K., 1979. Speech perception in early infancy: perceptual constancy for spectrally dissimilar vowel categories. The Journal of the Acoustical Society of America 66 6, 1668–79.

Kuhl, P.K., 1983. Perception of auditory equivalence classes for speech in early infancy. Infant behavior and development 6, 263–285.

Kuhl, P.K., 2004. Early language acquisition: cracking the speech code. Nature Reviews Neuroscience 5, 831–843.

Kuhl, P.K., Stevens, E., Hayashi, A., Deguchi, T., Kiritani, S., Iverson, P., 2006. Infants show a facilitation effect for native language phonetic perception between 6 and 12 months. Developmental science 9 2, F13–F21.

Kuhl, P.K., Tsao, F.M., Liu, H.M., 2003. Foreign-language experience in infancy: Effects of short-term exposure and social interaction on phonetic learning. Proceedings of the National Academy of Sciences 100, 9096–9101.

Lavechin, M., Bousbib, R., Bredin, H., Dupoux, E., Cristia, A., 2020. An open-source voice type classifier for child-centered daylong recordings, in: Interspeech.
Lavechin, M., Cristia, A., 2021. Babylogger versus LENA microphones study. https://gin.g-node.org/LAAC-LSCP/babylogger-vs-lena-data-public.

Lavechin, M., De Seyssel, M., Titeux, H., Bredin, H., Wisniewski, G., Cristia, A., Dupoux, E., 2022a. Can statistical learning bootstrap early language acquisition? a modeling investigation.

Lavechin, M., Métais, M., Titeux, H., Boissonnet, A., Copet, J., Rivière, M., Bergelson, E., Cristia, A., Dupoux, E., Bredin, H., 2022b. Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and c50 room acoustics estimation. arXiv preprint arXiv:2210.13248.

Levy, E.S., Strange, W., 2008. Perception of French vowels by American English adults with and without French language experience. Journal of phonetics 36, 141–157.

Li, R., Schatz, T., Matusevych, Y., Goldwater, S., Feldman, N.H., 2020a. Input matters in the modeling of early phonetic learning, in: CogSci.

Li, R., Schatz, T., Matusevych, Y., Goldwater, S., Feldman, N.H., 2020b. Input matters in the modeling of early phonetic learning, in: Proceedings of the Annual Conference of the Cognitive Science Society.

Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al., 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110.

Ludusan, B., Mazuka, R., Dupoux, E., 2021. Does infant-directed speech help phonetic learning? a machine learning investigation. Cognitive Science 45, e12946.

Lytle, S.R., Garcia-Sierra, A., Kuhl, P.K., 2018. Two are better than one: Infant language learning from video improves in the presence of peers. Proceedings of the National Academy of Sciences 115, 9859–9866.

Marklund, E., Schwarz, I.C., Lacerda, F., 2019. Amount of speech exposure predicts vowel perception in four-to eight-month-olds. Developmental Cognitive Neuroscience 36, 100622.

Maye, J., Werker, J.F., Gerken, L., 2002. Infant sensitivity to distributional information can affect phonetic discrimination. Cognition 82, B101–B111.

McMurray, B., 2022. The acquisition of speech categories: Beyond perceptual narrowing, beyond unsupervised learning and beyond infancy. Language, Cognition and Neuroscience.

McMurray, B., Kovack-Lesh, K.A., Goodwin, D., McEchron, W., 2013. Infant directed speech and the development of speech perception: Enhancing development or an unintended consequence? Cognition 129, 362–378.

Mehler, J., Bertoncini, J., Barriere, M., Jassik-Gerschenfeld, D., 1978. Infant recognition of mother's voice. Perception 7, 491–497.

Millet, J., Chitoran, I., Dunbar, E., 2021. Predicting non-native speech perception using the perceptual assimilation model and state-of-the-art acoustic models, in: Conference on Computational Natural Language Learning (CONLL).

Miyawaki, K., Jenkins, J.J., Strange, W., Liberman, A.M., Verbrugge, R., Fujimura, O., 1975. An effect of linguistic experience: The discrimination of [r] and [l] by native speakers of Japanese and English. Perception & Psychophysics 18, 331–340.

Miyazawa, K., Kikuchi, H., Mazuka, R., 2010. Unsupervised learning of vowels from continuous speech based on self-organized phoneme acquisition model, in: Interspeech.

Morton, J., Johnson, M.H., 1991. CONSPEC and CONLERN: a two-process theory of infant face recognition. Psychological review 98, 164.

Newman, R.S., 2005. The cocktail party effect in infants revisited: listening to one's name in noise. Developmental Psychology 41, 352.

Newman, R.S., Hussain, I., 2006. Changes in preference for infant-directed speech in low and moderate noise by 4.5-to 13-month-olds. Infancy 10, 61–76.

van den Oord, A., Li, Y., Vinyals, O., 2019. Representation learning with contrastive predictive coding. arXiv:1807.03748.

Paquette, N., Dionne-Dostie, E., Lassonde, M., Gallagher, A., 2018. Voice perception in newborns and infants. The Oxford Handbook of Voice Perception, 191.

Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., et al., 2011. The kaldi speech recognition toolkit, in: Automatic Speech Recognition and

Understanding (ASRU) workshop, IEEE Signal Processing Society.

Rivière, M., Joulin, A., Mazaré, P.E., Dupoux, E., 2020. Unsupervised pretraining transfers well across languages, in: International Conference on Acoustics, Speech and Signal Processing (ICASSP).

Rowe, M.L., 2012. A longitudinal investigation of the role of quantity and quality of child-directed speech in vocabulary development. Child development 83, 1762–1774.

Schatz, T., Feldman, N.H., Goldwater, S., Cao, X.N., Dupoux, E., 2021. Early phonetic learning without phonetic categories: Insights from large-scale simulations on realistic input. Proceedings of the National Academy of Sciences 118.

Schatz, T., Peddinti, V., Bach, F.R., Jansen, A., Hermansky, H., Dupoux, E., 2013. Evaluating speech features with the minimal-pair ABX task: analysis of the classical MFC/PLP pipeline, in: Interspeech.

Schneider, S., Baevski, A., Collobert, R., Auli, M., 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862.

Seidl, A., Onishi, K.H., Cristia, A., 2014. Talker variation aids young infants' phonotactic learning. Language Learning and Development 10, 297–307.

Singh, L., Rajendra, S.J., Mazuka, R., 2022. Diversity and representation in studies of infant perceptual narrowing. Child Development Perspectives 16, 191–199.

Sperry, D.E., Sperry, L.L., Miller, P.J., 2019. Reexamining the verbal environments of children from different socioeconomic backgrounds. Child development 90, 1303–1318.

Traer, J., McDermott, J.H., 2016. Statistics of natural reverberation enable perceptual separation of sound and space. Proceedings of the National Academy of Sciences 113, E7856–E7865.

Trott, S., Jones, C., Chang, T., Michaelov, J., Bergen, B., 2023. Do large language models know what humans know? Cognitive Science 47, e13309.

Tsuji, S., Cristia, A., 2014. Perceptual attunement in vowels: A meta-analysis. Developmental psychobiology 56, 179–191.

Tsuji, S., Cristia, A., Dupoux, E., 2021. Scala: A blueprint for computational models of language acquisition in social context. Cognition 213, 104779.

Vallabha, G.K., McClelland, J.L., Pons, F., Werker, J.F., Amano, S., 2007. Unsupervised learning of vowel categories from infant-directed speech. Proceedings of the National Academy of Sciences 104, 13273–13278.

Versteegh, M., Thiollière, R., Schatz, T., Cao, X.N., Miró, X.A., Jansen, A., Dupoux, E., 2017. The zero resource speech challenge 2017, in: Automatic Speech Recognition and Understanding (ASRU) workshop.

Vouloumanos, A., Werker, J.F., 2007. Listening to language at birth: evidence for a bias for speech in neonates. Developmental science 10 2, 159–64.

Warren, C., 2013. EchoThief impulse response library. http://www.echothief.com/.

Weisleder, A., Fernald, A., 2013. Talking to children matters: Early language experience strengthens processing and builds vocabulary. Psychological science 24, 2143–2152.

Werker, J.F., Tees, R.C., 1992. The organization and reorganization of human speech perception. Annual Review of Neuroscience 15, 377–402.