Learning to Speak Like a Child: Reinforcing and Evaluating a Child-level Generative Language Model (2025)

Abstract: A language model that can generate utterances that are appraised as being within a specific age of a young child who is beginning their language learning journey can be useful in scenarios where child-level language is needed, for example in virtual avatars, interactions with individuals who have disabilities, or developmental robotics. In this paper, we focus on an age range that is not represented in prior work: emergent speakers. We use the CHILDES database to train and tune language models of different parameter sizes using a group relative policy optimization reinforcement learning regime. Our goal is to find the most coherent, yet child-like language model while keeping the number of parameters to as few as possible. We evaluate using metrics of coherency, “toddlerality,” and an evaluation using human subjects who interact with two robot platforms. Our experiments show that even small language models (under 1 billion parameters) can be used effectively to generate child-like utterances.

## introduction a rich setting of spoken dialogue is the interaction between children and their caregivers. the primary and fundamental site of language use is in this childcaregiver interaction, which is a co-located spoken dialogue setting @xcite beginning with first words, short phrases, and simple sentences that later complex syntax and abstract words build on. early child speech is often produced in installments, requiring more dialogue turns to convey information. in contrast, most language models (lms) work on full utterances-not installmentsand have been trained on vast amounts of text, the majority of which likely comes from sources that are not written for children specifically, and certainly not in terms of direct transcriptions of child utterances. because lms are trained primarily with text, the data reflects language written by and for people who are in literate age ranges, roughly 6 years old or older (the age when u.s. children begin learning how to read). this is an unfortunate oversight because language phenomena from child-target sources can be very insightful and useful. @xcite showed that child texts help improve commonsense reasoning in lms because child texts don't assume domain knowledge; instead, 'commonsense' explanations are clearly stated. an important resource for child-caregiver interactions is the childes project @xcite , a dataset of childrencaregiver interactions with annotated transcriptions of speech as well as some gestures. childes has been used as a resource for recent work including learning about word acquisition @xcite , as well as early syntax and multlinguality @xcite . furthermore, childes has been used as a resource for pre-training lms including babyberta @xcite and for the babylm challenge, showing an increasing interest in compact, effective lms that are trained on only 10 or 100 million tokens taken from childproduced spoken transcriptions. 1 one outcome of the babylm challenge was the babystories model @xcite ) that showed improvements on downstream tasks using lms that are reinforced with stories written for children. taken together, this recent work focusing on child-level lms shows that there are benefits to using child-level data. an lm that can generate utterances finetuned to match the developmental language patterns of young children can be useful in scenarios requiring child-level communication. applications include virtual avatars, interactions with individuals who have disabilities, or developmental robotics. some lms have been trained on transcriptions of child speech from young, emergent speakers @xcite ), but they have not been evaluated to generate utterances that are age appro-priate for the youngest range of speakers, and furthermore have not been evaluated in interactive dialogue settings. in this paper we make two primary contributions: (1) we systematically explore how small lms can effectively be reinforced on small amounts of child-level data and be able to generate utterances that are regarded as being at a specific child level, with focus on emergent, toddler-level speech (age 1-3), and (2) construct a lm with as few parameters and with as little data as necessary to produce child-level dialogue utterances. we systematize our evaluations using coherency (explained below), as well as a human evaluation on two robot platforms in a dialogue setting. our results show that our method for reinforcement works effectively, enabling researchers to access a model that produces child utterances. ## related work small language models lms trained on huge datasets dominate, but many are turning to smaller lms that are tuned to do specific tasks well. small lms that are more child developmentally plausible are the focus of the babylm challenge @xcite @xcite . related work to ours include @xcite , a small model that was improved by using reinforcement learning on child texts; and @xcite and @xcite both explore how transformer-based lms can be effective, even with fewer than 10 million parameters on discriminative tasks. we follow this line of research, but instead of discriminative models, we work towards a harder generative model task that can produce child-level utterances which could be used for sds dialogue. child-level language models lms are increasingly used to identify early onset of certain linguistic abilities in children (e.g., determiner-noun onset @xcite ). more general-purpose lms have been trained on datasets derived from and designed for children including babyberta @xcite and @xcite , though the latter was trained on child-targeted texts, not on transcriptions of child speech. as part of the babylm challenge, other recent work explored just how small transformerbased lms could work on small amounts of data from childes including gpt-wee @xcite and electra-tiny @xcite . however, these lms are mostly discriminative, not generative. readability readability is an area of research that attempts to automatically determine the appropriate age or grade level a child needs to be in order to comprehend a given text. for example, a short sentence i am hungry can be understood by very young children, while the nomenclature of academic disciplines requires more knowledge of abstract words. approaches to the readability task include a number of formulae that use specific features such as number of sentences, average number of words in each sentence, and comparison to words on word lists maintained by educators, such as the spache @xcite , spache-allen @xcite , and flesch-kincaid formula @xcite . others have attempted to use lms coupled with more linguistically-derived features with varying degrees of success @xcite . however, the literature on readability focuses on children in their first years of education (kindergarten through 5th grade in the u.s.), roughly beginning at age 6. what happens before that-children who are in their first years of speaking, ages 1-5-is what we focus on in this paper. the most closely related to our work is @xcite controlling the language proficiency level of lm generation to a specific level, but focuses on readability (they used fleisch-kincaid to guide the loss function to fine-tune a lm), whereas we are interested in children ages 1-3, younger than the target ages for readability. moreover, we are interested not in what kids are able to read, but what they are able to say that is age appropriate. group relative policy optimization group relative policy optimization (grpo) is a deep reinforcement learning (rl) training regime that enables fine-tuning of high-capacity models such as large language models (llms) by leveraging reward models to guide learning @xcite . at each training step, grpo uses feedback from the reward model to compute gradients and update the policy weights, allowing for stable, preference-aligned optimization in high-dimensional action spaces. unlike other lm training regimes, grpo only requires an input prompt; at each step the model generates n outputs, which are then scored by the reward models, and the model weights are updated accordingly. to train an lm that can generate utterances within our targeted age range (1-3), we use a reinforcement learning (rl) regime of group relative policy optimization (grpo), following @xcite . as a preliminary step, we designed and fine-tuned a discriminator model which is to be used as our primary grpo reward model (rm). this discriminator model is tasked with rewarding the grpo fine-tuning process to encourage childlevel (main target is 1-3 years of age). we use the appropriate caregiver questions extracted from the childes dataset as the prompt inputs for the grpo task and we expect the model to respond with the appropriate child language response. we explain each aspect of our method below. ## experiment our experiment has two steps: (1) use pre-trained lms and determine the best, yet smallest architecture that we can use to effectively generate child utterances, and (2) using the best model size from (1) and pre-train a custom model using only child data. we explain each step in more detail. giving too much weight to rm-1 (childish) can lead to non-coherent model whereas giving too much weight to rm-2 (coherence) can make the training process fail to allow the model to reduce cognitive behavior that of a child. we omit rm-3 results to remove clutter, however all models converged to 100% score with rm-3 due to its simplicity. ## human evaluation our offline metrics show that our child-level generative models are coherent and speak at a toddlerlevel age. to further evaluate both models, we performed a human evaluation. robots & system because humans assign adultlevel expectations to dialogue systems that can speak @xcite , it is important that we evaluate models on multiple robot platforms, where one platform is perceived as child-age, and another that appears older. we opt for the cozmo robot which has been shown to have child-level qualities including morphology and voice @xcite . for the 'older' robot, we opt for misty ii since it is much larger than cozmo, but still smaller than a human toddler. we use the default voice for both misty ii and cozmo. we used the rrsds system for building dialogue systems on robotic platforms, which is built on retico @xcite . our system consisted of a usb microphone, whisper automatic speech recognition (asr) @xcite , an rrsds/retico lm module that takes asr transcriptions as text input and returns a response that is then uttered on the robot (each robot has a rrsds control module). we test two models that resulted from the above ex-figure 3 : depiction of our dialogue system that enables human participants to interact with a robot that uses a child-level lm to generate utterances. periment, smollm-350m and our toddler model (155m parameters, pre-trained on childes). the system is depicted in figure 3 . all modules that produce output are logged into a logging module for later analysis. task & procedure we recruited 19 participants to come into our lab and interact with our robots. each participant interacted with two of four possible robot+model pairs assigned at random: the robots were set on a table and participants were seated near the table so they could be faceto-face with the robot. we programmed the robots to perform small random movements to ensure to the participants that the robots were still functioning even when not speaking. moreover, constant, small random movements are closer to what humans expect from each other, children in particular. after signing an informed consent, 8 participants were tasked with speaking with the robot. we made sure that participants understood that they needed to help drive the conversation by asking questions, but we did not prime them to assume that the robot would act a certain age. after interacting with the robot for 15-20 minutes, they filled out a survey. metrics we used the standard human-robot interaction godspeed questionnaire @xcite which consists of 21 pairs of contrasting characteristics, each with a 5-point likert scale between them. some examples below: following @xcite , in addition to these questions, we also ask participants about their perceptions of the robots' age and grade level. we also asked that the participants rate coherence (specifically, if the responses 'made sense') on a scale of 1-5 (5=very coherent). we also include average age-of-acquisition (aoa) values for words spoken by participants and generated by the models. the aoa dataset is a list of words and the estimated age when those words are learned (for example, puppy is learned at 3 years four months on average, whereas quadratic is learned at 14 years old). in addition to questionnaires, we analyze the logs to determine several objective measures: the vocabulary of the participants and each robot+model, and the average aoa value for each robot+model pair. we hypothesize that the vocabulary of the human participants will be larger than that of the robot+model pairs, that the aoa values will be higher for the human-participant words compared to the robot+model words, and we anticipate that our model will be regarded as younger and more coherent than smollm-350. ## conclusion in this paper, we modeled, trained, and evaluated child-level lms. we evaluated multiple architectures using existing, pre-trained smollm models, and from that we determined the smallest architecture to pre-train a custom model on child-level data from childes. our training reinforced models to speak at a young child level (around 3 years of age), younger than prior models in the literature. our human evaluations using the models on two robot platforms further illustrate their usefulness in human-robot interaction settings, but more importantly provided additional evaluation about the perceived age, coherency, and competence of the models. the results suggest that the small model trained only on child-level data is comparable to a larger model trained on much larger amounts of adult-level text, though using an existing pretrained model and tuning it using our method to speak at child-level can be just as effective, albeit using a larger model. for future work, we will incorporate the childlevel lm into a human-robot interaction robot task where the robot and the human collaborate and the robot must learn from the human as it interacts.