C hi SC or: A Corpus of Freely-Told Fantasy Stories by D utch Children for Computational Linguistics and Cognitive Science (2023)

Abstract: In this resource paper we release ChiSCor, a new corpus containing 619 fantasy stories, told freely by 442 Dutch children aged 4-12. ChiSCor was compiled for studying how children render character perspectives, and unravelling language and cognition in development, with computational tools. Unlike existing resources, ChiSCor’s stories were produced in natural contexts, in line with recent calls for more ecologically valid datasets. ChiSCor hosts text, audio, and annotations for character complexity and linguistic complexity. Additional metadata (e.g. education of caregivers) is available for one third of the Dutch children. ChiSCor also includes a small set of 62 English stories. This paper details how ChiSCor was compiled and shows its potential for future work with three brief case studies: i) we show that the syntactic complexity of stories is strikingly stable across children’s ages; ii) we extend work on Zipfian distributions in free speech and show that ChiSCor obeys Zipf’s law closely, reflecting its social context; iii) we show that even though ChiSCor is relatively small, the corpus is rich enough to train informative lemma vectors that allow us to analyse children’s language use. We end with a reflection on the value of narrative datasets in computational linguistics.

## introduction all of us tell stories on a daily basis: to share experiences, contextualise emotions, exchange jokes, and so on. there is a rich tradition of research into how such storytelling develops during infancy, and its relations with various aspects of children's linguistic and cognitive development (for an overview see @xcite . chiscor (children's story corpus) was compiled to give a unique impulse to this tradition: it allows for (computationally) studying how children render character perspectives such as perceptions, emotions, and mental * equal contribution. this paper makes the following contributions. first, we release chiscor and describe its compilation, data, and annotations in detail (sections 2 and 3). second, we show how chiscor fuels future work on the intersection of language, cognition, and computation, with three brief case studies (section 4). we explore the dependency length minimization hypothesis @xcite with chiscor's language features and show that the syntactic complexity in children's stories is strikingly stable over different age groups. also, we extend emerging work on zipf's law in speech (e.g. @xcite and find that chiscor's token distribution approximates zipf better than a reference corpus consisting of language written by children, which we explain by appealing to the principle of least effort. furthermore, we show that chiscor as a small corpus is rich enough to be used with nlptools traditionally thought to require large datasets. we train informative lemma vectors with chiscor, that can be used to analyse how coherently children use specific lemmas of interest, and potential bias in their language use. together, our case studies demonstrate that even though storytelling is a cognitively challenging task, the language children employ is no less sophisticated (an observation also supported by @xcite . and although corpora of narratives are often smaller, we show that we can (and should) leverage nlp-tools to unravel linguistic and cognitive mechanisms at work in children's language productions. as discussed in section 5, we see this as an important stepping stone towards building more ecologically valid language models. ## background and relevance various resources of dutch child language exist. before the 2000s, corpora typically consisted of child speech gathered in unstructured home settings involving smaller numbers of younger children (e.g. @xcite . later, more structured language elicitation (e.g. with picture books) from larger samples of children was more common (e.g. @xcite , and recently we have seen large corpora documenting thousands of essays in school settings @xcite , and many hours of speech recordings in human-machine interaction contexts @xcite . although these resources are valuable, what is currently lacking is a corpus of speech samples that are i) produced freely in natural social settings, while being ii) sufficiently independent or 'decontextualised' to be a good reflection of children's capacities, and iii) containing metadata about children's backgrounds. the rest of this section will discuss these three characteristics, on the basis of which chiscor was compiled. ## case studies with chiscor we conduct three small case studies to illustrate chiscor's potential. since we aim to show chis-cor's versatility to the broader community, we draw in study 1 (section 4.1) on chiscor's own linguistic annotations and metadata; in study 2 leverage chiscor in a corpus linguistics-style analysis on zipf's law in child speech (section 4.2), and in study 3 show the feasibility of using chis-cor with nlp-tools that are traditionally thought to require larger corpora (section 4.3). ## discussion storytelling datasets are relatively scarce, which is a shortcoming in existing resources, given that live storytelling challenges children to leverage both linguistic, cognitive, and social competences to tell a story that engages an audience. these competences can be analysed through stories, manually or with computational tools, to learn more about child development. we demonstrated that chiscor has properties that other established language samples also have, such as a zipfian token distribution. moreover, chiscor's close fit to the zipfian curve testifies to the social context of the language contained in it and the principle of least effort that is likely at work there (section 4.2). in addition, even though storytelling is a cognitively demanding task, we demonstrated that the stories in chiscor are syntactically surprisingly complex, and we offered a tentative explanation why especially younger children may employ complex syntax, which could be related to chiscor's context of live storytelling in front of an audience (section 4.1). lastly, we have shown that chiscor can be used to learn a semantic vector space that is as intuitive as the semantic space of a much larger reference corpus (section 4.3). this opens up possibilities for using chiscor with tools that are traditionally deemed fit only for much larger corpora, to assess the coherence of contexts in which children use particular words of interest. for example, we found that words detailing cognitive and perceptual states were more clearly differentiated in chiscor compared to basiscript as a corpus of written child language. such words concern information that is critical to understand a plot that cannot be consulted again in live storytelling, possibly leading children to use these words more carefully and coherently. the social context of chiscor's narratives and its influence on language production invite us to reflect on a more general issue: the dominance of written (web) text in computational linguistics and nlp. researchers increasingly question scraping together increasingly larger uncurated and undocumented resources @xcite , that is, datasets without metadata, and it is subject to debate how helpful such large-scale written datasets are in e.g. understanding language acquisition and modelling cognition (e.g. @xcite . indeed, spoken language is different from written language in many ways, as linders and louwerse (2023) note: it is mainly acquired naturally (unlike writing) and predates writing in both the evolutionary and developmental sense. most critically, speech is typically situated in a social setting with other language users, evanescent, spontaneous, and grounded in a particular context, to mention just a few out of many defining characteristics. still, with large language models (llms) as prime current example of the reliance on large written datasets, such datasets have helped disclose what is in principle learnable from word cooccurrence statistics and a simple word prediction training objective, such as the capacity to represent language input hierarchically @xcite . although we should take llms serious as the current best yet data-hungry distributional learners we have @xcite , the next challenge is to achieve the same performance with more ecologically valid, smaller datasets and smaller neural architectures; here, corpora like chiscor could be part of the solution. since chiscor has information on the age groups of the children who produced the language, future work could, for example, partition chiscor to employ train and/or test sets that more realistically model children's language use at different stages of their development. and since chis-cor covers language from the speech domain, it provides an interesting opportunity to explore training language models on language with a different nature. still, we do not mean to claim that chiscor solves all issues regarding llms and training data, but we hope to contribute a dataset that can be a part of the move towards better datasets for computational linguistics, a dataset that, in the words of @xcite , 'is only as large as can be sufficiently documented'. lastly, we like to emphasize that since chiscor features high-quality audio besides text, it naturally opens directions for multi-modal research. for example, research on detecting characters' emotions will benefit from adding information on prosody. also, research aimed at improving speech-to-text models will benefit from the voices of 442 unique children of different ages, and accompanying transcripts, that can be used for fine-tuning existing speech-to-text models. ## conclusion this paper introduced chiscor as a versatile resource for computational work on the intersection of child language and cognition. chiscor is a new corpus of dutch fantasy stories told freely by children aged 4-12 years, containing high-quality language samples that reflect the social settings in which they were recorded in many details. we provided three case studies as examples of how chiscor can fuel future work: studying language development with chiscor's out-of-the-box age metadata and linguistic features, modelling zipf distributions with chiscor, and linking chiscor to common nlp-tools to study children's language in action. besides verbatim and normalised texts, chiscor comes with 442 high-quality audio samples of 442 children, metadata on the backgrounds of 148 children, annotations of character complexity, and extracted linguistic features that will be useful for a variety of researchers. in addition to dutch stories, chiscor comes with a small additional set of 62 english stories with the same additional metadata and annotations as for the dutch stories. four years have passed since we started compiling chiscor. we look back on many great moments with the children who were happy to share their fantasies and cleverly constructed plots with us. we encourage readers of this paper to have a look at the corpus--both for research purposes and for fun.