A surprisal–duration trade-off across and within the world’s languages (2021)

Abstract: While there exist scores of natural languages, each with its unique features and idiosyncrasies, they all share a unifying theme: enabling human communication. We may thus reasonably predict that human cognition shapes how these languages evolve and are used. Assuming that the capacity to process information is roughly constant across human populations, we expect a surprisal–duration trade-off to arise both across and within languages. We analyse this trade-off using a corpus of 600 languages and, after controlling for several potential confounds, we find strong supporting evidence in both settings. Specifically, we find that, on average, phones are produced faster in languages where they are less surprising, and vice versa. Further, we confirm that more surprising phones are longer, on average, in 319 languages out of the 600. We thus conclude that there is strong evidence of a surprisal–duration trade-off in operation, both across and within the world’s languages.

## introduction during the course of human evolution, countless languages have evolved, each with unique features. despite their stark differences, however, it is plausible that shared attributes in human cognition may have placed constraints on how each language is implemented. these constraints, in turn, may lead to compensations and trade-offs in the world's languages. for instance, if we assume a channel capacity @xcite ) in human's ability to process language (as posited by @xcite , we may make predictions about these trade-offs. additionally, if we assume this capacity to be uniform across human populations, these trade-offs will extend cross-linguistically. within languages, there is a direct connection between this channel capacity assumption and the uniform information density hypothesis (uid; @xcite @xcite , which predicts that speakers smooth the information rate in a linguistic signal so as to keep it roughly constant; by smoothing their information rate, natural languages can stay close to a (hypothetical) channel capacity. across languages, a unified channel capacity allows us to derive a specific instantiation of the compensation hypothesis @xcite , with information density (measured in, e.g., bits per phone) being compensated by utterance speed (in, e.g., milliseconds per phone). we may thus predict a trade-off between surprisal foot_0 and duration both within and across the world's languages. this trade-off has been studied amply within high resource languages @xcite @xcite . cross-linguistically, however, this trade-off has received comparatively little attention, with a few notable exceptions such as @xcite and @xcite . several factors have inhibited cross-linguistic studies of this kind. arguably, the most prominent is the sheer lack of data necessary to investigate the phenomenon. while massively cross-linguistic data abounds in the form of wordlists @xcite , surprisal is a context-dependent measure and, therefore, isolated word types are not enough for this analysis. further, as we have a specific hypothesis for why this trade-off should arise (humans' information processing capacity), we are not interested in simply finding any correlation between surprisal and duration. several confounds could drive such a correlation, but most of these are either trivially true or uninteresting from our perspective. therefore, a thorough analysis of this trade-off needs to control for these potential confounds. in this work, we investigate the surprisalduration trade-off by analysing a massively multi-lingual dataset of more than 600 languages @xcite . we present an experimental framework, controlling for several possible confounds, and evaluate the surprisal-duration trade-off at the phone level. we find evidence of a trade-off across languages: languages with more surprising phones compensate by making utterances longer. we also confirm mono-lingual trade-offs in 319 languages, out of 600; foot_1 within these languages, more surprising phones are pronounced with a significantly longer duration. this is the most representative evidence of the uniform information density hypothesis to date. moreover, we did not find evidence of a single language where the opposite effect is in operation (i.e. where more informative phones are shorter). given these collective results, we conclude there is strong evidence for a surprisal-duration trade-off both across and within the world's languages. ## surprisal and duration cross-linguistic comparisons of information rate go back at least 50 years. in a study comparing phonemes per second, @xcite found no statistical difference between the speech rate of english and japanese native speakers. in a similar study, den os @xcite compared dutch and italian and found no difference in terms of syllables per second, although italian was found to be somewhat slower in phones per second. such cross-linguistic comparisons, however, are not straightforward, since the range of speech rate can vary widely within a single language, depending on sentence length @xcite and type of speech (e.g. storytelling vs interview; @xcite . in a meta-analysis of these studies, @xcite concludes that carefully assembled speech databases would be necessary to answer this question. in this line, @xcite recently analysed the speech rate of 8 languages using a semantically controlled corpus. they found strong evidence towards non-uniform speech rates across these languages. this result is not surprising, however, given that natural languages vary widely in their phonology, morphology, and syntax. despite these differences, researchers have hypothesised that there exist compensatory relationships between the complexity of these components @xcite . for instance, a larger phonemic diversity could be compensated by shorter words @xcite or a larger number of irregular inflected forms could lead to less complex morphological paradigms @xcite . such a compensation can be thus seen as a type of balance, where languages compromise reliability versus effort in communication @xcite . one natural avenue for creating this balance would be a language's information rate. if this were kept roughly constant, the needs of both speakers (who prefer shorter utterances) and listeners (who value easier comprehension) could be accommodated. speech rate would then be compensated by information density, resulting in a form of surprisal-duration trade-off. indeed, pellegrino et al. (2011) and coupé et al. (2019) analogously, the uid hypothesis posits that, within a language, users balance the amount of information per linguistic unit with the duration of its utterance. this hypothesis has been used to explain a range of experimental data in psycholinguistics, including syntactic reduction @xcite and contractions, such as are vs 're @xcite . while this theory is somewhat under-specified with respect to its causal mechanisms, as we argue in @xcite , one of its typical interpretations is that users are maximising a communicative channel's capacity @xcite . if we assume this channel's capacity to be constant across languages, we may derive a cross-linguistic version of uid. such a hypothesis would predict, for instance, that speakers of languages with less informative phones will make them faster. under this specific interpretation, our study can be seen as evidence of uid as a cross-linguistic phenomenon. ## measuring surprisal to formalise our approach, we first present a standard measure of information content: surprisal. in the context of natural language, surprisal @xcite ) measures the shannon information content a linguistic unit conveys in context, which can be measured as its negative log-probability: unfortunately, this surprisal is not readily available, since we would need access to the true distribution p(s t | @xmath0. we will use an approximation p θ (s t | @xmath1, i.e. a phone-level model with estimated parameters θ. ## data we use the voxclamantis dataset for our analysis @xcite . this dataset is derived from spoken readings of the bible foot_3 and spans more than 600 languages from 70 language families, as shown in fig. 2 . 5 this dataset offers us a semantically controlled setting for our experiments, as it is composed of translations of a single text, the bible. this dataset contains automatically generated phone alignments and derived phonetic measures for all its languages (with both phone duration, and vowels' first and second formant frequencies). on average, there are approximately 9,000 utterances (or 20 hours of speech) per language, making it the largest dataset of its kind. phone labels were generated using grapheme-to-phoneme (g2p) tools and time aligned using either multilingual acoustic models @xcite or language-specific acoustic models @xcite . voxclamantis offers its phonetic measurements under three g2p models, which trade-off language coverage and quality. we will focus on two: foot_5 in order to study the trade-off hypothesis we require two measurements: phone durations and phone-level surprisals. as mentioned above, phone durations are readily available in voxclamantis. phone-level surprisals, on the other hand, are not, so we employed phone-level language models in order to estimate them (as detailed in §3). given both these values, we can now perform our crosslinguistic analysis. first, though, we will describe some data quality checks. filtering unitran. the phone and utterance alignments for the voxclamantis dataset were automatically generated and may be noisy due to both of these processes. the labels from the unitran g2p also contain inherent noise due to their deterministic nature. accordingly, we filter the data using the mean mel-cepstral distortion (mcd) as an implicit quality measure for the alignments. mcd is an edit-distance metric which evaluates the distance between some reference speech and speech synthesised using the alignments @xcite . we use the utterance-level mcd scores from the cmu wilderness dataset @xcite , removing all utterances with an mcd score higher than 7. this leaves us with 647 doculects from 600 languages in 69 language families. ## design choices there are several critical design choices that must be made when performing a cross-linguistic analysis of this nature. while some may at first seem inconsequential, they can have a large impact on down-stream results. specifically, we assume that there is a surprisal-duration trade-off which is caused by a capacity to process information, which should be roughly constant across human populations. we must thus control for other potential sources for this trade-off, which we deem to be uninteresting in this work. phone-level analysis. while there are good reasons for performing this analysis at the syllable-or word-level, we believe phones are advantageous for out study. @xcite , for instance, shows syllables are less prone than phones to be completely deleted in casual speech; syllables would thus allow more robust estimates of speech duration. nonetheless, languages that allow for more complex (and long) syllabic structures will naturally have more valid syllables. a larger number of syllables, in turn, will cause each syllable to be less predictable on average. 8 therefore, more complex syllables will be both longer and unpredictable. studying syllables can thus lead to trivial trade-offs which mainly reflect the methodology employed. a similar argument can be made against word-level analyses. 9 performing this type of analysis at the phone-level should alleviate this effect, making it the more appropriate choice. articulatory costs. whereas the range of effort used to produce individual phones may be smaller than in other linguistic hierarchies, there is still a considerable variation in the cost associated with each phone's articulation. for instance, @xcite argued that a phone's articulatory effort was related to its frequency. if this is indeed the case, a direct analysis of surprisal-duration pairs that does not control for articulatory effort could also lead to a trivial trade-off: the long and effortful phones will be less frequent and likely to be more unpredictable, having higher surprisals. to account for each phone's articulatory cost, we use mixed effects models in our analysis, and include phone identity as random intercept effects. ## generalised mixed effects throughout our experiments, we will use mixedeffects models; we provide a brief introduction here (see @xcite for a longer exposition). classical linear regressions models can be written as: in this section, we will first analyse the surprisalduration trade-off in individual languages. we will then perform an analysis with our full data, studying the trade-off both within and across languages with a single model. finally, in our last experiment we will average phone information per language to analyse a purely cross-linguistic trade-off. ## discussion the pressure towards a specific information rate (potentially set at a specific cognitive channel ca-pacity) has been posited as an invariant across languages. directly testing such a claim is perhaps impossible, as data alone cannot prove its universality. moreover, providing meaningful evidence towards this phenomenon requires a careful and comprehensive cross-linguistic analysis, which we attempt to perform in this work. in comparison to similar studies, such as those by @xcite and @xcite , we employ more sophisticated techniques to measure a linguistic unit's (in our case, a phone's) information content. moreover, we also employ more rigorous strategies for analysing the surprisal-duration relationship, controlling for several potential confounds. by introducing these improvements, we attain a more detailed understanding of the role of information in language production, both across and within languages. experimentally, we find that, after controlling for other artefacts, the information conveyed by a phone in context has a modest but significant relationship with phone duration. we see that this relationship is consistently positive across a number of investigated settings, despite being small in magnitude, meaning that more informative phones are on average longer. additionally, using two-tailed tests at α < 0.01 throughout our experiments, we find no language with a significant negative relationship between phone surprisal and duration. limitations and future work. in this work, we implemented a careful evaluation protocol to study the relationship between a phone's surprisal and duration in a representative set of languages. to perform our study in such a large number of languages, however, we rely on the automatically aligned phone measurements from voxclamantis, which contain noise from various sources. future work could investigate if biases in the dataset generation protocol could impact our results. further, voxclamantis data is derived from readings of the bible. future studies could extend our analysis to other settings, such as conversational data. ## conclusion in this work, we have provided the widest crosslinguistic investigation of phone surprisal and duration to date, covering 600 languages from over 60 language families spread across the globe. we confirm a surprisal-duration trade-off both across these analysed languages and within a subset of 319 of them, covering 53 language families. while there exist arguments against some of our design choices, our overarching conclusion is remarkably consistent across our analyses: the presence of a surprisal-duration trade-off is significant in language production. in other words, both across and within languages, phones carrying more information are longer, while phones carrying less information are produced faster. johannes dellert, thora daneyko, alla münch, alina ladygina, armin buch, natalie clarius, ilja grigorjew, mohamed balabel, hizniye isabella boga, zalina baysarova, roland mühlenbernd, johannes wahle, and gerhard jäger. 2020. northeuralex: a wide-coverage lexical database of northern eurasia. language resources and evaluation, 54:273-301. david fay and anne cutler. 1977. malapropisms and the structure of the mental lexicon. linguistic inquiry, 8(3):505-520. august fenk and gertraud fenk. 1980. konstanz im kurzzeitgedächtnis -konstanz im sprachlichen informationsfluß? zeitschrift für experimentelle und angewandte psychologie, 27(3):400-414. ivan fonagy and klara magdics. 1960. speed of utterance in phrases of different lengths. language and speech, 3(4):179-192. cécile fougeron and patricia a. keating. 1997. articulatory strengthening at edges of prosodic domains. the journal of the acoustical society of america, 101(6):3728-3740. austin f. frank and t. florain jaeger. 2008. speaking rationally: uniform information density as an optimal strategy for language production. in proceedings of the annual meeting of the cognitive science society, volume 30. dmitriy genzel and eugene charniak. 2002. entropy rate constancy in text. in proceedings of the 40th annual meeting of the association for computational linguistics, pages 199-206, philadelphia, pennsylvania, usa. association for computational linguistics. adam goodkind and klinton bicknell. 2018. predictive power of word surprisal for reading times is a linear function of language model quality. in proceedings of the 8th workshop on cognitive modeling and computational linguistics (cmcl 2018), pages 10-18, salt lake city, utah. association for computational linguistics.