Speakers enhance contextually confusable words (2020)

Abstract: Recent work has found evidence that natural languages are shaped by pressures for efficient communication — e.g. the more contextually predictable a word is, the fewer speech sounds or syllables it has (Piantadosi et al. 2011). Research on the degree to which speech and language are shaped by pressures for effective communication — robustness in the face of noise and uncertainty — has been more equivocal. We develop a measure of contextual confusability during word recognition based on psychoacoustic data. Applying this measure to naturalistic speech corpora, we find evidence suggesting that speakers alter their productions to make contextually more confusable words easier to understand.

## introduction a major open question in the study of natural languages is the extent to which pressures for efficient communication shape the online production choices of speakers or the system of forms and form-meaning mappings. @xcite famously noted that highly frequent words tend to be shorter and hypothesized that this could be explained in terms of pressures for efficient communication: the average cost of producing a word is lower than it would be otherwise. more recent work has formalized hypotheses about the effect of communicative pressures on language usage and design using tools from information theory @xcite and rational analysis @xcite . this work has found evidence that meanings are allocated to word types in a way that minimizes speaker effort @xcite , and that this appears to be at least partly explainable by online production choices @xcite . while this research offers evidence that lexicons and the production choices of speakers are shaped by pressures for efficient communication, other work examining how much words and lexicons are shaped by pressures for ensuring effective communication in the face of noise and uncertainty has been more equivocal. this work has found evidence that words with greater neighborhood size or density -that is, words that have a greater number of similar-sounding neighbors -have faster onset of production, and have lower overall durations. words with greater neighborhood density also take longer for listeners to recognize and comprehend, and have less acoustically distinctive vowels @xcite ; see @xcite . this work provides a challenge for communicatively-oriented models of production: words with greater numbers of similar-sounding neighbors seem likely to be more confusable, and therefore speakers would be predicted to decrease the likelihood of noise by, e.g., increasing their duration. however, this work does not directly estimate word confusability, instead using neighborhood density or an acoustic similarity measure as a proxy. it remains possible that greater word confusability is associated with phonetic enhancement, and that a more direct measure of confusability would reveal this relationship. in this paper, we present a measure of relative word confusability based on both a language model and psychoacoustic data, and we examine how well it predicts word durations in natural speech corpora. this measure differs from neighborhood density in three ways: 1) it is sensitive to edit type; 2) it considers words with edit distance greater than 1; and 3) it takes into account top-down expectations. the structure of the paper is as follows. we first present a derivation of a bayesian model of word recognition (broadly similar to @xcite that incorporates both linguistic context and a model of noise estimated from the gating data of @xcite . we use this speech recognition model to define a measure of confusability, and apply this measure to content words in the nxt-annotated subset of the switchboard corpus and in the buckeye corpus @xcite . we provide evidence that greater confusability is associated with longer duration. ## a model of word confusability we propose a simplified model of word confusability, in which there are two factors that will make word 𝑣 in context 𝑐 more vs. less confusable. on the one hand, a listener who has observed context 𝑐 has some 'top-down' beliefs and expectations about what 𝑣 will be before the speaker produces any acoustics for 𝑣. on the other hand, once the speaker has produced acoustics for 𝑣, there will be (in general ambiguous) 'bottom-up' acoustic cues that will usually underdetermine what the speaker's choice of 𝑣 actually was. the goal of the listener is then to combine their top-down expectations with their bottom-up observations to reason about which words are more vs. less likely to have been what the speaker intended. 1 we operationalize the perceptibility of word 𝑣 as the probability that the listener accurately recovers this word in situations where the speaker uses it; the confusability of a word is inversely related to its perceptibility. if a speaker has a model of the expected confusability of a given word, they can then decide to lengthen or shorten their particular production of the word token, balancing listener comprehension and their own effort. ## materials and methods we make use of two types of data: psychoacoustic gating data for estimating a noise model, and several corpora of natural speech for evaluating whether individuals increase the duration of more confusable words. ## results four model hyperparameters were selected using the switchboard and buckeye training sets: the order and direction of the n-gram model, the diphoneto-diphone channel pseudocounts, and the noise factor 𝜆. 11 backward bigram language models were found to perform best on the training sets, possibly due to distributional differences between these corpora and the fisher corpus, which was used for language model estimation. this is consistent with prior work in the area (e.g. @xcite . pseudocounts were set to 0.01, and the term 𝜆 was set to 2 -6 . figure 2 shows the frequency of modelcomputed confusability scores on the switchboard and buckeye test sets. figure 1 shows the relationship between confusability and word duration on the test sets. the first set of analyses include all of the covariates from section 3.5, except for unigram confusability. this allows us to determine whether there is an effect of word confusability on duration, independent of whether this effect is sensitive to context. greater confusability is associated with longer word durations on both the switchboard and buckeye training sets (@xmath0.001 for all analyses). after controlling for unigram confusability, contextual confusability remains associated with longer word durations on both the switchboard and buckeye training sets (@xmath1.001 for all analyses). ## discussion we draw two main conclusions from our results. first, we provide evidence that speakers lengthen words that are more confusable. this supports the hypothesis that variation and structure in natural languages are shaped not only by pressures for efficient signals, but also pressures for effective communication of the speaker's intended message in the face of noise and uncertainty @xcite @xcite . second, we provide large scale, naturalistic evidence for reduction and enhancement driven by contextual confusability. conversational context may make a speaker's intended message easier or harder to recover from ambiguous acoustics. the results suggest that speakers modulate their utterances in a manner that is sensitive to this effect of context, increasing duration when context makes the intended utterance harder to recover. the results complement previous work which demonstrates reduction and enhancement driven by contextual predictability (see e.g. seyfarth 2014). they also complement work which shows confusability-driven reduction and enhancement in targeted experimental manipulations (see e.g. @xcite @xcite . the study may help to resolve questions raised by previous work examining the effects of neighborhood density. that work found negative or null associations between word duration and neighborhood density and related measures (e.g. @xcite . the proposed confusability measure differs from neighborhood density in three ways: it is sensitive to edit type, words greater than two edits away, and top-down effects. these differences may account for the discrepancy in the effects of neighborhood density and confusability. under one hypothesis, neighborhood density effects reflect spillover of activation between words with overlapping subsequences of speech sounds (e.g. @xcite , @xcite , @xcite , @xcite ). this spillover is potentially sensitive only to levenshtein distance. in contrast, confusability is sensitive to fine-grained perceptual structure. when lexical neighbors differ in perceptually distinct segments, they will typically be nonconfusable. a second hypothesis is that the discrepancy arises from the role of top-down expectations in confusability. neighborhood effects are type-level phenomena: a word has the same neighbors no matter what context it appears in. confusability, on the other hand, is a token-level phenomenon: contextual expectations will change the confusability of a word. stable properties of the lexicon may determine which segment sequences undergo frequent articulatory rehearsal, and are reduced as a consequence. the confusability measure picks up on context-dependent variation, which rehearsal processes in the articulatory system may not be sensitive to. the study suggests several directions for future work. first, while there are advantages of using naturalistic speech data @xcite , it would be desirable to have experimental validation of the confusability measure and its relationship to speaker reduction. second, a lower-perplexity neural language model would provide better estimates of a word's confusability, but would first need to be validated on speech data. third, a more sophisticated channel model would allow for insertions and deletions, and better capture transitional coarticulatory cues @xcite . because speakers enhance or reduce their speech in ways other than changing duration (see e.g. @xcite @xcite , such a model would permit investigation of targeted enhancement and reduction in naturalistic data.