Improved pronunciation prediction accuracy using morphology (2021)

Abstract: Pronunciation lexicons and prediction models are a key component in several speech synthesis and recognition systems. We know that morphologically related words typically follow a fixed pattern of pronunciation which can be described by language-specific paradigms. In this work we explore how deep recurrent neural networks can be used to automatically learn and exploit this pattern to improve the pronunciation prediction quality of words related by morphological inflection. We propose two novel approaches for supplying morphological information, using the word’s morphological class and its lemma, which are typically annotated in standard lexicons. We report improvements across a number of European languages with varying degrees of phonological and morphological complexity, and two language families, with greater improvements for languages where the pronunciation prediction task is inherently more challenging. We also observe that combining bidirectional LSTM networks with attention mechanisms is an effective neural approach for the computational problem considered, across languages. Our approach seems particularly beneficial in the low resource setting, both by itself and in conjunction with transfer learning.

## introduction morphophonology is the study of interaction between morphological and phonological processes and mostly involves description of sound changes that take place in morphemes (minimal meaningful units) when they combine to form words. for example, the plural morpheme in english appears as @xcite , @xcite , and @xcite part of the work was done when d.s., n.c. and a.b. were at google. in phonology, e.g. in cops, cogs and courses. the different forms can be thought to be derived from a common plural morphophoneme which undergoes context dependent transformations to produce the correct phones. a pronunciation model, also known as a grapheme to phoneme (g2p) converter, is a system that produces a phonemic representation of a word from its written form. the word is converted from the sequence of letters in the orthographic script to a sequence of phonemes (sound symbols) in a pre-determined transcription, such as ipa or x-sampa. it is expensive and possibly, say in morphologically rich languages with productive compounding, infeasible to list the pronunciations for all the words. so one uses rules or learned models for this task. pronunciation models are important components of both speech recognition (asr) and synthesis (text-to-speech, tts) systems. even though end-to-end models have been gathering recent attention @xcite , often state-of-the-art models in industrial production systems involve conversion to and from an intermediate phoneme layer. a single system of morphophonological rules which connects morphology with phonology is well-known @xcite ). in fact computational models for morphology such as the two-level morphology of @xcite ; kaplan and @xcite have the bulk of the machinery designed to handle phonological rules. however, the approach involves encoding language-specific rules as a finite-state transducer, a tedious and expensive process requiring linguistic expertise. linguistic rules are augmented computationally for small corpora in @xcite , although scalability and applicability of the approach across languages is not tested. we focus on using deep neural models to improve the quality of pronunciation prediction using morphology. g2p fits nicely in the well-studied sequence to sequence learning paradigms @xcite , here we use extensions that can handle supplementary inputs in order to inject the morphological information. our techniques are similar to @xcite , although the goal there is to lemmatize or inflect more accurately using pronunciations. taylor and richmond (2020) consider improving neural g2p quality using morphology, our work differs in two respects. first, we use morphology class and lemma entries instead of morpheme boundaries for which annotations may not be as readily available. secondly, they consider bilstms and transformer models, but we additionally consider architectures which combine bilstms with attention and outperform both. we also show significant gains by morphology injection in the context of transfer learning for low resource languages where sufficient annotations are unavailable. ## background and related work pronunciation prediction is often studied in settings of speech recognition and synthesis. some recent work explores new representations @xcite @xcite , but in this work, a pronunciation is a sequence of phonemes, syllable boundaries and stress symbols @xcite . a lot of work has been devoted to the g2p problem (e.g. see @xcite ), ranging from those focused on accuracy and model size to those discussing approaches for data-efficient scaling to low resource languages or multilingual modeling @xcite @xcite . morphology prediction is of independent interest and has applications in natural language generation as well as understanding. the problems of lemmatization and morphological inflection have been studied in both contextual (in a sentence, which involves morphosyntactics) and isolated settings @xcite @xcite . morphophonological prediction, by which we mean viewing morphology and pronunciation prediction as a single task with several related inputs and outputs, has received relatively less attention as a language-independent computational task, even though the significance for g2p has been argued @xcite . @xcite show improved morphology prediction using phonology, and @xcite show the reverse. the present work aligns with the latter, but instead of requiring full morphological segmentation of words we work with weaker and more easily annotated morphological information like word lemmas and morphological categories. ## improved pronunciation prediction we consider the g2p problem, i.e. prediction of the sequence of phonemes (pronunciation) from the sequence of graphemes in a single word. the g2p problem forms a clean, simple application of seq2seq learning, which can also be used to create models that achieve state-of-the-art accuracies in pronunciation prediction. morphology can aid this prediction in several ways. one, we could use morphological category as a non-sequential side input. two, we could use the knowledge of the morphemes of the words and their pronunciations which may be possible with lower amounts of annotation. for example, the lemma (and its pronunciation) may already be annotated for an out-of-vocabulary word. often standard lexicons list the lemmata of derived/inflected words, lemmatizer models can be used as a fallback. learning from the exact morphological segmentation (taylor and richmond, 2020) would need more precise models and annotation @xcite . given the spelling, language specific models can predict the pronunciation by using knowledge of typical grapheme to phoneme mappings in the language. some errors of these models may be fixed with help from morphological information as argued above. for instance, homograph pronunciations can be predicted using morphology but it is impossible to deduce correctly using just orthography. 1 the pronunciation of 'read' (/ôi:d/ for present tense and noun, /ôed/ for past and participle) can be determined by the part of speech and tense; the stress shifts from first to second syllable between 'project' noun and verb. ## discussion we discuss our results under two themes -the efficacy of the different neural models we have implemented, and the effect of the different ways of injecting morphology that were considered. we consider three neural models as described above. to compare the neural models, we first note the approximate number of parameters of each model that we trained: for bilstm and bilstm+attn, the parameter size is based on neural architecture search i.e. we estimated sizes at which accuracies (nearly) peaked. for transformer, we believe even larger models can be more effective and the current size was chosen due to computational restrictions and for "fairer" comparison of model effectiveness. under this setting, bilstm+attn models seem to clearly outperform both the other models, even without morphology injection (cf. @xcite , albeit it is in the multilingual modeling context). transformer can beat bilstm in some cases even with the suboptimal model size restriction, but is consistently worse when the sequence lengths are larger which is the case when we inject lemma spellings and pronunciations. we also look at how adding lexical form information, i.e. morphological class and lemma, helps with pronunciation prediction. we notice that the improvements are particularly prominent when the g2p task itself is more complex, for example in english. in particular, ambiguous or exceptional grapheme subsequence (e.g. ough in english) to phoneme subsequence mappings, may be resolved with help from lemma pronunciations. also morphological category seems to help for example in russian where it can contain a lot of information due to the inherent morphological complexity (about 25% relative error reduction). see appendix b for more detailed comparison and error analysis for the models. our transfer learning experiments indicate that morphology injection gives even more gains in low resource setting. in fact for both the languages considered, adding morphology gives almost as much gain as adding a high resource language to the bilstm+attn model. this could be useful for low resource languages like georgian where a high resource language from the same language family is unavailable. even with the high resource augmentation, using morphology can give a significant further boost to the prediction accuracy. ## conclusion we note that combining bilstm with attention seems to be the most attractive alternative in getting improvements in pronunciation prediction by leveraging morphology, and hence correspond to the most appropriate 'model bias' for the problem from among the alternatives considered. we also note that all the neural network paradigms discussed are capable of improving the g2p prediction quality when augmented with morphological information. since our approach can potentially support partial/incomplete data (using appropriate missing or n/a tokens), one can use a single model which injects morphology class and/or lemma pronunciation as available. for languages where neither is available, our results suggest building word-lemma lists or utilizing effective lemma-tizers @xcite .