One Model to Pronounce Them All: Multilingual Grapheme-to-Phoneme Conversion With a Transformer Ensemble (2020)

Abstract: The task of grapheme-to-phoneme (G2P) conversion is important for both speech recognition and synthesis. Similar to other speech and language processing tasks, in a scenario where only small-sized training data are available, learning G2P models is challenging. We describe a simple approach of exploiting model ensembles, based on multilingual Transformers and self-training, to develop a highly effective G2P solution for 15 languages. Our models are developed as part of our participation in the SIGMORPHON 2020 Shared Task 1 focused at G2P. Our best models achieve 14.99 word error rate (WER) and 3.30 phoneme error rate (PER), a sizeable improvement over the shared task competitive baselines.

## introduction speech technologies are becoming increasingly pervasive in our lives. the task of graphemeto-phoneme (g2p) conversion is an important component of both speech recognition and synthesis. in g2p conversion, sequences of graphemes (the symbols used to write words) are mapped to corresponding phonemes (pronunciation symbols, e.g., symbols of the international phonetic alphabet). members of the special interest group on computational morphology and phonology (sigmorphon) have proposed a g2p shared task (sigmor-phon 2020 shared task 1) 1 involving multiple languages. in this paper, we describe our submissions to the shared task. organizers provide an overview of the task and submitted systems in gorman et al. (2020) (this volume). @xcite the shared task webpage is accessible at: https: //sigmorphon.github.io/sharedtasks/2020/task1 . the task was introduced with data from 10 languages, with an additional 5 'surprise' languages released during the task timeline. our goal was to develop an effective system based on modern deep learning methods as a solution. however, deep learning technologies work best with sufficiently large training data. hence, a clear challenge we came across is the limited size of the shared task training data for each of the 15 individual languages. to ease this bottleneck, we decided to view the task through a multilingual machine translation lens where we build a single model mapping from input to output across all the languages simultaneously. in this, we hypothesized that a multilingual model would allow for shared representations across the various languages that may be more powerful than individual representations of monolingual models. abundant evidence now exists for approaching machine translation tasks from a multilingual perspective (johnson et al., 2017a; dong et al., 2015; firat et al., 2016), which inspired our choice. in order to make use of unlabeled data, we also explore a straightforward self-training approach. in particular, we employ our trained models to convert sequences of multilingual unlabeled graphemes, taken from wikipedia data, into multilingual phonemes. we then select sequences of phonemes predicted with our models above a certain confidence threshold to augment the shared task training data, thus re-training our models with larger (gold and silver) training data from scratch. our models are based on the transformer architecture which exploits effective self-attention. we show that both our multilingual model and the self-trained variation outperform the results of the competitive baseline monolingual models provided by the task organizers. ultimately, we demonstrate how our simple modeling choices enable us to provide an effective solution to the problem in spite of the lowresource challenge. intrinsically, our approach also enjoys the simplicity of a single model rather than 15 different models. the rest of the paper is organized as follows: section 2 is a description of the shared task data, evaluation metrics, and baselines. section 3 introduces both our fully supervised, multilingual models (section 3.1) and self-trained model (section 3.2). we present our results in section 4. we provide an analysis of results and report on an ablation study in section 5. we overview related work in section 6, and conclude in section 7. ## task data, evaluation, and baselines the data provided by the organizers of the shared task are extracted from wiktionary foot_0 using the wikipron library (lee et al., 2020), and consist of 4,050 gold labeled graphemephoneme pairs for each of 15 languages, split into a training set (3,600 per language) and a development set (450 per language). the blind test data comprise 450 sources for each language. the data involves languages in the set {adyghe (ady), armenian (arm), bulgarian (bul), dutch (dut), french (fre), georgian (geo), modern greek (gre), hindi (hin), hungarian (hun), icelandic (ice), japanese hiragana (jpn), korean (kor), lithuanian (lit), romanian (rum), vietnamese (vie)}. @xcite this set of languages employ a variety of writing systems: alphabets (e.g. french), alphasyllabary (e.g. hindi), and syllabary (e.g. japanese hiragana), thus introducing enough diversity and modelling challenge. ## models as explained, our models are based on transformers and we offer two primary types of models, depending on how we supervise each. we first introduce fully supervised multilin-gual models, then we introduce our semisupervised models (also multilingual). our semi-supervised models follow a self-training set up. we now explain each of these models. ## conclusion we introduced a multilingual approach to g2p conversion, exploiting transformers in a fully supervised multilingual setting. strikingly, our choice to model all languages in a shared, nultilingual space reduces error rates (in wer and per) by almost one half. we also showed how an ensemble of individuallytrained multilingual transformers, is an improvement over non-ensemble models. we also leveraged multilingual wikipedia data via a self-training strategy, though due to time constraints we were not able to incorporate enough silver labeled data into training to see the results we had hoped for foot_9 . nevertheless, the multilingual models successfully surpassed all organizer-provided baselines on the task and compared favorably to several other submitted models. our future work includes scaling up our self-training with larger wikipedia data and choosing fully-trained models (e.g., in our case ones at 200k steps) to include in the ensemble.