Grapheme-to-Phoneme Conversion with a Multilingual Transformer Model (2020)

Abstract: In this paper, we describe our three submissions to the SIGMORPHON 2020 shared task 1 on grapheme-to-phoneme conversion for 15 languages. We experimented with a single multilingual transformer model. We observed that the multilingual model achieves results on par with our separately trained monolingual models and is even able to avoid a few of the errors made by the monolingual models.

## introduction grapheme-to-phoneme conversion is the task of predicting the phonemic representation for a given orthographic word, where a phoneme is the smallest unit of sound which can distinguish one word from another. in many languages, some phonemes have different realizations depending on their context, and these variants are called allophones. while the task is about predicting phonemes and not allophones, in fact most datasets (e.g., the datasets for hungarian, bulgarian, and armenian) also contain allophones. however, since the distribution of allophones conditioned on the context is learnable, this is not an issue. the shared task training data consists of 15 languages which have diverse phonologies, ranging from tonal languages to languages with glottalized consonants, and they are written in eight different writing systems. the data comes from the english version of wiktionary. each training set contains 3600 words, and each development and test set contains 450 words. the official metrics for the task are word error rate (wer) and phoneme error rate (per). a multilingual approach for grapheme-tophoneme conversion has been explored by @xcite . they propose a sequence-tosequence multilingual model that benefits from training on additional phonetic representations for the same language (which was not permitted in our shared task). the transformer @xcite ) with its attention mechanism has been applied very successfully to machine translation tasks, and it was also used for grapheme-to-phoneme conversion. @xcite suggested using a transformer-based approach for grapheme-tophoneme conversion and @xcite proposed a multilingual transformer model for languages with different writing systems by employing bytelevel input representation. in our submission to the shared task, we explore the performance of a multilingual transformer model with augmented input representation which can transduce a word from any language present in the training data into its ipa representation. ## approach we trained a multilingual model which can transduce a word in any of the 15 source languages into its ipa representation. multilingual models can be of the types many-to-one, one-to-many, or many-to-many. in our case, there are obviously multiple languages on the source side. on the target side, there is usually exactly one desired phoneme sequence for a given source word. superficially, we thus have a many-to-one problem. however, many character sequences exist in more than one language. for instance, the character @xmath0, and its pronunciation depends on the choice of language (/tɹaens.fɔɹ.meɪ.ʃən/ vs. /tʁɑ s.fɔʁ.ma.sjɔ /). this makes it a many-to-many problem for a subset of the data. the possibility of multiple desired sequences on the target side for a given source word makes it necessary to annotate the source words with the desired language. in our approach, we prefix each source word with its two-letter iso language code, followed by an underscore, e.g. 'fr_maison', or 'ka_ავტორი'. this is similar to the approach in @xcite . a side effect of our multilingual approach is that the size of the training data is increased from 3600 to 54000 (15 x 3600) samples. ideally, a model might profit from this enlarged dataset, and languages can learn from each other. given the various source-side writing systems and differences in phoneme sets across languages, we expect crosslanguage learning to be somewhat limited. the multilingual approach proposed here allows for language-specific preprocessing where needed. in our case, we only used a preprocessing step for korean, as outlined above in section 2.2.3. ## conclusion while other submissions outperformed our models, our per for uzh-3 is only 0.51 points higher than that of the winning model (ims). the difference in wer is slightly higher, with an increase of 2.53 points compared to the winning model. overall, this shows that a single multilingual model can achieve competitive results even in a setting with highly unrelated languages, by simply prefixing each word with its language code. in future work, we like to explore further how cross-language interference in a multilingual model influences performance both positively and negatively.