Low-Resource G 2 P and P 2 G Conversion with Synthetic Training Data (2020)

Abstract: This paper presents the University of Alberta systems and results in the SIGMORPHON 2020 Task 1: Multilingual Grapheme-to-Phoneme Conversion. Following previous SIGMORPHON shared tasks, we define a low-resource setting with 100 training instances. We experiment with three transduction approaches in both standard and low-resource settings, as well as on the related task of phoneme-to-grapheme conversion. We propose a method for synthesizing training data using a combination of diverse models.

## introduction in this system paper, we discuss the participation of the university of alberta team in the sigmor-phon 2020 task 1: multilingual grapheme-to-phoneme conversion @xcite . this is a sequence-to-sequence transduction task, in which a word, represented by a sequence of graphemes, must be converted into the sequence of phonemes representing its pronunciation. for example, given the french word connaissent the correct output is the phoneme sequence @xcite . following previous sigmorphon shared tasks, in addition to the standard setting with 3600 training examples for each language (which we refer to as the high-resource setting), we define a low-resource setting in which training data is limited to 100 examples. this emulates a plausible scenario of working with a low-resource language for which only a small quantity of reliable phonological data is available. for example, a typical ipa description of the phonological inventory of a single language contains about a hundred phonetic transcriptions of individual words @xcite . we analyze the relative performance of different systems depending on the size of the training data. the task of phoneme-to-grapheme (p2g) conversion is the inverse of grapheme-to-phoneme con-version (g2p), in which the goal is to predict the spelling of a word given its phonetic transcription @xcite . while g2p reflects the difficulty of reading, p2g may indicate the complexity of writing in a given language. training instances for one of the two tasks can easily be applied to the other one by simply reversing the input and output. we use the shared task datasets to investigate how systems designed for g2p perform on p2g. we also leverage raw text corpora to improve the accuracy on p2g, which indirectly leads to improvements on g2p as well. we develop a novel method of mitigating resource limitations by synthesizing additional training data using a combination of multiple g2p and p2g models. the underlying intuition is that a p2g model should be the inverse of the corresponding g2p model. since models trained on a small number of instances tend to have limited accuracy, we attempt to distinguish between the correct and incorrect predictions by ensuring that p2g model output matches the corresponding g2p model input. the precision of this approach is further improved by comparing predictions of different systems. figure 1 illustrates this idea. the principal contributions of this paper include a novel g2p data augmentation method that leverages multiple systems and text corpora, as well as a thorough comparison of several g2p and p2g systems in both low-resource and high-resource settings. ## prior work our methods build upon the prior work of the university of alberta teams on string transduction. di-rectl, a feature-based discriminative transducer, was originally designed for the g2p task @xcite . in directl+ @xcite , the feature set was augmented with joint n-grams defined on both source and target substrings. the system was applied to related tasks such as transliteration @xcite , morphological inflection @xcite , stemming @xcite , and cognate projection @xcite , proving to be particularly competitive in low-resource settings. dtlm @xcite , our principal tool in this work, is a successor of directl+, which incorporates target-side language models and a highprecision alignment. dtlm achieved state-of-theart results on several tasks in which plain word types constitute the transduction target strings. finally, our data augmentation approach is inspired by the self-training approach of @xcite . ## methods in this section, we first describe dtlm, a multipurpose string discriminative transduction system which we apply to both g2p and p2g tasks. we then introduce our approach to synthesizing additional training data from unannotated texts. ## development in this section, we describe our development experiments on both g2p and p2g with three different transduction systems and the synthetic training data. ## test results on the other hand, the data augmentation approach is remarkably successful in the low-resource setting, yielding an average wer improvement over 35% with respect to base transformer. we interpret these results as a strong proof-of-concept of the validity of our data augmentation approach; when training data is limited, it can dramatically improve the accuracy of neural models, without any change to their architecture. ## conclusion we have presented a novel data augmentation method that combines the strengths of multiple string transduction methods. we have also explored both g2p and p2g tasks in both the standard highresource setting, and a low-resource setting of our own design. the results demonstrate that the weakness of neural systems in low-resource settings can be mitigated through the application of data augmentation.