GE 2 PE : P ersian End-to-End Grapheme-to-Phoneme Conversion (2024)

Abstract: Text-to-Speech (TTS) systems have made significant strides, enabling the generation of speech from grapheme sequences. However, for low-resource languages, these models still struggle to produce natural and intelligible speech. Grapheme-to-Phoneme conversion (G2P) addresses this challenge by enhancing the input sequence with phonetic information. Despite these advancements, existing G2P systems face limitations when dealing with Persian texts due to the complexity of Persian transcription. In this study, we focus on enriching resources for the Persian language. To achieve this, we introduce two novel G2P training datasets: one manually labeled and the other machine-generated. These datasets comprise over five million sentences alongside their corresponding phoneme sequences. Additionally, we propose two evaluation datasets tailored for Persian sub-tasks, including Kasre-Ezafe detection, homograph disambiguation, and handling out-of-vocabulary (OOV) words. To tackle the unique challenges of the Persian language, we develop a new sentence-level End-to-End (E2E) model leveraging a two-step training approach, as outlined in our paper, to maximize the impact of manually labeled data. The results show that our model surpasses the state-of-the-art performance by 1.86% in word error rate, 4.03% in Kasre-Ezafe detection recall, and 3.42% in homograph disambiguation accuracy.

## introduction grapheme is the smallest functional unit of a language's writing system, phoneme is the smallest distinguishable sound unit of a language, and g2p is an important part of text-to-speech (tts) and automatic speech recognition (asr) @xcite . e2e tts systems using grapheme as input perform poorly on oov words and homograph disambiguation @xcite ; this phenomenon is more pronounced for low-resource languages. using g2p to convert the written form of text to pronunciation form, and leveraging this form as input to tts systems can considerably improve the intelligibility of the generated speech. g2p is similar with the machine translation (mt) task except that g2p is usually done on an isolated word tokenized at a character level. as a result of this character-level tokenization, transformers have performed poorly on g2p unlike in mt. however, it is shown that the reason behind this anomaly is the lack of information while updating model parameters, and it can be resolved by increasing the batch size @xcite . this finding has led to high performance and efficiency in transformer-based g2p models @xcite . following this success, knowledge transfer has been investigated through multilingual and multitask training @xcite , and grapheme pretraining @xcite . some research has also focused on transfer learning specifically for low-resource languages @xcite and data augmentation methods for training large models @xcite . although in real world applications, g2p is mainly employed for achieving better performance in low-resource tts, recent works on g2p systems mainly focus on high-resource languages like, english and pay less attention to the challenges of g2p for other languages. the persian language (a.k.a farsi) is a low-resource language known as one of the most challenging languages in this field @xcite @xcite due to its unique features. firstly, short vowels (/a/, /e/, and /o/) are not written in persian text resulting in a lack of information while generating the phoneme sequence. secondly, there are many homographs in persian due to the absence of short vowels e.g., /kerm/, /kerem/, and /karam/ are identical in written form. finally, kasre-ezafe, an /e/ sound connecting nouns to adjectives and descriptive nouns, is not written in persian text. as a result of the mentioned features, a persian g2p system requires: morphology and phonology to predict the omitted short vowels of each noun; syntactical relations to detect kasre-ezafe; and semantical knowledge to disambiguate homographs. therefore, unlike english g2p that uses words as input, persian g2p needs a phrase-level or sentencelevel input to achieve acceptable results. in this work, the main goal is to improve g2p accuracy and efficiency by employing sentence-level inputs. however, lack of data and evaluation standards appear to be the main obstacles to achieving this goal. we try to overcome these challenges by providing new training datasets and an evaluation benchmark tailored for specific features of the persian language. in this work, we introduce: ## related work the initial g2p systems used lexicons to map words to pronunciations @xcite . however, a comprehensive coverage of all words is not feasible as language varies over time, location, and usage domain. therefore, rule-based methods are employed alongside lexicons to alleviate this problem @xcite @xcite . although rule-based systems address the oov problem, they require expertise to design, making them challenging to implement @xcite . labeling words with phonetic labels is much easier compared to designing rules, leading to the use of probabilistic models to predict phoneme sequences of words @xcite . in addition, a variety of neural network models have been applied to this task, including rnns @xcite @xcite , cnns @xcite , and transformers @xcite with transformers demonstrating the best performance. following the success of transformers, the focus has shifted from model architecture to other methods aimed at improving performance. ## persian language persian, an indo-european language, uses the arabic script, which originates from the semitic language family with a vastly different phonetic system. this leads to inconsistencies between the written and spoken forms of persian, resulting in a lack of orthographic transparency. orthographic transparency is achieved when each grapheme corresponds to one and only one phoneme, and vice versa @xcite . in persian, each consonant can be represented by up to four different graphemes, and given that short vowels are typically not written, each grapheme can correspond to up to four different pronunciations. consequently, to manage this complexity and enable persian g2p, the task is divided into three subtasks: oov g2p, kasre-ezafe detection, and homograph disambiguation. ## datasets to address the issues discussed in section 3.4, we propose two datasets for training persian g2p at the sentence level, aiming to overcome all mentioned challenges using a single lm. these datasets include a manually labeled dataset ("farsdat aligned") and an automatically labeled dataset ("machine generated"). additionally, we propose two evaluation datasets, "homograph eval" and "kasre eval", to benchmark persian g2p models. "homograph eval" consists of challenging sentences that include homographs, while "kasre eval" contains challenging sentences featuring kasre-ezafe. statistics and data samples for all proposed datasets are available in ## experimental setup to address the challenges previously discussed and provide a persian end-to-end g2p model ( ge2pe ), we propose a byte-level transformer with one sentence as input. to mitigate the lack of data resources during training, we implement a two-step training process that optimizes the use of manually labeled data ("farsdat aligned"). in the following sections, we offer detailed explanations of our model architecture, baselines, proposed training methods, and evaluation metrics. ## results in the first experiment, we compare our proposed model to the multi-module model introduced by model per% wer% silver ge2pe 3.75 17.97 ge2pe 2.92 14.83 @xcite 2.96 16.69 to assess the effectiveness of our training method in maximizing the impact of manually labeled data, we calculated per and wer for both evaluation datasets in the first experiment. the results, summarized in unlike kasre-ezafe detection, there is no explicit token in the phoneme sequence of the training data to indicate the occurrence of homographs. thus, our model was not explicitly trained for homograph disambiguation. nevertheless, the language understanding gained through the g2p training process appears to enhance its performance in this task. ## conclusion with the recent growth of high-resource tts systems, the g2p module has been removed from the pipelines, and speech has been generated using graphemes in an e2e manner. however, phonemes are still needed to generate natural and intelligible speech for low-resource languages. although g2p is mainly used for these languages in real world applications, little work has been done on low-resource g2p. in this work, we emphasized the need for new data resources and conversion approaches for persian, a low-resource language, and provided new datasets for training and evaluating persian g2p with regard to three important persian g2p challenges: oov, kasre-ezafe detection, and homograph disambiguation. additionally, a new e2e model was introduced to address these persian g2p challenges and serve as a baseline for the newly proposed datasets. although using the proposed data, model, and training method led to state-of-the-art results in oov, kasre-ezafe detection, and homograph disambiguation, there is still room for improvement. the current work uses maximum likelihood loss to train the model for all tasks. however, adding a task-specific loss for kasre-ezafe detection can further improve the results. future work can also focus on augmenting data for homograph disambiguation and using task-specific loss for homograph disambiguation as well. these enhancements can further improve the results of the two tasks without any changes to the model architecture or training procedure.