Neural Text-to-Speech Synthesis for V õro (2023)

Abstract: This paper presents the first high-quality neural text-to-speech (TTS) system for Võro, a minority language spoken in Southern Estonia. By leveraging existing Estonian TTS models and datasets, we analyze whether common low-resource NLP techniques, such as cross-lingual transfer learning from related languages or multi-task learning, can benefit our low-resource use case. Our results show that we can achieve high-quality Võro TTS without transfer learning and that using more diverse training data can even decrease synthesis quality. While these techniques may still be useful in some cases, our work highlights the need for caution when applied in specific low-resource scenarios, and it can provide valuable insights for future low-resource research and efforts in preserving minority languages.

## introduction the advancements in neural text-to-speech (tts) technology have greatly improved the quality of speech synthesis for many languages. however, despite the potential benefits of tts for facilitating accessibility and language preservation, developing tts systems for low-resource languages remains challenging due to the limited availability of training data for these languages. võro, a finno-ugric minority language spoken in southern estonia, serves as a great example of a low-resource language that could benefit from tts technology. while linguistic resources for võro are limited, the language is closely related to estonian -a high-resource finno-ugric language with significantly more datasets, tools, and pretrained models. the goal of this paper is to present the first highquality neural tts system for võro and evaluate various low-resource nlp techniques for improving synthesis quality for the language. by leveraging existing estonian tts models and datasets, we investigate the impact of transfer learning from related languages and multi-speaker and multilingual approaches on the tts quality of võro. the main contributions of this paper are: ## background as neural text-to-speech models require vast amounts of data, existing research has proposed several approaches to mitigate the issue of insufficient training data. for example, several works have shown that cross-lingual pretraining improves the quality of low-resource tts systems @xcite . in a survey on multilingual strategies for lowresource tts, @xcite evaluated the usefulness of using multilingual datasets for improving low-resource language performance. they observed that for sequence-to-sequence models, including additional data from other languages is almost always beneficial and often overweighs the negative effect of having a lower ratio of target data in the entire training dataset. the authors also noted that there is no clear evidence that using supporting languages from the same language family is more beneficial but claimed that using a shared input representation space (such as phonemes) may be more important. at the same time, using closely related languages to boost low-resource performance has been successfully used for many text-based nlp tasks, including for developing finno-ugric machine translation systems that also include the võro language @xcite . unfortunately, the usage of neural methods for võro has so far been limited to this example. there is also no existing research on võru tts. while the estonian language institute and the võro institute have collaborated to create an hmm-based tts system for võro foot_1 , this work has not been described in research. ## methodology in this section, we present our methodology and experiment setup. our approach evaluates the benefits of low-resource tts approaches when training non-autoregressive transformer-based models @xcite . we focus on three common strategies -cross-lingual transfer learning from a pre-trained estonian tts model, combining data from multiple võro speakers, and including estonian data to create a multilingual system. additionally, we explore data augmentation to handle the orthographic variation of võro. ## results to assess the quality of the models, we conducted a mean opinion score (mos) @xcite ) evaluation 4 among volunteers from the võro community. the evaluators were required to know the võro language but did not have to be native speakers. of the 41 volunteers, 6 considered themselves native speakers, and 9 had a self-reported võru level of c1 or higher. many participants with lower levels of võru knowledge also mentioned that their passive language skills were higher as they mostly used võro when communicating with older family members who were native speakers. the evaluation used a subset of 50 random sentences per speaker (100 total per method) from the held-out dataset, and the samples were generated using pretrained hifigan @xcite models. the appropriate model for each speaker was selected by evaluating samples generated with multiple vocoder models. for the lowerpitched male speaker, we used a model trained on the vctk dataset @xcite , and for the child speaker, we used a model trained on the lj speech @xcite corpus and finetuned on tacotron 2 @xcite output. we also included ground truth samples from the held-out dataset and ground truth samples converter to mel-spectrograms and reconstructed by the same vocoder models. the evaluation results can be seen in in addition to scoring samples, participants were encouraged to comment on their overall impressions of speech quality and the evaluation process. many expressed a positive surprise about synthesis quality and mentioned the presence of tts artifacts, such as crackling, as their main evaluation criteria. some participants also noted that while almost all samples were intelligible, they did not always sound like a native võro speaker, especially when producing the glottal stop sound. unfortunately, as the participants did not know which models produced which samples, further analysis would be needed to assess whether all models are equally prone to this issue and whether it can also be observed in ground truth examples. firstly, it could be argued that the observations about the low negative performance impact of data imbalance by @xcite may not apply to non-autoregressive transformer-based systems, as the study focused on other methods, such as recurrent or convolutional neural networks. therefore, the performance drop in multilingual models could still be caused by an imbalance between the two languages in the dataset. alternatively, as our model size was dictated by the existing pretrained estonian models, it may lack sufficient capacity to work in a multilingual setting. additionally, it is possible that we should no longer consider võro a low-resource language in this task. based on initial testing with estonian datasets, we found that the required amount of speech data for transformer-based models to produce coherent speech is between 1-2 hours, and improvements from using more data are significantly less noticeable. similar observations about reduced data requirements for transformer-based models have also been recently reported by @xcite . in our case, we had 1.5 hours of speech per speaker, and it may have been sufficient for us not to benefit from additional data from other speakers. alternatively, as the two võro datasets contained identical sentences, they may not differ sufficiently to benefit from each other. however, a more detailed evaluation methodology could be considered to measure the effects on specific features of synthetic speech, such as prosodic variability or pronunciation mistakes. as our work focused on creating a high-quality system for võro without applying artificial constraints, such as using smaller subsets of the highresource datasets, these points were not explicitly explored in our work. however, in the future, low-resource tts strategies should be further reviewed specifically for transformer-based architectures and for different levels of resource constraint. until then, these strategies should be used with caution and evaluated for each specific lowresource scenario. ## conclusion this article presented the first high-quality neural text-to-speech system for the võro language. we explored the usage of estonian tts models and datasets to boost the performance of our lowresource use case. our results suggest that we can achieve highquality võro tts without transfer learning or using data from multiple speakers or closely related languages. while these techniques may still be helpful in some cases, we highlight the need for further research and evaluation when applied in specific low-resource scenarios.