Requirements and Motivations of Low-Resource Speech Synthesis for Language Revitalization (2022)

Abstract: This paper describes the motivation and development of speech synthesis systems for the purposes of language revitalization. By building speech synthesis systems for three Indigenous languages spoken in Canada, Kanien’kéha, Gitksan & SENĆOŦEN, we re-evaluate the question of how much data is required to build low-resource speech synthesis systems featuring state-of-the-art neural models. For example, preliminary results with English data show that a FastSpeech2 model trained with 1 hour of training data can produce speech with comparable naturalness to a Tacotron2 model trained with 10 hours of data. Finally, we motivate future research in evaluation and classroom integration in the field of speech synthesis for language revitalization.

## introduction there are approximately 70 indigenous languages spoken in canada, from 10 distinct language families @xcite . as a consequence of the residential school system and other policies of cultural suppression, the majority of these languages now have fewer than 500 fluent speakers remaining, most of them elderly. despite this, interest from students and parents in indigenous language education continues to grow (statistics canada, 2016); we have heard from teachers that they are overwhelmed with interest from potential students, and the growing trend towards online education means many students who have not previously had access to language classes now do. supporting these growing cohorts of students comes with unique challenges for languages with few fluent first-language speakers. a particular concern of teachers is to provide their students with opportunities to hear the language outside of 1 national research council canada 2 university of edinburgh 3 queen's university class. text-to-speech synthesis technology (tts) shows potential for supplementing text-based language learning tools with audio in the event that the domain is too large to be recorded directly, or as an interim solution pending recordings from firstlanguage speakers. development of tts systems in this context faces several challenges. most notable is the usual assumption that neural speech synthesis models require at least tens of hours of audio recordings with corresponding text transcripts to be trained adequately. such a data requirement is far beyond what is available for the languages we are concerned with, and is difficult to meet given the limited time of the relatively small number of speakers of these languages. the limited availability of indigenous language speakers also hinders the subjective evaluation methods often used in tts studies, where naturalness of synthetic speech samples is judged by speakers of the language in question. in this paper, we re-evaluate some of these challenges for applying tts in the low-resource context of language revitalization. we build tts systems for three indigenous languages of canada, with training data ranging from 25 minutes to 3.5 hours, and confirm that we can produce acceptable speech as judged by language teachers and learners. outputs from these systems could be suitable for use in some classroom applications, for example a speaking verb conjugator. ## indigenous language data although the term 'low resource' is used to describe a wide swath of languages, most indigenous languages in canada would be considered 'lowresource' in multiple senses of the word, having both a low amount of available data (annotated or unannotated), and a relatively low number of speakers. most indigenous languages lack transcribed audio corpora, and fewer still have such data recorded in a studio context. due to the limited number of speakers, creating these resources is 1 https://wiki.laptop.org/go/ instructions_for_implementing_a_new_language_% 22voice%22_for_speak_on_the_xo non-trivial: there are limited amounts of text from which a speaker could read, and there are few people available who are sufficiently literate in the languages to transcribe recorded audio. re-focusing speakers' limited time to these tasks presents a significant opportunity cost; they are often already over-worked and over-burdened in under-funded and under-resourced language teaching projects. as mentioned in §2.1, language technology projects that aim to assist language revitalization and reclamation efforts must be centered around the primary goals of those efforts and ensure that the means of developing the technology do not distract or work against the broader sociopolitical goals. a primary stress point for many natural language processing projects involving indigenous communities surrounds issues of data sovereignty. it is important that communities direct the development of these tools, and maintain control, ownership, and distribution rights for their data, as well as for the resulting speech synthesis models (keegan, @xcite . in keeping with this, the datasets described in this paper are not being released publicly at this time. to test the feasibility of developing speech synthesis systems for indigenous languages, we trained models for three unrelated indigenous languages, kanien'kéha ( §3.1), gitksan ( §3.2), and senćoŧen ( §3.3). ## research questions given the motivation and context for language revitalization-based speech synthesis, a number of research questions follow. namely, how much data is required in order to build a system of reasonable pedagogical quality? how do we evaluate such a system? and, how is the resulting system best integrated into the classroom? in §4.1, we discuss the difficulty of evaluating tts systems in low-resource settings. we then discuss preliminary results for english and indigenous language tts which show that acceptable speech quality can be achieved with much less training data than usually considered for neural speech synthesis ( §4.2). finally, we suggest possible directions for pedagogical integration in section §4.4. ## conclusion in this paper, we presented the first neural speech synthesis systems for indigenous languages spoken in canada. subjective listening tests showed encouraging results for the naturalness and acceptability of voices for two languages, kanien'kéha and gitksan, despite limited training data availability (3.5 hours and 35 minutes, respectively). more extensive evaluation on english shows that the fastspeech2 architecture can produce speech with similar quality to a tacotron2 system using a fraction of the amount of speech usually considered for neural speech synthesis. notably, a fast-speech2 voice trained on 1 hour of english speech achieved subjective naturalness ratings not significantly different from a tacotron2 voice using 10 hours of data, while a 3-hour fastspeech2 system showed no significant difference from a 24-hour tacotron2 voice. we attribute these results to the fact that fast-speech2 learns input token durations from forced alignments, rather than jointly learning to align linguistic inputs to acoustic features alongside the acoustic feature prediction task as in attention-based architectures such as tacotron2. given forced alignments of sufficient quality, which we found to be achievable even by training a montreal forced aligner model only on our limited indigenous language training data, this makes for more data-efficient training of neural tts systems than has generally been explored in previous work. these findings show great promise for future work in low-resource tts for language revitalization, especially as they come from systems trained from scratch on such limited data, rather than pre-training on a high-resource language and subsequent fine-tuning on limited target language data.