Developing multilingual speech synthesis system for O jibwe, Mi’kmaq, and Maliseet (2025)

Abstract: We present lightweight flow matching multilingual text-to-speech (TTS) systems for Ojibwe, Mi’kmaq, and Maliseet, three Indigenous languages in North America. Our results show that training a multilingual TTS model on three typologically similar languages can improve the performance over monolingual models, especially when data are scarce. Attention-free architectures are highly competitive with self-attention architecture with higher memory efficiency. Our research provides technical development to language revitalization for low-resource languages but also highlights the cultural gap in human evaluation protocols, calling for a more community-centered approach to human evaluation.

## introduction many world languages are currently endangered, especially those spoken by historically marginalized and indigenous communities. language revitalization and reclamation is an ongoing effort to ensure continued language vitality for community self-determination and well-being @xcite @xcite . among recent efforts of language revitalization, tts technology is valued as a potential tool to assist the education of indigenous languages, as tts models can flexibly synthesize diverse learning materials to guide pronunciation learning @xcite . in general, speech synthesis for indigenous languages is underdeveloped compared to the majority of languages. the main barrier to developing tts technologies for indigenous communities with oral traditions is still the lack of data @xcite . there are recent efforts to develop speech synthesis systems for low-resource and indigenous languages, including mundari @xcite , kanien'kéha (also known as mohawk; iroquoian), gitksan (tsimshianic), sen coten (coast salish) @xcite , plains cree (central algonquian) @xcite and ojibwe @xcite . yet there is still room for improvement and development in this space. in this study, we continue this line of effort and develop tts systems for ojibwe, mi'kmaq, and maliseet, the latter two of which haven't received any attention from the speech processing community yet. our study explicitly tackles several challenges in designing speech technology for indigenous communities. the code is available at: https://github.com/ shenrantomwang/tts . ## data collection languages we worked closely with speakers from three indigenous languages of canada: ojibwe, mi'kmaq, and maliseet. the three languages are genetically related. ojibwe is spoken around the great lakes of north america and is part of the central branch of the algonquian family, while mi'kmaq and maliseet are spoken in the maritimes and are classed within the eastern branch of the algonquian family. according to estimates from the 2021 statistics canada survey, there are 25,440 speakers of ojibwe, 9,000 speakers of mi'kmaq, and 790 speakers of maliseet @xcite . all language communities are actively involved in significant efforts to document and ensure the continued vitality of their languages. ## experiments training as these languages all use a phonetically transparent latin alphabet, we used a simple character-based tokenizer to tokenize all sentences. punctuations were all removed except for the apostrophe in ojibwe, which plays a role in ojibwe phonology. monolingual models were trained for each individual speaker, whereas multilingual models were trained on all speakers with different sequence mixing layers. all experiments were run on a single a100 40gb gpu for a fixed 200 epochs. full training details are available in appendix a. vocoder for waveform generation, we trained a vocos vocoder @xcite on all training samples. vocos is a frequency domain vocoder that closely matches the performance of timedomain vocoders like hifi-gan @xcite and diffusion-based vocoder like fregrad @xcite but with much higher throughput. since vocoder is not the focus, we provided their evaluation results in appendix d. ## results and discussions objective evaluation we perform our objective evaluation results with fundamental frequency root mean square error (f0 rmse), logamplitude rmse (las rmse), mel cepstral distortion (mcd), perceptual evaluation of speech quality (pesq), short-time objective intelligibility (stoi), voiced/unvoiced f1 (vuv f1) and mfcc frechet distance (fid), similar to contemprary works @xcite . results in while the self-attention matchatts dominates most objective metrics, other attention-free architectures also match its performance closely. no single model dominates all objective metrics. hydra's performance is particularly close to self-attention, suggesting that it is a strong competitor. its bidirectional nature also allows it to outperform mamba2. fnet underperforms all other models due to its parameter-free nature. in terms of computational efficiency, as shown in subjective evaluation despite these challenges, in evaluating the current work, we designed separate mean opinion score (mos) surveys for each language. for each tts voice, the survey included 10 generated utterances from each of the five models and 10 utterances of natural speech. the detailed design is described in appendix e. we were able to recruit three raters for ojibwe but one did not complete the survey. for mi'kmaq and maliseet, we were not able to obtain mos rating due to the limited number of speakers. generally speaking, the mos ratings are largely consistent with the objective metrics (see as recently discussed in @xcite , there are many challenges and questions to be raised when conducting a subjective evaluation of speech synthesis with indigenous communities. we also find that, due to the gap in cultural norms, the use of standard measures like mos and the current experimental paradigm may not always be viable in determining the quality of synthetic speech. for example, despite our instructions, one ojibwe rater rated 5 for all ojibwe nj's voices, regardless of whether it was natural or synthetic. we believe this may have been due to a reluctance to comment negatively on the voice, even when it was synthetic. the concept of participating in controlled experiments and judging synthetic voices, in general, is not a natural task, and cultural norms can amplify this. this implies that researchers working with indigenous communities should design more creative measures that also conform to the cultural norms of the relevant community. we plan to conduct such work as we continue development of these systems ## conclusion in this paper, we report our ongoing efforts to develop tts systems with and for the indigenous community. our experiments demonstrate that training multilingual tts models on similar languages can partially compensate for the lack of data for individual languages. in the future, we will be working with the relevant communities and schools to deploy these systems for indigenous language education.