M un TTS : A Text-to-Speech System for M undari (2024)

Abstract: We present MunTTS, an end-to-end text-to-speech (TTS) system specifically for Mundari, a low-resource Indian language of the Austo-Asiatic family. Our work addresses the gap in linguistic technology for underrepresented languages by collecting and processing data to build a speech synthesis system. We begin our study by gathering a substantial dataset of Mundari text and speech and train end-to-end speech models. We also delve into the methods used for training our models, ensuring they are efficient and effective despite the data constraints. We evaluate our system with native speakers and objective metrics, demonstrating its potential as a tool for preserving and promoting the Mundari language in the digital age.

## introduction india is home to approximately 1652 languages and 22 official languages written in different scripts @xcite . many of these languages are classified as lowresource, as native speakers are moving towards dominant languages that are supported by modernday technologies @xcite . according to a unesco report, india ranks fourth in the list of critically endangered languages with 41 languages @xcite . if a language becomes extinct we lose out a large part of the culture. motivated by these factors, many communities have self-initiated data collection and partnered with tech organizations to build technologies for their language @xcite . text-to-speech (tts) systems have been gaining a lot of importance as a vital language technology due to their applications in education, navigation, accessibility, voice assistants etc. @xcite . however, the development of tts systems for low-resource languages has several challenges * work done when the author was at microsoft 1 artifacts available at https://aka.ms/mun-tts @xcite . firstly, training current-day tts systems requires many hours of audio recordings and corresponding text transcriptions which is resource-intensive. secondly, carefully curating the training data such that it covers the phonetic complexity of the given language requires expert input. this becomes a problem especially when the available data is already scarce. thirdly, the availability of native speakers who are familiar with technology and can do audio recordings in a high-quality studio setup. fourthly, availability of enough native speakers, who could systematically evaluate these systems for subjective metrics. lastly, getting high-quality audio recordings can be very expensive. overcoming these challenges requires not only technical expertise to extract the most out of limited resources but also significant on-field operational efficiency to collect the right quality and quantity of data. no such data collection is possible without the active participation of the community and other stakeholders. in this work, we build a tts system for mundari. mundari is an austro-asiatic language spoken by munda tribes in the eastern indian states of jharkhand, odisha, and west bengal. according to the 2011 india census, there are ≈1m native speakers of this language (2011, archived from the original on 6 march 2021). mundari is mainly written in the dominant script of the region where it is spoken, viz. devanagari, odia, and bangla. in this study, we worked with the mundari spoken in jharkhand and written in the devanagari script. we collected audio recordings for 15,656 unique sentences in mundari. our mundari speech corpus consists of high-quality 26,868 audio recordings in male and female voices consisting of 27.51 hours. the average duration of the recordings in our dataset is 3.7 seconds, and the average sentence length is 8.4 words. using this data we train three tts systems: variational inference with adversarial learning for end-to-end (e2e) text-to-speech (vits) with 22khz sampling rate (vits-22k), vits with 44khz sampling rate (vits-44k) and fine-tune xtts v2 as well. we have audio samples from each of these systems evaluated by native speakers for subjective metrics. vits-44k gives the best overall performance with @xmath0.69 ± 1.18. ## data multiple steps were involved in the data collection process. first, the text data was obtained by translating a hindi corpus of 100,000 sentences obtained from the karya database. karya foot_0 is a data services organization that takes requests from clients and breaks down these complex requests into simple microtasks that users with little to no digital literacy can perform. we randomly selected 20,000 of these sentences and manually translated them to mundari. the translated mundari sentences were expressed using the devanagari script. the translators were instructed to prefer fluency of the sentences over faithfulness of the translations wherever they had to make a choice. the translated sentences were then validated for appropriateness by native speakers. this text corpus was then used as the final dataset for recording one male and one female speaker. the male and female speaker was selected from a pool of 12 speakers (6 male and 6 female), who were asked to complete a reading task online. after they submitted the speech samples, the speakers were then evaluated by native speakers and given a score for their reading efficiency and pronunciation. based on these scores, 3 male and 3 female speakers were shortlisted, and finally, from these 6 speakers, 1 male and 1 female speaker was selected after analyzing some voice quality features. the speakers, i.e., the voice artists, were instructed to record the sentences shown to them without any false starts, filled pauses, hiccups, or any other mistakes. all the recordings are done in a studio-quality room with a microphone connected to the karya crowdsourcing application for the convenience of collecting the data. the recordings' sampling rate is 44.1 khz with 32 bits per sample. finally, the curated text used to collect recordings contains 15,656 unique sentences. the average sentence length in the collected text corpus is 8.4 words. some duplication of sentences across speakers yields a total of 26,868 sentences and recordings in our final dataset. around 74% of the recordings in our dataset feature a female speaker, while the remaining 26% are attributed to a male speaker. we notice that the female recordings are, on average, slightly shorter than male recordings -females' being 3.62 seconds compared to males' 3.85 seconds. we present more details in ## conclusion in this work, we develop a tts system for a lowresource language, mundari, a low-resource language spoken by ≈ 1m people in india. we also analyze existing models for this language and evaluate popular multilingual and multi-speaker models by finetuning them. we show that the vits-44k model achieves a mean mos score of 3.69 and is evaluated as the best among the ones compared by native speakers. we release our model publicly and hope this research further promotes the development of speech systems for endangered and lowresource languages, aiding in bridging the digital divide in india.