End-to-End Speech Recognition for Endangered Languages of N epal (2024)

Abstract: This paper presents three experiments to test the most effective and efficient ASR pipeline to facilitate the documentation and preservation of endangered languages, which are often extremely low-resourced. With data from two languages in Nepal —Dzardzongke and Newar— we show that model improvements are different for different masses of data, and that transfer learning as well as a range of modifications (e.g. normalising amplitude and pitch) can be effective, but that a consistently-standardised orthography as NLP input and post-training dictionary corrections improve results even more.

## introduction of the 120+ distinct languages identified in the 2011 nepali census, at least 60 are endangered due to socio-political unrest, globalisation and environmental challenges. the 2015 earthquake and the global pandemic have had devastating effects on the tourist industry, which formed the major source of income for the country. long-lasting consequences include the increased migration away from the rural areas where many endangered languages are spoken towards kathmandu and other areas where the nepali language is dominant, as well as international destinations for education and employment. the loss of these languages also means the loss of unique cultural and religious identifiers. given this, there is a clear need for methods and tools to preserve linguistic and cultural diversity. a well-known challenge in language preservation, however, is the transcription bottleneck @xcite : transcribing one minute of audio requires at least an average of 40+ minutes @xcite . the transcription process is furthermore severely hindered by the fact that many endangered languages do not have written traditions or standardised orthographies. while advanced automatic speech-recognition (asr) tools are available, they are often ineffective for these extremely lowresource languages @xcite , due to the lack of good-quality training data. in this paper we present results from three experiments aimed at creating asr models for the endangered languages of nepal: (a) training models for two extremely low-resourced languages, dzardzongke (south mustang tibetan) and kathmandu valley newar, (b) testing the effectiveness of transfer learning for dzardzongke from the related standard tibetan language, and (c) testing other techniques that are useful to enhance low-resource asr such as sound and output manipulation, to measure their effectiveness on datasets of different sizes. ## methodology in this section we discuss our data collection and asr pipeline, followed by a description of our experiments. ## discussion from the results of all three experiments it becomes apparent that modifications are most useful up to around 90 minutes of 'monolingual' transcriptions. transfer learning in particular proved more effective at this stage than sound modifications, although the size of the standard tibetan datasets mattered less than expected. 5 the newar dataset exhibits a broad heterogeneity, encompassing a wide range of sources, whereas the dzardzongke data originates from a more specific geographical area with more data from one speaker, and, on average, shorter utterances, which could explain the higher newar wer of 50 (vs dzardzongke 32). additionally, the newar collection primarily features literary works, including readings of literature, theatrical performances, and discourses on religious or literary subjects that do not generalise well to more casual conversations that are also part of the same dataset. post-training corrections based on probabilistic spell checking from existing monolingual transcriptions is marginally effective for improving wer in dzardzongke, but would be more effective especially for recordings on new topics if a more comprehensive corpus were available. for newar, the lack of standardised, romanised spelling leads to higher word (but not character) error rates, but as shown in the previous section, these are not necessarily representative of actual qualitative errors in transcription. for both languages, as well as the standard tibetan datasets, an in-depth analysis of transcriptions results reveals the importance of a wellbalanced, varied dataset where incomplete transcriptions are filtered out to avoid artificially high error rates that make the models worse. although it is tempting with any low-resource language to utilise as many transcribed utterances as possible, those with too much noise or interference are clearly creating more problems later on. ## conclusion our main goal was to present a first test of the most effective and efficient asr pipeline to facilitate the documentation and preservation of endangered languages, which are often extremely low-resourced. for both dzardzongke and newar, model improvements are different for different masses of data, which helps to guide those who have to start transcriptions from scratch. we tested different modification techniques to see which would be most effective for small-size datasets and carefully evaluated and discussed the results. directions for future research include experiments with transfer learning for newar and further modifications and corrections once word lists in standardised orthographies have been created.