Enhance Contextual Learning in ASR for Endangered Low-resource Languages (2025)

Abstract: Automatic Speech Recognition (ASR) facilitates documenting endangered low-resource languages. While recent advances in acoustic modelling have been substantial, contextual learning remains underexplored. This study investigates the main factors that influence the integration of knowledge from language models (LMs) into state-of-the-art ASR models for endangered low-resource languages. Through experiments on five diverse low-resource languages, we find: 1) Fine-grained tokenization effectively improves ASR performance by addressing the prevalent unknown words and improving data usage efficiency; 2) The integration of transformer-based LMs into ASR systems surpasses that of N-gram LMs only in one language, even though they consistently achieve better results in language modelling tasks. 3) ASR performance is highly sensitive to language-specific optimization, as shown by a 43% performance degradation in one language due to parameter transfer across languages. We open-source our scripts to support further research and applications.

## introduction the threat of language endangerment continues to grow due to various external pressures, prompting linguists to actively document vulnerable languages. however, manual documentation processes are often impractical and time-intensive. automatic speech recognition (asr) models offer valuable support for language documentation, yet their effectiveness is hindered by the limited availability of supervised data. recent advancements indicate that multilingual self-supervised learning holds promise for developing asr systems tailored to endangered lowresource languages (mihajlik et al., 2023; li et al., 2024; taguchi et al., 2024; mainzinger and levow, 1 https://github.com/zl-ka/lm-lr-asr 2024; taguchi and chiang, 2024). among these approaches, fine-tuning the pre-trained wav2vec2 models @xcite with connectionist temporal classification (ctc) loss @xcite has emerged as a popular and effective strategy. compared to other pre-trained asr models, such as whisper @xcite , this approach often achieves superior performance, particularly in reducing character-level errors @xcite . this advantage can be attributed to its smaller parameter set and the extensive pre-training data, making it especially effective for low-resource settings. despite its strengths in acoustic modelling, this approach lacks contextual learning capabilities due to the conditional independence assumption inherent in ctc @xcite @xcite . to address this, previous research has integrated asr models with language models (lms) at the word level @xcite @xcite @xcite . however, word-level integration struggles with the high prevalence of unknown words in low-resource settings, where limited text data further impedes performance. additionally, prior studies have predominantly employed statistical n-gram lms for integration. however, transformer-based lms have demonstrated superior contextual learning capabilities compared to n-gram models for high-resource languages. while few studies have explored combining transformer-based lms with wav2vec2 and ctc fine-tuning @xcite , these investigations have focused on high-resource languages, leaving their potential for low-resource languages unexplored. differences in data availability and linguistic complexity underscore the need for further investigation. to fill these gaps, we explore lm integration for five low-resource languages from diverse language 1 families, considering differences in data size and source. our main contributions are: ## conclusion this study focuses on improving contextual learning in asr models for low-resource languages by examining tokenization granularity and the integration of transformer-based lms. the findings show that fine-grained tokenization enhances asr performance by addressing unknown words and increasing data usage efficiency. moreover, integrating transformer-based lms does not consistently outperform n-gram lms in boosting asr accuracy. finally, our results indicate that directly applying experimental settings to new languages harms performance, emphasizing the importance of language-specific optimizations. claytone sikasote, eunice mukonde, md mahfuz ibn alam, and antonios anastasopoulos. 2023. big-c: a multimodal multi-purpose dataset for bemba. in proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 2062-2078, toronto, canada. association for computational linguistics. chihiro taguchi and david chiang. 2024. language complexity and speech recognition accuracy: orthographic complexity hurts, phonological complexity doesn't. in proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: long papers), pages 15493-15503, bangkok, thailand. association for computational linguistics. chihiro taguchi, jefferson saransig, dayana velásquez, and david chiang. 2024. killkan: the automatic speech recognition dataset for kichwa with morphosyntactic information. in proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (lrec-coling 2024), pages 9753-9763, torino, italia. elra and iccl. silero team. 2024. silero vad: pre-trained enterprisegrade voice activity detector (vad), number detector and language classifier. https://github.com/ snakers4/silero-vad .