Reassessing Speech Translation for Low-Resource Languages: Do LLM s Redefine the State-of-the-Art Against Cascaded Models? (2025)

Abstract: Automatic speech translation (AST) promotes seamless communication among speakers of different languages. While current state-of-the-art models excel with high-resource languages, their performance on low-resource languages (LRLs) is not well-established. We investigate this by evaluating state-of-the-art models on 10 LRLs with varying data amounts (10-30+ hours). Through six finetuning strategies and experimenting with three main AST paradigms, we observe that: (1) The latest Large Language Models (LLMs) may struggle with LRLs. (2) Comprehensive experiments suggest that for LRLs, more AST finetuning data is not always beneficial. (3) Our 2-Stage with ASR corrector finetuning recipe can substantially improve AST performance on LRLs, achieving up to a 5.8x BLEU score boost on translating related languages to English, while on par with the best monolingual finetuning in BLEU score when translating the target language to English. (4) We share effective engineering practices, including how to effectively adapt AST models to unseen languages.

## introduction automatic speech translation directly converts speech from a source language into text or speech in a target language. the field has recently advanced at a rapid pace, driven by new paradigms like large-scale pre-training @xcite @xcite , large speech models, e.g. seamlessm4t @xcite ; large language models (llms), e.g. @xcite ; and speech-native audio llms, e.g. gpt-4o audio (openai, 2024), gemini 2.0 flash (google, 2025), etc. despite these progresses, many ast research centered on high-resource languages like english, french, german @xcite . therefore, a most recent investigation of the novel modeling paradigms for the low-resource languages (lrls) for ast is needed. ast for lrls is constrained by scarce training data. recent multilingual speech corpora like @xcite , @xcite , and fleurs @xcite enable novel ast paradigms for these languages. ast modeling paradigms fall into three categories: (1) cascaded approaches that apply automatic speech recognition (asr) followed by machine translation (mt), (2) multimodal mt approaches like @xcite that directly translate speech to text, and (3) multimodal large language models such as gemini 2.0 flash, which natively process text, speech and images, can perform direct speechto-text translation (s2tt). even for other llms which do not natively support audio input, mapping audio tokens to the token vocabulary @xcite can leverage mt capabilities, such as models like salmonn @xcite , qwen 2 audio @xcite and spire @xcite . we investigate which approach works best for lrls with small amounts of finetuning data. specifically, is the cascaded architecture superior with small data when compared to multimodal mt approaches like seamlessm4t and audio llms? we experiment with ten lrls from fleurs (five indic, five african) translating to english, choosing a translation direction @xmath0. we then compare performances across different ast paradigms against a high-resource language pair, i.e. @xmath1. for cascaded approach, we proposed various finetuning strategies for all the three main modeling paradigms of ast. through comprehensive experiments across 11 languages, we show that the best ast approach depends on the resource-level of the languages: for languages with slightly better data availability, such as all five indic languages and swahili, prompting gemini-2.0 flash llm works best. while for extremely low-resource languages, such as hausa and yorùbá, finetuning from large mt models or sequentially finetuning asr and mt models can be more effective. to summarize, our contributions include: we ensure full reproducibility by using only publicly available data and apis, and open-sourcing our code and recipes. 1 the central challenge in ast is data scarcity @xcite of high-quality paired (source speech, target text) data. conventional ast thus uses cascaded approaches @xcite that first transcribe speech via asr, then translate using mt. when it comes to the lrls, the challenge of data scarcity is more severe. multiple efforts address this challenge. corpora like fleurs and common voice (ardila et al., 2019) enable 1 https://github.com/mcgill-nlp/ast-lrl-speech ast for lrls, while naijavoices @xcite and bhasaanuvaad @xcite contribute data for african and indian languages, respectively, although wide gaps persist compared to high-resource languages. recent speech foundation models like wav2vec2 @xcite and multimodal llms (google, 2025) have transformed ast: @xcite and @xcite demonstrated the benefit of pre-training ast models on high-resource asr data to improve performance for low-resource language pairs. popular parameter-efficient finetuning methods such as lora @xcite , lightweight adapter @xcite , always require changing the model architecture. in contrast to these studies, our research concentrates on the curriculum design of finetuning, to uncover hidden factors within simple full finetuning methods. @xcite concluded that despite the rise of llms, ast still requires significant improvement, particularly in low-resource scenarios. multimodal benchmarks like superb @xcite cover many speech tasks but exclude ast, while msteb @xcite analyzes ast only at the language-family level. owls @xcite demonstrates scaling benefits for low-resource performance, which our whisper findings echo. we focus on broadly effective finetuning recipes and provide detailed analysis for low-resource african and indic languages, underexplored in prior surveys. multilingual finetuning on models like whisper (asr) and @xcite (ast) often degrades non-target languages, especially with monolingual finetuning. we propose an effective 2-stage finetuning curriculum that reduces this shift without architectural changes, much simpler than multi-stage methods proposed in @xcite . we also apply llm correction to asr components, previously used mainly in asr systems @xcite . trade-offs between cascaded and end-to-end systems remain debated, with methods lacking systematic evaluation across diverse lrls. our work aims to complement these efforts by providing a unified, cross-paradigm evaluation across lrls, comparing data efficiency and generalization across cascaded, multimodal mt, and audio-llm systems. ## conclusions our systematic comparison of cascaded asr+mt, multimodal speech translation, and audio-centric llms across 11 diverse languages yields several important insights: (1) our 2-stage ft strategy can improve translation performances on target language, and offer the additional performance benefit on regional related languages for both asr and mt, with a up to 5.8x boost in bleu on them than monolingual ft. this approach is particularly effective for meeting the demands of practical, realworld scenarios. (2) our 2-stage ft + asr corrector recipe can further improve wer across 9 of 10 languages, and carry on the additional gains to ultimate mt task. (3) while sota audio-llms excel on higher-resource languages, our evaluations unveil that they may struggle on truly low-resource languages such as african ones. finetuned whisper variants and seamlessm4t can match or exceed audio-llm performance by up to 10 bleu, suggesting the most reliable choices for ast of under-represented spoken languages. (4) our ablation study reveals that not always "the more finetuning data, the better" in low-resource asr. future work should focus on expanding high-quality parallel speech-text resources and developing regularized, domain-aware adaptation techniques to ensure robust translation across the full spectrum of the world's languages.