GMU Systems for the IWSLT 2025 Low-Resource Speech Translation Shared Task (2025)

Abstract: This paper describes the GMU systems for the IWSLT 2025 low-resource speech translation shared task. We trained systems for all language pairs, except for Levantine Arabic. We fine-tuned SeamlessM4T-v2 for automatic speech recognition (ASR), machine translation (MT), and end-to-end speech translation (E2E ST). The ASR and MT models are also used to form cascaded ST systems. Additionally, we explored various training paradigms for E2E ST fine-tuning, including direct E2E fine-tuning, multi-task training, and parameter initialization using components from fine-tuned ASR and/or MT models. Our results show that (1) direct E2E fine-tuning yields strong results; (2) initializing with a fine-tuned ASR encoder improves ST performance on languages SeamlessM4T-v2 has not been trained on; (3) multi-task training can be slightly helpful.

## introduction speech translation (st) is a task that aims to translate speech in one language into text in another language. it can be addressed by either an endto-end (e2e) st model or a cascaded system that combines an automatic speech recognition (asr) model and a machine translation (mt) model. recent advances in e2e st have been driven by the development of large multilingual models trained on large amounts of multilingual datasets (seamless @xcite . similar trends can be observed in asr @xcite and @xcite as well. despite these models have covered a wide range of languages, many low-resource languages remain underrepresented and are not yet well supported by existing models. the iwslt low-resource speech translation shared tasks @xcite ; ahmad 1 we release our code for reproducibility: https:// github.com/mct10/iwslt2025_lowres_st . speech encoder text decoder e2e st fine-tuning cascaded st fine-tuning asr mt hello! hello! hello! muli shani! speech encoder text decoder text encoder text decoder speech encoder text decoder speech encoder text encoder text decoder muli shani! muli shani! e2e st fine-tuning with in-domain pretraining initialize multi-task training initialize initialize hello! optionally initialize optionally initialize figure 1 : illustration of our seamlessm4t-v2 finetuning strategies. speech encoder, text encoder, and text decoder refer to the corresponding components of @xcite @xcite are designed to advance st technology for low-resource languages. to address the challenge of data scarcity, previous submissions have explored various pre-trained models, including multilingual self-supervised speech models such as xlsr @xcite , multilingual asr models such as whisper @xcite , multilingual mt models such as nllb @xcite , and multilingual st models such as @xcite . these pre-trained models were then fine-tuned on st datasets for low-resource languages. among them, seamlessm4t-v2 has demonstrated superior performance, according to last year's evaluations @xcite . this paper describes gmu submissions to the iwslt 2025 low-resource speech translation task @xcite . our work focuses on fine-tuning the seamlessm4t-v2 model (seamless @xcite for all language pairs except levantine arabic-to-english. we finetuned the model for both e2e and cascaded systems. for e2e st fine-tuning, we explored multiple strategies, including multi-task training with mt and knowledge distillation objectives, as well as initializing model components with those from fine-tuned asr and/or mt models, trying to utilize all available datasets. figure 1 illustrates our strategies. our results show that direct e2e fine-tuning seamlessm4t-v2 yields strong performance across all languages pairs, except quechua, which has too little training data. for languages not seen during seamlessm4t-v2 pre-training, we show that finetuning the model on asr data and initializing the st encoder with the asr encoder improves performance significantly. we also show that multi-task training offers some performance gains when the mt model significantly outperforms the e2e st model. ## task descriptions the iwslt 2025 low-resource st task (abdulmumin et al., 2025) covers 10 language pairs: (north) levantine dialectal arabic to english (apc-eng), tunisian arabic dialect to english (aeb-eng), bemba to english (bem-eng), fongbe to french (fonfra), irish to english (gle-eng), bhojpuri to hindi (bho-hin), estonian to english (est-eng), maltese to english (mlt-eng), marathi to hindi (mar-hin), and quechua to spanish (que-spa). in each of these language pairs, the source language is lowresource while the target language is high-resource. we trained systems for all language pairs except for apc-eng. 2 formally, e2e st is defined as translating a speech utterance x sp in the source language into text y in the target language. for cascaded st, a source speech utterance x sp is first transcribed into text x text in the source language using an asr model, which is then translated into the targetlanguage text y using an mt model. the datasets we used are summarized in a 2-way st data sample is represented as a tuple (x sp , y), while a 3-way st data sample refers to a triple (x sp , x text , y). 3-way st datasets are available for aeb-eng, bem-eng, est-eng, mlt-eng, and que-spa. the other languages are provided with 2-way st datasets. among these, est-eng has the largest dataset with more than 1,000 hours of speech. both aeb-eng and bem-eng have more than 100 hours of data, while datasets for other languages are limited and having only about 10 hours of speech. in addition, the organizers provide pointers to additional asr and mt datasets. an asr data sample is represented as (x sp , x text ), while an mt data sample is represented as (x text , y). it is evident that both asr and mt datasets can be derived from 3-way st datasets. the task allows submissions under two conditions: constrained and unconstrained. under the constrained condition, only the provided dataset can be used and no pre-trained models are allowed. the unconstrained condition allows the use of any models and any datasets. all of our submissions fall under the unconstrained condition. ## methods our methods focus on fine-tuning the seamlessm4t-v2 model (seamless @xcite . we explore 4 different fine-tuning strategies: (1) e2e st fine-tuning; although the mt components of seamlessm4t-v2 are initialized by the nllb model @xcite , seamlessm4t-v2 has been trained on less languages and supports mt for only 4 out of the 10 language pairs in this shared task. in contrast, the nllb model supports mt for all 10 pairs. to evaluate whether the smaller language coverage of seamlessm4t-v2 impacts performance, we additionally fine-tuned an nllb model on the mt datasets, using it as the mt baseline. section 3.1 introduces the nllb and seamlessm4t-v2 models. section 3.2 through section 3.5 elaborate our fine-tuning strategies. 3.1 base models nllb @xcite . nllb is a multilingual mt model supporting over 200 languages, including all language pairs in this shared task. the model is available in two architecture variants: a sparsely gated mixture-of-experts (moe) one and a set of dense transformer models. the dense transformer architecture comprises a text encoder and a text decoder. while the moe variant (nllb-200) achieves the strongest performance, it has 54.5b parameters and is not practical for fine-tuning. therefore, in our experiments, we choose the 1.3b dense transformer model distilled from nllb-200, referred to as nllb-200-distilled-1.3b. @xcite . seamlessm4t-v2 is the state-ofthe-art foundation model for st. while it supports into-speech translation, we only focus on its intotext translation capabilities for the purpose of this shared task. seamlessm4t-v2 is composed of a speech encoder, a text encoder, and a shared text decoder. its large variant has 2b parameters in total and we refer to it as seamlessm4t-v2-large. the speech encoder is pre-trained on 4.5m hours of unlabeled audio with the w2v-bert 2.0 objective. the text encoder and decoder are initialized by the nllb model. during fine-tuning, a multitask training strategy is employed, incorporating asr, mt, st, and knowledge distillation (kd) objectives. we also explore this strategy in our ex-periments. the model supports 101 languages for speech input and 96 languages for text input and output. among the low-resource languages in this shared task, seamlessm4t-v2 supports est, gle, mar, and mlt, but does not support aeb, bem, bho, fon, and que. ## experiments we describe the additional datasets we used in section 4.1. in section 4.2, we describe the fine-tuning hyperparameters. the evaluation metrics are described in section 4.3. ## results and analysis we first present the asr and mt performance in section 5.1 and 5.2, respectively. then, we summarize the st performance in section 5.3. the ablation study of using additional datasets is presented in appendix b. ## conclusion and future work in this paper, we describe gmu systems for the iwslt 2025 low-resource st shared task. we focus on fine-tuning the seamlessm4t-v2-large model and explore four fine-tuning strategies. we find that e2e st fine-tuning performs best on languages with asr support. for languages without asr support, we can fine-tune the model on in-domain asr datasets first and then initialize the st encoder with the asr encoder, which significantly improves performance. multi-task training and cascaded systems are not as good as e2e finetuning in general. we hypothesize that it is because seamlessm4t-v2-large is strong enough on st, and the fine-tuned mt performance is not strong enough to provide useful additional performance gains. for future work, we could explore better pretraining methods to mitigate the gap between the speech encoder and the text decoder @xcite . we could also explore the use of speech large language models, as large language models have recently achieved success in mt tasks @xcite . decoder input embedding, and a text decoder output embedding (also termed lm_head). these three embeddings are intended to share the same weight matrix. however, in the official codebase, the lm_head is accidentally untied from the other two embeddings during model initialization, resulting in additional 262m trainable parameters. in contrast, the huggingface codebase still ties all three embeddings.