JHU IWSLT 2025 Low-resource System Description (2025)

Abstract: We present the Johns Hopkins University’s submission to the 2025 IWSLT Low-Resource Task. We competed on all 10 language pairs. Our approach centers around ensembling methods – specifically Minimum Bayes Risk Decoding. We find that such ensembling often improves performance only slightly over the best performing stand-alone model, and that in some cases it can even hurt performance slightly.

## introduction and background despite many recent advances in deep learning and artificial intelligence, challenges in low-resource and dialectal speech translation still preclude highquality automated translation systems for many language communities. cross-lingual transfer and multilingual models have allowed for recent progress in scarce data settings, but performance still lags significantly behind that of higher resource languages @xcite . due to a lack of training resources, low-resource languages systems tend to generate hypotheses with higher variance than is seen in higherresourced conditions. in other words, different models might generate diverse outputs; hence a single system might not be optimal in all scenarios. this motivates attempting to select the best option from multiple potential systems-i.e., ensembling. for this year's iwslt low-resource speech translation campaign @xcite , we, the johns hopkins university (jhu) team decided to focus on minimum bayes risk decoding (mbr) with the interest in exploring in-depth how combining methods across a range of language pairs can improve performance in a low-resource setting @xcite . following our approach from last year (robinson et al., 2024), we submitted systems for all language pairs, with a focus on seeing how robust our methods are across a wide range of data settings and typologically diverse languages. rather than focusing on a specific language, our line of inquiry was geared towards broader exploration, with the interest of discovering language-agnostic trends in mind. our approach in 2024 focused on fine-tuning pretrained models whisper @xcite ), nllb @xcite , and seam-lessm4t v2 @xcite for both asr+mt cascading and end-to-end speech translation. we also incorporated joint training for language pairs with common targets, as well as finetuning with a regularization technique known as intra-distillation @xcite @xcite . in this year's submission, we similarly gather a variety of different systems for each translation language pair. we use these to obtain a diverse set of outputs for the language pairs sampled from various checkpoints of the different cascaded and end-to-end fine-tuned systems. to maximize variety of systems, and following our submission from last year, we experimented with combining fine-tuning data for language pairs with a common target language, and with use of additional or supplementary training data. rather than comparing these diverse systems directly, however, we ensembled them into a single inference method. we used mbr to select the best performing candidate translation from the resulting pool for each source audio. our decision to attempt an ensemble approach was inspired by other team submissions from last year. our submissions performed best in last year's evaluation campaign @xcite for irish-to-english (gle-eng), bemba-to-english (bem-eng), and bhojpuri-to-hindi (bho-hin) translation. however the teams that performed best for levantine arabic-to-english (apc-eng), maltese-to-english (mlt-eng), and quechua-to-spanish (que-spa) all employed ensemble models. ben kheder et al. (2024) ensembled 26 model checkpoints for their apc-eng system, and e. ortega et al. (2024) ensembled 10 checkpoints for their que-spa system. for their mlt-eng system, @xcite ensembled cascade systems with end-to-end models (just as we attempt to do in this work). ## methodology and experiments our methodology is illustrated in figure 1 . given any language pair, we have a number of cascade and end-to-end systems (three in the figure). these systems employ either whisper st, whisper asr with nllb mt, or seamlessm4t v2 st, and may have other more minor variations differentiating them. the number of systems varies between one and four, depending on the language pair. the systems we use for each language pair are listed in we keep three fine-tuning checkpoints from the final model of each of the systems (nllb in the case of the bi-model cascade) and for each input audio, we sample five hypothesis translations from each system checkpoint, resulting in a total of 15 hypotheses per system for each input. in the case of language pairs like apc-eng, for which we ensembled four different systems, this amounted to 15 * 4 = 60 hypotheses for each input audio, which are reduced to a single hypothesis via the mbr process. ## results and conclusions the results of our different language systems are in in we also note that while there are significant score differences between different systems (such as seamlessm4t v2 vs. whisper or cascaded vs. end-to-end), training with combined language data or supplementary data (i.e. shrutilipi) also did not cause any drastic score increases. given the scantness of these results, we conclude that methods such as ensembling and multilingual training either have limited use in some low-resource speech translation settings, or that they require more creative and effective applications than those we explored here in order to be optimally useful. we encourage researchers to explore such creative applications of these techniques, as well as other techniques to improve low-resource systems, in the future.