IIITH - BUT system for IWSLT 2025 low-resource B hojpuri to H indi speech translation (2025)

Abstract: This paper presents the submission of IIITH-BUT to the IWSLT 2025 shared task on speech translation for the low-resource Bhojpuri-Hindi language pair. We explored the impact of hyperparameter optimisation and data augmentation techniques on the performance of the SeamlessM4T model fine-tuned for this specific task. We systematically investigated a range of hyperparameters including learning rate schedules, number of update steps, warm-up steps, label smoothing, and batch sizes; and report their effect on translation quality. To address data scarcity, we applied speed perturbation and SpecAugment and studied their effect on translation quality. We also examined the use of cross-lingual signal through joint training with Marathi and Bhojpuri speech data. Our experiments reveal that careful selection of hyperparameters and the application of simple yet effective augmentation techniques significantly improve performance in low-resource settings. We also analysed the translation hypotheses to understand various kinds of errors that impacted the translation quality in terms of BLEU

## introduction speech translation (st) transforms spoken language into written text in a different language, serving as a critical component in breaking down communication barriers. while significant advancements have been made for high-resource language pairs @xcite , developing effective st systems for low-resource and dialectal languages remains challenging due to scarce parallel data, inconsistent orthography, and substantial linguistic variation. we address the low-resource scenario of bhojpuri speech to hindi text translation within the indian linguistic context. despite being spoken by over 50 million people, bhojpuri suffers from limited quality and diversity in available speech-text corpora (wikipedia contributors, 2025). in contrast, hindi possesses relatively abundant resources, making it an ideal target language for translation. our system, developed for the iwslt 2025 shared task, addresses these resource disparities through a combination of transfer learning from linguistically similar languages, data augmentation techniques, and hyperparameter optimization. recent end-to-end st models, such as the speech-to-text transformer @xcite and seamlessm4t @xcite effectively replace traditional cascade pipelines by jointly modelling asr and mt, reducing error propagation and latency. however, these approaches typically require substantial labelled data unavailable for most low-resource languages, often leading to overfitting and poor generalisation. our contributions include: the remaining of this paper is organized as follows: section 2 provides a comprehensive review of related work, section 3 describes our system including hyperparameter optimization, data augmentation techniques, and joint fine-tuning approach, section 4 details our experimental setup, section 5 presents results and analysis, and section 6 concludes with future directions. ## related work low-resource speech translation (st) has garnered significant attention through iwslt shared tasks @xcite @xcite . the field has evolved from traditional pipeline approaches @xcite to end-to-end architectures such as listen attend and spell (las) @xcite , fairseq s2t @xcite , @xcite , and transfer learning based methods @xcite . in @xcite , explored fine-tuning selfsupervised models by incorporating a linear layer for the st task, which streamlined workflows while maintaining specialized strategies for low-resource scenarios. @xcite , implemented various data augmentation techniques including audio stretching, back-translation, and paraphrasing. contemporary approaches to low-resource st can be categorized into several methodological frameworks. the setu-dcu submission @xcite enhanced st robustness through ctc loss integration and rigorous data cleaning protocols. @xcite incorporated pseudolabelling techniques to expand their training corpus. the jhu iwslt 2024 system @xcite demonstrated the efficacy of whisperstyle large models with domain-adaptive pretraining methodologies. meanwhile, the quespa team @xcite implemented ensemble decoding with cross-lingual knowledge transfer mechanisms. seamlessm4t @xcite presents a comprehensive approach handling asr, mt, and st across more than 100 languages, though its performance on genuinely low-resource languages necessitates substantial adaptation strategies. our approach focuses on systematically exploring hyperparameter optimization and data augmentation techniques for low-resource speech translation. unlike previous work that often applies general strategies such as default hyperparameter val-ues, generic data augmentation, standard transfer learning without language-specific considerations and using default model architectures and training strategies without adaptation to low-resource constraints, we conduct a comprehensive investigation specifically tailored to the challenges of the bhojpuri-hindi language pair. our experimentation includes a detailed analysis of learning rates, batch sizes, label smoothing values, and warmup periods, and used two data augmentation techniques (speed perturb, specaug). additionally, we examine how cross-lingual transfer from marathi can supplement these optimizations, leveraging the linguistic proximity between these indo-aryan languages. ## results and analysis this section presents an analysis of the experimental results obtained from our primary and contrastive models. the primary model encompasses an optimized combination of hyperparameter configuration and data augmentation techniques that yielded the highest bleu score in our evaluations. as demonstrated in for our contrastive model, we implemented a multilingual fine-tuning strategy that jointly trained on marathi and bhojpuri data using the hyperparameter configuration that previously achieved the highest performance (36.4 bleu). this multilingual model initially underperformed compared to our monolingual system, potentially due to catastrophic forgetting. to mitigate this issue, we conducted additional fine-tuning on bhojpuri data exclusively for various epoch counts. as shown in ## conclusion our submission to the iwslt 2025 evaluation campaign for low-resource and dialectal speech translation advances bhojpuri-hindi st through a combination of hyperparameter optimisation, data augmentation, and cross-lingual joint fine-tuning. by leveraging the seamlessm4t medium model (1.2b parameters) and systematically exploring optimal training configurations, we demonstrate significant performance gains despite the challenges posed by limited parallel data. our results show that even established techniques like specaugment and speed perturbation, when carefully implemented, can lead to substantial improvements in low-resource speech translation tasks, expanding our effective training data threefold. additionally, we found that joint training with marathi-a linguistically related indo-aryan language-followed by sequential bhojpurispecific adaptation provides an effective strategy to mitigate data sparsity and improve generalisation across diverse speech patterns. for future work, we plan to explore more sophisticated augmentation techniques such as noise injection, pitch shifting, and cross-speaker synthe-sis. we also intend to investigate self-supervised pretraining on monolingual bhojpuri speech data and further extend our cross-lingual approach to additional indo-aryan languages. as low-resource st continues to evolve, we believe that modular, linguistically informed adaptation pipelines will play a key role in advancing the real-world applicability of such systems for under represented language communities.