Effectively combining Phi-4 and NLLB for Spoken Language Translation: SPRING Lab IITM ’s submission to Low Resource Multilingual I ndic Track (2025)

Abstract: This paper presents the methodologies implemented for Spoken Language Translation for the language pairs Hindi-English, Bengali-English and Tamil-English for the Low Resource Multilingual Indic Track of The International Conference on Spoken Language Translation (IWSLT) for 2025. We adopt a cascaded approach and use a fine-tuned Phi-4 multimodal instruct model for Automatic Speech Recognition(ASR) and a fine-tuned NLLB model for Machine Translation(MT).

## introduction india is home to an incredibly diverse linguistic landscape, with over 100 officially recognized languages and thousands of dialects spoken across its vast geographic and cultural expanse. this linguistic richness reflects the countrys deep-rooted cultural heritage and regional diversity, but it also presents considerable challenges for natural language processing (nlp) applications, particularly for tasks like automatic speech recognition (asr) and machine translation (mt). unlike monolingual or relatively linguistically homogeneous countries, india's multilingualism requires nlp systems to handle a wide variety of linguistic variations from phonetic and grammatical differences to script variations and regionspecific vocabulary. asr systems must account for varied accents, pronunciation patterns, and speech styles, while mt systems are required to preserve contextual accuracy and fluency across structurally and syntactically diverse language pairs. this diversity becomes even more complex due to situations like speakers alternate between lan-guages mid-sentence, highly inflected words and complex verb conjugations, and lexical words from one language are integrated into another. such features are common in indian speech and text, making the development of robust, generalizable asr and mt models particularly challenging. despite these obstacles, these technologies are crucial for millions of indians who do not speak english or other dominant languages. asr and mt systems can help bridge the communication barrier, enabling equal access to information, digital services, and opportunities across indias diverse population. the "low resource multilingual indic" track proposed by iwslt 2025 tasked participants with developing innovative methods to work with the sparse and varied resources available for three indian languages: hindi, bengali, and tamil. the participants were required to submit under various different conditions constrained or unconstrained, end-to-end or cascaded, and monolingual or multilingual. our team participated under the unconstrained, cascaded, and monolingual category for the language pairs hindi to english, bengali to english, and tamil to english. this paper outlines the implementation of our asr and mt systems designed for these language pairs. 2 asr 2.1 datasets ## final evaluation after fine-tuning and evaluating the asr and mt model separately, we conducted a final evaluation to test how the models were performing collectively with models namely: whisper-large-v2, data2vec-aqc foot_2 and nllb. we cascaded the finetuned phi-4 multimodal instruct (marah abdin, 2024) with nllb and our fine-tuned nllb model (3.3b), whisper-large-v2 @xcite with nllb (nllb team, 2022) and our finetuned nllb model (3.3b) and data2vec-aqc with nllb and our fine-tuned nllb model (3.3b), the metrics used for the evaluation were bleu and chrf++, the results for the bengali to english translation are listed in on final evaluation on the iwslt 2025 leaderboard, with chrf as the ranking metric, we find that we perform the best on indic to english translations, achieving first place for both hindi to english and bengali to english, and achieving second place for tamil to english. however, we rank last on english to indic evaluation. together, these limitations explain the performance gap between the two translation directions and emphasize the need for multilingual strategies and additional data in future work.