Empowering Low-Resource Language Translation: Methodologies for B hojpuri- H indi and M arathi- H indi ASR and MT (2024)

Abstract: The paper describes our submission for the unconstrained track of ‘Dialectal and Low-Resource Task’ proposed in IWSLT-2024. We designed cascaded Speech Translation systems for the language pairs Marathi-Hindi and Bhojpuri-Hindi utilising and fine-tuning different pre-trained models for carrying out Automatic Speech Recognition (ASR) and Machine Translation (MT).

## introduction india boasts a vast linguistic variety, with more than 100 official languages and numerous dialects spoken all throughout the nation. natural language generation (nlg) tasks such as automated speech recognition (asr) and machine translation (mt), are greatly hampered by the tremendous variety. for millions of indians who do not speak english or other commonly spoken languages, asr and mt can be crucial in bridging the language gap and granting them access to information and services in a multi-linguistic country like india. however, the creation of asr and mt systems is a challenging endeavour due to the inherent features of indian languages, such as rich morphology, the occurrence of code-switching, and borrowing from other languages. the 'dialectal and low-resource track'proposed by iwslt 2024 requires the participants devise creative approaches to leverage the disparate resources available for 8 dialectal and low-resource languages. the participants are required to sub-mit under two conditions -namely constrained and unconstrained. the constrained condition should contain systems that are trained only on the datasets provided by the organizers while the unconstrained condition can contain systems trained with any resource including pre-trained and multilingual models. our team participated in the unconstrained condition for the language pairs -marathi to hindi and bhojpuri to hindi. this paper will discuss the implementation details of our asr and mt systems for the above-mentioned language pairs. 2 related work automatic speech recognition(asr) and machine translation(mt) in low-resource languages have been the subject of extensive research in recent years. several approaches have been proposed to address the challenges associated with lowresource languages in asr and mt. for instance, multilingual training has been identified as an effective approach for compensating for the limited amount of data in low-resourced asr @xcite . additionally, transfer learning methods have been used to develop end-to-end asr systems for low-resource languages, demonstrating their influence in addressing the challenges of low data levels @xcite . furthermore, the use of self-supervised speech recognition models has been hindered by the requirement for considerable labeled training data, which poses a challenge for their application to low-resource languages @xcite . in the context of mt, the scarcity of parallel data for low-resource languages has been identified as a significant challenge @xcite . neural machine translation (nmt) systems, which require large amounts of training data, face difficulties in creating high-quality systems for low-resource languages @xcite . however, research efforts have been directed towards improv-ing low-resource nmt, with studies exploring techniques such as teacher-free knowledge distillation to enhance performance in low-resource languages @xcite . the encoder-decoder framework for nmt has also been found to be less effective for low-resource languages, highlighting the need for specialized approaches to address the challenges of low-resource machine translation @xcite . the use of transfer learning has shown effectiveness in addressing the challenges of low-resource nmt, particularly in scenarios where parallel data is limited @xcite . additionally, the development of multilingual nmt systems has contributed to improving the quality of translation, especially for low-resource language pairs, enabling zero-shot translation and allowing the translation of language pairs never seen in training @xcite . automatic speech recognition (asr) and machine translation (mt) for indian languages have gained significant attention in recent years. the development of asr systems for indian languages has been a focus of research, with studies addressing low-resource challenges @xcite , multilingual and code-switching asr systems (diwan, 2021), and the impact of multilingual representations on asr and keyword search @xcite . research has also been conducted on asr for specific indian languages such as hindi, marathi, bengali, and oriya @xcite . furthermore, the potential of asr to aid individuals with speech disabilities, such as dysarthria, has been explored @xcite . in the realm of mt, efforts have been made to improve the quality of translations for indian languages through techniques such as transliteration and partof-speech tagging ( @xcite . moreover, the development of mt systems for indian languages and their approaches have been a subject of interest @xcite . research has also delved into rule-based machine translation and inflection rules for specific indian languages like marathi @xcite . the significance of asr and mt for indian languages is underscored by the need to break language barriers and facilitate inter-lingual communication @xcite . furthermore, the development of asr and mt systems for indian languages is crucial for addressing the diverse linguistic landscape of the country and enabling access to information and services for non-english speakers. in conclusion, the research on asr and mt for indian languages has made substantial progress, addressing challenges related to low-resource settings, multilingualism, and specific language requirements. these advancements are pivotal in enabling effective communication, accessibility, and inclusivity for indian language speakers. ## datasets we utilized the datasets provided by the track organizers as indicated below: ## conclusion and future work in this paper, we have presented our speech translation systems for the dialectal and low-resource track of iwslt 2024 employing a cascaded approach using fine-tuned models for both asr and mt. our submission trailed by a chrf2 score of 20 in comparison to the best submission in marathi-hindi task(unconstrained). in bhojpuri-hindi task(unconstrained) our submission trailed the best submission by a chrf2 score of 8.4 . our future work will comprise of using dataaugmentation techniques and fine-tuning multiple pre-trained multilingual models and exploring more speech translation models for low-resource indian languages.