UM IWSLT 2024 Low-Resource Speech Translation: Combining M altese and N orth L evantine A rabic (2024)

Abstract: The IWSLT low-resource track encourages innovation in the field of speech translation, particularly in data-scarce conditions. This paper details our submission for the IWSLT 2024 low-resource track shared task for Maltese-English and North Levantine Arabic-English spoken language translation using an unconstrained pipeline approach. Using language models, we improve ASR performance by correcting the produced output. We present a 2 step approach for MT using data from external sources showing improvements over baseline systems. We also explore transliteration as a means to further augment MT data and exploit the cross-lingual similarities between Maltese and Arabic.

## introduction there are a variety of challenges inherent in spoken language translation for low-resource languages. by definition, these languages have very limited data available to use for natural language processing (nlp) tasks. the majority of current work on nlp targets just 20 out of the approximately 7,000 languages used worldwide, leading to a significant gap in research and negative impacts on excluded speech communities @xcite . while most machine learning tasks are performed using vast amounts of data, models for low-resource languages must be adapted to work with less data or other strategies must be employed to augment the existing data. in this paper, we present our submission for the 2024 iwslt low-resource shared task. concretely, we submit two systems for speech translation to english, from maltese and north levantine arabic. the main motivation for focusing on these two languages is their similarity to one another which we aim to exploit to improve the performance of our pipeline speech translation system. as a well-known case of diglossia, the arabic language has a notable distinction between the formal variety used in written communication, political speech, and the educational system -known as modern standard arabic (msa) -and the informal varieties primarily used in spoken communication -collectively referred to as dialectal arabic (da). these dialects exhibit considerable diversity influenced by geographical and socio-economic factors, diverging significantly from msa in phonology, morphology, lexicon, and syntax @xcite . on the other hand, maltese is a semitic language derived from siculo-arabic @xcite , with a notable mutual intelligibility with tunisian da @xcite . its evolution independently from the arab world -particularly its substantial influence from italian and english and its use of a modified latin alphabet -makes it a distinct language and not an arabic dialect. we split the task of spoken language translation into two sequential tasks consisting of automatic speech recognition (asr) and machine translation (mt). this process transforms speech in a lowresource language into text in a high-resource language, namely english for this shared task. at the same time, splitting the task into asr and mt allows us to exploit existing multilingual models and source larger corpora for each sub-task to improve their performance in the target language. all of our code is made publicly available. 1 2 related work in order to examine past approaches to lowresource spoken language translation, this literature review includes an overview of previous iwslt low-resource track submissions and our asr and mt systems as well as our innovative approach to data augmentation through transliteration. ## speech translation pipeline following our evaluation on individual tasks in sections 3 and 4, we now combine both systems by first getting the transcription using an asr system and then passing this transcription through the mt system to get the translation. the best-performing asr and mt systems on our validation sets were selected for the pipelines. for maltese, we only choose the mt system trained with the 2 stage training, opus-100+cv+masri (which we refer to as mlt lat ) and combine it with the asr systems with the 3-gram, 4-gram, and 6-gram models, to compose our primary, contrastive 1, and contrastive 2 systems, respectively. for north levantine arabic, we use the only trained model for asr, paired with all 3 mt systems which made use of 2 stage training, namely msa+apc+aeb, msa+apc+aeb+mlt ara , and msa+apc, to compose our primary, contrastive 1, and contrastive 2 systems, respectively. the results on the north levantine arabic data are shown in ## conclusion overall, this paper presented our findings for maltese and north levantine arabic spoken language translation into english with a pipeline system in the unconstrained setting for the 2024 iwslt lowresource track shared task. for our approach we fine-tune a wav2vec 2.0 xls-r model for asr, and an nllb model for mt. we enhance the asr model by correcting the outputs with a language model. moreover, we augment the mt data from additional sources and employ a two-stage finetuning process to improve performance. additionally, we exploit the cross-lingual similarities between maltese and arabic by transliterating maltese to arabic script, observing interesting performance boosts. in terms of limitations, the lack of training data for north levantine arabic impeded the progress of our asr system. by using msa to train our arabic asr models, the resulting system struggled with non-standard pronunciation and dialectspecific variation. furthermore, the absence of testing data for tunisian arabic hindered our models considering its close similarity with maltese. more general improvements could be undertaken in future work such as hyper-parameter tuning and supplementing currently available data with backtranslation. rather than relying solely on parallel data, implementing backtranslation with larger monolingual corpora holds promise for improving the mt systems discussed in this paper.