ALADAN at IWSLT 24 Low-resource A rabic Dialectal Speech Translation Task (2024)

Abstract: This paper presents ALADAN’s approach to the IWSLT 2024 Dialectal and Low-resource shared task, focusing on Levantine Arabic (apc) and Tunisian Arabic (aeb) to English speech translation (ST). Addressing challenges such as the lack of standardized orthography and limited training data, we propose a solution for data normalization in Dialectal Arabic, employing a modified Levenshtein distance and Word2vec models to find orthographic variants of the same word. Our system consists of a cascade ST system integrating two ASR systems (TDNN-F and Zipformer) and two NMT modules derived from pre-trained models (NLLB-200 1.3B distilled model and CohereAI’s Command-R). Additionally, we explore the integration of unsupervised textual and audio data, highlighting the importance of multi-dialectal datasets for both ASR and NMT tasks. Our system achieves BLEU score of 31.5 for Levantine Arabic on the official validation set.

## introduction speech translation (st) systems play a crucial role in facilitating communication across languages and dialects, enabling access to information and services for diverse linguistic communities. how ever, developing accurate st systems for dialec tal arabic poses significant challenges due to the scarcity of annotated data and the lack of standard ized orthography. in particular, dialectal variants such as levantine arabic (apc) and tunisian ara bic (aeb) are severely underresourced in terms of automatic speech recognition (asr) and neural machine translation (nmt) datasets. these limitations present a major bottleneck in the development of highquality st systems and many works in previous iwslt evaluations @xcite explored various transfer techniques on the acoustic level by finetuning pretrained speech en coders such as the wav2vec 2.0 @xcite and hubert @xcite for asr, or neural models such as nllb200 @xcite and mbart @xcite for nmt. the use of modern standard arabic (msa) datasets like mgb2 for dialect transfer @xcite has also been proven effective. in recent years, more sophisticated asr archi tectures such as the zipformer @xcite emerged as a more effective alternative to other transformerbased architectures like conformers @xcite and branchformer @xcite . in nlp, large language models (llms) @xcite have demonstrated strong performance across var ious tasks in mainstream languages, yet a notable constraint persists in their limited support for low resource languages and dialects. building upon these novelties, we propose an approach that leverages pretrained models and multidialectal resources for dialectal arabic st. we adopt a cascade st system comprising two asr systems (tdnnf and zipformer) and two nmt modules derived from pretrained models (nllb200 and commandr). additionally, we develop a generic text normalization methodology for dialectal arabic and integrate crowdsourced nmt data and multidialectal datasets like padic @xcite to supplement the limited training data. the outcomes for levantine arabic (apc) are reported on the iwslt2024 valid and test2024 sets, while the results for tunisian arabic (aeb) are provided for both validation and test set test1 and dev published in iwslt2022. ## experiments this section describes our experimental settings, used data and results.