Data Augmentation for M altese NLP using Transliterated and Machine Translated A rabic Data (2025)

Abstract: Maltese is a unique Semitic language that has evolved under extensive influence from Romance and Germanic languages, particularly Italian and English. Despite its Semitic roots, its orthography is based on the Latin script, creating a gap between it and its closest linguistic relatives in Arabic. In this paper, we explore whether Arabic-language resources can support Maltese natural language processing (NLP) through cross-lingual augmentation techniques. We investigate multiple strategies for aligning Arabic textual data with Maltese, including various transliteration schemes and machine translation (MT) approaches. As part of this, we also introduce novel transliteration systems that better represent Maltese orthography. We evaluate the impact of these augmentations on monolingual and mutlilingual models and demonstrate that Arabic-based augmentation can significantly benefit Maltese NLP tasks.

## introduction maltese is the only semitic language written in the latin script and the only one that is an official language of the european union. it retains a core semitic structure but has undergone extensive lexical borrowing and structural influence from italian and english. despite its roots in north african arabic, modern maltese and arabic are now orthographically and lexically distant, posing unique challenges for leveraging arabic nlp resources to support maltese. given the low-resource status of maltese, recent work has focused on leveraging multilingual language models and transfer learning @xcite . in this paper, we pursue a complementary strategy: enriching maltese datasets through augmentation derived from arabic resources. while doing so, we address the divergence in script and phonology between arabic and maltese, by considering multiple layers arabic @xmath0. uroman awqft alsyara fy altryq. chartx uqft alsjara fi altriq. morphtx awqefat is-sejjara fit-teriq. ## our transliteration systems in this section, we outline the two arabic-to-maltese transliteration systems we developed, with all mapping rules listed in appendix a. ## experimental setup our goal is to improve cross-lingual transfer to maltese by transliterating arabic and using it for data augmentation. our few-shot setup involves fine-tuning on arabic data, followed by maltese. section 4.1 outlines the datasets used, and section 4.2 details the models and input processing. appendix c includes more technical details on our fine-tuning setup. ## transliteration fine-tuning results we first compare the different transliteration systems extrinsically with models fine-tuned on original arabic as well as fine-tuning only on maltese data. results are shown in ## cascaded arabic training in section 5, our results show that multilingual models do not benefit from cross-lingual transfer capabilities as much when fine-tuning with transliterated arabic, instead of original arabic. we hypothesise that this is due the model's pre-training on arabic in arabic script rather than latin. to test this, we conduct further experiments where we cascade multiple stages of fine-tuning on arabic, with increasing similarity to maltese, before the final stage of maltese fine-tuning. therefore, we first start with a fine-tuning step on original arabic data, followed by transliterated arabic data, and lastly a final phase of maltese fine-tuning. to simplify the setup, we choose one transliteration system based on observed performance trends from section 5 -morphtx for ner and chartx for sa. furthermore, we consider another variant where we also fine-tune on machine translated arabic after fine-tuning on original and transliterated arabic. we compare these cascaded approaches against fine-tuning with only one stage of arabic finetuning -original, transliterated, and machine translated arabic. the results are shown in with bertu, cascaded arabic fine-tuning does not help over a single phase of arabic fine-tuning, as the best result is obtained with machine translation for ner and transliteration for sa. this is likely due bertu's maltese pre-training with little to no arabic data. this is supported by the relatively lower scores observed for bertu when fine-tuned with original arabic. in contrast, both mbert and mbertu achieve the best performance when fine-tuned on arabic data in a cascaded approach. for mbertu, the best result is obtained with original and transliteration cascading, whereas the full cascade yields the best result for mbert. interestingly, when mbert and mbertu are fine-tuned solely with either transliterated arabic or translated arabic, worse results are generally obtained compared to original arabic fine-tuning, but by combining all approaches in one pipeline, we are better able to unlock the models' cross-lingual transfer capabilities. ## conclusion and future work in this work, we presented transliteration systems from arabic to maltese. our experimental results highlight that the effectiveness of transliteration depends on the model's exposure to the target language. we find that a monolingual maltese model benefits from transliterated arabic data, while multilingual models are able to make cross-lingual transfer links without transliteration. however, cascading the original and transliterated data during the fine-tuning process proves to be beneficial over fine-tuning with only one of these. future work includes exploring unsupervised or self-supervised methods to better align arabic and maltese representations without heavy reliance on parallel data or linguistic analysis. we also plan to investigate advanced machine translation and neural transliteration models that capture deeper morphological and phonological patterns. additionally, we plan on applying these augmentation techniques to other facets of maltese nlp such as language modelling.