Exploring the Impact of Transliteration on NLP Performance: Treating M altese as an A rabic Dialect (2023)

Abstract: Multilingual models such as mBERT have been demonstrated to exhibit impressive crosslingual transfer for a number of languages. Despite this, the performance drops for lowerresourced languages, especially when they are not part of the pre-training setup and when there are script differences. In this work we consider Maltese, a low-resource language of Arabic and Romance origins written in Latin script. Specifically, we investigate the impact of transliterating Maltese into Arabic scipt on a number of downstream tasks: Part-of-Speech Tagging, Dependency Parsing, and Sentiment Analysis. We compare multiple transliteration pipelines ranging from deterministic character maps to more sophisticated alternatives, including manually annotated word mappings and non-deterministic character mappings. For the latter, we show that selection techniques using n-gram language models of Tunisian Arabic, the dialect with the highest degree of mutual intelligibility to Maltese, yield better results on downstream tasks. Moreover, our experiments highlight that the use of an Arabic pre-trained model paired with transliteration outperforms mBERT. Overall, our results show that transliterating Maltese can be considered an option to improve the cross-lingual transfer capabilities.

## introduction the availability of multilingual models has facilitated the development of nlp tools for many lowresource languages. their appeal not only comes from this universal language representation but also through leveraging data from related languages @xcite @xcite . despite this, multilingual models may fall short especially for lowerresourced languages @xcite . in particular, @xcite show that the cross-lingual transfer capabilities are hampered due to script differences between the target language and the related language which is part of the multilingual model pre-training. however, they show that performance is dramatically improved by transliterating the target language to the same script as the related language. unlike translation, transliteration is a relatively cheap process which maps characters in a given script to another. in this work, we focus on maltese, a lowresource hybrid/mixed language of semitic origin but written in latin script. although it retains a strong semitic, specifically arabic, component in its grammar, it borrows heavily from romance (italian) and english. this motivates us to explore the impact that transliterating maltese into arabic script could have on a number of downstream tasks. besides using large language models based on arabic text, this experimental setup opens up interesting research questions about the impact of high ambiguity from arabic orthographic choices, such as dropping diacritics which may lower outof-vocabulary, but also increase ambiguity. despite its arabic roots, transliterating maltese to arabic script is not trivial because of the strong non-arabic influences on maltese and its evolution independent of the arab world for a significant number of years @xcite . that said, arabictransliterated maltese can be deemed as an arabic dialect with a higher degree of italian codeswitching. as such, unlike @xcite , we do not just rely on multilingual language models for cross-lingual transfer, but also make use of arabic language models, specifically camelbert @xcite . we compare its performance to multilingual bert @xcite and monolingual maltese bert @xcite . empirically, we show that there are differences in the cross-lingual transfer capabilities of arabic models and multilingual models for maltese. the main contributions of this work are as follows. we present various transliteration pipelines from maltese to arabic script ranging from simple one-to-one character maps to more sophisticated alternatives that explore multiple possibilities or make use of manually annotated linguistic constructions. we show that the sophisticated systems are consistently better than simpler systems, quantitatively and qualitatively. we also show that despite its hybrid nature, transliterating maltese can be considered as an option to improve the cross-lingual transfer capabilities. the rest of the paper is organized as follows. we present a discussion of motivating linguistic background (section 2) followed by our approach for transliterating maltese to arabic script (section 3). we present our evaluation in sections 4 and 5. ## our transliteration system we present a maltese-to-arabic transliteration system with a number of variants that we evaluate in sections 4 and 5. the transliteration system contains two operations: mapping and ranking. maltese text tokens and characters are mapped from latin script to one or more alternatives in arabic script (section 3.2). then, a separate component ranks the choices or uses a deterministic hardcoded baseline (section 3.3). ## human readability evaluation in this section, we investigate how readable transliterated maltese is to native arabic speakers. we compare four settings: the deterministic system against the non-deterministic system (tunisian arabic word model) with both none and full token mappings. we sample 50 instances from the mudt training set @xcite . for each example, we provide the original sentence, a translation extracted from google translate as a reference, and each of the alternative transliterations (see example in the evaluator reported that reading maltese written in the arabic script allowed them to easily recognize shared words between maltese and tunisian arabic. for instance, the evaluator did not recognize the maltese word kien, but when transliterated into the arabic script as 'he was', it was evident. however, italian-origin words presented a reading challenge, e.g. maltese akkuża (italian accusa) was garbled. the evaluator also pointed out that none of the transliteration models were capable of handling we presented a maltese-to-arabic transliteration system as a tool to leverage cross-lingual transfer from arabic. as evidenced by our empirical results, a non-deterministic system with signals from the target language helps in choosing a better transliteration alternative, especially in ambiguous cases. moreover, incorporating human-annotated transliterations of a set of closed-class of words is beneficial in downstream performance, especially in lower-resource settings. our experimental setup exploited an arabic language model for cross-lingual transfer, instead of a multilingual model such as mbert. results show promising results, giving better performance than multilingual models. this echoes the findings by @xcite , and we encourage further research to investigate ways to effectively leverage resources from linguistically related languages. future work should investigate the use of large sentence-level contexts in mapping selections. exploring cross-lingual transfer from italian and english is also an interesting direction, including the use of few-shot and zero-shot learning. it would be interesting to also investigate these cross-lingual transfer techniques through transliterated arabic.