A Unified Model for A rabizi Detection and Transliteration using Sequence-to-Sequence Models (2020)

Abstract: While online Arabic is primarily written using the Arabic script, a Roman-script variety called Arabizi is often seen on social media. Although this representation captures the phonology of the language, it is not a one-to-one mapping with the Arabic script version. This issue is exacerbated by the fact that Arabizi on social media is Dialectal Arabic which does not have a standard orthography. Furthermore, Arabizi tends to include a lot of code mixing between Arabic and English (or French). To map Arabizi text to Arabic script in the context of complete utterances, previously published efforts have split Arabizi detection and Arabic script target in two separate tasks. In this paper, we present the first effort on a unified model for Arabizi detection and transliteration into a code-mixed output with consistent Arabic spelling conventions, using a sequence-to-sequence deep learning model. Our best system achieves 80.6% word accuracy and 58.7% BLEU on a blind test set.

## introduction the term transliteration generally refers to the process of mapping the orthographic symbols used in one script into another. the types of mapping schemes can range from strict one-to-one transliterations @xcite , to named-entity transliteration which can be partially constrained by spelling conventions of the target language @xcite , and all the way to the spontaneous romanizations used in many places around the world, in competition with local languages and spelling traditions. in the arab world, the latter of these phenomena is often called arabizi, 1 since it is used primarily in social media (sms and chat), it's notoriously noisy and inconsistent although there are some common conventions. and because it is written in the roman script, it encourages written linguistic code mixing between arabic and english (or french). furthermore, since it is not an official orthography, arabizi is a resource poor language form compared to modern standard arabic and some of its dialects, where there are more resources written in arabic script. as such, many researchers work on mapping arabizi to arabic script as part of general text normalization. this task includes two components: identifying the language, and mapping it to the target script accordingly. for example, the codemixed sequence w aletli tuesday 'and she-said-to-me tuesday' would be mapped to tuesday wqalt ly tuesday foot_1 'and-she-said to-me tuesday'. while the input and output here have the same number of words, they are not aligned one-to-one as arabic spelling rules define words boundaries differently, an additional complexity of the task. there have been a number of efforts in natural language processing (nlp) that worked on this interesting phenomenon using a range of techniques from classical machine learning and n-gram language models @xcite @xcite to sequence-to-sequence neural models @xcite @xcite . the various previous solutions differ in @xcite whether and how they modeled the subtask of arabizi detection (which words are arabic written in roman script, and which are english/french), (b) what amount of context they use, and (c) how they define the arabic script target (whether to allow token merges and splits or follow arabizi word boundaries). in this paper we present the first effort on a unified model of arabizi detection and transliteration into a code-mixed output with consistent arabic spelling conventions, using neural sequence-to-sequence models. our best system achieves 80.6% word accuracy and 58.7% bleu score on a blind test set -9.9%, and 15.9% absolute improvements, respectively, over a simple but robust baseline. our system's code is public. 3 but the data must be acquired from the ldc (see section 5). we discuss related work in sections 2, and arabizi challenges and task definition in section 3. we present our system architecture and approach in section 4, and our experiments and results in section 5. ## arabizi challenges and task definition our task of mapping code-mixed egyptian arabizi and english input into code-mixed egyptian arabic and english poses a number of challenges. arabizi noise and ambiguity while there are some common conventions for arabizi-to-arabic mapping, they are not strictly followed by far. this is further exacerbated by typical noisy spelling in spontaneous social media text. as a result, we have many-to-many mappings at the word level. for example, the arabic word hbyby 'my beloved' appears in the arabizicorpus paired with 24 different arabizi spellings: (in order of frequency) habibii, hbebe, habibi, 7abiby, 7abibi, hbebee, 7abeby, 7biby, habbii, 7bibi, 7abibii, hbybyy, hbbee, hbbebe, habiibii, habbyy, bbe, 7apipy, 7abiibii, 7abibyy, 7abebe, 7abby, 3abiibii, and 3abibi. similarly, the arabizi word arfo is paired with three different arabic words: ςarfh'i know it', qrfwa 'they made me loathe [something]' and qrfh 'its nastiness'. we mitigate the noise and variability in modeling the arabizi-to-arabic transliteration as a character level sequence-to-sequence process. we also make use of word-level embeddings using fasttext @xcite which models subword units within a bigger word-based context. additionally, we handle common variations in social media text such as inconsistent capitalization and emphatic repetitions through global lower casing and repetition elision to two characters, e.g., arabizi habiiiiiibiiiiii is preprocessed to habiibii. ## conclusion and future work we presented a character-level seq2seq model for arabizi detection and transliteration together into a code-mixed output with a consistent arabic spelling convention. our best system improves over the baseline by 9.9% in word accuracy and 15.9 bleu points on a blind test set. we report on a number of experiments including some negative results. in the future, we plan to integrate our best model in an open source toolkit for supporting arabic nlp. furthermore, we want to work on arabizi text from a range of arabic dialects. finally, we want to integrate our system into other arabic nlp applications, such as machine translation from arabizi to demonstrate the relevance of arabizi-to-arabic transliteration.