Identifying Code-switching in A rabizi (2022)

Abstract: We describe a corpus of social media posts that include utterances in Arabizi, a Roman-script rendering of Arabic, mixed with other languages, notably English, French, and Arabic written in the Arabic script. We manually annotated a subset of the texts with word-level language IDs; this is a non-trivial task due to the nature of mixed-language writing, especially on social media. We developed classifiers that can accurately predict the language ID tags. Then, we extended the word-level predictions to identify sentences that include Arabizi (and code-switching), and applied the classifiers to the raw corpus, thereby harvesting a large number of additional instances. The result is a large-scale dataset of Arabizi, with precise indications of code-switching between Arabizi and English, French, and Arabic.

## introduction arabizi is a writing system for (primarily dialectal) arabic that uses the roman alphabet. it is ubiquitous on social media outlets, and has many characteristics of social media writings in other languages (e.g., slang, tendency towards the spoken register, spelling errors, abbreviations, character repetition, use of emoticons, etc.) the use of the roman alphabet facilitates (and perhaps even encourages) code-switching: moving between arabic (represented in arabizi) and other languages, notably english and french, sometimes even within the same sentence. code-switching is becoming more and more prevalent as the world's population is becoming more multilingual @xcite . it is a natural phenomenon that is triggered by linguistic, sociolinguistic, psycholinguistic, demographic, and contextual prompts, and has been studied mainly in the spoken language until recently. with the ubiquity of text online, however, code-switching is beginning to be investigated also in the written language (e.g., @xcite @xcite . such research has various practical applications, both for understanding and for generation of codeswitched language @xcite . our main interest is in code-switching phenomena in arabizi; in order to better understand them, a large dataset of arabizi is required. the main goal of this work is to construct a largecorpus of arabizi utterances, potentially including instances of code-switching between arabizi and english, french, or arabic written in the arabic script. the dataset is based on social media posts from two outlets: twitter and reddit. to collect the data, we implemented a classifier that can identify sentences containing words in arabizi, arabic, english, and french, and used it to filter out texts harvested from the two outlets. we describe the dataset and the methods we used to curate it (section 3). we then discuss the challenge of determining the language id of words in multilingual texts, and describe classifiers that can accurately predict such language tags, based on a schema we developed for the task (section 4). we extend the word-level classifiers to sentence-level ones, assigning a complex tag to each sentence that indicates the presence of words from various categories (i.e., languages) in it (section 5). finally, we use the classifiers to extract additional instances of sentences with arabizi (and with code-switching) from our raw corpus (section 6). this paper makes several contributions: 1. we release a large-scale corpus of twitter and reddit posts that include arabizi; 2. we introduce a novel annotation scheme that determines the language of words in multilingual utterances; specifically, we advocate a unique tag for words that can be included in more than one mental lexicon (and hence trigger code-switching); 3. we release a portion of the dataset, manually annotated according to this annotation scheme; and 4. we provide highlyaccurate classifiers that can determine the language id tags of words in this corpus; the classifiers were used to identify hundreds of thousands of additional sentences that are very likely to include arabizi in general and code-switching with arabizi in particular. we expect these resources, which are all publicly available, foot_0 to be instrumental for future research in code-switching and in arabizi. ## related work arabizi has attracted some interest in recent years, and various works address the tasks of detecting it and converting arabizi to the arabic script. darwish (2014) used word-and sequence-level features to identify arabizi mixed with english and achieved 98.5% accuracy on the identification task. he argued that classifying a word as arabizi or english has to be done in context, and thus employed sequence labeling using conditional random fields (crf) for classification. the data were selected from twitter, by querying (three) commonly used arabizi words and then extracting the user ids of all the authors of the resulting tweets, obtaining all their tweets, under the assumption that authors who use arabizi once may use it often. then, tweets in which most of the words contained arabic letters were filtered out. this resulted in 522 tweets consisting of 5207 tokens, of which 1203 were in arabizi. @xcite compiled a corpus of more than half a million pages from an algerian newspaper website, from which they extracted almost 7m tokens which were annotated for language, using three tags: arabic, french, or other. more recently, @xcite compiled a corpus of arabic mixed with moroccan darija, in which tokens were assigned to seven categories: three for languages, and then mixed (morphemes from more than one language in the same token), named entity, ambiguous and other. in total, 223k tokens were annotated. the task of transliterating arabizi to arabic was addressed by @xcite , who employed finite-state transducers, a language model and morphological processors for arabic. they used a dataset consisting of 1500 words only. this approach was then extended to the tunisian dialect @xcite . the transliteration task was applied to the tunisian dialect in a more recent work @xcite , using contemporary machine-learning techniques, but the datasets remained relatively small. @xcite addressed the joint task of identifying arabizi and transliterating it to the arabic script, reporting high word accuracy on a large (1m token) dataset. @xcite trained an svm classifier to identify arabizi in multilingual twitter data. he assumed that in order to tag a tweet as arabizi it should have more arabizi words than english words. the best results were obtained using three features: (1) the languages as detected by langdetect; two arabizi datasets were recently compiled and released @xcite : lad, a corpus of 7.7m tweets written in arabizi; and salad, a randomly-selected subset of lad, containing 1700 tweets, manually annotated for sentiment analysis. the tweets were harvested using twint: twitter intelligence tool, by setting 48 common words in egyptian as seeds. this work focused mainly on the egyptian dialect, and the manually-annotated dataset is rather small. @xcite built the first north-african arabizi treebank. it contains 1500 sentences, fully annotated with morpho-syntactic and universal dependency codes, with full translation at both the word and the sentence levels. it is also supplemented by 50k unlabeled sentences collected using web-crawling. the texts reflect the algerian dialect, and contain 36% french tokens. recently, this dataset was extended by adding transliterations of all the arabizi tokens, as well as sentence-level annotations of sentiment and topic @xcite . @xcite focused on the task of identifying arabizi (and romanized bereber) in social media texts, reporting near-perfect accuracy using very simple character-ngram features. the data were collected from north-african sources and reflect these dialects. more recently, @xcite used deep learning methods to identify the language of words in tunisian social media texts. they defined five categories for the classification (tunisian dialect words, foreign langauge words, punctuation, symbols, and emoticons) and reported almost perfect accuracy on this task. one of our goals in this work is to create a large dataset of sentences containing arabizi, potentially mixed with words in other languages, focusing on the egyptian and lebanese dialects. unlike much existing work, we annotate our dataset at the word level, thereby yielding a richer annotation that clearly outlines sentences with codeswitching. our language id annotation scheme acknowledges the difficulty of assigning language id tags to words that may be shared by more than one mental lexicon; such words, which include proper names and cognates, are assumed to trigger code-switching @xcite @xcite @xcite . we then use our annotated dataset to train classifiers that we employ to extract more code-switched arabizi instances from reddit and twitter, thereby extending the scope of our dataset significantly. ## data collection we conjectured that social media outlets, particularly reddit and twitter, would include a sizable amount of arabizi utterances. to identify them, we modified the method suggested by @xcite , which has subsequently been used also to harvest code-switched data from reddit @xcite . first, we identified some reddit fora ('subreddits') where we expected to find arabizi used. these included r/arab, r/arabs, r/egypt, r/jordan, r/lebanon, and r/syria. we downloaded the entire collection of the above subreddits. the resulting (raw) reddit dataset consisted of 3,584,915 sentences, 59,593,594 words and 72,305 authors. for twitter, we followed darwish (2014) and defined a few dialectal arabic seed words that we expected to occur with high frequency in arabizi texts, focusing on the egyptian dialect (where we expected to find code-switching with english) and the lebanese dialect (where we expected mixed french). these seed terms are listed in appendix a. we located and retained tweets that included any of the seed words in our list. we then extracted the user ids of authors of such texts, under the assumption that authors that use arabizi in some tweets are likely to use it elsewhere, too; and we included all tweets authored by these users in our corpus. the resulting (raw) twitter dataset con-sisted of 2,466,642 sentences (22,530,044 words) authored by 1090 users: 936 egyptians and 154 lebanese. we used nltk @xcite for sentence boundary detection and tokenization. as the tokenizer did not split emojis from other tokens, we added a simple post-processing step to make sure all emojis were standalone tokens. we removed extra spaces and separated arabic letters from non-arabic ones. we also shortened adjacent repeated letters to only two (e.g., we converted 'ahhhhh edaaa thankkk youuuu' to 'ahh edaa thankk youu'). next, we aimed to identify sentences containing arabizi in the raw dataset. we first utilized a number of language identification tools, including spacy @xcite ), google's langdetect (we used the python port), langid @xcite , and fasttext @xcite . unsurprisingly, they all failed to detect arabizi with acceptable accuracy. to evaluate the accuracy of existing language id tools on arabizi we selected 100 sentences from the annotated arabizi dataset of @xcite : the first 50 sentences containing only arabizi words from the egypt dataset, and the first 50 from the lebanon dataset. we applied the above-mentioned classifiers to these 100 sentences; since none of the tools was trained on arabizi data, none predicted arabizi. but they did not predict arabic, either: instead, langdetect defaulted to somali 43 times, (and indonesian 25 times); langid detected english, spanish-castilian, indonesian, and swahili for 50 of the sentences; fasttext preferred english and spanish; and spacy identified half of the sentences as somali or indonesian. we therefore resorted to defining our own language id detection model, which we specifically tuned to identifying arabizi (in addition to english and french). we developed a dedicated scheme for tagging words in a mixed-language dataset (section 4.1), manually tagged a sizeable number of sentences reflecting the various language combinations witnessed in the dataset (section 4.2), and then used the manually annotated subset to train classifiers (sections 4.3-4.4) that can assign language id tags to words in unseen texts. finally, we extended the annotation from words to sentences (section 5) in order to devise an efficient extractor for more instances of code-switched arabizi from our corpus. we now detail these stages. ## word level classification some existing work on arabizi focused on identifying the language of a sentence, or a larger chunk of text. for example, tobaili (2016) defined a tweet as arabizi if it contained at least 50% arabizi tokens. in contrast, we focus on identifying the language of each individual token in the corpus, as our main motivation is to prepare a dataset suitable for research on code-switching, which may of course be intra-sentential. as mentioned above, existing tools for word-level language id fail miserably when arabizi is concerned. we begin by discussing the challenges involved in word-level annotation of multilingual texts (section 4.1), detail the manual annotation (section 4.2), and then discuss our classifiers, both statistical (section 4.3) and neural (section 4.4). ## identifying code-switching the word-level annotation immediately facilitates the identification of code-switching: a sentence with at least one word in arabizi and one in either english or french necessarily includes a switch. to ## conclusion we described a classifier that identifies words in arabizi, english, arabic, and french in multilingual sentences from social media. we applied the classifier to a large set of sentences collected from twitter and reddit, and produced a huge dataset of more than 880k automatically-annotated arabizi sentences, of which over 446k include codeswitching with either english or french. we are now ready to use this dataset for a largescale corpus-based investigation of theoretical research questions in cognitive linguistics. specifically, we are interested in the correlation between shared words, as defined in our annotation scheme, and code-switching. we leave such investigations for future work.