Nâbra: S yrian A rabic Dialects with Morphological Annotations (2023)

Abstract: This paper presents Nâbra (نَبْرَة), a corpora of Syrian Arabic dialects with morphological annotations. A team of Syrian natives collected more than 6K sentences containing about 60K words from several sources including social media posts, scripts of movies and series, lyrics of songs and local proverbs to build Nâbra. Nâbra covers several local Syrian dialects including those of Aleppo, Damascus, Deir-ezzur, Hama, Homs, Huran, Latakia, Mardin, Raqqah, and Suwayda. A team of nine annotators annotated the 60K tokens with full morphological annotations across sentence contexts. We trained the annotators to follow methodological annotation guidelines to ensure unique morpheme annotations, and normalized the annotations. F1 and \kappa agreement scores ranged between 74% and 98% across features, showing the excellent quality of Nâbra annotations. Our corpora are open-source and publicly available as part of the Currasat portal https://sina.birzeit.edu/currasat.

## introduction dialectal arabic (da) content dominates informal writings in emails, social media, blogs, and social messaging. interest in building computational resources for arabic dialects has been in the rise to provide both (i) annotated corpora @xcite @xcite @xcite @xcite and (ii) morphological dialect analyzers @xcite @xcite @xcite . in this paper, we present nâbra , a set of corpora that complement existing arabic dialect corpora by covering several dialect variants of syrian arabic. nâbra covers dialects from 10 syrian localities including aleppo, damascus (a.k.a. shami) , deir-ezzur, hama, homs, huran, latakia, figure 2: examples of typical suffixes in syrian dialects. mardin, raqqah, and suwayda. nâbra was collected from several sources including social media posts, scripts of movies and series, lyrics of songs, and local proverbs. nine annotators worked on annotating 6k sentences with 60,021 tokens with full morphological annotations. each word was annotated using: prefix(s), stem, and suffix(s), part of speech (pos), dialect lemma, msa lemma, person, number, gender, gloss, and synonyms; in addition to the sub-dialect it belongs to. we adopted the same annotation methodology used to annotate the palestinian curras2 and the lebanese baladi corpora @xcite , which we also used with the four corpora of lisan @xcite . as we will discuss later, we adopted the sama tagsets @xcite , but we introduced new prefixes and suffixes that are commonly used in syrian dialects (figures 1 and 2 ). ## related work there are several annotated corpora and lexicographic resources for msa. the ldc's penn arabic treebank patb @xcite consists of about consists of 791,210 tokens collected from several news sources. patb annotations include: tokenization, segmentation, pos tagging, lemmatization, diacritization, english gloss and syntactic structure. the ldc ontonotes 5 @xcite is another msa corpus collected from news sources, consisting of about 330k tokens, which are annotated in the same way as the patb. ontonotes 5 also contains multiple layers of annotation, including the patb annotation layer. the prague arabic dependency treebank (ar-padt) @xcite is a treebank that contains morphological annotations for a corpus of msa text. these annotations include lemmas, partof-speech tags, and other morphological features. ar-padt contains about 224k words. the ldc's sama is a stem database @xcite , which is an extension of bama @xcite , designed only for morphological modeling. it contains stems and their lemmas and compatible affixes. it contains about 40k lemmas. the lexicographic database at birzeit university (jarrar and amayreh, 2019) provides a large set of msa lemmas, word forms, and morphological features, which are linked with the arabic ontology @xcite using the w3c lemon model @xcite . ## syrian as a levantine dialect the levantine family of dialects can be linguistically split across the north including lebanon and syria, and the south including palestine and jordan. during the seventh century, arabic spread across the area, which spoke western aramaic before then @xcite . aramaic is a semitic language continuum spoken during antiquity throughout the levant where it served as the lingua-franca. aramaic survives today through modern dialects such as turoyo syriac and western neo-aramaic spoken in parts of syria. it also survives more subtly in the noticeable substratum underlying levantine dialects that differ from msa on several linguistic characteristics such as phonology, syntax, morphology, and lexicon. this additionally motivates the development of morphologically annotated resources for levantine dialects. in the sequel, we briefly review the differentiating factors between levantine dialects, syrian dialects, and msa. ## variant syrian dialects syrian arabic dialects are used in daily communication among most syrians. some of them are closer to iraqi dialects, and the rest are closer to the levantine southern levantine dialects. here, we review the most famous dialects spoken in syria. the shami dialect is the dominant dialect in the damascus area and is the most widespread and used syrian dialect. as the dialect of the capital, it dominates syrian series and films which are widely accepted, appreciated, and spread in the arab world. it is used in dubbing and translation of foreign series (turkish and hindi). the latakia dialect is spoken across the coast in latakia and tartous. it is a mixture of arabic, syriac, and phoenician. it is characterized by the strong pronunciation of the letter /q , and also features the letter /m before verbs to denote the present tense in all its forms, e.g. the raqqa dialect is one of the closest dialects to classical arabic in terms of vocabulary. raqqa enjoys a distinguished location on the shores of the euphrates river. it is home ( /dyār ) mudar, who are arabs from the north. mudar were displaced to the euphrates island several centuries before islam. the raqqa syllables sound commensurate to the corresponding classical arabic syllables. for example, the pronunciation of /k results in a thirsty /ǧ as in /kānt pronounced as /ǧānt . the letter /q is pronounced /k similar to yemeni dialects as in /qā↩ (earth) pronounced as /kā↩ . the deir-ezzur dialect aka. as /āldyryh is in proximity to the euphrates as well, and preserves most of the phonetic aspects of standard arabic. the significantly different phonemes are /q , /k and / , while there is no different in the gingival sounds. the homs dialect varies slightly across several rural and urban areas in the homs district. this is mainly due to the habitual diversity of the countryside including a sizeable turkman population. this paper covers the dominant variant in the city of homs. the homs dialect is characterized by pronouncing the first letter in a word as if it has a dammah /u diacritic (inclusion). this includes the name of the city /h . ims . , pronounced with a kasra /i dialect everywhere else. it also flips gender when it comes to masculine second-person / inti (you-male in homsi) and feminine second person / inta (you-female in homsi). it also differs in the pronunciation of the letter /ǧ as they phonetically annex a silent /d resulting in a /dǧ sound. the hama dialect is spoken in the central syrian governorates. it is a good representative of the syrian levantine dialects and close to the shami one, as it tends to be soft and long in speech. it is distinguished by its eloquence and stretch in speech. al-hader (city in hama) variant of the hama dialect is the most prominent variant. the hauran dialect is spoken south of the damascus countryside down to the ajloun mountains in jordan including daraa. it is an ancient arabic dialect spoken by multiple arab tribes, where each of them has some distinguishing phonetic characteristics. the al-suwayda dialect is spoken in jabal al-arab. the harshness of the mountain environment is reflected in the dialect's tone. it is taut, clear, and possesses a fast rhythm. syllable notes exit soundly and eloquently. the concept of /ālmd . āfh played a major role in preserving the strength of the dialect. therein, prominent, cultured, and experienced speakers exchange arguments. this highly contributed to the rigor of the dialect and brought it closer to standard and classical arabic. the mardini dialect takes its name from the city of mardin in /ālh . skh . it is also called /ālǧzrāwyh in relevance to the /ālfrātyh island. the dialect contains many turkish, persian, and aramaic words. ## nâbra corpora collection we manually collected about 6,000 sentences with 60k tokens from facebook, blogs, popular proverbs, syrian films and series, local poetry, and lyrics of popular local songs in several syrian dialects to build nâbra. the distribution relatively follows the order of dialect demographics. the shami dialect is the richest with 17.3k tokens, used as primary dialect in damascus, the capital, and in various syrian tv series and films. nâbra contains 9.2k aleppo tokens collected from popular stories on facebook and from vocal poetry. coastal latakia features 7.9k tokens collected from film dialogues such as /rsāyl šfhyh -/qmrān wzytwnh (voice letters, qumran and zeitouna) and series such as /d . y↩h d . āy↩h (lost town). we also added common proverbs. suwayda dialect features 3.2k tokens from the /ālhrbh series. for homs and hama we collected jokes, and food discussions from social media blogs. the raqqa, huran, and mardin dialects feature the remaining 6.3k, 3.8k, and 1.6k tokens, respectively. we manually collected texts from social media for raqqa and huran. we found blogs documenting raqqa. we used blogs and traditional stories for raqqa, vocal poetry and lyrics of popular folklore songs for mardini, and scenes from the bedouin series for huran dialects. we noticed that the collected data reflected spontaneous dialect documentation all across, contrary to what one would expect. films and series were no less spontaneous than blogs and social media. as arabic is diacritic-sensitive @xcite , we did not remove any diacritics we tokenized the text of nâbra so that each token has a tuple with the following information. ## annotation methodology and features we followed a semi-automated methodology, with an integrated productivity tool, friendly to nonprogrammers, to annotate nâbra. ## evaluation and agreement before evaluating nâbra, we normalized the annotations to unify variant annotations that are equivalent. these variants occur due to human mistakes such as typos ( /mās . y instead of /mād . y ), ordering of tags in sequences of tags, and inconsistent use of separators and spacing. another source of variants is tokens with no feature values in the existing annotated dialects. annotators have to come up with novel values. we detected these tag values, ranked them based on their frequencies, and clustered them based on their edit distance from each other. then we reviewed them and unified them across nâbra and its features. we developed a small suite of vba scripts empowered with regular expressions to check for these variants and correct them automatically where possible. if automatic correction is not possible and human attention is required, then our reference annotators interfere to correct it. ## conclusion this paper presents nâbra, a morphologically annotated corpora of syrian arabic dialects. the corpora contain about 60k tokens from 10 syrian dialects, collected from social media platforms, movies and series, common proverbs, and song lyrics and poetry. and other arabic corpora, we chose to annotate the corpora using sama tagsets. to evaluate the quality of the corpora, we used the f 1 and kappa scores which show high agreement. we plan to use nâbra to extend wojood @xcite by annotating the corpora for named entity recognition, similar to what we did with curras and baladi.