Introducing A large T unisian A rabizi Dialectal Dataset for Sentiment Analysis (2021)

Abstract: On various Social Media platforms, people, tend to use the informal way to communicate, or write posts and comments: their local dialects. In Africa, more than 1500 dialects and languages exist. Particularly, Tunisians talk and write informally using Latin letters and numbers rather than Arabic ones. In this paper, we introduce a large common-crawl-based Tunisian Arabizi dialectal dataset dedicated for Sentiment Analysis. The dataset consists of a total of 100k comments (about movies, politic, sport, etc.) annotated manually by Tunisian native speakers as Positive, negative and Neutral. We evaluate our dataset on sentiment analysis task using the Bidirectional Encoder Representations from Transformers (BERT) as a contextual language model in its multilingual version (mBERT) as an embedding technique then combining mBERT with Convolutional Neural Network (CNN) as classifier. The dataset is publicly available.

## introduction while being so diverse and rich, arabic language and particularly arabic dialects, are still under represented and not yet fully exploited. arabizi is a term describing a system of writing arabic using english characters. this term comes from two words "arabi" (arabic) and "engliszi" (english). arabizi is the representation of arabic sounds using latin letters and numbers to replace the non existing equivalent ones @xcite . particularly in tunisia, this way of writing was introduced in @xcite as "tunizi". in this paper, we introduce a large tunizi dataset available freely in order to help tunisian and other researchers interested in the natural language processing (nlp) field. this dataset contains 100k tunizi comments collected from crowd-sourcing 1 https://zenodo.org/record/4275240.ydew1-oo9h4 and built for the sentiment analysis task. indeed, the dataset's comments are manually annotated by tunisian native speakers as positive, negative, or neutral. this dataset can be also used for other nlp subtasks such as dialect identification, named entities recognition, etc. in this paper, we present the motivation of our work, the tunizi way of writing, previous related work, data composition including the data collection, preprocessing, annotation, and the data statistics. then, we present experiments performed using our dataset for the sentiment analysis (sa) task. finally, we present the conclusion and future work. ## tunizi tunizi was referred in @xcite as "the tunisian arabic text written using latin characters and numerals rather than arabic letter". this type of writing is most used on social media platforms since it presents an easier way to communicate. also, since the tunisian dialect already contains french and english words, people tend to use tunizi for easier typing of non formal texts. however, not all arabic letters have a latin equivalent representation such as the character " " or " ". these characters are expressed using numbers instead in the tunizi representation. tunisians have high contact culture where online social networks play a key role in facilitating social communication @xcite . for instance, due to the linguistic situation specific to each country, tunisians generally use the french background when writing in tunizi, whereas, other arabic countries like egypt would use english. for example the word would be written in tunizi as "chrit" whereas as "ishtareet" in arabizi. in arabic arabizi tunizi 7 7 5 or 7' 5 or kh d' or dh dh $ or sh ch t' or th or 4 th 4' gh or 8 3 3 8 9 ## related work in @xcite , a tunisian dataset entitled tsac (tunisian sentiment analysis corpus) was presented. this dataset combines 17366 positive/negative tunisian facebook comments written in arabic and tunizi. tsac includes only 9196 comments written in tunizi. however, tunizi comments are not balanced with 3856 negative comments (41%) and 5340 positive comments (59%). the doc2vec algorithm was used to produce document embeddings of tunisian arabic and tunizi comments. the generated embeddings were fed to train a multi-layer perceptron (mlp) classifier performing a f-measure values of 78%. in @xcite , authors proposed a transliteration method from arabizi to msa in order to evaluate sentiment analysis of arabizi algerian dialect. the best achieved performances was 87% of f-score using the support vector machine. in @xcite , a work on lebanese arabizi was performed. in order to create sentiment analysis experiments, a dataset was collected from twitter and manually labelled as positive and negative. the dataset resulted in 2.9k tweets: 801 positive, 881 negative, and 1.2k neutral arabizi tweets. comments used for sentiment analysis experiments are of size 1.6k balanced as 800 positive, and 800 negative tweets. a sentiment analysis task is, then, performed using a lexicon-based approach. the lexicon created was first of size 1.6k words and was expanded until achieved 80k words. the lexiconbased approach using the original lexicon classified only 23% of the tweets. unclassified tweets are classified as positive or negative randomly. the classification using the original lexicon achieved 60% f1-score. classification achieved a coverage of 63% of the tweets with an f1-score of 78% in one of the expanded versions of the lexicon. in @xcite , a tunisian arabizi sentiment analysis dataset called "tunizi" was introduced. this work provided 9k sentences annotated as positive and negative. tunizi is a balanced and representative dataset that covers different topics collected from youtube social network. however, the dataset is not enough large to be used for deep learning approaches. in @xcite , a work focuses on the arabizi form of writing. the work creates a large dataset (lad) of size 7.7m written in arabizi collected from arabizi tweets using egyptian keywords as seeds. lad was used for pretraining a bert language model called "baert". a subset of lad entitled salad of size 1700 was annotated for sentiment analysis. the subset was used for fine-tuning baert. salad contains tweets annotated as positive, negative, neutral, conflict and conflicttxtvsem (tweets where the comments have a conflict in the text and the emoji symbol) . however, the dataset is not balanced and very limited to be used for deep learning approaches. tunizi dataset @xcite was also used for fine-tuning baert model. fine-tuning on tunizi and salad (3 classes) performed 83.8% and 74.4% of f1-score respectively. in order to perform better performances, such experiments would require larger annotated datasets. in @xcite , the authors present a multi-task sequence prediction system for tunisian arabizi. a dataset was annotated for several subtasks: text classification, tokenization, pos tagging and encoding of tunisian arabizi into coda* arabic orthography. nevertheless, the dataset was not annotated for sentiment analysis task. previous work proposed relatively small datasets to be used for deep learning approaches. hence, our contributions can be summarized as: the first release of a tunizi large-scale common-crawl-based ## tunisian arabizi dataset this dataset is composed of one instance presented as text comments, and labeled as positive, negative, or neutral. in this work, we use crowd-sourcing from social media until we reached 100k comments. ## experiments to evaluate the dataset, two experiments were conducted: we decided to use the bidirectional encoder representations from transformers (bert) as a contextual language model in its multilingual version (mbert) as an embedding technique. mbert contains more than 10 languages including english, french and arabic. using the bert tokenizer, we map words to their indexes and representations in the bert embedding matrix. these representations are used to feed the mbert classifier. filter sizes used for all experiments are 2,3,4,5,6. before preprocessing after preprocessing 3saall to9telll nmout 3liha!!! 3saall to9telll nmout 3liha odkhlou lel google www.google.com odkhlou lel google ma3ejbetnich el 7al9a lyoum ma3ejbetnich el 7al9a lyoum since the neutral class have less comments than the positive and negative ones, the same experiments were conducted on only two classes of the dataset: positive and negative. hence statistics of the new sub-dataset are presented in ## conclusion in this work, we present the largest dataset dedicated for the tunisian arabizi, annotated as positive, negative, and neutral and preprocessed for sentiment analysis subtask. such datasets can be used for further nlp activities. we also experiment deep learning models using our dataset. this dataset could improve performances of several works dedicated for the tunisian dialect in the nlp field. as a future work, we plan to build and release other large underrepresented african and arabic datasets in order to help the nlp community further build machine learning and deep learning models.