Sarcasm and Sentiment Detection in A rabic language A Hybrid Approach Combining Embeddings and Rule-based Features (2021)

Abstract: This paper presents the ArabicProcessors team’s system designed for sarcasm (subtask 1) and sentiment (subtask 2) detection shared task. We created a hybrid system by combining rule-based features and both static and dynamic embeddings using transformers and deep learning. The system’s architecture is an ensemble of Naive bayes, MarBERT and Mazajak embedding. This process scored an F1-score of 51% on sarcasm and 71% for sentiment detection.

## introduction automatic sarcasm detection is a sub-discipline of sentiment analysis. this process, however, presents specific challenges @xcite related to the fact that sarcasm can present opposite polarities between surface and intended sentiment. macmillan english dictionary show the complexity of this phenomenon by defining it as "the activity of saying or writing the opposite of what you mean, or of speaking in a way intended to make someone else feel stupid or show them that you are angry". this imposes additional layers to go beyond the methods used so far for sentiment analysis; since in addition to the sentimental polarity of the text, it will also be necessary to consider, among others, pragmatic factors @xcite , based on context and background knowledge @xcite , behavior modelling @xcite and the linguistic theory of incongruity @xcite . this observation is even more accentuated when it comes to the treatment of arabic sources. undoubtedly, the scarcity of data available in arabic relative to english and the different arabic dialects, for example, makes the detection of sarcasm even more complicated. should sarcasm in the arabic language be treated the same way it is treated in the english language? @xcite is sarcasm also present in arabic the same way in its different dialects? this complexity is reflected in the scarcity of studies on automatic arabic sarcasm detection, such that first works @xcite and @xcite dealing with this problem, were recently published in 2017 and 2019. shared task on sarcasm and sentiment detection in arabic @xcite is a new step in exploring this problem, aimed at both sarcasm detection (subtask 1) and the analysis of sentimental polarity (subtask 2). in this paper, we present our contribution to this challenge, by illustrating a novel approach based on a hybrid system that combines rule-based features and both static and dynamic embeddings using transformers and deep learning. in the next sections, we describe used data in section 2, describe our system in section 3, present and discuss our results in section 4, and finally summarize our work in section 5. ## data to train our models, we were provided with the same dataset for both subtasks; arsarcasm-v2 @xcite . it comprises of four variables, namely; the text of the tweet, a boolean label for sarcasm, sentimental polarity label (positive, neutral and negative) as well as a dialect label. in the cause of this shared task, a total of 12,548 tweets were made available to us, of which 2168 are labeled as sarcastic tweets, representing 17.3% of all tweets (figure 1 ). therefore, it is an unbalanced dataset, as is the case for the majority of sarcasm and irony datasets @xcite . with regards to sentiment distribution, approximately half were neutral, while 37% were negative and 17% positive (figure 1 ). it is noteworthy, however, that the vast majority of tweets were written in msa (68%) and dominantly in egyptian dialect (figure2). with respect to the sentimental polarity label, we established overriding amount of negative sentiments among sarcastic tweets (figure 3 ). moreover, it was noted that the egyptian dialect, which has the same provided dataset general distribution with non-sarcastic tweets, it represents the same share as msa among sarcastic tweets. in analysing the most common words among sarcastic tweets, we noted two main groups, political personalities, or entities or events ( sissi, trump, morsi,«‫ﻼب‬q‫-»اﻧ‬coup d'etat-, hilary, etc.) and punctuation dominated by quotation marks, colons, replies (@) and exclamation mark. we further observed that averagely, there are more words in sarcastic tweets (18 words) than in the non-sarcastic ones (15 words). sarcastic tweets are, however, on an average, shorter with 5.6 characters as against 6.6 characters for other tweets. the same applies to stopwords, which are more frequent among sarcastic tweets, reaching an average of 2.1 stopwords as against 1.7 for the rest of the tweets. the most common emoticons in sarcastic and non-sarcastic tweets include the "face with tears of joy", the "new moon face", and the "pensive face". the emoticon showing a "face with tears of joy" sums up to 32% of sarcastic tweets, far ahead of the rest of the emoticons, while it sums up to only 7% of non-sarcastic tweets. regarding the interjection ‫»ﻫ‪h‬ﻪ«‬ with n repetition of the letter «þ‫,»ﻫ‬ equivalent to «haha», we found out that it is more present in sarcastic tweets with 4% compared to other tweets not exceeding 0.5%. these findings are a strong indication of the potential efficiency of the use of features extracted from tweets. indeed, this would be a factor for sarcasm detection. in that regard, we analyzed the features presented in the following section. ## results and discussion we oriented the development of our system towards the use of an ensemble model including marbert, bi-lstm with mazajak embedding and gaussian naive bayes. gaussian naive bayes was chosen after comparison with logistic regression, svm, xg-boost, lightgbm, and random forest was done. results obtained for individual models on the 5-fold cross validation, and ensembled one are reported in ## conclusion in this paper, we have described our contribution to the wanlp'21 sarcasm and sentiment detection in arabic subtasks. the system's architecture is an ensemble of gaussian naive bayes, marbert and mazajak embedding. by recording an f1-score of 51% at subtask 1, and 71% at subtask 2, this approach demonstrates the efficiency of our hybrid system in the challenging field of sarcasm and sentiment detection for arabic texts.