D eep B lue AI at WANLP - EACL 2021 task 2: A Deep Ensemble-based Method for Sarcasm and Sentiment Detection in A rabic (2021)

Abstract: Sarcasm is one of the main challenges for sentiment analysis systems due to using implicit indirect phrasing for expressing opinions, especially in Arabic. This paper presents the system we submitted to the Sarcasm and Sentiment Detection task of WANLP-2021 that is capable of dealing with both two subtasks. We first perform fine-tuning on two kinds of pre-trained language models (PLMs) with different training strategies. Then an effective stacking mechanism is applied on top of the fine-tuned PLMs to obtain the final prediction. Experimental results on ArSarcasm-v2 dataset show the effectiveness of our method and we rank third and second for subtask 1 and 2.

## introduction social media is growing as a communication medium where people can express online their feelings and opinions on a variety of topics in ways they rarely do in person. detecting sarcasm and sentiment in text have become a crucial element for decision-makers and business leaders as well as for common users to understand public opinion. the significant role of the arab region in international politics and in the global economy have led to the investigation on the task of detecting sarcasm and sentiment in arabic. the task involves detecting whether a piece of text expresses a positive, a negative, or a neutral sentiment; and whether it is sarcasm or non-sarcasm. sarcasm or irony is a form of speech that, in the context of sentiment analysis, mostly takes place when the speaker expresses a positive opinion but actually aims to complain about the opinion target @xcite . sarcasm is particularly hard to detect in arabic language due to the use of positive indicators to express negative emotions. sentiment detection is also challenging because the same sentiment word may have different polarity according to the domain @xcite . there are two main approaches for the monolingual approach of sarcasm and sentiment detection in arabic: corpus-based and lexicon-based @xcite . then combination of these two can be referred to as the hybrid approach. the corpus-based method, typically trains sentiment classifiers. several supervised learning algorithms (e.g. svm) @xcite have been used to classify the sentiment label into positive or negative. these algorithms require handcrafted features including part-of-speech (pos) tags and social media-driven features. recently, deep learning techniques such as recurrent neural network (rnn) @xcite and convolutional neural network (cnn) @xcite emancipate researchers from feature engineering. the lexiconbased method commonly determines the sentiment or polarity of opinion by evaluating the sentiment words in the document or the sentence which is used when the data is unlabelled. it often uses predefined dictionaries of annotated sentiment terms to label each word in the document by its sentiment @xcite . then lexicon scores are used as input features to the classifier and thus plays an important role in the hybrid approach @xcite . however, more recent word embedding techniques, such as fasttext, elmo, bert, are yet to be fully explored for sarcasm and sentiment detection in arabic despite having pretrained arabic vesrions of them publicly available like emlo-formanylangs @xcite , @xcite . therefore, in this paper, we introduce our system for sarcasm and sentiment detection in arabic, which leverages multiple plms with several training strategies. there are two main steps for our system, (i) fine-tuning two kinds of our experiments, merging plms in total, indicate that our method successfully utilizes weaker plms as well as high-performing plms. as a result, our system ranks third and second for the subtask 1 and 2 of sarcasm and sentiment detection in arabic, wanlp-2021 @xcite . ## data in this paper, we use the dataset called arsarcasm-v2 @xcite . this is an arabic sarcasm detection dataset based on several other arabic sentiment analysis datasets including arsarcasm (abu farha and magdy, 2020), semeval's 2017 @xcite and astd @xcite . for the annotation process, the annotators were asked to provide three labels for each tweet including sarcasm (sarcastic or nonsarcastic), sentiment (positive, negative or neutral) and dialect (egyptian, gulf, levantine, maghrebi or modern standard arabic). each tweet was annotated by at least three different annotators. arsarcasm-v2 contains 12,548 training samples with annotations and 3,000 testing samples without annotations. since there is no official validation set, we use 7-fold cross-validation on the training dataset for performance estimation of our model. ## results and discussion evaluation metrics as mentioned in the evaluation procedure of wanlp-2021 sarcasm task, the f-score of the sarcastic class is the official metric for subtask 1 and f-pn (marco average of the fscore of the positive and negative classes) is the official one for subtask 2. plms with training strategies as shown in ## official submission we submitted the scores predicted by the ensemble method introduced above. the official ranking have been released and the top five teams are pre-sented in ## conclusion in this paper, we propose a top performing model for the task of sarcasm and sentiment detection. we fine-tune two kinds of pre-trained language models including xlm-r and arabert with different training strategies contains task-adaptive pretraining and knowledge distillation. then we stack them with a simple linear regression model in subtask 1 and a svm model in subtask 2. experimental results show the validity of this ensemble method and we rank third and second for subtask 1 and 2.