A ra BERT and Farasa Segmentation Based Approach For Sarcasm and Sentiment Detection in A rabic Tweets (2021)

Abstract: This paper presents our strategy to tackle the EACL WANLP-2021 Shared Task 2: Sarcasm and Sentiment Detection. One of the subtasks aims at developing a system that identifies whether a given Arabic tweet is sarcastic in nature or not, while the other aims to identify the sentiment of the Arabic tweet. We approach the task in two steps. The first step involves pre processing the provided dataset by performing insertions, deletions and segmentation operations on various parts of the text. The second step involves experimenting with multiple variants of two transformer based models, AraELECTRA and AraBERT. Our final approach was ranked seventh and fourth in the Sarcasm and Sentiment Detection subtasks respectively.

## introduction during the last two decades, work on subjective language processing has been very common in literature. the work on sentiment analysis was a major theme that was pursued during this time. sentiment analysis is a process, according to @xcite , in which we extract out and examine the emotional stance in a particular piece of text. with the introduction of user-driven networks such as websites for social media, the work on sentiment analysis flourished. most of this work was based on english, while not much attention was gathered by the arabic language. the work on arabic sentiment analysis was initiated by @xcite , but as compared to english it still needs development. this can be due to the many complexities pertaining to the language, including the broad range of dialects @xcite and the complicated language morphology @xcite . some of the prominent problems while tackling the task of sentiment analysis are domain dependency, negation handling, lack of sarcasm and knowledge @xcite . sarcasm is described as a form of verbal irony designed to convey disdain or ridicule @xcite . there has been a lot of work on the identification of english sarcasm, including datasets, like the works of @xcite @xcite @xcite and detection systems like @xcite @xcite . work on arabic sarcasm is, to our knowledge, restricted to the work of @xcite , an irony detection task @xcite , and sarcasm datasets by @xcite . this paper puts forth the approach we applied to handle the wanlp-2021 shared task 2. the paper is ordered in the following manner: the problem statement, along with details of the arsarcasm-v2 dataset are presented in section 2. the methodology that we propose as our solution is described in section 3. the experiments which were carried out, dataset statistics, system settings and results of the experiments are provided in section 4. the paper ends with a brief section that talks about the conclusion and future directions of our research, in section 5. ## task definition the wanlp-2021 shared task 2 @xcite is based on a text classification problem, based on identifying sentiment and sarcasm in arabic tweets. the provided training and test datasets have a total of 12,548 and 3,000 tweets respectively. the shared task is divided into two subtasks: subtask 1 (sarcasm detection): the aim is to identify whether a tweet is sarcastic or not. given a tweet, the task is to return true if there is sarcasm present in the tweet and false otherwise. this is a binary classification problem. precision/recall/fscore/accuracy are the evaluation metrics, where f-score of the sarcastic class is the official metric of evaluation. subtask 2 (sentiment analysis): the aim is to identify the sentiment of a tweet by assigning one of three labels (positive, negative, neutral). given a tweet, the task is to return pos if the tweet has a positive sentiment, neg if the tweet has a negative sentiment or neu if the tweet has a neutral sentiment. this is a multiclass classification problem. precision/recall/f-score/accuracy are the evaluation metrics, where macro average of the f-score of the positive and negative classes (f-pn) is the official metric of evaluation. ## methodology this section describes the process we employed to tackle the task. the process is divided into two steps: data preprocessing and transformer based models application. the first step involves processing the provided arsarcasm-v2 dataset to convert it into a format better processed by the models. the second step involves experimenting with different models to decide which model performs the best for the arsarcasm-v2 dataset. details about these steps have been provided in the following sub-sections. ## experiments we split the provided training dataset in a 90:10 ratio to create our training and validation splits. this is followed by experimenting with eight transformer based models, for which performance on the validation split is compared. the pre trained models are fine tuned on the training split and metrics for the validation split are calculated. the final test set predictions are made from the model that performs the best on the validation split, among the eight models. this section includes the dataset distribution, system settings, results and a brief analysis of the system, for both the subtasks: sarcasm and sentiment detection. ## conclusion and future work in this paper, we present our strategy to approach the eacl wanlp-2021 shared task 2. we tackle the task in two steps. in the first step, the arsarcasm-v2 dataset is pre-processed by altering different parts of text. this is followed by running experiments with multiple variants of two transformer based models pre-trained on arabic text, araelectra and arabert. the final submissions for the tasks of sarcasm and sentiment detection are based on that variant of model which performs the best. our approach fetches a private leaderboard rank 7 and 4 in the sarcasm and sentiment detection tasks respectively. as future scope, we plan to explore other features which may be relevant for this task, and inculcate ensemble learning taking into consideration both word vector based and transformer based embeddings.