Sarcasm and Sentiment Detection In A rabic Tweets Using BERT -based Models and Data Augmentation (2021)

Abstract: In this paper, we describe our efforts on the shared task of sarcasm and sentiment detection in Arabic (Abu Farha et al., 2021). The shared task consists of two sub-tasks: Sarcasm Detection (Subtask 1) and Sentiment Analysis (Subtask 2). Our experiments were based on fine-tuning seven BERT-based models with data augmentation to solve the imbalanced data problem. For both tasks, the MARBERT BERT-based model with data augmentation outperformed other models with an increase of the F-score by 15% for both tasks which shows the effectiveness of our approach.

## introduction sarcasm is a form of figurative language, where the speaker expresses his/her thoughts in a sarcastic way. the process of sarcasm detection relies on understanding people's true sentiments and opinions. application of sarcasm detection can be beneficial in several nlp applications, such as marketing research, opinion mining and information categorization. in recent years, sarcasm detection has received considerable attention in the nlp community @xcite . different approaches were used for sarcasm detection; early approaches for sarcasm detection used feature-based machine learning models @xcite . recently, deep learning methods have been applied for this task @xcite . for a comprehensive survey on sarcasm and irony detection see @xcite . however, work on arabic sarcasm detection is still in its early stages with very few works. there are few efforts on arabic sarcasm detection such as the works of @xcite ; @xcite and the recent efforts to build standard datasets for sarcasm detection such as @xcite and @xcite . the current small size of the shared task labeled data-set and its imbalance nature makes it extremely difficult to build effective detection systems. also, the context of the current sarcasm tweets does not have enough information to decide on its state which indeed makes the tasks more challenging. in this paper, we describe the system submitted for the shared task on sarcasm detection and sentiment analysis in arabic. we approached this challenge first by experimenting with different classical machine learning classifiers such as support vector machines (svms), xgboost, random forest that are trained on tf-idf features. then, we experimented with different deep neural networks (dnns) along with character and wordlevel features. finally, we conducted experiments on several bert-based models such as marbert and qarib. we took the most promising bert-based models results for subtask 1 and subtask 2 on the training set, which was marbert for both of them and tested it with a new augmented dataset. the rest of the paper is organized as follows: section 2, describes the dataset. in section 3, we describe our approach in tackling the problem. section 4 provides and discusses the results of subtask 1 and subtask 2. and in section 5, we provide a conclusion of our work. ## dataset the arsarcasm-v2 dataset @xcite released for both shared tasks by the competition organizers is the same containing 12,548 training tweets. in addition to 3000 tweets for testing. the dataset was annotated for sarcasm detection task (subtask 1) with the label "true" for sarcastic tweets and "false" for not sarcastic tweets. for the second shared task (subtask 2) on sentiment analysis the labels are (neu, neg or pos) for neutral, negative or positive, respectively. ## system this section provides a description of the different data preprocessing steps, models we used in the experiments, and our data augmentation approach. ## results and discussion in this section, we present and analyse the results of our experiments for subtask 1 and 2. ## conclusion in this paper we experimented with seven bert-based models and we augmented the shared task data set to identify the sentiment of a tweet or detect if a tweet is sarcasm. we achieved promising results for sarcasm detection and sentiment identification by marbert model with data augmentation. our error analysis indicates certain types of errors in the dataset and in the annotation that can be addressed in the future. last but not least, if we had more time to work on this shared task, we will build a lexicon for sarcasm, try other approaches for data augmentation and also revise the annotation in the dataset, since we found several tweets were misannotated.