Multi-task Learning Using a Combination of Contextualised and Static Word Embeddings for A rabic Sarcasm Detection and Sentiment Analysis (2021)

Abstract: Sarcasm detection and sentiment analysis are important tasks in Natural Language Understanding. Sarcasm is a type of expression where the sentiment polarity is flipped by an interfering factor. In this study, we exploited this relationship to enhance both tasks by proposing a multi-task learning approach using a combination of static and contextualised embeddings. Our proposed system achieved the best result in the sarcasm detection subtask.

## introduction social media platforms, such as twitter, provide the public with an opportunity to share their thoughts and feelings with others through short posts of text. this has created a rich area of research to examine for linguistic expressions used on social media and explore expressions of emotion @xcite . 'sarcasm' is defined as a figurative type of language where the expression is meant to communicate the opposite of its literal meaning. within sa, detecting sarcasm is vital as sarcasm typically implies a negative feeling although the positive language is used. this conflict between what is said and what is meant creates a complicated challenge for sa tasks @xcite . compared to english, only a few studies have been made on arabic sarcasm detection. among the studies completed in this area are research by @xcite and a shared task study by @xcite . recently, efforts have been made by @xcite and @xcite to create standard datasets to support sarcasm detection. in the wanlp-2021 workshop, a shared task ('sarcasm and sentiment detection in arabic') @xcite was organised to contribute to the development of this area. two sub-tasks are included in the shared task. the first will be to identify whether a tweet is sarcastic or not, and the second will be to determine if the tweet is expressing a neutral, positive or negative feeling. some substantial difficulties are presented by the shared task. specifically, the targeted classes are distributed in an unbalanced way and the tweets used are in various dialects and even sub-dialects. arabic dialects can be significantly different and they lack the established language rules found in modern standard arabic (msa). to deal with these difficulties, we propose a combined approach that uses three models. by using this combination, we believe we will be able to benefit from the strengths of each model and overcome their weaknesses. the remainder of this paper is organised as follows. section 2 comprises an overview of the methodology for our proposed system. section 3 describes the dataset for the shared task. section 4 provides experimental results, including the experimental set-up though which the proposed system was tested and a discussion of the outcomes. finally, section 5 concludes the paper with suggestions for future research. ## method as described above, our proposed methodology consists of three models. these are the static word embeddings (swe) , contextualized embeddings (marbert) and multi-task learning (mtl) models. the following three subsections explain each of these and how our proposed system incorporates them. ## data description the shared task's organisers released the train and test datasets (arsarcasm-v2) for both tasks, namely, subtask 1 -sarcasm detection and subtask 2 -sentiment analysis @xcite . the train dataset contained 12,548 tweets while the test dataset consisted of 3,000 tweets. for subtask 1, the tweets were manually annotated using the term 'true' for sarcastic tweets and 'false' for tweets that were not sarcastic. for subtask 2, the tweets were designated as 'neg' for negative class, 'neu' for neutral and 'pos' for positive. an overview of the dataset is provided in tables 2 and 3 . training 2168 10380 12 , 548 test 821 2179 3000 ## conclusions in this work, we described an mtl approach using a combination of static and contextualised word embeddings. the proposed system achieved the best result in sarcasm detection subtask and ranked 11th in sa task. in future studies, we will investigate the impact of sarcasm on sa task to enhance the performance of our proposed system for sa task. additionally, advanced pre-possessing techniques should be applied as we observed some multi-words need to be segmented and treated. also, instead of removing all punctuation symbols, some of these marks (e.g. question marks and exclamation marks) may be useful especially for the sarcasm detection task.