A lex U - AL at S em E val-2022 Task 6: Detecting Sarcasm in A rabic Text Using Deep Learning Techniques (2022)

Abstract: Sarcasm detection is an important task in Natural Language Understanding. Sarcasm is a form of verbal irony that occurs when there is a discrepancy between the literal and intended meanings of an expression. In this paper, we use the tweets of the Arabic dataset provided by SemEval-2022 task 6 to train deep learning classifiers to solve the sub-tasks A and C associated with the dataset. Sub-task A is to determine if the tweet is sarcastic or not. For sub-task C, given a sarcastic text and its non-sarcastic rephrase, i.e. two texts that convey the same meaning, determine which is the sarcastic one. In our solution, we utilize fine-tuned MARBERT (Abdul-Mageed et al., 2021) model with an added single linear layer on top for classification. The proposed solution achieved 0.5076 F1-sarcastic in Arabic sub-task A, accuracy of 0.7450 and F-score of 0.7442 in Arabic sub-task C. We achieved the 2^{nd} and the 9^{th} places for Arabic sub-tasks A and C respectively.

## introduction sarcasm is ubiquitous phenomenon on the social web, and is difficult to be analysed automatically and manually by humans because of its nature. sarcasm data can be very confusing to computer systems which use it to perform tasks such as sentiment analysis, opinion mining, author profiling, and harassment detection @xcite @xcite . @xcite show that the sentiment polarity classification performance on nonsarcastic tweets is much better than on sarcastic ones, in the context of semeval. sentiment polarity classification is used widely in industry, driving marketing, administration, and investment @xcite . so it is important to create models for sarcasm detection. a comparatively small dataset is a challenge we faced when working on the arabic dataset that makes it difficult to train complex models. we used transfer learning to treat this issue. by using a transfer learning, a pre-trained model for some task on a large dataset can be used as a starting point in another task which improves the performance. it is used in a wide range of natural language processing (nlp) tasks. word embeddings such as word2vec @xcite , fasttext @xcite and glove @xcite can be used to initialize vectors learnt form large dataset. recently, bidirectional encoder representations from transformers (bert) @xcite became the most popular nlp approach to transfer learning. google ai language team pretrained bert model and fine-tuned it for a large range of tasks, such as question answering and language inference where it achieved state-of-the-art performance. mar-bert is built using the same network architecture as bert base @xcite , without the next sentence prediction (nsp). marbert is trained on a large twitter dataset. therefore, we utilize fine-tuned marbert to solve this challenge. this paper is organized as follows. in section 2, we discuss relevant related works in sarcasm detection. we describe our proposed system in section 3. the models implementation details are explained in section 4. section 5 describes the dataset for the shared task. then we report and analyze the evaluation results in section 6. finally, we provide our conclusions in section 7. ## related work a weak supervision and manual labelling methods can be used for the annotation process. a weak supervision method is to consider the texts sarcastic, if they meet predefined criteria, like including specific tags (e.g. #sarcasm, #irony) @xcite . as pointed out by @xcite , noisy labels can be induced by this labelling method. manual labelling is collecting texts and presenting them to human annotators for labelling @xcite @xcite . this can lead to a problem when annotator perception differs from author intention, as further outlined by @xcite . compared to english, only a few studies have been made on arabic sarcasm detection. among the studies completed in this area are research by @xcite @xcite @xcite @xcite . @xcite show the effect of sociocultural variables on sarcasm communication online, which makes the performance of models trained on english unpredictable, if they are trained on other languages. @xcite @xcite relay on the two labelling methods mentioned above. recently, efforts have been made by @xcite @xcite to create standard datasets to support sarcasm detection. in the semeval-2022 workshop, a shared task ('isarcasmeval: intended sarcasm detection in english and arabic') @xcite was organised to contribute to the development of this area using a new labelling method that avoids the limitations of previous labelling methods. ## proposed system semeval 2022 task 6 focuses on detecting sarcastic tweets. the task supports arabic and english languages and we tackle the problem on arabic language. this task has three sub-tasks. ## data description as mentioned above, annotator perception may differ from author intention. to overcome this problem, the authors annotated the data themselves, which is a new method to collect data introduced by the task's organisers. arabic and english datasets are collected using this method. for each sarcastic text, they provide a non-sarcastic rephrase to convey the same intended message, for english and arabic datasets. eventually, for english dataset, linguistic experts label each tweet to one of the ironic speech categories outlined by @xcite : sarcasm, irony, satire, understatement, overstatement, and rhetorical question. the dialect label of the text is included for the arabic dataset. ## implementation we trained the proposed solution model using the given arabic dataset. we divided its training data into 80% for training, 10% for validation and 10% for testing. in this section, we discuss the details of the different deep learning models we built or fine-tuned 1 . for our solution, we fix the tweets length to 64 by truncating longer tweets and padding shorter ones. this length is selected as the max value of the arabic training tweets lengths after tokenization. after that each token in the input tweet is replaced with its vector representation obtained from a pretrained word embeddings model. we used pretrained marbert to initialize the words embeddings but these representations are then updated during the training of the deep learning models. the huggingface 2 pytorch implementation includes a set of interfaces designed for a variety of nlp tasks. though these interfaces are all built on top of a trained bert models, each has different top layers and output types designed to accomodate their specific nlp task. we used bertforsequenceclassification which is the normal marbert model with an added single linear layer on top for classification that we used as a sentence classifier. ## experiment results in this section we report and discuss the results of the proposed solution when evaluated on our testing data. moreover, we show the results of our submission to semeval 2022 task 6 on arabic data. our model ranked the second out of 32 participants for sub-task a and the 9 th out of 13 participants for sub-task c, semeval 2022 task 6 (isar-casmeval: intended sarcasm detection in english and arabic). ## conclusion we used the fine-tuned marbert model in our submissions to semeval 2022 task 6. we participated in the a and c sub-tasks for sarcasm detection in arabic tweets. our proposed approach is ranked the 2 nd and the 9 th in sub-tasks a and c, respectively. for future work, we explore the impact of building deeper neural networks with multiple convolutions or recurrent layers applied sequentially on the input text.