JCT at S em E val-2022 Task 6-A: Sarcasm Detection in Tweets Written in E nglish and A rabic using Preprocessing Methods and Word N-grams (2022)

Abstract: In this paper, we describe our submissions to SemEval-2022 contest. We tackled subtask 6-A - “iSarcasmEval: Intended Sarcasm Detection In English and Arabic – Binary Classification”. We developed different models for two languages: English and Arabic. We applied 4 supervised machine learning methods, 6 preprocessing methods for English and 3 for Arabic, and 3 oversampling methods. Our best submitted model for the English test dataset was a SVC model that balanced the dataset using SMOTE and removed stop words. For the Arabic test dataset our best submitted model was a SVC model that preprocessed removed longation.

## introduction the rapid development of various types of social networks allows the development of increasingly offensive language in general and sarcasm in particular. sarcasm is the use of words that mean the opposite of what you want to say especially to insult someone, show irritation, or be funny foot_0 . sarcastic language can harm individuals or groups of people and may cause harmful effects on society. thus, it is important to develop highquality systems capable of detecting sarcastic expressions automatically. sarcasm detection is a difficult task not only for a computer but even for a human being. highquality performance of this task requires understanding the context of the situation, the relevant culture, and in some cases the specific issue or people involved in this situation @xcite . moreover, the noisy nature of social media texts especially twitter messages make the detection task even harder. therefore, it is an interesting and challenging task to detect sarcasm using supervised machine learning (ml) methods and natural language processing (nlp) tools. furthermore, @xcite show a significant drop in sentiment polarity classification performance when processing sarcastic tweets, compared to non-sarcastic ones. in this paper, we describe our research and participation in subtask 6-a for sarcasm detection in tweets written in two languages: english and arabic. the full description of task 6 in general and 6-a in particular including the datasets and the participating teams is given in abu farha et al. @xcite . the structure of the rest of the paper is as follows. section 2 introduces a background concerning sarcasm detection, text preprocessing, and text classification with imbalanced classes. section 3 describes subtask 6-a and its datasets. in section 4, we present the submitted models and their experimental results. section 5 summarizes and suggests ideas for future research. ## related research most prior textual sarcasm detection datasets have been annotated by using either manual labeling or a weak supervision method. in the first approach, sarcasm labels are provided by human annotators (e.g., @xcite @xcite . however, such labels represent annotator perception, which may differ from the author intention, as further pointed out by @xcite . in the second approach, texts are labeled as sarcastic if they meet predefined criteria, e.g., including specific tags (e.g. #irony, #sarcastic, #sarcasm) @xcite . however, this method can lead to noisy labels because of various reasons @xcite . to overcome these disadvantages, @xcite introduced another method. in their method, the sarcasm labels for texts are provided by the authors themselves. most of the sarcasm detection studies are in the english language @xcite @xcite @xcite @xcite @xcite . there are also some studies in arabic @xcite @xcite . for more information about various issues concerning sarcasm detection please refer to survey papers such as @xcite , sarsam et al. (2020), @xcite , and moores and mago (2022). ## task and datasets description tables 1-4 present various statistical details about the training and test sets for english and arabic. the analysis of the details presented in tables 1 and 2 show that the training and test sets for english are highly imbalanced, with a ratio of about 25:75 and 14:86 (non-sarcastic:sarcastic), respectively. we tried to balance the dataset in our experiments using the oversampling methods that we have mentioned above. the analysis of the details presented in tables 3 and 4 show a similar picture. the training and test sets for arabic are highly imbalanced, with a ratio of about 39:61 and 14:86 (nonsarcastic:sarcastic), respectively. also, for arabic, we tried to balance the dataset in our experiments using the oversampling methods that we have mentioned above. both for english and arabic, the test datasets are even more imbalanced than the compatible training datasets. ## summary and future research in this paper, we describe our models and submissions to subtask 6-a of semeval-2022: sarcasm detection in twitter messages written in english and arabic using preprocessing methods and word n-grams. we applied 4 supervised ml methods, 6 preprocessing methods for english and 3 for arabic, and 3 oversampling methods. our best submitted model for the english test dataset was an svc model that balanced the dataset using smote and removed stop words. for the arabic test dataset, our best submitted model was an svc model that preprocessed longation removal. there are various ideas for future research that are connected to the nature of twitter messages as follows: (1) the use of skip character ngrams because they serve as generalized ngrams that allow us to overcome problems such as noise and sparse data @xcite , which are common to twitter messages and (2) many twitter messages contain acronyms. acronym disambiguation might enable better classification @xcite . another idea that may lead to better classification is to use additional feature sets such as stylistic feature sets @xcite .