I 2 C at S em E val-2022 Task 6: Intended Sarcasm Detection on Social Networks with Deep Learning (2022)

Abstract: In this paper we present our approach and system description on iSarcasmEval: a SemEval task for intended sarcasm detection on social networks. This derives from our participation in SubTask A: Given a text, determine whether it is sarcastic or non-sarcastic. In our approach to complete the task, a comparison of several machine learning and deep learning algorithms using two datasets was conducted. The model which obtained the highest values of F1-score was a BERT-base-cased model. With this one, an F1-score of 0.2451 for the sarcastic class in the evaluation process was achieved. Finally, our team reached the 30th position.

## introduction sarcasm is a form of mockery intended to imply the opposite or to express displeasure. it can be classified as irony, satire, understatement, overstatement, rhetorical question and sarcasm itself. social media platforms allow people to express themselves and speak about several different topics via text, emojis and multimedia. many companies collect information about people's opinions to aid in marketing decisionmaking about their products. sarcasm detection is relevant in data analysis because it can be wrongly interpreted due to its nature, becoming potentially harmful to different computational systems. for example, if a computer system understands the sentence "the only thing i got from college is a caffeine addiction", its literal meaning would not be understood, as it refers to how stressful college can be. defining a phrase as sarcastic is a really challenging task to solve for the text mining community because there are many details that need to be considered such as the writer's personality and the sentence context. for instance, depending on the ideology of the person, "trump, that respectful man" could be sarcastic or not. in isarcasmeval: task 6, subtask a (abu farha et al. 2022) participants had to decide if a particular tweet includes a sarcastic connotation or not. ## background two sample datasets were used to train the model. the first one was the original dataset without any preprocessing. this dataset is composed of 3467 rows, with 2600 0-value rows (nonsarcastic) and 867 1-value rows (sarcastic). the second sample dataset (tweet -rephrase) consists of a phrase and rephrase contrast in which the aim was to adequately learn the differences between a phrase with a sarcastic connotation and that same sentence expressing the same thing but in a literal way. this dataset is composed of 1734 rows: 867 1-value rows and 867 0-value rows, where these last ones are rephrases related to each tweet taken as a 1-value row. so, these two datasets follow the same structure: in the second dataset, the column "rephrase" was added to the list of tweets. the rephrase example for the tweet below would be: the issue of sarcasm detection has been addressed by various authors in recent years. in the last several years, researchers have been working on a technique to analyze social media data in order to identify possible undisclosed information so it can be useful to assess new patterns and make important decisions through it @xcite . sentiment analysis is also used for sarcasm detection @xcite , dividing a sentence into phrases and judging the situation and the sentiment separately. in @xcite , the authors present a review of methodologies and techniques of sarcasm detection. they concluded that deep learning is the most common technique to identify sarcasm. ## system overview in this section, the models used to solve the sarcasm detection issue will be described. due to the nature of the problem, a decision was made not to carry out any preprocessing. the reason is that we think that sarcasm interpretation requires the original phrase to be considered in its entirety, so letter cases, emojis and punctuation were kept. ## experimental setup many libraries were imported to develop the experiments. some of them are: "numpy" @xcite )", a fundamental package for scientific computing; "spacy" @xcite , a library for advanced natural language processing; "nltk" @xcite , a tool-kit to work with human language information; "keras" @xcite , a neuralnetwork library; "scikit-learn", which contains simple and efficient tools for predictive data analysis and "pandas" (mckinney 2010), used for manipulating data. for all the experiments, the datasets were split using 80% to train and 20% to test. we use a stratify approach. finally, when the test dataset was released, the whole train dataset was used to train the final model through bert-base-cased, which was the best algorithm to complete the task. the metrics obtained to evaluate our model are accuracy and f1-score. ## results results shown in the leaderboard with the test dataset yielded a 0.2451 f1-score. finally, our team reached the 30 th position. ## conclusions in this paper our approach to solve task 6 (isarcasmeval) -subtask a: given a text, determine whether it is sarcastic or non-sarcastic, in english, has been described. the main idea was to check models of deep learning algorithms such as bert or lstm neural network trained with the dataset tweet -rephrase. after training and analyzing each model and comparing deep learning with machine learning techniques, bert-base-cased model obtained higher results f1-score and accuracy. in the future, a bert model will be trained using a larger corpus and more innovative techniques such as the application of sentiment analysis in broken-down sentences.