G et S mart MSEC at S em E val-2022 Task 6: Sarcasm Detection using Contextual Word Embedding with G aussian model for Irony Type Identification (2022)

Abstract: Sarcasm refers to the use of words that have different literal and intended meanings. It represents the usage of words that are opposite of what is literally said, especially in order to insult, mock, criticise or irritate someone. These types of statements may be funny or amusing to others but may hurt or annoy the person towards whom it is intended. Identification of sarcastic phrases from social media posts finds its application in different domains like sentiment analysis, opinion mining, author profiling, and harassment detection. We have proposed a model for the shared task iSarcasmEval - Intended Sarcasm Detection in English and Arabic (CITATION) by SemEval-2022 considering the language English based on ELmo embeddings for Subtasks A and C and TF-IDF vectors and Gaussian Naive bayes classifier for Subtask B. The proposed model resulted in a F1 score 0.2012 for sarcastic texts in Subtask A, macro-F1 score of 0.0387 and 0.2794 for Subtasks B and C respectively.

## introduction in the internet era, specifying user comments, views and opinions through social media has become very common and these may not be specified directly and can include indirect phrases with implicit meanings @xcite . the posts may also represent undesirable characteristics using positive words and they may not be formal in nature. it is common to use abbreviations, uncommon, ambiguous and multilingual words in social media posts and no predefined structures are explicitly defined for sarcastic messages @xcite . so the identification of sarcastic posts cannot be considered as a direct process. exhaustive understanding of the contextual meaning of the posts is considered as an important factor in identifying sarcastic posts. sarcasm detection is considered as a challenging task associated with sentiment analysis. sarcasm detection also plays an important role in analysing the voice of the customer based on which major decisions will be taken. sarcasm requires some shared knowledge between speaker and audience and it is considered as a profoundly contextual phenomenon. using sarcastic phrases in sentences is a common way of ironic or satirical speech for the common man. that being said, social media platforms, such as twitter, facebook and youtube contain millions of tweets and comments that include sarcastic phrases, thus making it a field of study under the domain of nlp. for example, a user can change a supposedly negative comment using positive words as in the sentence, "it is awesome to go to bed at 3 am #not". in such cases it becomes important to ensure that the right sentiment is drawn out of the sentence through proper analysis @xcite . the shared task isarcasmeval was part of se-meval 2022 @xcite and there were three subtasks associated with it. two of the subtasks -subtask a and subtask b, were conducted for both english and arabic languages, while subtask c only pertains to english. we as a team participated in all the three subtasks associated with the language english. ## data set the data set that we used to implement sarcasm detection was the training and the test dataset that was provided by the organisers of the shared task. each instance of the training dataset had the following informations attached to it: ## system description the proposed methodology uses elmo embedding based convolutional neural network model for implementing the subtasks a and c with an embedding layer followed by two dense layers. the subtask b has been implemented using tf-idf based gaussian naive bayes classifier. it stands for embeddings from language models and is a novel way to represent words in vectors or embeddings @xcite . in elmo the syntax and semantics of the word and the linguistic context associated with it are modelled as a deep contextualised word representation. huge text corpus had been used to pretrain the model and is constructed using deep bidirectional models foot_0 . it is implemented with 4096 units and the input embedding transform using 2048 convolutional filters. the context of the word usage forms the base for the word representation. elmo word representations take the entire input sentence into equation for calculating the word embeddings. the architecture of elmo model 2 is shown in figure 1 . ## results the metrics that were considered for the evaluation of all the three subtasks were precision, recall, accuracy and macro-f1 score. precision represents the ratio of the number of correct positive results to the number of positive results predicted by the classifier. the recall measures the model's ability to detect positive samples. the higher the recall, the more positive samples are detected. classification accuracy is the ratio of number of correct predictions to the total number of input samples. f1 score is an overall measure of a model's accuracy that combines precision and recall. a high f1 score means that the classification has resulted with low number of false positives and low false negatives. the proposed model resulted in an f1 score for sarcastic texts as 0.2012 based on which subtask a was evaluated and we were ranked 36 on the leaderboard. accuracy was the metric that was used for the evaluation of subtask c, and our model achieved an accuracy of 0.34 with rank 15 on the leaderboard. the f1 score that we obtained for this subtask was 0.2794 and these are tabulated in ## conclusions sarcasm detection has become an important area of research as it is interlinked with different areas of application that includes sentiment analysis, opinion mining, offensive and hate speech detection. having this in mind semeval-2022 had come up with the task of sarcasm detection which was represented by three subtasks namely sarcasm detection, identifying the type of irony associated with the sarcastic text and identifying whether the text or its rephrase is sarcastic. we have applied elmo embedding based convolutional neural network model for implementing the binary subtasks a and c. gaussian naive bayes classifier based on tf-idf vectors was used to implement subtask b. all the three subtasks were implemented considering the language english. the performance of the models used, were not up to the mark and it is found from the task overview that the transformer models when applied over this tasks provides better results. dataset for sarcasm detection could be created with contextual information which can help in ef-fectively detecting sarcasm. usage of hybrid approaches where different machine learning and deep learning models are combined can also facilitate efficient detection of sarcasm from text. often it could be observed that sarcasm is not in the text, but could be detected from the intonation or facial expression, which has made mulitmodal sarcasm detection also as a promising research area.