T ech SSN at S em E val-2022 Task 6: Intended Sarcasm Detection using Transformer Models (2022)

Abstract: Irony detection in the social media is an upcoming research which places a main role in sentiment analysis and offensive language identification. Sarcasm is one form of irony that is used to provide intended comments against realism. This paper describes a method to detect intended sarcasm in text (SemEval-2022 Task 6). The TECHSSN team used Bidirectional Encoder Representations from Transformers (BERT) models and its variants to classify the text as sarcastic or non-sarcastic in English and Arabic languages. The data is preprocessed and fed to the model for training. The transformer models learn the weights during the training phase from the given dataset and predicts the output class labels for the unseen test data.

## introduction sarcasm is a form of verbal irony that occurs when there is a discrepancy between the literal and intended meanings of an utterance. this is often used to express the opposite meaning of the words spoken. this is used frequently while making fun of someone or something, and is used in a variety of contexts, like casual conversation, memes, or even public speaking, to convey a variety of meanings, providing a certain level of depth and sophistication to the communication of the language. sarcasm is present in all overcontemporary social media networks and may reduce the efficiency of systems that perform operations on these sarcastic data such as sentiment analysis, opinion mining, author profiling, and harassment detection @xcite @xcite . it generates misleading conclusions, due to its nature to imply different meaning than what is intended on the surface. even in semeval, @xcite shows that there is a significant drop in system performance when processing sarcastic text data, in comparison to non-sarcastic data. these systems are used in industry, driving marketing, administration, and investment decisions @xcite . this clearly shows that developing models to find and detect sarcasm is becoming more important by the day. the isarcasmeval task 6 for semeval 2022 @xcite is comprised of three sub-tasks: to classify the input text as sarcastic or not, in english (subtaska english) and arabic (sub-task a arabic), and further classify sarcastic text into categories (subtask b), and given two phrases with same meaning, identify the sarcastic one (sub-task c english, subtask c arabic). of these, the techssn team has attempted to solve subtaska english, subtask a arabic, and subtask b. ## related work a lot of the previous sarcasm detection datasets have been annotated using a weak supervision method. in weak supervision, text data is classified as sarcastic only if it meets a certain set of conditions that are decided upon prior to the collection and analysis of the data. this includes using tags (e.g. #sarcasm, #irony) @xcite to perform the above mentioned classification. however, this can result in noisy labels for many reasons, as demonstrated by @xcite . other work makes use of manual labelling, where sarcasm labels are provided by human annotators @xcite @xcite . but, this can mean that labels are subjective in nature, i.e. labels may reflect annotator perception, which may differ from the meaning intended by the author, as pointed out by @xcite . moreover, a lot of sarcasm detection work applies only to the english language and, because of the socio-cultural aspects of sarcastic communication @xcite , it is doubtful that the models trained to detect sarcasm in the english language could do the same task with the same effectiveness on other languages such as arabic (where most of the sarcasm detection is carried out using the above-mentioned weak supervision). we have participated for irony and sarcasm detection semeval task in @xcite and used multilayer perceptron model to find the ironic and sarcastic tweets. we have used cnn, rnn, lstm, bert and colbert models for offensive language detection in earlier semeval workshop tasks @xcite , (sivanaiah et al., 2020), @xcite in which bert models provides better results than other machine learning and deep learning models. ## results and discussions the test dataset was provided by the semeval-2022 organizers and was given to different models for each subtask and the results are listed in tables 1 to 3 . colbert model gives better accuracy for the english language than arabic. for subtask b we have used multilabel classifier to predict the output. figure 2: pretraining and fine tuning bert -source:(devlin et al., 2018) models f1-score accuracy bert 0.2558 0.2936 colbert 0.2637 0.7407 ## conclusion it is obvious that the detection of sarcasm and its various categories is important as it is pivotal in computations like sentiment analysis, opinion mining, author profiling, or harassment checking. this can become increasingly difficult and tedious to compute, as sarcasm is extremely subjective as its nature itself is implying and contradictory, and the amount of data to be analyzed is getting larger and complex by the day. it is also a big step for natural language processing (nlp) as it can tremendously help in the creation of more sophisticated virtual models f1-macro roberta 0.0596 semeval-2022 task 6 is comprised of three sub-tasks of which the techssn team has participated in subtask a english, subtask a arabic and subtask b. deep learning models like bert, robert, and colbert were used to carry out the tasks successfully. the team was able to obtain the 27th rank in task a -english, 29th rank in task a -arabic, and 17th rank in task b. results show that bert based models perform better on average than other conventional models like logistic regression decision tree, support vector machines etc. because of bert's multilingual nature, sarcasm detection can be carried out in other languages across the globe. we would like to investigate further and apply these models to other languages. the accu-racy could potentially be improved by using more advanced and efficient pre-processing techniques.