TUG - CIC at S em E val-2021 Task 6: Two-stage Fine-tuning for Intended Sarcasm Detection (2022)

Abstract: We present our systems and findings for the iSarcasmEval: Intended Sarcasm Detection In English and Arabic at SEMEVAL 2022. Specifically we take part in Subtask A for the English language. The task aims to determine whether a text from social media (a tweet) is sarcastic or not. We model the problem using knowledge sources, a pre-trained language model on sentiment/emotion data and a dataset focused on intended sarcasm. Our submission ranked third place among 43 teams. In addition, we show a brief error analysis of our best model to investigate challenging examples for detecting sarcasm.

## introduction according to the freedictionary 1 , sarcasm is a cutting statement to express contempt or ridicule, often using words to convey a meaning that is the opposite of their literal or actual meaning. due to its ambiguous nature, sarcasm plays an important role for resolution of several nlp tasks, such as sentiment analysis @xcite @xcite , hate speech @xcite , disagreement classification @xcite , opinion mining @xcite ) among others. one example of benefits of modeling sarcasm is presented by @xcite about the use of sarcastic tweets to improve sentiment analysis. in this work we present our submission to isar-casmeval at semeval-2022 for subtask a (given a text, determine whether it is sarcastic or nonsarcastic). our solution system is at the top three teams among 43 teams. the proposed approach uses the pre-trained language model well informed of this task, because it uses sentiment/emotion data 1 https://www.thefreedictionary.com/ and we enhanced it with a dataset about intended sarcasm texts. the rest of the document goes as follows: section 2 overviews some related works about sarcasm detection tasks. in section 3, we give a detailed description of our conducted experiments, including the dataset used. section 4 summarizes our results and offers an interpretation to our findings. finally, section 5 presents our conclusion, contributions and future work. ## related works beside the importance of detecting sarcasm for industry applications related to understanding comments on social networks or commercial products reviews @xcite , sarcasm detection has received great attention from the nlp community, influencing the creation of diverse approaches, benchmark datasets and evaluation methods. the methods for modeling sarcasm range from rule-based system and statistical approaches by exploiting handcrafted features using traditional machine learning @xcite to modern techniques using deep learning architectures together with word embeddings and pretrained language models @xcite @xcite . previous attempts for benchmarking sarcasm detection were proposed by "semeval-2018 task 3: irony detection" 2 , alta shared task 2019 3 , figlang 2020 4 and wanlp 2021 5 . they crafted their datasets either using weak supervision tech-niques like scraping tweets having the #irony, #sarcasm hashtags, or by manual labeling from third party annotators, which leads to several shortcomings as exposed in oprea and magdy (2020), instead the dataset used in this work (isarcasm) was crafted by asking the authors themselves to provide the sarcastic/non-sarcastic labeling for their tweets. ## experiments our experiments aim to determine whether a given text is intended to be sarcastic or non-sarcastic. the metric for ranking systems on the competition is the f1-score for the sarcastic class (positive label) only. ## conclusion we present our systems for the task of sarcasm detection using a variety of knowledge sources that explore the capabilities of pre-trained language models to understand pragmatic phenomena such as sarcasm. we found that although these knowledge sources help to shed light over the sarcasm detection task, still more external knowledge would be required to correctly classify difficult cases that require a deeper understanding of real world context. based on our analysis, we plan to further examine the impact of including say emoji modelling to measure its influence, especially over cases that show a contradiction on expressed sentiments.