LT 3 at S em E val-2022 Task 6: Fuzzy-Rough Nearest Neighbor Classification for Sarcasm Detection (2022)

Abstract: This paper describes the approach developed by the LT3 team in the Intended Sarcasm Detection task at SemEval-2022 Task 6. We considered the binary classification subtask A for English data. The presented system is based on the fuzzy-rough nearest neighbor classification method using various text embedding techniques. Our solution reached 9th place in the official leader-board for English subtask A.

## introduction sarcasm (or irony) can be defined as a trope or figurative language use whose actual meaning is different from what is literally enunciated @xcite . the task of sarcasm detection can be connected with various challenges in the natural language processing (nlp) field, from sentiment analysis to hate speech detection. however, this task is more complicated by its nature. even for a human, sarcasm detection could be a challenging issue. it can be represented in different shapes, with voice, gestures, mimic, etc. so, text alone may not be sufficient to detect whether a given utterance is sarcastic or not. it makes the labeling of such datasets quite complicated @xcite . the semeval competition is an annual event that provides a set of challenges for researchers in different aspects of the nlp field. this year we participated in semeval-2022 task 6 called "isar-casmeval" that considers sarcasm detection in two languages: english and arabic @xcite . we tackled subtask a for english, where for a given text, we should determine whether it is sarcastic. as described by the authors of the dataset, text writers provided the labels by themselves to exclude subjective labeling. the dataset contains text, its binary label for subtask a (sarcastic or not), a non-ironical rephrase of the provided text with an explanation of why is it sarcastic for subtask c (we did not use it in our experiments), and a set of binary classes for six types of sarcasm (irony, satire, rhetorical question, etc.) for subtask b. we note that the size of the non-irony class for subtask a is three times bigger than the size of the irony class (2,601 instances and 867 instances). a task related to sarcasm detection is emotion detection issue, which we considered in our previous papers. in @xcite and @xcite , we addressed the intensity task provided by semeval-2018 task 1 @xcite . our first paper used the weighted k nearest neighbor (wknn) classification approach with corresponding text cleaning and embedding steps. in the second paper, we tuned the preprocessing steps and, instead of wknn, we considered the fuzzy-rough nearest neighbor (frnn) classification model with ordered weighted average (owa) operators ( @xcite , @xcite , @xcite ). the final approach has the shape of an ensemble of frnn models based on several strong embedding techniques. the solution described in this paper is based on the methodology presented in @xcite , but it is fine-tuned for the isarcas-meval dataset and task. our code is provided at the github repository foot_0 . the remainder of this paper has the following parts: in section 2 we provide a step-by-step description of our system, including text preprocessing and the used embedding methods, the classification model and its ensemble, and the evaluation metric. in section 3 we provide results obtained by cross-validation, and identify the best setup ap-plied on the test data. it also provides an error analysis of the output and post-competition experiments for model improvement. section 4 presents conclusions and future steps. ## system description figure 1 gives an overview of our full method, which encompasses the following steps: we embed the raw text with the twitter-roberta-base for irony detection model @xcite , then we use the output vectors in the owa-frnn classification method to obtain a prediction of classes. ## results this section consists of several parts. in subsection 3.1 we provide an overview of our experiments, using cross-validation on the train data to identify the best model for irony detection and evaluate its result for test data. in subsection 3.2, we describe error analysis and illustrate the explainability of our approach. we also performed additional experiments after the competition was finished and labels for the test data released to improve our scores and present this process in subsection 3.3. ## conclusion & future work in this paper, we presented our model for the is-arcasmeval competition. our solution uses the instance-based classification method owa-frnn and a roberta-based model for irony recognition as an embedding method. we fine-tuned the best setup on the train data with cross-validation and obtained the ninth place on the test data in the competition leader-board. our approach is explainable in a way that we can trace back the test instance and find the training instances that determined the predicted class to explore some patterns. for example, we observe the significance of the tweet topic and even of particular keywords. in the future, the provided solution may be improved by additional text preprocessing techniques or roberta-based model fine-tuning using additional datasets.