ISD at S em E val-2022 Task 6: Sarcasm Detection Using Lightweight Models (2022)

Abstract: A robust comprehension of sarcasm detection iscritical for creating artificial systems that can ef-fectively perform sentiment analysis in writtentext. In this work, we investigate AI approachesto identifying whether a text is sarcastic or notas part of SemEval-2022 Task 6. We focus oncreating systems for Task A, where we experi-ment with lightweight statistical classificationapproaches trained on both GloVe features andmanually-selected features. Additionally, weinvestigate fine-tuning the transformer modelBERT. Our final system for Task A is an Ex-treme Gradient Boosting Classifier trained onmanually-engineered features. Our final sys-tem achieved an F1-score of 0.2403 on SubtaskA and was ranked 32 of 43.

## introduction sarcasm is the use of irony-which communicates the opposite of what is said-to humorous and derisive effect @xcite . on the web, sarcasm is ubiquitous-not least because social media users often apply sarcasm to incorporate a sardonic sense into their statements @xcite . this poses a substantial challenge to artificial systems evaluating tasks including sentiment analysis @xcite . it is already a challenge enough for human annotators to determine what is intended to be taken at face-value or not in context-lacking text; it is even more difficult for nlp systems to distinguish between what should be taken literally and what is sarcastic. task 6 of @xcite ) provides an environment to build systems to approach these challenges. in particular, subtask a in task 6 ( ## dataset in subtask a, we train our models on the official semeval-2022 task 6 english training set, which was curated from a set of tweets. each sentence (examples in ## conclusion our models were trained to determine whether texts were sarcastic or not. for the most part, our models struggled to detect sarcasm in text--as was expected, given that the task was quite challenging even for humans. we find that the models that achieve the highest degree of success in detecting sarcasm were gaussiannb and xgbclassifier models. we also find that using manual features, as listed in finally, fine-tuning bert achieves reasonable results while detecting sarcasm. however, this method is still inferior to a lightweight approach. overall, our best model, the xgbclassifier with manually engineered features, did not perform significantly better than the logistic regression model. our results demonstrate that boosting algorithms can predict sarcasm in text to a moderate degree of success.