Sarcasm Detection using Context Separators in Online Discourse (2020)

Abstract: Sarcasm is an intricate form of speech, where meaning is conveyed implicitly. Being a convoluted form of expression, detecting sarcasm is an assiduous problem. The difficulty in recognition of sarcasm has many pitfalls, including misunderstandings in everyday communications, which leads us to an increasing focus on automated sarcasm detection. In the second edition of the Figurative Language Processing (FigLang 2020) workshop, the shared task of sarcasm detection released two datasets, containing responses along with their context sampled from Twitter and Reddit. In this work, we use RoBERTa_{large} to detect sarcasm in both the datasets. We further assert the importance of context in improving the performance of contextual word embedding based models by using three different types of inputs - Response-only, Context-Response, and Context-Response (Separated). We show that our proposed architecture performs competitively for both the datasets. We also show that the addition of a separation token between context and target response results in an improvement of 5.13% in the F1-score in the Reddit dataset.

## introduction sarcasm is a sophisticated form of speech, in which the surface meaning differs from the implied sense. this form of expression implicitly conveys the message making it hard to detect sarcasm in a statement. since speech in sarcasm is dependent in context, it is tough to resolve the speaker's intentions unless given insights into the circumstances of the sarcastic response. these insights or contextual information may include the speaker of the response, the listener of the response, and how its content relates to the preceding discourse. both authors contributed equally to the work. recognizing sarcasm is critical for understanding the actual sentiment and meaning of the discourse. the difficulty in the recognition of sarcasm causes misunderstandings in everyday communication. this difficulty also poses problems to many natural language processing systems, including summarization systems and dialogue systems. therefore, it is essential to develop automated sarcasm detectors to help understand the implicit meaning of a sarcastic response. the sarcasm detection shared task held at the second edition of the figurative language processing (figlang 2020) workshop proposes two datasets from different social media discourse platforms for evaluation. the first dataset contains user conversations from twitter, while the second dataset contains reddit conversation threads. both datasets contain contextual information in the form of posts of the previous dialogue turns. their primary aim is to understand the importance of conversational contextual information in improving the detection of sarcasm in a given response. in this work, we explore the use of contextualized word embeddings for detecting sarcasm in the responses sampled from reddit as well as twitter. we outline the effect of adding contextual information, from previous dialogue turns, to the response, for both the datasets. we further explore the importance of separation tokens, differentiating discourse context from the target response while detecting sarcasm. the use of other turns of dialogue as contextual information has been explored well in previous works. @xcite investigated the use of "author and addressee" features apart from the conversation context, but observed only a minimal impact of modeling conversation context. @xcite investigated using "pre" and "post" messages from debate forums and twitter conversations to identify whether rhetorical questions are used sarcastically or not. they observe that using the "post" message as context improved the f1-score for the sarcastic class. @xcite showed that sequence labeling algorithms outperform traditional classical statistical methods, obtaining a gain of 4% in f1-score on their proposed dataset. the use of pre-trained contextual word representations has been explored in multiple tasks in nlp, including text classification. recently released models such as bert @xcite and roberta @xcite , exploit the use of pre-training and bidirectional transformers to enable efficient solutions obtaining state-of-the-art performance. pre-trained embeddings significantly outperform the previous state-of-the-art in similar problems such as humor detection @xcite , and subjectivity detection @xcite . ## approach this section outlines the approach used to detect sarcasm in the reddit and twitter datasets. our approach utilizes contextualized word embeddings with three different inputs, as explained in the following paragraphs. the use of pretrained contextualized word embeddings has been applied to achieve state-of-theart results for various downstream tasks @xcite . @xcite proposed bert that leverages context from both left and right representations in each layer of the bidirectional transformer. the model is pretrained and released in the public domain, and can be trained in a simpler, yet efficient manner without having to make significant architectural changes for a specific task. roberta @xcite ) is a replication study of bert, trained on a dataset twelve times larger with bigger batches as compared to bert. roberta makes use of larger byte-pair encoding(bpe) vocabulary that helps to achieve better results than bert on various downstream tasks. we use robert a large and finetune it for the task using varying hyperparameters for different inputs. we use the following three different types of input to study the effect of context on the performance of roberta large to detect sarcasm: ## conclusion this work presents the importance of context while detecting sarcasm from responses sampled from twitter and reddit. our proposed architecture using three different inputs performs competitively for both datasets showing that the addition of contextual information to target response improves the performance of the fine-tuned contextual word embeddings for detecting sarcasm. we further show that the addition of a separation token between context and target response also performs com-petitively, markedly showing an improvement of 5.13% in the f1-score of reddit dataset. future works include exploring different contextual cues, including user-specific attribute information and extending this hypothesis to other figurative speeches like irony detection and humor detection.