Applying Transformers and Aspect-based Sentiment Analysis approaches on Sarcasm Detection (2020)

Abstract: Sarcasm is a type of figurative language broadly adopted in social media and daily conversations. The sarcasm can ultimately alter the meaning of the sentence, which makes the opinion analysis process error-prone. In this paper, we propose to employ bidirectional encoder representations transformers (BERT), and aspect-based sentiment analysis approaches in order to extract the relation between context dialogue sequence and response and determine whether or not the response is sarcastic. The best performing method of ours obtains an F1 score of 0.73 on the Twitter dataset and 0.734 over the Reddit dataset at the second workshop on figurative language processing Shared Task 2020.

## introduction we are living in the age of social media. many consider it as a revolution. social media creates a variety of new possibilities; for instance, today, people can express their thought with just a tap of a finger. in the twitter platform, people are twitting around 500 million tweets per day, and it is estimated that over 2.8 million comments are posted on the reddit every single day. this vast amount of data present an enormous opportunity for businesses, and researchers alogn with a significant number of challenges. many companies and researchers have been interested in these data to investigate the opinion, emotions, and other aspects of them. the usage of informal language and noisy content within social media presents many difficulties toward the opinion and emotion analysis problems. one of the main challenges in this criteria is the appearance of figurative language such as sarcasm. the sarcasm can alter the meaning of the sentence ultimately, and consequently, make the opinion analysis process error-prone. for instance, criticism may use positive words to convey a negative message. in recent years there was a growing trend to address the sarcasm detection problem among natural language processing (nlp) researchers. many approaches tackle the sarcasm detection problem by considering contextual information, instead of using utterance solely. for instance, bamman and smith ( 2015 the sarcasm detection shared task foot_0 is aimed to detect sarcasm based on the conversation context. given the current utterance and conversation history, the models are expected to decide if the utterance is sarcastic or not. we test our models on the dataset from both twitter and reddit. both utterance and the conversation history have been used as input. we applied the transformer-based model and adopted aspect-based sentiment analysis approaches to address the problem. the remnant of this paper is organized as follows: section 2 reviews related work. section 3 describes the datasets. section 4 explains our methodology. section 5 shows the results of each dataset in detail. lastly, section 6 provides conclusions and our plans for future research. ## related works there have been several attempts to solve the sarcasm detection problem with rule-based approaches. @xcite presented two rulebased classifiers for two different types of tweet structure, the first one used to detect sarcasm in tweets that have a contradiction between negative sentiment and positive situation. the second classifier applied to tweets that start with interjection words. the former classifier applied parsed-based lexicon generation to identify phrases that display sentiment, and indicate the sarcastic label whenever a negative phrase occurs in a positive sentence. the latest classifier used interjections and intensifiers that occur together in order to predict sarcasm. maynard and greenwood (2014) suggests that hashtag sentiment is an essential symbol of sarcasm, and authors often used hashtags to emphasize sarcasm. they propose the tweet is sarcastic whenever the hashtags' sentiments do not agree with the rest of the tweet. besides the requirement for in-depth knowledge of the domain and much manual work, rule-based methods are usually not the best performers in terms of prediction quality. because of the high cost of the rule-based methods, many researchers put there focus on machine learning approaches. different types of models and features have been adopted to tackle this problem. @xcite addressed the problem in both supervised and unsupervised settings. they utilized naïve bayes as a classifier and c-means clustering, which is one of the most widely used fuzzy clustering algorithms. @xcite adopted a sequence labeling techniques (svm-hmm and searn) and indicated that sequence labeling algorithms outperform the classification algorithms in conversational data. with the development of computational hardware and deep learning in recent years, many deep learning methods have been proposed to address the sarcasm detection problem. @xcite proposed a convolutional neural network-based architecture that jointly learns and exploits embeddings for the users' content and utterances. ghosh and veale (2016) used the complication of the convolutional neural network and recurrent neural network. they used two layers of convolutional neural network, followed by two layers of long short-term memory(lstm). the output of lstm layers fed to a fully connected neural network in order to produce a higher-order feature set. @xcite proposed a novel multi-dimension question answering network in order to detect sarcasm. they utilized conversation context information. a deep memory network based on bilstm and attention mechanisms have been adopted to extract the factors of sarcasm. ## dataset two corpora were used in sarcasm detection shared task, which both of them are balanced. the twitter corpus consists of 5000 data samples for the train and 1800 for the test set. on the other hand, reddit corpus contains 4400 data samples for the train and 1800 for the test set. training datasets have four columns: ## methodology in this section we describe models and technquies that we used to address sarcasm detection problem. ## results on twitter corpus, the performance of nbsvm as a simple model is quite impressive. as long as the data is from social media and might contain informal and misspelling content using character n-gram tfidt matrix can yield excellent performance. as features, we used different combinations of 'response' column, 'context' column. however, taking the 'response' column as a feature solely produced the best result. we also test models with and without preprocessing steps. however, adding the preprocessing step did not show a significate change in the results. as expected, bert achieved the second-best position on the scoreboard. we used a different set of features, but again using the 'response' column solely scored the best among others. furthermore, lcf-bert, which is an aspect-based sentiment classification method, scored the best on the twitter dataset because aspect-based sentiment classification methods consider input data as two different sections and try to learn them interactively. the complete results with more details are shown in unlike our experience on the twitter dataset, nbsvm did not perform well on the reddit dataset. it appears that the reddit dataset is more complicated and challenging than twitter. however, using an additional dataset, around 1 million data points, boosted the nbsvm result around 7 percent. for nbsvm, we used the same feature as it was used for the twitter dataset. bert performance was the best on this dataset, regardless of additional data. for bert-svm and bert-lr, we only utilized the 'response' column as input. moreover, for xl-net, we used the 'response' column with 100,000 random data points from the additional dataset. furthermore, for the aspect-based sentiment analysis models, we used the last dialogue 'context' column and 'response' column as our input. the complete results with more details are shown in ## conclusion our proposed methods ranked 5 out of 37 groups for the reddit dataset and ranked 25 out of 36 for the twitter dataset. this result shows the strength of the bert pre-trained model on sarcasm detection and its combination with aspect-based sentiment analysis models, which take data as two separate parts and learn them interactively. also, additional data can improve performance slightly better. it is noteworthy to mention that nbsvm performance as a simple baseline with the tfidf matrix with character n-gram was quite impressive. for future work, a combination of contextual and character-based embedding could lead to better performance. moreover, since social media content usually contains misspelling and informal data, more complicated preprocessing techniques like social media content normalization might be more helpful than proposed techniques.