Sarcasm Detection in Tweets with BERT and G lo V e Embeddings (2020)

Abstract: Sarcasm is a form of communication in which the person states opposite of what he actually means. In this paper, we propose using machine learning techniques with BERT and GloVe embeddings to detect sarcasm in tweets. The dataset is preprocessed before extracting the embeddings. The proposed model also uses all of the context provided in the dataset to which the user is reacting along with his actual response.

## introduction sarcasm is defined as a sharp, bitter or cutting expression or remark and is sometimes ironic @xcite . to identify if a sentence is sarcastic, it requires analyzing the speaker's intentions. different kinds of sarcasm exist like propositional, embedded, like-prefixed and illocutionary @xcite . among these, propositional requires the use of context. the most common formulation of sarcasm detection is a classification task @xcite . our task is to determine whether a given sentence is sarcastic or not. sarcasm detection approaches are broadly classified into three types @xcite . they are: rule based, deep learning based and statistical based. rule based detectors are simple, they just look for negative response in a positive context and vice versa. it can be done using sentiment analysis. deep learning based approaches use deep learning to extract features and the extracted features are fed into a classifier to get the result. statistical approach use features related to the text like unigrams, bigrams etc and are fed to svm classifier. in this paper, we use bert embeddings @xcite and glove embeddings @xcite as features. they are used for getting vector representation of words. these embeddings are trained with a machine learning algorithm. before extracting the embeddings, the dataset also needs to be processed to enhance the quality of the data supplied to the model. ## literature review there have been many methods for sarcasm detection. we discuss some of them in this section. under rule based approaches, @xcite use hashtag sentiment to identify sarcasm. the disagreement of the sentiment expressed by the hashtag with the rest of the tweet is a clear indication of sarcasm. @xcite identify sarcasm in similes using google searches to determine how likely a simile is. @xcite look for a positive verb and a negative situation phrase in a sentence to detect sarcasm. in statistical sarcasm detection, we use features related to the text to be classified. most approaches use bag-of-words as features @xcite . some other features used in other papers include sarcastic patterns and punctuations @xcite , user mentions, emoticons, unigrams, sentiment-lexicon-based features @xcite , ambiguity-based, semantic relatedness @xcite , n-grams, emotion marks, intensifiers @xcite , unigrams @xcite , bigrams @xcite , word shape, pointedness @xcite , etc. most work in statistical sarcasm detection relies on different forms of support vector machines(svms) @xcite . @xcite uses naive bayes and decision trees for multiple pairs of labels among irony, humor, politics and education. for conversational data,sequence labeling algorithms perform better than classification algorithms @xcite . they use svm-hmm and searn as the sequence labeling algorithms @xcite . for a long time, nlp was mainly based on statistical analysis, but machine learning algorithms have now taken over this domain of research providing unbeaten results. dr. pushpak bhattacharyya, a well-known researcher in this field, refers to this as "nlp-ml marriage". some approaches use similarity between word embeddings as features for sarcasm detection. they augment these word embedding-based features with features from their prior works. the inclusion of past features is key because they observe that using the new features alone does not suffice for an excellent performance. some of the approaches show a considerable boost in results while using deep learning algorithms over the standard classifiers. @xcite use a combination of cnn, rnn, and a deep neural network. another approach uses a combination of deep learning and classifiers. it uses deep learning(cnn) to extract features and the extracted features are fed into the svm classifier to detect sarcasm. ## dataset we used the twitter dataset provided by the hosts of shared task on sarcasm detection. initial analysis reveals that this is a perfectly balanced dataset having 5000 entries. there are an equal number of sarcastic and non-sarcastic entries in it. it includes the fields label, response and context. the label specifies whether the entry is sarcastic or non-sarcastic, response is the statement over which sarcasm needs to be detected and context is a list of statements which specify the context of the particular response. the test dataset has 1800 entries with fields id(an identifier), context and response. most of the time, raw data is not complete and it cannot be sent for processing(applying models). here, preprocessing the dataset makes it suitable to apply analysis on. this is an extremely important phase as the final results are completely dependent on the quality of the data supplied to the model. however great the implementation or design of the model is, the dataset is going to be the distinguishing factor between obtaining excellent results or not. steps followed during the preprocessing phase are: @xcite • check out for null values -presence of null values in the dataset leads to inaccurate predictions. there are two approaches to handle this: both training and test data are preprocessed with the above methods. once the above preprocessing steps have been applied we are ready to move to the model development. ## methodology in this section, we describe the methods we used to build the model for the sarcasm detection. ## results the result was measured using the metric fmeasure. f-measure is a measure of a test's ac- ## conclusion sarcasm detection can be done effectively using word embeddings. they are extremely useful as they capture the meaning of a word in a vector representation. even though bert gives contextual classifier f-measure linear support vector classifier 0.679 logistic regression 0.690