DUCS at S em E val-2022 Task 6: Exploring Emojis and Sentiments for Sarcasm Detection (2022)

Abstract: This paper describes the participation of team DUCS at SemEval 2022 Task 6: iSarcasmEval - Intended Sarcasm Detection in English and Arabic. Team DUCS participated in SubTask A of iSarcasmEval which was to determine if the given English text was sarcastic or not. In this work, emojis were utilized to capture how they contributed to the sarcastic nature of a text. It is observed that emojis can augment or reverse the polarity of a given statement. Thus sentiment polarities and intensities of emojis, as well as those of text, were computed to determine sarcasm. Use of capitalization, word repetition, and use of punctuation marks like '!' were factored in as sentiment intensifiers. An NLP augmenter was used to tackle the imbalanced nature of the sarcasm dataset. Several architectures comprising of various ML and DL classifiers, and transformer models like BERT and Multimodal BERT were experimented with. It was observed that Multimodal BERT outperformed other architectures tested and achieved an F1-score of 30.71%. The key takeaway of this study was that sarcastic texts are usually positive sentences. In general emojis with positive polarity are used more than those with negative polarities in sarcastic texts.

## introduction according to the collins dictionary foot_0 "sarcasm is speech or writing which actually means the opposite of what it seems to say". fox @xcite report that sarcasm is challenging to identify, even for humans. people often use gestures like rolling of eyes or heavy tonal stress to express sarcasm @xcite , in speech or in-person communication. intonation and stress in speech, too, are strong indicators of sarcasm @xcite . also, the context in the form of shared knowledge between the speaker and the audience can be leveraged to detect sarcasm @xcite . but most of the time the tone or the context is missing in text data, especially tweet data. some users attempt to use capitalization like "oh yeah" or repetitions like "woowww" to indicate tonal intensity. but, most of the time these cues are not enough, especially in absence of context to gauge sarcasm. this makes sarcasm detection a challenging task. in this work, team ducs participated in semeval 2022 task 6 : isarcasmeval subtask a, @xcite , to determine if the given text is sarcastic or not. the english text training dataset contains 3468 sentences. each text is labeled as sarcastic or non-sarcastic by the text-author. of the 3468 sentences 867 are sarcastic and 2601 are non-sarcastic. in the dataset corresponding to a sarcastic text, a rephrased non-sarcastic version has also been provided. these sarcastic texts are labeled as one of sarcasm, irony, satire, understatement, overstatement, and rhetorical question. these ironic speech categories are determined by a linguist expert. the target task is to determine whether a text is sarcastic or not on the test dataset which contains 1400 sentences. for sarcasm detection in the above-discussed dataset, an attempt was made to derive cues from the text itself. it is observed that in real scenarios, the absence of labels by users like #not or #sarcasm or any other contextual information makes sarcasm detection challenging @xcite . thus, in this work, the authors attempted to use the information available in the text itself. emoji occurrences in the text are captured to study their impact in discerning sarcasm. sentiment intensity and polarity of both the emojis and the text were computed to train the classifiers for sarcasm classification. this paper is organized as follows. section 2 discusses related work in sarcasm detection. in section 3, system overview, the proposed approach to incorporate polarity and intensity of emojis along with text sentiment polarity and intensity is discussed. section 4, describes the experimental setup and the data augmentation approach used for the given dataset. the results of the experiments conducted are reported in section 5. in the last section conclusion, the key takeaways and learnings are discussed. ## related work the sarcastic nature of the text can deceive classifiers as well as humans, given it conveys the opposite of what it means. many researchers have attempted to use text sentiment and emojis for sarcasm detection. @xcite use word and emoji embeddings simultaneously to train deep learning models with attention layer for sarcasm detection. pamungkas and patti (2018) utilize structural features and affective features of tweets for irony detection. structural features include the presence of hashtags, links, emojis, quotes, etc. the counts of mentions, exclamations, upper-case, intensifiers, links, along with the counts of verbs, nouns, and adjectives are also used along with the use various affective resources to capture sentiment polarity and emotions. @xcite used an ensemble approach with lstm for representing emojis and hashtags, cnn-lstm for representation of cases, stop-words, punctuation, and sentiments. mlp was trained with facebook's infersent embeddings @xcite while svm was trained with emotion and stylometric features. a decision tree with adaboost served as the base estimator for the ensemble. this ensemble was used to predict sarcasm on the conversational text where context was available. sundararajan and palanisamy (2020) propose a rule-based classifier with an ensemble of 20 different features. they observed that the features based on sentiment can predict sarcasm better in combination with contradictory features. in most work so far researchers have tried to capture sentiment and linguistic features to identify sarcasm. a few researchers have attempted to include emojis in sarcasm classification. but, how the usage of emojis with text impacts or conveys sarcasm is yet to be explored in depth. it needs to be studied how frequent is sentiment incongruence of emojis in the text they are used with and can the polarity of emojis and text help deduce sarcasm. hence in this work, the authors focus on these features -emoji sentiment and text sentiment as well as their polarities to understand how this combination impacts sarcasm. this approach is discussed in the next section. ## system overview in the given dataset, situational context is not available. although linguistic markers are provided, they may not be available in real scenarios, eg when the users post on social media. in the absence of context, it is important to take into account other factors that may point to sarcasm. for instance, the surface sentiment of a text is often used in many sarcasm classifiers @xcite . @xcite , report that most often sarcasm is used with an apparently positive statement to produce a negative impact. it is noted that the users may use intensity in the form of capitalization, repeated letters, etc. to express sarcasm @xcite . in this work, to identify sarcasm, the sentiment polarity of the text and intensity in the form of sentiment score was incorporated for the sarcasm classification task. intensity as a feature is used as it may help capture intonation or stress a user may want to express. it has been reported that features like emojis can augment as well as alter the sentiment polarity @xcite . @xcite observed that emoji incongruity with the text may help improve the irony classification. thus, this work investigates whether in sarcastic texts users use in-congruent emojis with the text to express sarcasm. and whether the use of emojis in the text helps in detecting sarcasm? ## experimental setup for sarcasm detection, this work employs various machine and deep learning classifiers. the proposed set up is also tested with transformer models like bert and multimodal bert. first, the text is cleaned to remove stop words, newlines, and spaces. the cleaned text is then converted to lower case and then tokenized to prepare for loading the word embeddings. pre-trained glove embeddings, @xcite of 100 dimensions are used to represent the tokenized text. 80% of the dataset was used for training while the remaining 20% was used for validation. data augmentation this dataset was highly imbalanced with 867 sarcastic and 2601 nonsarcastic texts, i.e. (3:1 ratio for non-sarcastic: sarcastic texts). so the dataset was augmented to improve this ratio of sarcastic and non-sarcastic texts. the sarcastic texts were oversampled with nlpaug python library foot_3 for text augmentation. the contextual bert embeddings were used to synthesize new texts from existing texts in the dataset. two different augmented datasets were created. in the first augmented dataset (aug1 training set), each sarcastic text was synthesized once to create one more text. in this set, the ratio of non-sarcastic: sarcastic texts was 2:1. in the next augmented dataset (aug2 training set), each sarcastic text was synthesized to create two more texts, thus resulting in the ratio of 1:1 for non-sarcastic to sarcastic texts. for example, if the tweet was, "see brexit is going well". then in the first augmented text (aug1 training set) the original tweet (mentioned above) was retained and a synthesized tweet in the form "can see brexit negotiations is going extremely well" was added to the dataset. this was done for every sarcastic tweet, making the ratio of non-sarcastic:sarcastic texts as 2:1. in the aug2 training set) the number of synthesized texts was increased to two. so two synthesized tweets apart from the original tweet were appended to the dataset making the ratio of non-sarcastic:sarcastic as 1:1. the computed sentiment polarity, text sentiment score, emoji sentiment polarity, and emoji sentiment score were replicated from the original texts in the synthesized texts. ## conclusion in this work, the authors took their first steps in understanding how the sentiment of emojis alongwith the text sentiment helps in detecting sarcasm. since sarcasm is difficult to detect without context, authors attempted to uncover information implicit in the text. for this the intensity and polarity of both the available emojis and the text itself were used to identify sarcasm. it was observed that sarcastic texts generally have positive polarity. sarcastic texts employed emojis with positive polarities more than the negative emojis. this information can be used further to improve results of sarcasm classification. the given dataset had relatively fewer texts with emojis, thus, more work is required to fully capitalize on the proposed approach. this motivates the authors to explore more sarcasm datasets with emojis to exhaustively study the impact of emojis in identifying sarcasm.