How effective is incongruity? Implications for code-mixed sarcasm detection (2021)

Abstract: The presence of sarcasm in conversational systems and social media like chatbots, Facebook, Twitter, etc. poses several challenges for downstream NLP tasks. This is attributed to the fact that the intended meaning of a sarcastic text is contrary to what is expressed. Further, the use of code-mix language to express sarcasm is increasing day by day. Current NLP techniques for code-mix data have limited success due to the use of different lexicon, syntax, and scarcity of labeled corpora. To solve the joint problem of code-mixing and sarcasm detection, we propose the idea of capturing incongruity through sub-word level embeddings learned via fastText. Empirical results show that our proposed model achieves an F1-score on code-mix Hinglish dataset comparable to pretrained multilingual models while training 10x faster and using a lower memory footprint.

## introduction sarcasm is defined as a sharp remark whose intended meaning is different from what it looks like. for example, "i am not insulting you. i am describing you." could mean that the speaker is insulting the audience, but the receiver does not get it. sarcasm usually involves ambivalence (also known as incongruity which means words/phrases having contradictory implications @xcite and difficult to comprehend. though english is used as a way to communicate and exchange messages, majority of the people still use the mother language to express themselves on social media @xcite . according to one study @xcite , more than 50% posts on twitter are written in a language other than english. code-switching (also known as codemixing) is a writing style in which the author uses words from different languages either in the same * work done while the author was an intern at iit indore sentence (called intra-sentential) or different sentences (called inter-sentential) switching. an example of code-switching is:"he said kal karte hai kaam"" (gloss: he said tomorrow we'll do the work). the studies such as @xcite show that people use code-switch language when trying to convey comicality, satire or humor. motivated by previous studies, we plan to detect sarcasm in code-mix languages. though many studies exist to detect sarcasm in unimodal and multimodal data @xcite @xcite @xcite , methods to detect sarcasm in code-mix data are limited and have not been explored much. @xcite @xcite . it is due to several challenges such as ambiguous words, variable lexical representation, word-level code-mixing, reduplication, and wordorder. to solve some of these issues, we present a deep-learning based architecture to capture incongruity in code-mix data. our proposed model achieves competitive performance as compared to pre-trained multilingual models (fine-tuned on the code-mix sarcasm detection task) with significantly fewer parameters and faster training time. our contributions are as follows: ## model architecture code-mix language contains noisy words mixed with different languages and this might lead to out-of-@xmath0. so, we use fasttext skipgram @xcite for learning subword level representation from the code-mix data. we hypothesise that subword level representation is able to handle ambiguous words, variable lexical representation, and word-level code-mixing. for example, the ambiguous word "to" may be present in both hindi and english language. learning subword representation alleviates the problem of encoding the word "to" differently for both the languages. further, variable length words like {"gharr", "gharrr", "gharrrr" will be split into tokens {"gha","har", "arr","rrr" } and uniquely represented using only these subwords tokens. code-mix words like "chapless" (mix of bengali "chap" and english "less") are also represented via sub-words "chap" and "less". the proposed model architecture is shown in fig. 1 . each sentence s is represented by its embedding @xmath1 our findings suggest that gelu activation yields better results for attention based mechanisms. note that the above formulation is a form of bilinear similarity popularly used in metric learning. to calculate the attention score α i , @xmath2, 2, . . . , n) for each word, we take the mean of each row (contrary to max of rows as in @xcite ) and apply the softmax for normalization. self-attention approach though captures the incongruity in the sentence, it misses the sentence's compositionality which is essential for sarcasm detection as suggested in @xcite . therefore, sentence embedding is passed to the bilstm encoder @xcite and the hidden states of the forward lstm and backward lstm are concatenated as