R eactive S upervision: A N ew M ethod for C ollecting S arcasm D ata (2020)

Abstract: Sarcasm detection is an important task in affective computing, requiring large amounts of labeled data. We introduce reactive supervision, a novel data collection method that utilizes the dynamics of online conversations to overcome the limitations of existing data collection techniques. We use the new method to create and release a first-of-its-kind large dataset of tweets with sarcasm perspective labels and new contextual features. The dataset is expected to advance sarcasm detection research. Our method can be adapted to other affective computing domains, thus opening up new research opportunities.

## introduction sarcasm is ubiquitous in human conversations. as a form of insincere speech, the intent behind a sarcastic utterance is integral to its meaning. perceiving a sarcastic utterance as genuine will often result in a complete reversal of the intended meaning, and vice versa @xcite . it is therefore crucial for affective computing systems and tasks, such as sentiment analysis and dialogue systems, to automatically detect sarcasm from the perspective of the author as well as the reader in order to avoid misunderstandings. @xcite recently pioneered the study of intended sarcasm (by the author) vs. perceived sarcasm (by the reader) in the context of sarcasm detection tasks. the training of models for these tasks requires large amounts of labeled sarcasm data, with twitter becoming a major source due to its popularity as a social network as well as the huge amounts of conversational text its users generate. previous works describe three methods for collecting sarcasm data: distant supervision, manual annotation, and manual collection. distant supervision automatically collects "inthe-wild" sarcastic tweets by leveraging author- * corresponding author: shmueli@iis.sinica.edu.tw generated labels such as the #sarcasm hashtag @xcite . this method generates large amounts of data at low cost, but labels are often noisy and biased @xcite . to improve quality, manual annotation asks humans to label given tweets as sarcastic or not. since finding sarcasm in a large corpus is "a needle-in-ahaystack problem" @xcite , manual annotation can be combined with distant supervision @xcite . still, low inter-annotator reliability is often reported @xcite , resulting not only from the subjective nature of sarcasm but also the lack of cultural context @xcite . moreover, neither method collects both sarcasm perspectives: distant supervision collects intended sarcasm, while manual annotation can only collect perceived sarcasm. lastly, in manual collection, humans are asked to gather and report sarcastic texts, either their own @xcite or by others @xcite . however, both manual methods are slower and more expensive than distant supervision, resulting in smaller datasets. to overcome the above limitations, we propose reactive supervision, a novel conversation-based method that offers automated, high-volume, "inthe-wild" collection of high-quality intended and perceived sarcasm data. we use our method to create and release the spirs sarcasm dataset foot_0 . ## reactive supervision reactive supervision exploits the frequent use in online conversations of a cue tweet -a reply that highlights sarcasm in a prior tweet. figure 1 ( left panel) shows a typical exchange on twitter: c posts a sarcastic tweet. unaware of c's sarcastic intent, b replies with an oblivious tweet. lastly, a person example cue regular expression example author sequences 1st i was only being sarcastic lol ˆa[ˆa] * (a)[ˆa] * @xmath0 ab, aba, abaa 3rd she was just being sarcastic! ˆaa * b[ab] * (c)[ab] * @xmath1 2nd ˆ(a)a * (b)(a * )@xmath2