Rhetorical Device-Aware Sarcasm Detection with Counterfactual Data Augmentation (2025)

Abstract: Sarcasm is a complex form of sentiment expression widely used in human daily life. Previous work primarily defines sarcasm as a form of verbal irony, which covers only a subset of real-world sarcastic expressions. However, sarcasm serves multifaceted functions and manifests itself through various rhetorical devices, such as echoic mention, rhetorical question and hyperbole. To fully capture its complexity, this paper investigates fine-grained sarcasm classification through the lens of rhetorical devices, and introduces \textbf{RedSD} , a \textbf{R} h \textbf{E} torical \textbf{D} evice-Aware \textbf{S} arcasm \textbf{D} ataset with counterfactually augmented data.To construct the dataset, we extract sarcastic dialogues from situation comedies (i.e., sitcoms), and summarize nine rhetorical devices commonly employed in sarcasm. We then propose a rhetorical device-aware counterfactual data generation pipeline facilitated by both Large Language Models (LLMs) and human revision. Additionally, we propose duplex counterfactual augmentation that generates counterfactuals for both sarcastic and non-sarcastic dialogues, to further enhance the scale and diversity of the dataset.Experimental results on the dataset demonstrate that fine-tuned models exhibit a more balanced performance compared to zero-shot models, including GPT-3.5 and LLaMA 3.1, underscoring the importance of integrating various rhetorical devices in sarcasm detection. Our dataset is avaliable at https://github.com/qqHong73/RedSD.

## introduction sarcasm is a subtle and peculiar form of sentiment expression, often employed to criticize or ridicule a person, situation or idea. refer to the formal description of sarcasm as presented in a dictionary of modern english usage @xcite : i love to be ignored. "sarcasm does not necessarily involve irony, and irony has often no touch of sarcasm. but irony... is so often made the vehicle of sarcasm... the essence of sarcasm is the intention of giving pain by (ironical or other) bitter words." with the universal existence of sarcasm, sarcasm detection (sd) plays a vital role in tasks such as sentiment analysis, opinion mining, and hate speech detection @xcite , all of which depend on accurately capturing genuine human sentiments @xcite . however, understanding sarcasm requires commonsense knowledge and logical reasoning @xcite . additionally, in text-only settings, sarcasm detection models are particularly sensitive to the presence or absence of contextual cues @xcite . even for people, it is not always easy to identify sarcasm in a single tweet without prior conversational context @xcite , highlighting the need to construct a high-quality corpus of sarcasm in dialogue. most prior work defines sarcasm as a way of verbal irony where someone says the opposite of what they mean @xcite @xcite @xcite . consequently, automatic sarcasm detection methods predominantly focus on exploring the incongruity between positive sentiment and negative situation @xcite , or evaluating the inconsistency between the actual intention and the literal content @xcite , as illustrated in figure 1 . despite their effectiveness, a notable concern is whether current sarcasm detection models are robust enough to capture the complexity and diversity of sarcasm. recently, some researchers have argued that this narrow definition of sarcasm provides a foundation that is neither necessary nor sufficient for sarcasm to occur @xcite . meanwhile, several studies have embarked on fine-grained sarcasm detection. for example, @xcite operationalize classes of sarcasm in the form of rhetorical question and hyperbole, abu @xcite propose to further label each text into one of the categories: sarcasm, irony, satire, understatement, overstatement, and rhetorical question, and @xcite extend the mustard dataset with sarcasm types that specify the necessary information or modality for sarcasm detection. while these studies have advanced our understanding of sarcasm, they fail to encompass the full spectrum of sarcastic expressions or delve into the intrinsic nature of sarcasm itself. therefore, it is imperative to explore a more nuanced and comprehensive classification system for sarcasm. since sarcasm often manifests through various rhetorical devices that simultaneously contribute to its complex and multifaceted nature, this work explores fine-grained sarcasm classification through the lens of rhetorical devices and integrating them into sarcasm detection. specifically, we introduce redsd, a rhetorical device-aware sarcasm dataset with counterfactually augmented data. inspired by @xcite , we utilize sitcom corpus to extract sarcastic dialogues with various rhetorical devices. as shown in figure 1 , each sarcastic dialogue may involve multiple rhetorical devices. to learn more robust sarcasm detection models, we employ chatgpt (openai, 2023) to generate counterfactuals and incorporate rhetorical devices into the prompts. after automatically filtering and and human revision, we ultimately cu-rate a new sarcasm detection dataset with an equal number of sarcastic and non-sarcastic dialogues. to further enhance the scale and diversity of our dataset, we propose duplex counterfactual augmentation, which generates counterfactuals for both sarcastic and non-sarcastic dialogues. in summary, our contributions are as follows: ## methodology and dataset in this section, we present our rhetorical device-aware counterfactual sarcasm detection framework, which consists of three phases: rhetorical device-aware data collection (rddc, §3.1), counterfactual data augmentation (cda, §3.2) and duplex counterfactual augmentation (dca, §3.3), as illustrated in figure 2 . in addition, we conduct dataset analysis in §3.4. ## counterfactual quality evaluation to evaluate the quality of the counterfactual data generated by cda, we define the following quantitative metrics. 1) label-flip rate (lfr). lfr calculates the percentage of counterfactual data that flip the original label (sarcasm) to the target label (non-sarcasm). 2) plausibility. plausibility measures the logical coherence of the context and whether the content aligns with commonsense knowledge. 3) context adequacy (ca). ca refers to the extent to which the context provides sufficient information to support the decision. 4) content preservation (cp). cp assesses the semantic similarity between sarcastic data and counterfac-tual data using bertscore @xcite . we randomly select 50 samples to assess counterfactual quality in three experimental settings. 1) regular, which provides only basic input-output examples without sarcastic segments or rhetorical devices. 2) rhetoric-guided, which includes both sarcastic segments and rhetorical devices, along with icl examples to guide reasoning. 3) human-refined, which extends rhetoric-guided by incorporating human review and revision. two annotators, blinded to the experimental settings, independently assess the generated data using metrics 1-3. for plausibility and ca, we adopt a 5-point likert scale (1: very poor; 5: excellent). as shown in while the human-refined setting exhibits marginally lower scores in plausibility and ca, it achieves the highest cp score. on the one hand, human revision prioritizes minimal modifications to preserve the original intention of the sarcastic dialogue, which may slightly compromise logical consistency or contextual coherence. on the other hand, since the context remains identical and only the sarcastic utterance is rewritten, annotators may have subconsciously rated plausibility and ca higher when exposed to the same context. ## conclusion in this paper, we study fine-grained sarcasm classification through the lens of rhetorical devices. we introduce a novel sarcasm dataset that incorporates various rhetorical devices, and propose an counterfactual data generation pipeline facilitated by both llms and human revision. we conduct a series of experiments with our dataset to benchmark baseline systems, demonstrating the necessity and effectiveness of integrating rhetorical devices for improved sarcasm detection. in summary, our work contributes to paving the way for more nuanced analysis of this intricate linguistic phenomenon.