Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques (2025)

Abstract: Sarcasm is a complex linguistic and pragmatic phenomenon where expressions convey meanings that contrast with their literal interpretations, requiring sensitivity to the speaker’s intent and context. Misinterpreting sarcasm in collaborative human–AI settings can lead to under- or overreliance on LLM outputs, with consequences ranging from breakdowns in communication to critical safety failures. We introduce Sarc7, a benchmark for fine-grained sarcasm evaluation based on the MUStARD dataset, annotated with seven pragmatically defined sarcasm types: self-deprecating, brooding, deadpan, polite, obnoxious, raging, and manic. These categories are adapted from prior linguistic work and used to create a structured dataset suitable for LLM evaluation. For classification, we evaluate multiple prompting strategies—zero-shot, few-shot, chain-of-thought (CoT), and a novel emotion-based technique—across five major LLMs. Emotion-based prompting yields the highest macro-averaged F1 score of 0.3664 (Gemini 2.5), outperforming CoT for several models and demonstrating its effectiveness in sarcasm type recognition. For sarcasm generation, we design structured prompts using fixed values across four sarcasm-relevant dimensions: incongruity, shock value, context dependency, and emotion. Using Claude 3.5 Sonnet, this approach produces more subtype-aligned outputs, with human evaluators preferring emotion-based generations 38.46% more often than zero-shot baselines. Sarc7 offers a foundation for evaluating nuanced sarcasm understanding and controllable generation in LLMs, pushing beyond binary classification toward interpretable, emotion-informed language modeling.

## introduction sarcasm is defined as the use of remarks that convey the opposite of their literal meaning. understanding sarcasm requires an intuitive grasp of humor and social cues, posing a challenge for natural language processing (nlp) tasks such as humanlike conversation @xcite . sarcasm is a pragmatic act, where meaning depends not only on words but also on speaker intent, emotional tone, and shared context. large language models (llms) generally perform poorly on sarcasm classification and generation tasks due to the subtlety and context dependence of sarcastic language @xcite . traditional sentiment analysis and machine learning techniques also struggle with these challenges. this work introduces a novel sarcasm benchmark grounded in the seven recognized types of sarcasm and proposes an emotion-based approach for both classification and generation. we examine whether llms can demonstrate pragmatic reasoning. in contrast to prior rule-based and template-driven methods, which often produced rigid outputs @xcite , and even more recent deep learning models that still fall short in capturing subtlety and social nuance @xcite , our technique aims to improve contextual relevance and expressive range in sarcastic generation. ## related work while prior benchmarks @xcite focus on binary detection by evaluating state-of-theart (sota) large language models (llms) and pretrained language models (plms), @xcite real-world agents require subtype sensitivity. according to @xcite , @xcite first introduced a seven-type classification of sarcasm based on observational studies of classroom discourse. @xcite then refined these categories into operational definitions tailored for social-interview data, providing clear examples and criteria. @xcite subsequently applied this refined taxonomy in an irony and sarcasm detection system for publicfigure speech. ## conclusion we present sarc7, the first benchmark to evaluate both the detection and controlled generation of seven nuanced sarcasm subtypes, framing the task as a test of an llm's pragmatic competence. our classification experiments show that while chainof-thought prompting yields the highest accuracy, emotion-based prompts achieve a superior macroaveraged f1 score (0.3664 with gemini 2.5). a human baseline (κ = 0.6694) confirms the inherent difficulty of subtypes like brooding and deadpan. for generation, structured prompts specifying dimensions like incongruity and emotion improved subtype alignment by 38% over zero-shot baselines with claude 3.5 sonnet. by benchmarking finegrained performance, sarc7 moves beyond binary detection and lays the groundwork for more natural, context-sensitive dialogue agents with potential for future multimodal and cross-lingual extensions.