“Laughing at you or with you”: The Role of Sarcasm in Shaping the Disagreement Space (2021)

Abstract: Detecting arguments in online interactions is useful to understand how conflicts arise and get resolved. Users often use figurative language, such as sarcasm, either as persuasive devices or to attack the opponent by an ad hominem argument. To further our understanding of the role of sarcasm in shaping the disagreement space, we present a thorough experimental setup using a corpus annotated with both argumentative moves (agree/disagree) and sarcasm. We exploit joint modeling in terms of (a) applying discrete features that are useful in detecting sarcasm to the task of argumentative relation classification (agree/disagree/none), and (b) multitask learning for argumentative relation classification and sarcasm detection using deep learning architectures (e.g., dual Long Short-Term Memory (LSTM) with hierarchical attention and Transformer-based architectures). We demonstrate that modeling sarcasm improves the argumentative relation classification task (agree/disagree/none) in all setups.

## introduction user-generated conversational data such as discussion forums provide a wealth of naturally occurring arguments. the ability to automatically detect and classify argumentative relations (e.g., agree/disagree) in threaded discussions is useful to understand how collective opinions form, how conflict arises and is resolved @xcite @xcite @xcite @xcite . linguistic and argumentation theories have thoroughly studied the use of sarcasm in argumentation, including its effectiveness as a persuasive device or as a means to express an ad hominem we study whether modeling sarcasm can improve the detection and classification of argumentative relations in online discussions. we propose a thorough experimental setup to answer this question using feature-based machine learning approaches and deep learning models. for the former, we show that combining features that are useful to detect sarcasm @xcite @xcite with state-of-theart argument features leads to better performance for the argumentative relation classification task (agree/disagree/none) (section 5). for the deep learning approaches, we hypothesize that multitask learning, which allows representations to be shared between multiple tasks (e.g., here, the tasks of argumentative relation classification and sarcasm detection), lead to better generalizations. we investigate the impact of multitask learning for a dual long short-term memory (lstm) network with hierarchical attention @xcite (section 4.2) and bert (bidirectional encoder representations from transformers) @xcite , including an optional joint multitask learning objective with uncertainty-based weighting of task-specific losses @xcite (section 4.3). we demonstrate that multitask learning improves the performance of the argumentative relation classification task for all settings (section 5). we provide a detailed qualitative analysis (section 5.1) to give insights into when and how modeling sarcasm helps. we make the code from our experiments publicly available. 1 the internet argument corpus (iac) @xcite ) can be found for public acess here: foot_1 2 related work argument mining is a growing area of research in computational linguistics, focusing on the detection of argumentative structures in a text (see @xcite for an overview). this paper focuses on two subtasks: argumentative relation identification and classification (i.e., agree/disagree/none). some of the earlier work on argumentative relation identification and classification has relied on feature-based machine learning models, focusing on online discussions @xcite @xcite @xcite and monologues @xcite @xcite . @xcite proposed a set of lexical, syntactic, semantic, and discourse features to classify them. on the same essay dataset, @xcite utilized contextual information to improve the accuracy. both @xcite and @xcite used integer linear programming (ilp) based joint modeling to detect argument components and relations. @xcite introduced sentence similarity and accommodation features, whereas @xcite presented how entailment between text pairs can discover argumentative relations. our argumentative features in the feature-based model are based on the above works (section 4.1). we show that additional features that are useful in sarcasm detection @xcite enhance the performance on the argumentative relation identification and classification tasks. in addition to feature-based models, deep learning models have been recently used for these tasks. @xcite proposed a pointer network, and @xcite offered lstm+attention network to predict argument components and relations jointly, whereas @xcite exploited adaptive pretraining @xcite for bert to identify argument relations. we use two multitask learning objectives (argumentative relation identification/classification and sarcasm detection), as our goal is to investigate whether identifying sarcasm can help in modeling the disagreement space. majumder et al. (2019); chauhan et al. (2020) used multitask learning for sarcasm & sentiment and sarcasm, sentiment, & emotion, respectively, where a direct link between the corresponding tasks is evident. finally, analyzing the role of sarcasm and verbal irony in argumentation has a long history in linguistics @xcite @xcite . we propose joint modeling of argumentative relation detection and sarcasm detection to empirically validate sarcasm's role in shaping the disagreement space in online conversations. while the focus of our paper is not to provide a state-of-the-art sarcasm detection model, our feature-based models, along with the deep learning models for sarcasm detection are based on state-ofthe-art approaches. we implemented discrete features such as pragmatic features @xcite , diverse sarcasm markers @xcite , and incongruity detection features @xcite . the lstm models are influenced by @xcite ; @xcite , where the function of contextual knowledge is used to detect sarcasm. lastly, transformer models such as bert and roberta have been used in the winning entries for the recent shared task on sarcasm detection @xcite . in our research, for both kinds of deep-learning models, the best results are obtained by using the multitask setup, showing that multitask learning indeed helps improve both tasks. ## data our training and test data are collected from the internet argument corpus (iac) @xcite . this corpus consists of posts from conversations in online forums on a range of controversial political and social topics such as evolution, abortion, gun control, and gay marriage @xcite . multiple versions of iac corpora are publicly available, and we use a particular subset, marked as iac orig , collected from @xcite . this consists of around 10k pairs of conversation turns (i.e., prior turn pt and the current turn ct) that were annotated using mechanical turk for argumentative relations (agree/disagree/none) and other characteristics such as sarcasm/non-sarcasm, respect/insult, nice/nastiness. median cohen's κ is 0.5 across all topics. for agree/disagree/none relations the annotation was a scalar judgment on an 11 point scale @xcite where "-5" indicates a high disagreement move, "0" indicates none relation, and "5" denotes a high arg. rel. sarcasm # of turns a s 315 (33%) n s 638 (67%) d s 2207 (57%) n s 1696 (43%) n s 2285 (44%) n s 2841 (56%) each "current turn" that is part of @xmath0,@xmath1. ## experimental setup we present the computational approaches to investigate whether modeling sarcasm can help detect argumentative relations. as our goal is to provide a comprehensive empirical investigation of sarcasm's role in argument mining rather than propose new models, we explore three separate machine learning approaches well-established for studying argumentation and figurative language. first, we implement a logistic regression method that exploits a combination of state-of-the-art features to detect argumentative relations as well as sarcasm (section 4.1). second, we present a dual lstm architecture with hierarchical attention and its multitask learning setup (section 4.2). third, we discuss experiments using the pre-trained bert models and our multitask learning architectures based on it (section 4.3). ## results and discussion for lstm models, we see that multitask learning helps, lstm m t uncert showing a 2.8% improvement over the single model lstm attn , which is statistically significant. moreover, we notice that the improvement for the agree (a) and disagree (d) classes is 5.1%, with just a small reduction for the none (n ) class (0.7%). for bert, we notice better results when performing multitask learning, while the best performing model is obtained from bert m t uncert where we experimented with the dynamic weighting of task-specific losses during the training process @xcite . the performance increase is consistent across all three classes. the difference in performance among each setup is sta-lrargf lrargf +sarcf pronouns: i. my (both a), your(s) (d); discourse: so, because, for (all a), incidentally, particularly, although (all d); disagree lexicon: disagree, differ (both d);agree lexicon: agreed (a); entailment relation; negation (d) pronouns: mine, my (both a), you (d); discourse: then (a), though, however (both d); modal: will (a); punctuation: multiple question marks (both a and d); tag question: "are you", "do you" (both d); hyperbole: wonderful (a), nonsense, biased (both d); liwc dimensions: anxiety, assent, certainty (all d); sentiment incongruity (d); interj: so, agreed (both a) between the three classes -a, d, and n -we observe the lowest performance on the a class. this is unsurprising, given the highly unbalanced setting of the training data (a occurs less than 10% of times in the iac orig , see in sum, these improvements through multitask learning over single task argumentative relation detection indicate that modeling sarcasm is useful in modeling the disagreement space in online discussions. this provides an empirical justification to existing theories that study sarcasm's impact in modeling argumentation, persuasion, and argument fallacies such as ad hominem attacks. finally, we notice that multitask learning also improves the performance on the sarcasm detection task (results are presented in the appendix). ## conclusion and future work linguistic and argumentation theories have studied the use of sarcasm in argumentation, including its effectiveness as a persuasive device or as a means to express an ad hominem fallacy. we present a comprehensive experimental study for argumentative relation identification and classification using sarcasm detection as an additional task. first, in discrete feature space, we show that sarcasm-related features, in addition to argument-related features, improve the accuracy of the argumentative relation identification/classification task by 3%. next, we show that multitask learning using both a dual lstm framework and bert helps improve performance compared to the corresponding single model by a statistically significant margin. in both cases, the dynamic weighting of task specific losses performs best. we provide a detailed qualitative analysis by investigating a large sample manually and show what characteristics of sarcasm are attended to, which might have guided the correct prediction on the identification of the argumentative relation/classification task. in the future, we aim to study this synergy further by looking at sarcasm as well as the persuasive strategies (e.g., ethos, pathos, logos), and argument fallacies (e.g., ad hominem attack that was also noticed by habernal et al. (2018