Unsupervised stance detection for arguments from consequences (2020)

Abstract: Social media platforms have become an essential venue for online deliberation where users discuss arguments, debate, and form opinions. In this paper, we propose an unsupervised method to detect the stance of argumentative claims with respect to a topic. Most related work focuses on topic-specific supervised models that need to be trained for every emergent debate topic. To address this limitation, we propose a topic independent approach that focuses on a frequently encountered class of arguments, specifically, on arguments from consequences. We do this by extracting the effects that claims refer to, and proposing a means for inferring if the effect is a good or bad consequence. Our experiments provide promising results that are comparable to, and in particular regards even outperform BERT. Furthermore, we publish a novel dataset of arguments relating to consequences, annotated with Amazon Mechanical Turk.

## introduction in the context of decision making it is crucial to compare positive and negative effects that result from a potential decision. indeed, arguing for or against something because of its possible consequences is a frequent form of argumentation @xcite . in this paper, we address the classical stance detection problem paying special attention to such arguments. stance detection, also called stance classification, is the task to decide whether a text is in favor of, against, or unrelated to a given topic. this problem is related to opinion mining, but while opinion mining focuses on the sentiment polarity explicitly expressed by a text, stance detection aims to determine the position that the text holds with respect to a topic that is generally more abstract and might not be mentioned in the text. as such, in stance detection, texts can transmit a negative sentiment or opinion, but be in favor of the targeted topic. for example, the text holocaust denial psychologically harms holocaust survivors expresses a negative opinion, but its stance towards criminalization of holocaust denial is positive. 1 recently, the problem of stance detection has received growing attention from the scientific community, as shown by the recent survey of @xcite . most approaches tackle this problem by learning stance classification models for each topic. while this can achieve good results, new models need to be trained for each new topic of interest, generally entailing large annotation studies. while we admit that a one-size-fits-all approach to stance detection is currently unfeasible, we take a different perspective. rather than targeting topicdependent models, we target a subclass of arguments. specifically, we focus on arguments that have been classified by @xcite under the argument from consequences scheme. they contain a premise of the form if a is brought about, then good (bad) consequences will (may plausibly) occur, and a conclusion a should (not) be brought about. in most real-life arguments of this type, the consequences are expressed, but the interpretation that they are good or bad, as well as the conclusion, are most often implicit. the task of stance detection is then to determine if the argument is against or in favor of a. our solution to find the stance of such arguments revolves around extracting and analyzing cause-effect relations in order to infer if the consequences are good or bad. we conducted an amazon mechanical turk (amt) study, in which we crowdsourced annotations for 1894 arguments extracted from debatepedia. we compared our system's performance to a sentiment analysis baseline and a fine-tuned bert model. the results show that our results are comparable and, in some settings, even better than bert's. 2 aside from not needing annotated training data, we stress the advantage of our approach for providing human-understandable explanations to the results, and to provide, as a byproduct, cause-effect relations between concepts brought up in arguments. the paper is structured as follows. section 2 positions our contributions with respect to related literature. section 3 presents our proposed approach. section 4 describes our crowdsourced dataset, which we use in section 5 to evaluate our approach. lastly, section 6 concludes the paper. ## related work stance detection has been studied on various types of formal texts such as congressional debates @xcite and company-internal discussions @xcite . however, like most recent related work on the topic, we are particularly interested in informal texts from online social media. the vast majority of previous approaches proposes supervised methods, using traditional machine learning algorithms @xcite @xcite @xcite and more recently, various deep neural networks architectures @xcite @xcite . these approaches, most of which have been triggered by a recent semeval shared task foot_2 @xcite , learn topic-specific models. thus, new topics require new models whose training entails large user annotation studies. in contrast, we propose a fully unsupervised, topic-independent method, and rather target a particular but frequent class of claims, those that refer to consequences. among the unsupervised approaches, the most prominent one is this of @xcite , which got extended by @xcite and @xcite . however, they focus on non-ideological topics (usually products, e.g., iphone vs. galaxy). in contrast, we target ideological topics (e.g., gay marriage, abortion) whose stance is harder to detect due to less fre-quent use of sentiment words and a wider variety of brought up issues and arguments @xcite . on the one hand, these works extract topic aspects (e.g., screen resolution, battery) and polarities towards these aspects, a step that is unfeasible for ideological topics. on the other hand, like these works, we also use syntactic rules, but not for pairing aspects to opinions, but for extracting triples that correspond to statements about effects over opinion words. another class of stance detection approaches uses the context of the post, such as its relations to other posts in the debate, the network of authors, or the author's identity @xcite @xcite . by contrast, we target claim-topic pairs in isolation. another aspect that sets our work apart from most related work is that, except for the approaches that target tweets, most focus on longer texts while we consider short, one-sentence claims. in this regard, but not only, the stance detection work that is closest to ours is the partly supervised system of @xcite . they also propose a topicindependent solution to stance detection for short claims without considering context, but they do not specifically address arguments from consequences. while they follow a similar sequence of steps as we do, they propose different approaches for each step. for instance, they propose a supervised approach to detect the target of a claim's opinion, while we do it in an unsupervised manner. they focus primarily on detecting contrastive relations between phrases, while our focus is on detecting effects. in this last regard, the works can be considered complementary. regarding the analysis of arguments from consequences, @xcite provide and use scheme dependent templates to analyze the structure of arguments. their work is rather conceptual and focuses on annotations. very recently, al-khatib et al. (2020) built, on similar intuitions as ours, an approach for creating argumentation knowledge graphs based on cause-effect relations. their work comes to reinforce the usefulness of addressing arguments from consequences. to sum up, our contribution is three-fold: (i) we propose a fully unsupervised approach for stance detection, focusing on arguments that refer to consequences; (ii) we define rules over grammatical dependencies that exploit sentiment as well as ef-fect words in order to determine good and bad consequences; (iii) we publish a new stance detection dataset that labels claims that refer to consequences, and which was crowdsourced on amt. ## our approach given an argumentative claim and a topic, our task is to detect the stance that the claim has with respect to the topic. statements such as the claim or topic usually express a positive (favorable) or negative (unfavorable) position to a concept that we call the target. as such, the target is a phrase that belongs to the statement. in the example shown topic: medical marijuana dispensaries claim: legalizing medical marijuana does not increase use and abuse at the core of our approach resides what we call the effect triple. the effect triple is a triple of the @xmath0, dir ), (p, eff ), (o, sent) >. the (t, dir ) pair represents the target t of the statement and if the statement refers to a magnification (@xmath1.g. legalizing medical marijuana), or a reduction (@xmath2.g. banning medical marijuana). the (p, eff ) pair represents the predicate p that has t as the subject, together with the effect eff that it has over the object o. the effect can be positive (@xmath3. lastly, the (o, sent) pair represents the object over which t has the effect p . we expect the sentiment of an object to reflect whether it is generally regarded as a good thing (@xmath4. our approach's core idea is to distill such an effect triple from the claim and use it to infer the claim's stance towards t c . we further determine (t t , dir) to infer the topic's stance towards t t . using these stances, together with the relation between the claim's and the topic's target, we finally decide the claim's stance with respect to the topic. we now describe the lexicons we use as well as each of these steps in more detail. ## dataset generation to evaluate our approach, we need stance annotated topic-claim pairs, as well as annotations if the topicclaim pair refers to a consequence or not. ## conclusion and future work we propose a fully unsupervised method to detect the stance of arguments from consequences in online debates. the method exploits grammatical dependencies and lexicons to identify effect words and their impact. for our evaluation, we annotated arguments from debatepedia regarding their stance and whether they involve consequences or not. the results we obtained are motivating. our method is comparable to bert while being more robust. besides the future extensions of this approach that we mentioned in our results discussion and error analysis, this work opens several interesting research paths. mainly, its good performance on the claims that refer to consequences reinforces our intuition that designing systems tailored for particular argumentation schemes might be a good alternative to topic-specific models. therefore, we plan to complement this work with approaches for other frequently applied schemes such as arguments by expert opinion and arguments by example.