Detecting Dissonant Stance in Social Media: The Role of Topic Exposure (2022)

Abstract: We address dissonant stance detection, classifying conflicting stance between two input statements. Computational models for traditional stance detection have typically been trained to indicate pro/con for a given target topic (e.g. gun control) and thus do not generalize well to new topics. In this paper, we systematically evaluate the generalizability of dissonant stance detection to situations where examples of the topic have not been seen at all or have only been seen a few times. We show that dissonant stance detection models trained on only 8 topics, none of which are the target topic, can perform as well as those trained only on a target topic. Further, adding non-target topics boosts performance further up to approximately 32 topics where accuracies start to plateau. Taken together, our experiments suggest dissonant stance detection models can generalize to new unanticipated topics, an important attribute for the social scientific study of social media where new topics emerge daily.

## introduction a prevalent theory about human reasoning, the argumentative theory, is that its primary function is to support argumentation of one's stance or belief @xcite . new arguments come up on a daily basis and thus new topics for stance emerge. however, most current approaches to stance detection are restricted to wellestablished topics, and thus are limited in their applications, such as improving educational strategies to facilitate learning @xcite or tracking political opinions on the latest concerns @xcite . as a step toward stance detection, unrestricted to particular topics, we study the problem of identifying (dis)agreement between two statements under pre-chosen as well as unseen topics @xcite @xcite (henceforth, dissonant stance detection). given two claims c 1 , c 2 under topic t, the task is to classify them into either (i) consonance if the stance suggested by c 1 towards t is the same as that by c 2 , (ii) dissonance if the stance suggested by c 1 towards t is the opposite to that by c 2 , or (iii) neither (see here, we question the necessity of the target topic by exploring the impact of non-target topics on transformer-based models. over a corpus of 34 diverse topics, we conduct a large-scale empirical evaluation on the role of exposure to topics. our contributions include: (a) the evaluation of the role of exposure to other topics when detecting statements with dissonant stance for a target topic using transformer-based models; (b) we show that topic-independent (topicindep) dissonant stance detection models, which are not exposed to the target topic, can perform as well as those trained on a target topic when exposed to as few as 4 nontarget topics during training ( §3); (c) we show that adding more non-target topics further boosts the performance, beginning to reach a plateau at approximately 24 to 32 non-target topics, evaluating several transformer-based models; (d) we demonstrate that a topic-independence dissonance model, trained only on pairs of social media posts, can transfer to a different social media domain and variant of task (finding dissonance within phrase pairs of a single post) with a novel small annotated dataset. ## related work stance detection is conventionally modeled as identifying the stance expressed by a statement towards a target topic @xcite @xcite @xcite @xcite we generalize conventional stance detection as dissonant stance detection or contrast detection. beyond generalized stance detection, identification of dissonance in language has other social scientific applications such as detecting cognitive dissonance @xcite . generalized stance has previously studied between two short concise statements and without evaluation for the amount of topic exposure @xcite . 1 on the other hand, other work has considered stance detection models in a cross-target settings @xcite @xcite @xcite @xcite . some approaches achieve this through incorporation of external lexical or world knowledge @xcite or using adversarial training to eliminate topic-specific information @xcite . however, most of these studies use a corpus comprising a small number of topics, such as the six topics of semeval-2016 task 6 @xcite . importantly, despite these promising results, the question of whether the topic needs to be included at all has remained opened as well as the degree of non-target topic exposure. ## conclusions this paper weighs in on a key problem as nlp is increasingly used for studies of social science: the role of exposure to a diverse set of social or political topics and the ability to generalize to new topics. to this end, we have proposed and studied the problem of dissonant stance detection in the topicindep/intopicfew setting. we find that models continue to improve under a "topic independent setting" but start plateauing at around 8 non-target topics. our experiments also revealed that topicindep/intopicfew dissonant stance detection models trained on only a small number of non-target topics already perform as well as those trained on a target topic, and that adding more nontarget topics further boosts performance. further, we find the model trained on the debate forum, where statements are from distinct users, generalizes to a new domain and finding dissonant statements from the same person. taken together, these results suggest transformer-based dissonant stance detection model can generalize to unseen topics and domains.