SSR : Utilizing Simplified Stance Reasoning Process for Robust Stance Detection (2022)

Abstract: Dataset bias in stance detection tasks allows models to achieve superior performance without using targets. Most existing debiasing methods are task-agnostic, which fail to utilize task knowledge to better discriminate between genuine and bias features. Motivated by how humans tackle stance detection tasks, we propose to incorporate the stance reasoning process as task knowledge to assist in learning genuine features and reducing reliance on bias features. The full stance reasoning process usually involves identifying the span of the mentioned target and corresponding opinion expressions, such fine-grained annotations are hard and expensive to obtain. To alleviate this, we simplify the stance reasoning process to relax the granularity of annotations from token-level to sentence-level, where labels for sub-tasks can be easily inferred from existing resources. We further implement those sub-tasks by maximizing mutual information between the texts and the opinioned targets. To evaluate whether stance detection models truly understand the task from various aspects, we collect and construct a series of new test sets. Our proposed model achieves better performance than previous task-agnostic debiasing methods on most of those new test sets while maintaining comparable performances to existing stance detection models.

## introduction the task of stance detection aims to predict the stance of the text towards the given target. it is crucial for various downstream tasks including fact verification, rumor detection, etc. it has a wide application in analyzing political opinions and product reviews. existing works usually treat this task as a text pair classification problem and many design target-tweet interaction structures @xcite to learn target-aware stance representations. however, @xcite have shown 1 refers to targets that a given tweet expresses opinions on. tweet: hilarity of the day: hillary said she went 'above and beyond' in transparency. really? what about the 30k deleted emails? #semst given target: hillary clinton gold stance: against target 1: hillary clinton target 2: feminist movement target 3: atheism that those models @xcite can achieve superior performances only using the tweet. those end-to-end stance detection models treat the stance reasoning process as a black box and are prone to rely on bias features in the dataset instead of learning the underlying task. for instance, in figure 1 , the bert model still predicts against even when the target is changed to an unrelated target like atheism. meanwhile, common stance detection models perform poorly on out-of-distribution datasets @xcite and unseen targets, which calls for debiasing stance detection models to get rid of spurious correlations in the datasets. while @xcite made the first attempt to reveal dataset bias in stance detection, mitigating bias in other natural language understanding (nlu) tasks has been extensively explored. the key challenge in debiasing is how to discriminate between genuine and bias features. one line of work @xcite implicitly hypothesized that features, learnt by small models or by large models at earlier steps, could potentially be bias features. in addition, others @xcite @xcite tried data augmentation to break spurious correlation in the training data and treated features learnt on the augmented data as genuine features. another line of work @xcite adopted multi-task learning with auxiliary tasks, ## approach in this section, we present the ssr model that employs simplified stance reasoning process to combat dataset bias in stance detection. we first describe the basic text encoder that we use to encode tweet, target and target-tweet pairs. then, we introduce two intermediate sub-tasks based on our observation of the stance reasoning process. after that, we elaborate on why and how we simplify the introduced two sub-tasks. these sub-tasks are implemented by maximizing the mutual information between the opinioned target and the tweet. finally, we show how we combine these two sub-tasks for final stance predictions. ## conclusion in this paper, we propose to utilize the stance reasoning process as task knowledge to guide the discrimination between genuine and bias stance features. to alleviate demands for token-level intermediate annotations, we simplify the stance reasoning process where labels for proposed subtasks can be automatically inferred without additional annotations. to evaluate whether stance detection models understand the task from various aspects, we collect and construct 6 new test sets. empirical results show that our model outperforms task-agnostic debiasing methods on 4/6 new test sets while maintaining comparable performances to existing stance detection models on in-domain datasets. zhao wang and aron culotta. 2021. robustness to spurious correlations in text classification via automatically generated counterfactuals. proceedings of the aaai conference on artificial intelligence, 35(16):14024-14031. hanqi yan, lin gui, gabriele pergola, and yulan he. 2021. position bias mitigation: a knowledge-aware graph model for emotion cause extraction. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 3364-3375, online. association for computational linguistics. linyi yang, jiazheng li, padraig cunningham, yue zhang, barry smyth, and ruihai dong. 2021. exploring the efficacy of automatically generated counterfactuals for sentiment analysis. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 306-316, online. association for computational linguistics. yi-ting yeh and yun-nung chen. 2019. qainfomax: learning robust question answering system by mutual information maximization. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pages 3370-3375, hong kong, china. association for computational linguistics.