Exploiting Sentiment and Common Sense for Zero-shot Stance Detection (2022)

Abstract: The stance detection task aims to classify the stance toward given documents and topics. Since the topics can be implicit in documents and unseen in training data for zero-shot settings, we propose to boost the transferability of the stance detection model by using sentiment and commonsense knowledge, which are seldom considered in previous studies. Our model includes a graph autoencoder module to obtain commonsense knowledge and a stance detection module with sentiment and commonsense. Experimental results show that our model outperforms the state-of-the-art methods on the zero-shot and few-shot benchmark dataset–VAST. Meanwhile, ablation studies prove the significance of each module in our model. Analysis of the relations between sentiment, common sense, and stance indicates the effectiveness of sentiment and common sense.

## introduction stance detection aims to identify the authors' attitudes or positions (pro (support), con (oppose), neu (neutral)) towards a specific target such as an entity, a topic. @xcite @xcite @xcite . it is crucial for understanding opinions and analyzing how opinions are presented in texts regarding specific issues, and much work has been done building stance detection models @xcite @xcite . there are two salient challenges to the task. first, obtaining rich annotated data in stance detection is time-consuming and laborintensive. to address this issue, @xcite propose the dataset vast containing various topics for few-shot and zero-shot stance detection tasks, requiring the model to classify the stance of topics unseen in the training set. second, the topic is often not explicitly mentioned in the document, resulting in difficulty. considering text : i totally agree with this premise. as a younger person i was against nuclear power (i was in college during 3 mile island) but now it seems that nuclear should be in the mix. fission technology is better, and will continue to get better if we actively promote its development. the prospect of fusion energy also needs to be explored. if it's good enough for the sun and the stars, it's good enough for me. text : example 2 topic : nuclear power stance : pro example 1 topic : olympics stance : pro the games should proceed. athletes have made tremendous sacrifices to qualify and be prepared. it would be cruel to deny them their chance. in the future the games should be held in countries within the top say 15 gdp per capita. figure 1 : examples for stance detection vast. figure 1 example 1, the document does not explicitly contain the topic 'olympics', but 'games' and 'athlete' implicitly refer to the topic. existing work incorporates external knowledge to solve the challenges @xcite . for example, cke-net achieves the state-of-the-art results for zero-shot stance detection, which uses pre-trained model bert and commonsense knowledge graph on conceptnet @xcite . however, such a method only considers the knowledge relations between documents and topics (i.e., the commonsense knowledge in two-hop directed paths on the conceptnet from documents to topics), limiting the generalization of adding other types of related knowledge. in figure 1 example 1, the word 'games' can also represent the computer programs in a different document. such knowledge cannot be used for that document if no relation between 'game' and 'computer program' can be learned from the relations between documents and topics in the dataset. we consider incorporating two types of general knowledge, including common sense and sentiment. first, we incorporate commonsense knowledge into the stance detection model using a graph autoencoder module. we take a pre-training method to train the graph autoencoder, separately to the stance detection module. second, stance detection is significantly influenced by the sentiment information @xcite @xcite (case study can be seen in appendix). in figure 1 example 2, the document contains many positive words like 'good', and 'better' regarding the topic 'nuclear power', which implies a pro stance. however, little existing work has considered sentiment knowledge for zero-shot stance detection. we use the sentimentaware bert (sentibert henceforth) to extract the sentiment information, assisting in classifying the stances of topics. existing work on injecting knowledge into nlp models can be broadly classified into two categories. one uses a graph encoder to integrate structural knowledge into a neural encoder @xcite @xcite and the other injects knowledge by using training losses to tune model parameters @xcite @xcite . in our work, we consider the former for commonsense knowledge and the latter for sentiment due to the sources of information. in the component of knowledge graph encoding, a graph autoencoder consisting of relational graph convolutional network (rgcn) encoders @xcite and a dismult decoder @xcite is trained using negative sampling to obtain the relations of concepts on the commonsense knowledge graph. we inject sentiment knowledge encoded by sentibert into bert using a cross attention module and tuning the fusing process by the training loss of the stance detection. our model achieves the state-of-the-art performance on the benchmark dataset vast @xcite in both zero-shot and fewshot stance detection, improving the performance on many challenging linguistic phenomena such as sarcasm and quotations. we analyze the performance of our model with respect to different sentiment and common sense features, finding that the data with the corresponding sentiment and stance pairs (i.e., (pos, pro) and (neg, con)) are the easiest part for models to classify; in addition, increased commonsense knowledge leads to improved performance of the stance detection model. to our knowledge, we are the first to incorporate both sentiment and common sense into zero-shot stance detection model. the code has been released https://github.com/luoxiaoheics/stancecs . ## related work stance detection, also known as stance classification @xcite , stance identification @xcite , stance prediction @xcite , debate-side classification @xcite , and debate stance classification @xcite , aims to identify the stance of the text author towards a target (an entity, event, idea, opinion, claim, topic, etc.) either explicitly mentioned or implied within the text. for the initial task of stance detection, models are trained an individual classifier for each topic @xcite @xcite @xcite or only a small number of topics are both in training and evaluation sets @xcite @xcite . however, given rich and varying topics, data annotation can be time-consuming and laborintensive. researchers attempt to solve the task in a cross-target setting @xcite , training the model in a topic and testing it on another one, and propose several weakly supervised approaches using unlabeled data related to the test topics @xcite @xcite . other studies propose the tasks of zero-shot and few-shot stance detection, which requires training the model in data of several topics and testing it on some unseen topics @xcite . @xcite propose to solve the task using a topic-grouped attention net, which uses the relation between the training and evaluation topics in an unsupervised way, and they also analyze the relationship between sentiment and stance from the perspective of the model by corrupting sentences with replacing sentiment words. @xcite use human rationales as attribution priors to provide faithful explanations of models. @xcite propose to incorporate commonsense knowledge to learn the relations between different topics utilizing a compgcn (a variant of graph convolution networks). however, it limits the content of knowledge (only knowledge from documents to stances in the training data). our model differs from such a method in that our model adopts the related concepts of both documents and topics and uses a pre-trained graph autoencoder to obtain commonsense information. adversarial learning is also applied to solve the zero-shot task by using unlabeled raw data @xcite . unlike the above work, we consider integrating external knowledge for zero-shot stance detection, including sentiment and commonsense information that are rarely considered. to our knowledge, we are the first to systematically incorporate sentiment and commonsense knowledge into the stance detection model and analyze the relationship between them (in section 4.5 and 4.6). ## method the architecture of our model is illustrated in figure 2, which contains two components: (1) knowledge graph encoding, which integrates commonsense knowledge from conceptnet (section 3.1); ## conclusion we proposed a stance detection model incorporating commonsense knowledge and sentiment information, achieving state-of-the-art zero-shot and few-shot stance detection results on the standard dataset. the ablation study showed the significance of each module, such as knowledge graph autoencoder, sentibert, and bert. we also analyzed the relation between sentiment/common sense and stance, which indicate the effectiveness of this external knowledge.