Transitive Consistency Constrained Learning for Entity-to-Entity Stance Detection (2024)

Abstract: Entity-to-entity stance detection identifies the stance between a pair of entities with a directed link that indicates the source, target and polarity. It is a streamlined task without the complex dependency structure for structural sentiment analysis, while it is more informative compared to most previous work assuming that the source is the author. Previous work performs entity-to-entity stance detection training on individual entity pairs. However, stances between inter-connected entity pairs may be correlated. In this paper, we propose transitive consistency constrained learning, which first finds connected entity pairs and their stances, and adds an additional objective to enforce the transitive consistency. We explore consistency training on both classification-based and generation-based models and conduct experiments to compare consistency training with previous work and large language models with in-context learning. Experimental results illustrate that the inter-correlation of stances in political news can be used to improve the entity-to-entity stance detection model, while overly strict consistency enforcement may have a negative impact. In addition, we find that large language models struggle with predicting link direction and neutral labels in this task.

## introduction detecting polarity from text has been widely studied in different forms, such as sentence-level @xcite or aspect-level sentiment analysis @xcite , target-oriented stance detection @xcite , and structured analysis @xcite @xcite . some recent efforts explore a streamlined and informative form, entity-to-entity stance detection @xcite ## entity-to-entity stance detection frameworks entity-to-entity stance detection identifies the stance between a pair of entities, as well as the source and target through a directed link. in this section, we introduce two basic frameworks for this challenge. one is based on relation classification using entity pair representations, while the other is to generate the stance autoregressively. ## transitive consistency constrained learning inter-connected stances may be correlated, especially in political news, as shown in figure 1 . we hope to capture the correlation from optimizing the the transitive stance inference requires multiple inter-correlated entity pairs in a context, while most existing resources only annotate one entity-toentity stance at the sentence level. therefore, we propose a simple sentence pair sampling method that helps obtain data for transitive inference. then we introduce the constrained learning method, which can be added to both classification-based and generation-based methods to capture the transitive correlation. the overall framework is illustrated in figure 2 . ## related work earlier efforts on stance detection primarily focus on some specific targets with rich training and testing data @xcite @xcite . a typical model in this setting is built for each target separately @xcite @xcite @xcite , or cross-target stance detection, where we have pre-defined leaveout targets to test the model generalization to targets that do not have training data @xcite @xcite . more recent efforts also study zero-shot or few-shot stance detection on a large number of targets @xcite @xcite @xcite . this setting requires model to generalize to a large number of unseen targets. recently, another line of research studies a more objective form of stance detection, entity-to-entity stance detection @xcite , where we analyze the stance from one entity to another entity in the text. our work follows this direction and studies the consistency between related entity-to-entity stances and uses this consistency to help model training, compared to previous work @xcite that tackles the stances individually. on the other hand, stance detection can be considered as a simplified task of structured sentiment analysis @xcite , which identifies opinion holders, targets, expressions, and polarities into dependency structures. typical stance detection setups assume that the opinions are from the author, and models only need to consider the target. while entity-to-entity stance detection combines holders, targets, and polarities with more streamlined, directed link labels. the consistency assumption between related stances is also related to multi-target stance detection @xcite . multi-target stance detection is to detect a stance pair for a multi-target (e.g., a pair of targets), assuming that when expressing the stance to one target, it also implies stances to a related target. similar consistency constraints have also been discussed on polarity link prediction in social networks @xcite and event relation extraction @xcite . the focus of @xcite is the network-based link prediction, which is quite different from the text-based analysis. @xcite performs event-event relation extraction (temporal and hierarchical). the document-level annotation provides consistent labels and therefore they do not have steps that we have to create data for consistency training. ## conclusion in this paper, we present a method that models the transitive consistency constraints during training to help train entity-to-entity stance detection models. our proposed methods first sample sentence pair to conduct stance transitivity inference, and model the constraints as the similarity between the inferred and directly predicted stance. experiments show that this constrained learning helps improve both classification-and generation-based models. further analysis indicates that the constrained learning is sensitive to the balance factor which controls the enforcement of constraints during training. we also find large language models may not perform reliable complex structured predictions, especially on neutral and directed samples.