Target Really Matters: Target-aware Contrastive Learning and Consistency Regularization for Few-shot Stance Detection (2022)

Abstract: Stance detection aims to identify the attitude from an opinion towards a certain target. Despite the significant progress on this task, it is extremely time-consuming and budget-unfriendly to collect sufficient high-quality labeled data for every new target under fully-supervised learning, whereas unlabeled data can be collected easier. Therefore, this paper is devoted to few-shot stance detection and investigating how to achieve satisfactory results in semi-supervised settings. As a target-oriented task, the core idea of semi-supervised few-shot stance detection is to make better use of target-relevant information from labeled and unlabeled data. Therefore, we develop a novel target-aware semi-supervised framework. Specifically, we propose a target-aware contrastive learning objective to learn more distinguishable representations for different targets. Such an objective can be easily applied with or without unlabeled data. Furthermore, to thoroughly exploit the unlabeled data and facilitate the model to learn target-relevant stance features in the opinion content, we explore a simple but effective target-aware consistency regularization combined with a self-training strategy. The experimental results demonstrate that our approach can achieve state-of-the-art performance on multiple benchmark datasets in the few-shot setting.

## introduction stance detection is intended to identify the attitude of opinions towards certain targets, where labels can be favor, against, and neutral. for example, the opinion "true equality allows all to be born." is against the target "legalization of abortion". the settings of stance detection can be generally divided into in-target and cross-target ones. specifically, in-target stance detection aims to train an exclusive classifier for prediction on the same set * zheng lin is the corresponding author. different from common classification tasks (e.g., sentiment classification), the identification of stance is heavily dependent on the specific target @xcite . as a target-oriented task, the key problem of few-shot stance detection is how to thoroughly exploit the target-relevant information from the limited labeled data and the unlabeled data. existing methods like supervised contrastive learning @xcite and semisupervised learning @xcite prove significant effectiveness in the few-shot setting. when only a few labeled samples are provided, supervised contrastive learning (scl) hopes to improve the representation ability of the model to a certain extent by leveraging class label information. however, such an objective ignores target label information, the crucial clue, which plays a vital role in stance detection. when the unlabeled data is available, semi-supervised learning (ssl) like selftraining and consistency regularization, is a way of bringing unlabeled data into full play, which helps to overcome the scarcity of sufficient annotated data. however, self-training algorithm @xcite only encourages the student network to mimic the teacher network's label predictions simply. moreover, consistency regularization @xcite constrains the model to make consistent predictions of the same example under some task-agnostic data augmentation strategies. neither of them digs deeply into target-relevant stance features in the opinion content. to further tackle the challenges above, we propose a novel semi-supervised framework with target-aware contrastive learning and consistency regularization (stcc). first, we introduce a targetaware contrastive learning objective to consider both target and stance label information, which promotes the distinction and isolation of samples from different targets, as well as different classes. since the target-aware contrastive learning objective learns more distinguishable representations, the model trained here can be used in semi-supervised learning as a better teacher model. second, we combine two approaches to ssl: consistency regularization and self-training. specifically, we design a simple but effective target-aware data augmentation strategy for consistency regularization, i.e., masking the corresponding target for every unlabeled example, so that the model can perceive the target-related information contained in the content as much as possible. moreover, our proposed target-aware contrastive learning objective can be naturally extended to semi-supervised learning with additional unlabeled data. specifically, the pseudo-labels generated by the teacher model are used for the contrastive learning objective. finally, we conduct comprehensive experiments on two large datasets of semeval-2016 and covid-19-stance to verify the effectiveness of our framework for few-shot stance detection. the experimental results demonstrate that our approach achieves the state-of-the-art performance 1 . ## related work stance detection stance detection aims to identify the attitude from an opinion towards a certain target. incipient studies focus on in-target stance detection @xcite @xcite @xcite , which only train the model and perform the prediction on a single target. @xcite investigated the multi-target training and knowledge distillation in the stance detection task. data augmentation @xcite has been used for in-target setting to improve performance in fully supervised learning. crosstarget stance detection @xcite @xcite hopes to transfer knowledge between related targets, which attempts to mitigate the lack of labeled training data for a new target. @xcite focused on semi-supervised user stance detection using the information from tweets posted by users and their followers, whereas we do not consider specific user information. @xcite studied few-shot cross-lingual stance detection, which transferred the knowledge from english resources to non-english scenarios. recently, allaway and mckeown (2020) defined zero-shot and few-shot stance detection, according to which the targets have no or very few training examples. and, they present a new dataset vast, which consists of thousands different targets. however, vast includes a wide range of similar expressions for one target (e.g., "guns on campus" versus "firearms on campus"). the situation above makes the source of the model's benefit too ambiguous to trace. therefore, semeval-2016 and covid-19-stance are relatively much more accessible for studying in the few-shot setting, compared with vast. contrastive learning in recent years, contrastive learning has made significant progress in self-supervised representation learning, both in the cv @xcite and nlp @xcite domains. @xcite introduced supervised contrastive learning (scl), which further ex-tended the self-supervised contrastive learning to the fully-supervised setting by leveraging label information. @xcite integrated the scl objective for fine-tuning pre-trained language models, which significantly improves the performance in the few-shot learning settings. in stance detection, @xcite proposed a hierarchical contrastive learning loss to take both the data types and the stance labels into account. they subdivided the data types by judging whether a sample is sensitive to its corresponding target based on the self-supervised learning pretext task. however, this strategy is not suitable for the few-shot setting, which further reduces the target-related information availability to the model. semi-supervised learning consistency regularization, pseudo-labeling, and self-training are all important components of semi-supervised learning. consistency regularization @xcite constrains the model to make consistent predictions of the same example under varied noises. and, pseudo-labeling @xcite selects those unlabeled data with high confidence as a form of entropy minimization. these methods use the model being trained to generate pseudo-labels instead of a separate teacher model pre-trained on labeled data. self-training @xcite ) allows a teacher model pre-trained on labeled data, and then applies the combination of labeled and pseudo-labeled data to retrain a student model. recent, semi-supervised learning methods have combined those techniques to some extent. @xcite ; rizve et al. (2021) combine the self-training process with pseudo-labeling, while using curriculum labeling and uncertainty-aware techniques to improve the filter process for unlabeled data. @xcite unify the consistency regularization and pseudolabeling. however, these task-agnostic methods cannot adequately mine target-relevant stance features in the opinion content for stance detection. ## conclusion in this paper, we focus on in-target few-shot stance detection to alleviate the demand for human supervision, and propose a novel target-aware semisupervised framework with contrastive learning and consistency regularization. the target-aware contrastive learning objective performs well on both labeled and unlabeled data, which promotes the model's ability to distinguish various classes and targets. moreover, our proposed target-aware consistency regularization is validated to be more efficient in mining target-relevant stance features in the content. experiments on two popular benchmarks demonstrate the effectiveness and consistent improvements over baselines.