Generative Data Augmentation with Contrastive Learning for Zero-Shot Stance Detection (2022)

Abstract: Stance detection aims to identify whether the author of an opinionated text is in favor of, against, or neutral towards a given target. Remarkable success has been achieved when sufficient labeled training data is available. However, it is labor-intensive to annotate sufficient data and train the model for every new target.Therefore, zero-shot stance detection, aiming at identifying stances of unseen targets with seen targets, has gradually attracted attention. Among them, one of the important challenges is to reduce the domain transfer between seen and unseen targets. To tackle this problem, we propose a generative data augmentation approach to generate training samples containing targets and stances for testing data, and map the real samples and generated synthetic samples into the same embedding space with contrastive learning, then perform the final classification based on the augmented data. We evaluate our proposed model on two benchmark datasets. Experimental results show that our approach achieves state-of-the-art performance on most topics in the task of zero-shot stance detection.

## introduction in recent years, social media websites have become an important platform for people to express their opinions on different targets (or topics in some literature) ranging from politics, government policies, movies, sports and social issues, etc @xcite . more often, users tend to take a stance, in favor, against or neutral towards a particular target @xcite . the task of stance detection aims to automatically identify the stance represented by the users in numerous texts. it is of great significance to applications like argument mining, fake news detection, public opinion analysis and has caught the attention of researchers. remarkable success has been achieved when sufficient labeled training data is available in the task of stance detection @xcite ; siddiqua in zssd, the annotation data of the target to be tested will not appear in the training step of the model. @xcite . most of the existing research work relies on a large number of labeled data for a specific target. however, in reality, new targets are constantly produced, and it is unpractical to label sufficient data for each target. based on the idea of transfer learning, some researchers use cross-target stance detection to train one classifier adapting from the current target to "new" target @xcite . however, this adaptation depends on the correlation between targets, and still needs annotation data of "new" targets. to solve the problem of missing annotating data of "new" targets in the real world scenario, the task of zero-shot stance detection (zssd), aiming at classifying stances for a large number of unseen targets without training data, has emerged @xcite . a great challenge in the task of zssd is the generalization ability of the model due to the lack of labeled data of specific targets. some studies try to introduce external knowledge (e.g. knowledge graph) to capture the correlation between targets @xcite . however, due to the existence of domain specific features, directly transferring stance features from seen targets to unseen targets may not result in good performance. to tackle this problem, we propose generating highquality training data for unseen targets based on the training data of seen targets. in order to ensure the quality of the generated samples, we use the discriminator and generator in generative adversarial networks (gans) for adversarial learning @xcite . in addition, we conduct hybrid contrastive learning on the synthesized samples and the ground-truth target samples @xcite , to remove the noise data irrelevant to the target and improve the quality of generated samples @xcite . through the instance-level and the class-level contrastive learning, the knowledge in different spaces can be effectively transferred. experimental results on two public zssd data sets show that our method can significantly outperform various competitive baselines. the ablation experiment proves the effectiveness of each part of our proposed model. the main contributions of our work can be summarized as follows: ## relate work existing research on stance detection can be roughly divided into target-specific and cross-target stance detection @xcite . for target-specific stance detection, most studies rely on a large number of labeled data for the specific target to train classifiers through different deep learning models @xcite @xcite . while cross-target stance detection is based on the correlations between user's stances on different targets. researchers proposed many approaches, such as attention-based model @xcite , memory-based model @xcite and graph-based model @xcite , to capture the underlying knowledge transferred between targets. further, @xcite proposed a novel data augmentation approach by predicting the masked token and replacing a mentioned target with another that achieved promising performance on multi-target stance detection. unlike the above tasks, zero-shot stance detection aims learning a classifier that is evaluated on a large number of completely new targets. @xcite introduced a new dataset of news article comments for zero-shot stance detection and proposed a topic-grouped attention model to implicitly construct relationships between the seen and unseen targets. inspired by adversarial learning for domain adaptation, @xcite extracted target-invariant transformation features by adversarial learning. in addition, by introducing commonsense knowledge, @xcite make use of the structural-level and semantic-level information of relational subgraphs to strengthen generalization capability of the model. the above work focused on using existing features to embed a text and any possible attribute description into their corresponding latent representations. the main goal of this embedding based approach is to map textual features and attribute descriptions into a common embedding space by using projection functions, which are learned by deep networks @xcite . however, the method based on embedding is more inclined to predict the seen class labels as the output, it will cause the problems of distribution deviation and domain shift. in order to overcome this problem, some works generate training data for the unseen classes through the generation model @xcite @xcite . by augmenting the data of the target domain, zero-shot classifier can be trained on all samples of known and unknown classes. our work is based on these considerations. ## methodology in this section, before introducing our proposed generative data augmentation framework with contrastive learning (gda-cl for short), we will first define the task of zssd. the overall model is shown in figure 2 , which consists of three parts, namely training data generation, hybrid contrastive learning and fine-tuning in stance classification. ## fine-tuning in stance classification as the generator model is trained on the labeled data of seen targets, the generated texts x g may contain some noise unrelated to the unseen target. therefore, directly applying all the generated data d g to the training classifier c may lead to the prediction biases. in this section, we make a data selection strategy to the generated data, and train the classifier with the filtered data set. ## experiments in this section, we perform experiments to answer the following research questions: rq1. can the generative data augmentation approach effectively improve the performance of zero-shot stance detection? if so, how much improvement is our method compared with other baselines? rq2. can generative adversarial networks (gans) and contrastive learning help to improve the quality of the generated texts? rq3. how sensitive is our method to the parameters? ## conclusion in this article, we propose a generative data augmentation model that generates high-quality training data for unseen targets by adversarial learning and contrastive learning, for the task of zero-shot stance detection. we conducted a series of experiments to evaluate our approach against several state-of-the-art models on two benchmark datasets and found that our method outperforms the base-lines significantly. through qualitative analysis and visual analysis, we show that the generated texts for unseen targets have good fluency while maintaining semantics.