Venue: NAACL 2024

# LLM-Driven Knowledge Injection Advances Zero-Shot and Cross-Target Stance Detection

Zhao Zhang1,3, Yiming Li2,3, Jin Zhang1,*, Hui Xu1

###### Abstract

Stance detection aims at inferring an author's attitude towards a specific target in a text. Prior methods mainly consider target-related background information for a better understanding of targets while neglecting the accompanying input texts. In this study, we propose to prompt Large Language Models (LLMs) to explicitly extract the relationship between paired text and target as contextual knowledge. We then inject such LLM-driven knowledge into a generation model BART to exploit the rich contexts and semantics. Moreover, to further enhance the decoding capability of BART, a novel prototypical contrastive scheme is designed to align input contents with stance labels. Our experimental results demonstrate the state-of-the-art performance across several publicly available datasets, showcasing effectiveness in both zero-shot and cross-target stance detection scenarios. We publicly release our code to facilitate future research.1

Footnote 1: [https://github.com/zhangzhao219/LKI-BART](https://github.com/zhangzhao219/LKI-BART)

\begin{tabular}{l l}  & **Target:** Joe Biden \\  & **Text:** To correct all am a fRepublican and will vote for gBoeliden. \\  & **Prior Target-Related Knowledge:** Joe Biden was a member of the Democratic Party. (Q) contains confusing information that the author and Joe Biden belong to different parties \\  & **Our LLM-Driven Knowledge:** Despite his Republican affiliation, the author has made a decision to vote for Joe Biden. (Q) resolves the confusion by analyzing the relationships between the fact and target \\ \end{tabular}

## 1 Introduction

The objective of stance detection is to ascertain an individual's stance regarding a specified target in a text, which may either be explicitly mentioned or implied only Kucuk and Can (2022). The significance of stance detection is evident in its role across various fields, such as predicting election and referendum outcomes Kawinitranon and Singh (2021), classifying rumors Lin et al. (2021), detecting fake news Hanselowski et al. (2018), and identifying instances of disinformation Hardalov et al. (2022).

Stance detection can be divided into three categories according to the availability of test targets: 1) target-specific: stance detection for fixed targets Hasan and Ng (2014); 2) cross-target: stance detection for related targets Augenstein et al. (2016); 3) zero-shot: stance detection for unseen targets Allaway and McKeown (2020). Among these tasks, the latter two are more challenging since test targets are inaccessible in the training data. One promising solution is to incorporate background knowledge as additional information, which previous works mainly focus on He et al. (2022). Despite the understanding of specific targets being improved, such approaches solely consider target-related knowledge but ignore the relationship between texts and targets, which might mislead the stance detection model as shown in Figure 1.

Recently, LLMs, such as ChatGPT released by OpenAI2, have shown superior natural language understanding performance. In this work, we specially design the prompt to make the most of ChatGPT's internal knowledge and reasoning capabilities for extracting the relationship between texts and targets explicitly. We refer to the consequent response as LLM-driven knowledge, a snippet of which can also be found in Figure 1. Compared to prior ones, LLM-driven knowledge is more specific and context-rich for stance detection.

Footnote 2: [https://openai.com/chatgpt](https://openai.com/chatgpt)

In order to bridge the semantic gap between LLM-driven knowledge and stance labels, we adopt a bidirectional autoregressive model - BART Lewis et al. (2020); Wen and Hauptmann (2023) as our backbone. As a result, the detection scheme is reformulated as a conditional stance label generation task, where the condition consists of a partially filled template, the input text, and knowledge derived from LLMs. This transformation allows us

Figure 1: An example comparing our LLM-driven knowledge with previous target-related knowledge.



to go beyond direct classification and leverage rich semantics for stance decoding.

Based on the above framework, we further devise a prototypical contrastive learning scheme to align stance representations with label semantics. Specifically, we adopt class-wise prototypes to model the feature space. Then a contrastive loss is optimized, forcing stance representations to be appealed to the corresponding prototype and repelled to prototypes of other stances. As a result, per-class representations can be clustered compactly on the feature space, making it easier to learn a mapping from stance representations to label semantics.

We refer to our method as LLM-Driven **K**nowledge **I**njection BART (LKI-BART), and an overview is depicted in Figure 2. Extensive experiments on two benchmark datasets suggest that LKI-BART achieves state-of-the-art performance on zero-shot and cross-target stance detection tasks. Additional ablation studies also indicate the effectiveness of each technique, namely LLM-driven knowledge, BART backbone, and prototypical contrastive learning, in LKI-BART.

## 2 Related Work

**Zero-Shot and Cross-Target Stance Detection with External Knowledge.** In addition to designing algorithms to learn transferable features, several works introduce external knowledge for a deeper understanding of unseen targets. Liu et al. (2021) involved commonsense knowledge based on the graph structure, Zhu et al. (2022) incorporated external knowledge from Wikipedia. In recent studies, He et al. (2022); Wen and Hauptmann (2023); Hanley and Durumeric (2023) have reached a consensus on the usage of external knowledge. However, none of them considers explicitly acquiring the relationship between texts and targets, which may result in redundant and confusing information.

**Utilize LLMs for Stance Detection.** Recently, some works have directly used LLMs for stance detection. Zhang et al. (2023a,b) explores various methods to prompt ChatGPT for stance reasoning. Lan et al. (2023) adopts distinct LLM-based Agents to create a collaborative stance detection system. Also, some studies have expressed a negative opinion on using LLMs for stance detection. Zhu et al. (2023) uses ChatGPT to annotate stance labels but obtains inferior results. Cruickshank and Ng (2023) conclude that overall accuracy is not much better than supervised models.

## 3 Approach

### Problem Formulation

Formally, given a text \(x\) and a target \(t\), the stance detection task aims to identify the stance \(y\) that \(x\) expresses towards \(t\). \(y\) is basically in the collection \(S=\{favor,against,neutral\}\), which may vary in different datasets. The detection model is trained to infer \(y\) given \(x\) and \(t\) with parameter \(\theta\). If knowledge \(k\) is involved, the formulation will be:

\[f(x,t,k;\theta)=y\]

### LLM-Driven Knowledge

To acquire LLM-driven knowledge for stance detection, we employ a partially filled zero-shot prompt for each input text in every dataset, as shown in Appendix A. Specifically, our prompt is designed with three aspects in mind. First, we instruct the LLM to _list keywords_. In that case, the subsequent detection model may pay more attention to these words. Second, we prompt the LLM to _analyze implied emotions and rhetorical devices_, as these may be strong pieces of evidence for the expressed stance. Finally, unlike previous studies, we ask the LLM to _briefly analyze the stance_ rather than produce exact answers directly, reducing the risks of intrinsic hallucination in LLMs, which may mislead the subsequent detection model into generating incorrect predictions. Besides, we add a brief description of the dataset at the beginning, so that LLM may find more background information concerning the dataset by retrieving its internal knowledge.

### BART Backbone

As LLM-driven knowledge may cover rich contextual information, it is vital to associate such information with stance semantics. Recently, generative modeling has shown great potential by leveraging pre-training objectives to decode the answer (Radford et al., 2019). Inspired by prior works (Wen and Hauptmann, 2023), we inject LLM-driven knowledge into BART (Lewis et al., 2020), an auto-regressive transformer, for stance detection.

In that case, the task is reframed as a denoising one, which takes \(h(x,t,k)\) as input and generates an output sequence \(u\) containing stance labels. Specifically, \(h(x,t,k)\) is a combination of input text \(x\), target \(t\) and LLM-driven knowledge \(k\) with special tokens "
 <stance> is the stance for the target \(t\) 

  \(x\) 

 \(k\) 
", and \(u\) is formulated as "
 <stance> is the stance 

for the target \(t
^{*}\). Note that the <stance> placeholder is kept in input while being replaced by the predicted stance in output.

Finally, the BART model is trained by maximizing the log-likelihood over the whole sequence.

\[\mathcal{L}_{gen}=-\sum_{i=1}^{|\boldsymbol{u}|}\log p(u_{i}|,\boldsymbol{u}_{< i},h(x,t,k);\theta)\]

where \(p(u_{i}|,\boldsymbol{u}_{<i},h(x,t,k);\theta)\) is the probability to select a token \(u_{i}\) at step \(i\) given the input \(h(x,t,k)\) and previously generated tokens \(\boldsymbol{u}_{<i}\).

### Prototypical Contrastive Learning

One may notice that \(\mathcal{L}_{gen}\) maximizes the likelihood over the entire sentence. In fact, more optimizations may be beneficial especially when decoding the stance label. To this end, we decouple the stance embedding \(\boldsymbol{z}\in\mathbb{R}^{embed\_size}\) (the embedding used to generate the stance token) from BART decoder outputs. We then project it into a low-dimensional vector \(\hat{\boldsymbol{z}}\in\mathbb{R}^{low\_embed\_size}\) to prevent \(\boldsymbol{z}\) losing much semantics and being overcorrected. To regularize \(\hat{\boldsymbol{z}}\) to be more discriminative in the latent space, inspired by Li et al. (2021a), we introduce the concept of prototypes, they are widely adopted in data-efficient learning Li et al. (2024) and can be viewed as the representatives of class-wise embeddings. However, different from Li et al. (2021a), we view prototypes as the representatives of class-wise projected embeddings instead of class-agnostic ones. By interacting with prototypes, a contrastive loss is employed to increase the intra-class similarity but decrease the inter-class similarity of projected stance embeddings. In the following, we will detail how to estimate class-wise prototypes and formulate the contrastive loss.

Online Prototype UpdateFor each stance class \(c\), we randomly initialize a vector \(\boldsymbol{v}_{c}\) as its prototype before normalizing it into a unit one at the beginning. Along the training progress, we update \(\boldsymbol{v}_{c}\) at each step in a moving average manner by,

\[\boldsymbol{v}_{c}\leftarrow\text{Normalize}(\beta\boldsymbol{v}_{c}+(1- \beta)\boldsymbol{v}_{c}^{\prime})\]

where \(\beta\) is a momentum coefficient, \(\text{Normalize}(\cdot)\) is the normalization function, and \(\boldsymbol{v}_{c}^{\prime}\) is the centroid of embeddings belonging to class \(c\) in the batch.

Prototypical Contrastive LossTo compute the loss, we firstly obtain the embedding-to-class cosine similarity score \(s_{j}=<\hat{\boldsymbol{z}},\boldsymbol{v}_{j}>\). Then, we optimize the following loss,

\[\mathcal{L}_{con}=-\sum_{c=1}^{C}y_{c}\log\frac{\text{exp}(\frac{s_{c}}{ \gamma})}{\sum_{j=1}^{C}\text{exp}(\frac{s_{j}}{\gamma})}\]

where \(\gamma\) is a scalar temperature parameter and \(\boldsymbol{y}\) is the one-hot label for the current sample. If it belongs to class \(c\), optimizing \(\mathcal{L}_{con}\) will maximize \(s_{c}\) but minimize \(s_{j}\) (\(j=1,\cdots,C\) and \(j\neq c\)), thereby pulling together \(\hat{\boldsymbol{z}}\) and class \(c\)'s prototype \(\boldsymbol{v}_{c}\) while pushing away \(\hat{\boldsymbol{z}}\) from prototypes of other classes. Consequently, a well-structured feature space is modeled with stance embeddings from the same class clustering together, making it easier to reach semantic alignment.

Relation with JointCLAlthough previous work, namely JointCL Liang et al. (2022b), also introduced the concept of prototypes, our approach is

Figure 2: Overview of our proposed LKI-BART.

 

fundamentally distinctive from it. As for the purpose of prototypes, JointCL obtains class-agnostic prototypes to build a graph dedicated to adapting and refining representations of unseen targets, while we rely on class-specific prototypes to regularize the feature space. Besides, JointCL still follows the traditional supervised contrastive loss Khosla et al. (2020), forcing larger cosine similarity between samples that share the same stance label, while prototypes are not involved in contrastive loss at all. By contrast, our method encourages large similarity scores between a sample's embedding and its corresponding prototype, which is less computationally intensive and more robust, since potential mis-calibration can be alleviated by viewing prototypes as representatives of class-wise embeddings.

The overall loss function of LKI-BART considering both \(\mathcal{L}_{gen}\) and \(\mathcal{L}_{con}\) is defined as follows:

\[\mathcal{L}=\lambda_{l}\cdot\mathcal{L}_{gen}+(1-\lambda_{l})\cdot\mathcal{L} _{con}\]

where \(\lambda_{l}\) is involved to balance the optimization.

## 4 Experiments

We evaluate our LKI-BART on VAST Allaway and McKeown (2020) and P-Stance Li et al. (2021), please refer to Appendix B and Appendix C for details of the training datasets as well as our experimental setup.

### Results

#### 4.1.1 Zero-Shot Stance Detection

We test our model on VAST, where the model is trained on thousands of targets and evaluated on targets not present in the training data.

**Baselines.** We compare our model with StSQA Zhang et al. (2023), GDA-CL Li and Yuan (2022), PT-HCL Liang et al. (2022), JointCL Liang et al. (2022), BS-RGCN Luo et al. (2022), COLA Lan et al. (2023), TarBK-BERT Zhu et al. (2022), WS-BERT-Single He et al. (2022), CondGen Wen and Hauptmann (2023), KASD-BERT Li et al. (2023) and TATA Hanley and Durumeric (2023).

**Results and Analysis.** The results of LKI-BART on VAST are presented in Table 1. Notably, LKI-BART exhibits a significant performance improvement over prior approaches, validating its effectiveness on the zero-shot stance detection task.

**Ablation Study on LLM-Driven Knowledge.** To validate the effectiveness of our proposed LLM-driven knowledge, we integrate different components of it into our model as well as target-related knowledge from Wikipedia He et al. (2022). The results are presented in Table 2, showing that each component contributes to the performance gain and the combination of them can complement each other and generally yields a superior result beyond Wiki knowledge only. Besides, we observe a significant improvement with the "Analyse stance" part. This is because instructing LLMs to analyze the stance directly seems to be the most straightforward way. Nevertheless, the other components also play essential roles. In cases where "Analyse stance" leads to an incorrect answer, our LKI-BART can still correct it with the guidance of other parts, suggesting its robustness.

**Ablation Study on BART Backbone.** As LLM-driven knowledge may cover rich contextual information, it is vital to associate such information with stance semantics. Therefore, we choose a seq2seq generative model BART as our backbone. To further verify this choice, we try two alternatives of BART to examine its effects in Table 3: 1) feed the LLM-driven knowledge into a BERT Devlin et al. (2019) classification model (denoted as BERT); 2) feed the LLM-driven knowledge into another LLM and use its few-shot in-context abili

\begin{table}
\begin{tabular}{c c c c c} \hline  & \multicolumn{5}{c}{**VAST**} \\

- & 68.9 \\ GDA-CL & 59.8 & 62.3 & 89.3 & 70.5 \\ PT-HCL & 61.7 & 63.5 & 89.6 & 71.6 \\ JointCTL & 64.9 & 63.2 & 88.9 & 72.3 \\ BS-RGCN & 60.8 & 67.4 & 89.5 & 72.6 \\ COLCA & 73.4 & 77.2 &
- & 73.4 \\ TarBK-BERT & 65.7 & 63.9 & 91.2 & 73.6 \\ WS-BERT-Single &
- & 75.3 \\ CondGen &
- & 76.4 \\ KASD-BERT &
- & 76.8 \\ TATA & 69.5 & 71.1 & 90.5 & 77.1 \\ \hline LKI-BART & 75.1 & 72.9 & 90.7 & **79.6** \\ \hline \end{tabular}
\end{table}
Table 1: Experimental results on VAST.

\begin{table}
\begin{tabular}{c c c c c} \hline \multirow{2}{*}{**Component**} & \multicolumn{4}{c}{**VAST**} \\  & Sup & Aga & Neu & Avg \\ \hline None & 65.9 & 68.4 & 90.1 & 74.8 \\ Wiki & 71.4 & 71.8 & 87.4 & 76.9 \\ Keywords & 67.2 & 70.3 & 89.6 & 75.7 \\ Implied emotions & 67.6 & 72.5 & 90.6 & 76.9 \\ Rhetorical devices & 67.7 & 71.0 & 89.9 & 76.2 \\ Analyse stance & 71.2 & 71.9 & 89.3 & 77.4 \\ All & **75.1** & **72.9** & **90.7** & **79.6** \\ \hline \end{tabular}
\end{table}
Table 2: Ablation study on different knowledge types or components as contextual information for BART model.

 

ties to directly infer the stance (denoted as few-shot LLM). As shown, training a seq2seq BART separately performs best, as it can learn how to leverage the knowledge provided by LLMs more effectively.

#### 4.1.2 Cross-Target Stance Detection

We adopt P-Stance for cross-target stance detection evaluation, where our model is trained on one target and tested on another related target.

**Baselines.** We compare our model with BiCE Augenstein et al. (2016), CrossNet Xu et al. (2018), BERTweet Li et al. (2021) and WS He et al. (2022).

**Results and Analysis.** The results of our model on P-Stance are shown in Table 4. We can observe that LKI-BART outperforms the previous best model by 10-15 F1-pts. Interestingly, we note that among the three targets, our model performs best on JB, followed by DT and BS. We speculate that this discrepancy arises from JB being the current president and DT being the former president. In that case, LLMs possess more internal information about JB and DT thereby producing more informative knowledge on them after prompting.

**Feature Visualizations.** Figure 3 shows the t-SNE van der Maaten and Hinton (2008) visualization of stance embeddings (the definition is given in Section 3.4) from the vanilla BART model and our LKI-BART model on the test set. Specifically, the vanilla BART model is based on the generation framework but trained without LLM-driven knowledge or prototypical contrastive loss. As shown, the visualization of the vanilla BART shows basically no clusters, while embeddings with our LKI-BART are gathered according to their labels.

**Ablation Study on Training Strategies.** To validate the effects of our proposed strategies, we conduct several experiments. The corresponding results are listed in Table 5. As seen, LC greatly boosts the detection ability by explicitly modeling relationships between texts and targets while PCL can further bring about improvements as it helps to regularize the feature space. Besides, our PCL surpasses its variant SCL, which may be easily influenced by in-batch outliers due to its sample-to-sample contrastive formula.

## 5 Conclusion

In this paper, we propose to collect LLM-driven knowledge to incorporate connections between input texts and unseen targets for zero-shot and cross-target stance detection. A generation framework BART is adopted to better leverage LLM-driven knowledge for detection and a prototypical contrastive loss is optimized for better alignment between input materials and stance semantics. Combining all the above techniques, our LKI-BART finally achieves state-of-the-art performance on VAST and P-Stance datasets.

\begin{table}
\begin{tabular}{l c c c c} \hline  & \multicolumn{4}{c}{**P-Stance**} \\ \cline{2-5}
**Target** & CrossNet & BERTweet & WS & LKI-BART \\ \hline DT\(\rightarrow\) JB & 56.67 & 58.88 & 68.30 & **85.02** \\ DT\(\rightarrow\) BS & 50.08 & 56.50 & 64.40 & **79.57** \\ JB\(\rightarrow\)DT & 60.43 & 63.64 & 67.70 & **80.74** \\ JB\(\rightarrow\) BS & 60.81 & 67.04 & 69.00 & **79.65** \\ BS\(\rightarrow\) DT & 52.99 & 58.75 & 63.60 & **80.91** \\ BS\(\rightarrow\) JB & 62.51 & 72.99 & 76.80 & **85.56** \\ \hline \end{tabular}
\end{table}
Table 4: Experimental results on P-Stance, where JB, DT, BS are short for Joe Biden, Donald Trump, and Bernie Sanders, respectively.

Figure 3: The t-SNE visualization of stance embeddings from the vanilla BART (left) and our LKI-BART (right). We use random initialization with perplexity as 50, blue stands for AGAINST label, and red stands for FAVOR label.

\begin{table}
\begin{tabular}{c c c c c} \hline  & \multicolumn{4}{c}{**VAST**} \\ \cline{2-5}
**Backbone** & Sup & Aga & Neu & Avg \\ \hline few-shot LLM & 68.7 & 67.8 & 72.4 & 69.6 \\ BERT & 73.5 & **74.3** & 85.2 & 77.7 \\ BART & **75.1** & 72.9 & **90.7** & **79.6** \\ \hline \end{tabular}
\end{table}
Table 3: Ablation study on different backbones.

 



### Limitations

LKI-BART relies on knowledge generated by LLMs. Due to constraints in budget and time, we only experiment with GPT-3.5-turbo from Azure. We encourage further exploration by researchers to compare various LLMs and prompt formats.

## Ethical Statement

In this research, it's crucial to acknowledge the potential limitations of LLMs. Although Azure has made significant progress to guard against abuse and unintended harm, ChatGPT may also produce biased information on certain targets as many other LLMs, especially on targets related to people. However, we do not adopt any additional processing for LLM-driven knowledge, while other parts of the training data come from publicly available datasets that are commonly employed in prior research. We keep fair and honest in our analysis of experimental results. Additionally, our LKI-BART is extremely lightweight and allows the reproduction of the experiments on common GPUs. We have made our code accessible for future investigations.

## References

* A. Alaway and K. McKeown (2020)Zero-shot stance detection: a dataset and model using generalized topic representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, pp. 8913-8931. External Links: Link Cited by: SS4.
* I. J. Cruickshank and L. Ng (2023)Use of large language models for stance classification. External Links: 2306.07886 Cited by: SS4.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171-4186. External Links: ISBN 978-3-319-1422-1, Document Cited by: SS1.
* I. Habernal, H. Wachsmuth, I. Gurevych, and B. Stein (2018)The argument reasoning comprehension task: identification and reconstruction of implicit warrants. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, Louisiana, pp. 1930-1940. External Links: Link, Document Cited by: SS1.
* H. W. A. Hanley and Z. Durumeric (2023)Tata: stance detection via topic-agnostic and topic-aware embeddings. External Links: 2306.07886 Cited by: SS1.
* A. Hanselowski, A. PVS, B. Schiller, F. Caspelherr, D. Chaudhuri, C. M. Meyer, and I. Gurevych (2018)A retrospective analysis of the fake news challenge stance-detection task. In Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA, pp. 1859-1874. External Links: Link, Document Cited by: SS1.
* M. Hardalov, A. Arora, P. Nakov, and I. Augenstein (2022)A survey on stance detection for mis- and disinformation identification. In Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, United States, pp. 1259-1277. External Links: Link, Document Cited by: SS1.
* K. E. Hasan and V. Ng (2014)Why are you taking this stance? identifying and classifying reasons in ideological debates. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 751-762. External Links: Link, Document Cited by: SS1.
* Z. He, N. Mokhberian, and K. Lerman (2022)Infusing knowledge from Wikipedia to enhance stance detection. In Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, Dublin, Ireland, pp. 71-77. External Links: Link, Document Cited by: SS1.
* K. Kawinitranon and L. Singh (2021)Knowledge enhanced masked language model for stance detection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online, pp. 4725-4735. External Links: Link, Document Cited by: SS1.
* P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan (2020)Supervised contrastive learning. Advances in neural information processing systems33, pp. 18661-18673. External Links: Link, Document Cited by: SS1.
* D. Kucuk and F. Can (2022)A tutorial on stance detection. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, WSDM '22, New York, NY, USA, pp. 1626-1628. External Links: Link, Document Cited by: SS1.



Xiaochong Lan, Chen Gao, Depeng Jin, and Yong Li. 2023. Stance detection with collaborative role-infused llm-based agents.
* Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7871-7880, Online. Association for Computational Linguistics.
* Li et al. (2023) Ang Li, Bin Liang, Jingqian Zhao, Bowen Zhang, Min Yang, and Ruifeng Xu. 2023. Stance detection on social media with background knowledge. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 15703-15717, Singapore. Association for Computational Linguistics.
* Li et al. (2021a) Junnan Li, Pan Zhou, Caiming Xiong, and Steven C. H. Hoi. 2021a. Prototypical contrastive learning of unsupervised representations.
* Li and Yuan (2022) Yang Li and Jiawei Yuan. 2022. Generative data augmentation with contrastive learning for zero-shot stance detection. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 6985-6995, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_, pages 271-275.
* Li et al. (2021b) Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Jayaaraman Nair, Diana lnkpen, and Cornelia Carageaa. 2021b. P-stance: A large dataset for stance detection in political domain. In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_, pages 2355-2365, Online. Association for Computational Linguistics.
* Liang et al. (2022a) Bin Liang, Zixiao Chen, Lin Gui, Yun He, Min Yang, and Ruifeng Xu. 2022a. Zero-shot stance detection via contrastive learning. In _Proceedings of the ACM Web Conference 2022_, WWW '22, page 2738-2747, New York, NY, USA. Association for Computing Machinery.
* Liang et al. (2022b) Bin Liang, Qinglin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, and Ruifeng Xu. 2022b. JointCL: A joint contrastive learning framework for zero-shot stance detection. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 81-91, Dublin, Ireland. Association for Computational Linguistics.
* Lin et al. (2021) Hongzhan Lin, Jing Ma, Mingfei Cheng, Zhiwei Yang, Liangliang Chen, and Guang Chen. 2021. Rumor detection on Twitter with claim-guided hierarchical graph attention networks. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 10035-10047, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Liu et al. (2021) Rui Liu, Zheng Lin, Yutong Tan, and Weiping Wang. 2021. Enhancing zero-shot and few-shot stance detection with commonsense knowledge graph. In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_, pages 3152-3157, Online. Association for Computational Linguistics.
* Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization.
* Luo et al. (2022) Yun Luo, Zihan Liu, Yuefeng Shi, Stan Z. Li, and Yue Zhang. 2022. Exploiting sentiment and common sense for zero-shot stance detection. In _Proceedings of the 29th International Conference on Computational Linguistics_, pages 7112-7123, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners.
* van der Maaten and Hinton (2008) Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. _Journal of Machine Learning Research_, 9(86):2579-2605.
* Wen and Hauptmann (2023) Haoyama Wen and Alexander Hauptmann. 2023. Zero-shot and few-shot stance detection on varied topics via conditional generation. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 1491-1499, Toronto, Canada. Association for Computational Linguistics.
* Xu et al. (2018) Chang Xu, Cecile Paris, Surya Nepal, and Ross Sparks. 2018. Cross-target stance classification with self-attention networks. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 778-783, Melbourne, Australia. Association for Computational Linguistics.
* Zhang et al. (2023a) Bowen Zhang, Daijun Ding, and Liwen Jing. 2023a. How would stance detection techniques evolve after the launch of chatgpt?
* Zhang et al. (2023b) Bowen Zhang, Xianghua Fu, Daijun Ding, Hu Huang, Yangyang Li, and Liwen Jing. 2023b. Investigating chain-of-thought with chatgpt for stance detection on social media.
* Zhu et al. (2022) Qinglin Zhu, Bin Liang, Jingyi Sun, Jiachen Du, Lanjun Zhou, and Ruifeng Xu. 2022. Enhancing zero-shot stance detection via targeted background knowledge. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_, SIGIR '22, page 2070-2075, New York, NY, USA. Association for Computing Machinery.

 

Yiming Zhu, Peixian Zhang, Ehsan-Ul Haq, Pan Hui, and Gareth Tyson. 2023. Can chatgpt reproduce human-generated labels? a study of social computing tasks.

## Appendix A Prompt Details

Assuming that you are a social expert, you are given a short Passage of **{dataset_description}** below, please think step by step, extract the keywords in the passage, analyze the author's implied emotions, rhetorical devices, etc., finally briefly analyze the author's stance on the Target, paying attention to giving the process of analysis without giving the conclusion.

Passage:**{text}**

Target:**{target}**

## Appendix B Datasets

Allaway and McKeown (2020) construct a dataset named VAST with varied topics for evaluating zero-shot stance detection, the original examples of VAST are collected from Habernal et al. (2018) under Apache-2.0 license3, while P-Stance by Li et al. (2021b) is a commonly adopted benchmark for cross-target stance detection, under MIT license4.

Footnote 3: [https://github.com/UKPLab/argument-reasoning-comprehension-task/blob/master/LICENSE](https://github.com/UKPLab/argument-reasoning-comprehension-task/blob/master/LICENSE)

Footnote 4: [https://github.com/chuchun8/Pstance/blob/main/LICENSE](https://github.com/chuchun8/Pstance/blob/main/LICENSE)

1) VAST contains 18,548 comments from New York Times "Room for Debate" section with 5,630 different targets for zero-shot and few-shot stance detection. Each instance can be classified as Favor, Against, or Neutral. The statistics are summarized in Table 6.

2) P-Stance contains 21,574 political tweets with stance annotations for "Donald Trump", "Joe Biden", and "Bernie Sanders". Each tweet is annotated with a stance label "Favor" or "Against". The statistics are summarized in Table 7.

## Appendix C Experimental Setup

We use unmodified part-base from huggingface.co5. All our experiments are carried out on a single NVIDIA A100 40G with 50 epochs, which generally take about 3 hours for training. We use a base learning rate of 5e-6 with a warm-up proportion of 0.1 and AdamW Loshchilov and Hutter (2019) is adopted as the optimizer. The training batch size is defined as 64. For prototypical contrastive learning, hyper-parameters \(\tau\), \(\lambda_{l}\), and \(\beta\) are set to 0.1, 0.8, and 0.99 respectively. Test results are reported based on the best overall F1 performance on the development set, using the averaged results from 5 different random seeds. Following the experimental setup by previous works, we use macro-F1 as the evaluation metric.

\begin{table}
\begin{tabular}{c|r r r} \hline  & **Train** & **Dev** & **Test** \\ \hline \# Examples & 13,477 & 2,062 & 3,006 \\ \# Unique Comments & 1,845 & 682 & 786 \\ \# Zero-shot Topics & 4,003 & 383 & 600 \\ \# Few-shot Topics & 638 & 114 & 159 \\ \hline \end{tabular}
\end{table}
Table 6: Statistics of VAST dataset.

\begin{table}
\begin{tabular}{c c|r r r} \hline  & & **Trump** & **Biden** & **Sanders** \\ \hline
**Train** & Favor & 2,937 & 2,552 & 2,858 \\  & Against & 3,425 & 3,254 & 2,198 \\
**Dev** & Favor & 365 & 328 & 350 \\  & Against & 430 & 417 & 284 \\
**Test** & Favor & 361 & 337 & 343 \\  & Against & 435 & 408 & 292 \\ \hline \end{tabular}
\end{table}
Table 7: Statistics of P-Stance dataset.