Guiding Computational Stance Detection with Expanded Stance Triangle Framework
Year: 2023

Abstract:
Stance detection determines whether the author of a piece of text is in favor of, against, or neutral towards a specified target, and can be used to gain valuable insights into social media. The ubiquitous indirect referral of targets makes this task challenging, as it requires computational solutions to model semantic features and infer the corresponding implications from a literal statement. Moreover, the limited amount of available training data leads to subpar performance in out-of-domain and cross-target scenarios, as data-driven approaches are prone to rely on superficial and domain-specific features. In this work, we decompose the stance detection task from a linguistic perspective, and investigate key components and inference paths in this task. The stance triangle is a generic linguistic framework previously proposed to describe the fundamental ways people express their stance. We further expand it by characterizing the relationship between explicit and implicit objects. We then use the framework to extend one single training corpus with additional annotation. Experimental results show that strategically-enriched data can significantly improve the performance on out-of-domain and cross-target evaluation.

heading: 1 Introduction
text: Stance (and its variant stancetaking) is a concept defined as a linguistically articulated form of social action whose meaning is construed within language, interaction, and sociocultural value (Biber and Finegan, 1988; Agha, 2003; Du Bois, 2007; Kiesling, 2022). Its subject can be the speaker in a conversation or the author of a social media post, and its object can be in the form of an entity, concept, idea, event, or claim.1
The stance detection task in natural language processing aims to predict the stance of a piece
1There are various definitions of stance and stancetaking in pragmatics and sociolinguistics field. In this article, we follow the generic “stance act” defined by Du Bois (2007).
of text toward specified targets. Stance detection is commonly formulated as a classification problem (Küçük and Can, 2020), and is often applied to analyzing online user-generated content such as Twitter and Facebook posts (Mohammad et al., 2016; Li et al., 2021). When given the text and one specified target (i.e., stance object), a classifier is used to predict a categorical label (e.g., Favor, Against, None). Along with social networking platforms’ growing impact on our lives, stance detection is crucial for various downstream tasks such as fact verification and rumor detection, with wide applications including analyzing user feedback and political opinions (Glandt et al., 2021). For example, during the pandemic of COVID-19, it was essential to understand the public’s opinion on various initiatives and concerns, such as getting booster vaccinations and wearing facial masks. The insight from stance analysis could help public health organizations better estimate the expected efficacy of their mandates, as well as proactively detect pandemic fatigue before it leads to a serious resurgence of the virus.
While state-of-the-art results have been achieved on text classification by adopting data-driven neural approaches, especially utilizing recent largescale language backbones (Devlin et al., 2019; Liu et al., 2019), stance detection remains challenging;
3987
there is a substantial gap between human and machine performance.2 One challenge comes from the ubiquitous indirect referral of targeted stance objects. When interacting socially online, people express their subjective attitude with brevity and variety: they often do not directly mention the final target, but mention its related entities, events, concepts, or claims. As examples shown in Table 1, unlike aspect-based sentiment analysis, where aspect terms are usually explicitly stated in sentences, targets specified for stance labeling can be flexibly assigned. For instance, in a tweet about COVID-19, while “Dr. Fauci” is not mentioned, one can infer that the user stands for him from the support of “wearing a mask” and “science”. Therefore, target-aware context understanding requires capturing the relationship of explicitly-mentioned objects and various targets, but existing models lack such capability.
Another challenge stems from limited annotated data for stance detection. When training on a corpus constructed with a small number of targets from a single domain, data-driven approaches cannot generalize well on out-of-domain samples and unseen targets (Allaway and Mckeown, 2020; Kaushal et al., 2021). Meanwhile, due to low data diversity and the spurious correlation caused by single target labeling, models are prone to over-fit on superficial and biased features (e.g., sentimentrelated lexicon). The strong baselines are observed to solely rely on the input text (e.g., tweets) but neglect the specified target (Ghosh et al., 2019; Kaushal et al., 2021), and fail to make correct predictions when we change the targeted object. As shown in Figure 1, the classifier always produces the same output Favor, even when irrelevant targets such as “CD Disk” are indicated.
In this work, we investigate solutions for the aforementioned challenges from a linguistic perspective. The pragmatic and linguistics studies provide us with detailed theories of how humans perform stancetaking (Du Bois and Kärkkäinen, 2012; Kiesling et al., 2018), and help us identify the key components and inference paths for stance analysis. The “Stance Triangle” (Du Bois, 2007) is one of the most influential and generic linguistic frameworks.
2Recent empirical evaluation studies show that large language models (LLMs) can provide reasonable results on various NLP tasks including stance detection. However, the performance of adopting zero-shot and few-shot inference with LLMs is still lower than task-specific fine-tuned approaches, and LLMs require great amounts of computational resources (Ziems et al., 2023; Bang et al., 2023).
As shown in Figure 1, it presents three stancetaking acts: a subject (i.e., the stance holder) evaluates an object, positions themselves and others, and aligns with other subjects. While this model covers the important aspects of stancetaking, its broadness leaves the operationalization of stance in practical use cases under-specified (Kiesling, 2022). Regarding stance analysis of social networking platforms, modeling the implication of targets is important, but it is not well-formulated in the triangle framework. Therefore, we expand it by delineating the relationship between explicit and implicit objects, and outline two paths to complete the human-like inference. Aside from using the expanded framework for qualitative analysis, we further utilize it for strategic annotation enrichment, which shows strong potential to improve the robustness and generality of data-driven approaches. In summary, our contributions of this work are as follows:
• We make the first attempt to expand the linguistic framework “stance triangle” for improving computational stance detection, by characterizing the relationship and labels of explicit and implicit objects.
• We conduct qualitative analysis following the expanded framework on tweet stance detection, and outline the primary aspects and inference paths.
• We leverage the proposed framework to enrich the annotation of a single-domain corpus, and empirically demonstrate its effectiveness in improving the performance of out-of-domain and cross-target generalization.

heading: 2 The Stance Triangle Framework
text: In linguistics, stance is originally defined as “the overt expression of an author’s or speaker’s attitudes, feelings, judgments, or commitment concerning the message” (Biber and Finegan, 1988). Sociolinguists further emphasized inter-subjective relations, which refers to how speakers position themselves with their own or other people’s views, attitudes, and claims toward particular objects or ideas in ongoing communicative interaction (Haddington et al., 2004; Jaffe et al., 2009). In this way, the core concept of stance is embedded with inherently dialogic and inter-subjective characteristics. When other interlocutors in the context indicate a stance on objects, the current speaker may take the same polarity or against their stance in various ways (Du Bois, 2007).
Given that stance is firmly grounded in the communicative interaction and stancetaking is crucial for the social construction of meaning in different discourses, Du Bois (2007) proposed a holistic analytic framework named “Stance Triangle”, which describes the communication situation in which two speakers create intersubjectivity through their evaluation of an object.
As shown in Figure 1, in the stance triangle, the three key components regarding stancetaking are located at the vertices, namely the current stance holder (Subject 1), what the stance is about (Object), and the other stance holders in context (Subject 2). The edges are then categorized into three types to reveal expression acts among interlocutors and objects. The evaluation and positioning are both Affective Expression (Du Bois and Kärkkäinen, 2012; Kiesling et al., 2018). Evaluation is tied to affect (Du Bois and Kärkkäinen,
2012). It refers to a person expressing an overt emotional reaction to or displaying an affective orientation toward the object. Positioning often occurs as a consequence of evaluation. The positioned subject is the one who makes the evaluation toward a specific object. By invoking an evaluation, the subject presents himself/herself as taking a particular affective orientation toward an object via the act of stancetaking. Alignment is used to highlight the similarities and differences of interlocutors’ stance. In communicative interaction, one could find his/her own stance is compared and contrasted with others. As communication goes on, interlocutors may line up affective stances on the shared object. Alternatively, they negotiate their stance and stay differential. Alignment thus becomes an essential part of intersubjectivity in dynamic dialogical interaction.
The stance triangle provides insights into the dialogic nature of stance and serves as an analytical framework to understand how stance is taken in interactions. There are many qualitative studies that are inspired by this framework and applied the fundamental theory to analyze various resources, such as story narration (Bohmann and Ahlers, 2022), and social media text (Simaki et al., 2018). While previous work demonstrates the success of the stance triangle in linguistics and social constructionism, to the best of our knowledge, few works use the linguistic framework to facilitate computational stance detection.

heading: 3 Our Proposed Linguistic Framework
text: While this triangle model describes the important aspects of stancetaking, its broadness leaves a few limitations when we adopt it to analyze online usergenerated content. First, the triangle model mainly
Example A: Greater is He who is in you than he who is in the world. - 1 John 4:4 Explicit Object: “1 John 4:4” (Quotation) Stance Label: Favor
Specified Target: “God” Explicit Mention: No Label Alignment: 1 Stance Label: Favor Specified Target: “Atheism” Explicit Mention: No Label Alignment: -1 Stance Label: Against
Example B: We remind ourselves that love means to be willing to give until it hurts - Mother Teresa Explicit Object: “Mother Teresa” (Quotation) Stance Label: Favor
Specified Target: “the unborn” Explicit Mention: No Label Alignment: 1 Stance Label: Favor Specified Target: “Abortion” Explicit Mention: No Label Alignment: -1 Stance Label: Against
(2) As Path-B shown in Figure 2, the current stance holder may align or dis-align with other stance holders. For Example A shown in Table 3, the speaker quoted a sentence from one chapter “John 4:4” of the Bible. This presents the speaker’s belief in God, and one can infer that the label of the specified target “Atheism” is Against. Moreover, regarding stance analysis of online social networking platforms such as Twitter, the alignment act can be extended to include quotations, re-tweet behavior, and the ‘Like’ button.
The expanded linguistic framework describes the key components, expression acts, and inference flows of stance detection, and it is helpful for qualitative and quantitative analysis, especially for the implicitly-mentioned stance objects. More importantly, the expanded framework sheds light on the challenging parts of computational stance detection. For instance, some domain knowledge is necessary for reasoning the label alignment between explicit objects and specified targets, and current tweetrelated datasets do not provide particular labeling of re-tweet and quotations. This framework paves the way for further research extension.

heading: 4 Theory-inspired Practice: Strategic Annotation Enrichment
text: Various corpora with target-aware stance annotation are constructed to facilitate computational solutions for stance detection (Mohammad et al., 2016; Allaway and Mckeown, 2020; Li et al., 2021; Glandt et al., 2021). However, most of them only adopt a simple annotation scheme, where a single target and its corresponding label are provided. Some recent datasets adopt a multi-target annotation (Kaushal et al., 2021), but the paired target number is limited.
According to our linguistic framework, modeling the implication of a specified object is important. Therefore, we enrich the annotation of one corpus from a single domain by adding multi-target stance labeling on explicit and implicit objects. We
select the data from SemEval2016 Task-6 A “tweet stance detection” (Mohammad et al., 2016) as it serves as a benchmark in many previous works (Küçük and Can, 2020). As shown in Table 4, it is built on tweets about a set of politics-relevant targets (e.g., politicians, feminism movement, climate change), and each sample only has one specified target with a human-annotated stance label.
We first obtain a sample subset where the specified target is not explicitly mentioned in the text. Next, to obtain explicit objects in an extractive manner, we apply an off-the-shelf constituency parsing model,3 and collect all noun phrases in the constituency tree. To reduce extraction noise, we filter out the noun-phrase candidates with some criteria (e.g., being not near the verbs in the sentence, being shorter than 4 characters, and being started with hashtag token and “@user”).
Then linguistic annotators are invited to label the stance polarity on the explicit objects. To reduce superficial target-related patterns and biases from single target labeling, and emphasize object-object relationship, here we propose and adopt an adversarial multi-target strategy, namely selecting the explicit object that shows a stance dis-alignment to the specified target (e.g., “unborn children” and “abortion” of Example B in Table 2). This adversarial multi-target labeling can encourage models to condition their prediction more on the given target, as well as learn some correlation between explicit and implicit objects. We obtain 1,500 paired samples, where the original training size is 2,914. Note that we do not introduce any new data to the training set, but enrich the existing corpus. Similar to previous work (Mohammad et al., 2016), our four linguistic annotators participate in the enrichment task (see Appendix Table 9 for more details), and the Cohen’s Kappa score calculated for interannotator agreement is 0.79 for stance labeling, and according to Uebersax (1982), this score represents a reasonable agreement level.
3https://demo.allennlp.org/constituency-parsing

heading: 5 Experiments on Computational Stance Detection

heading: 5.1 Task Definition
text: Given x = {w1, w2, .., wn} (n denotes the token number) as one input text, and t = {t1, t2, .., tm} (m denotes the token number) as the target, the stance detection model is to predict the classification label (e.g., Favor, Against, None). In our experimental setting, we use the 3-class scheme, as the ‘None’ label is necessary for practical use cases. Note that in some stance detection corpora, they introduce ‘Neutral’ as the third label. To uniform the 3-class labeling for extensive evaluation, we merge ‘None’ and ‘Neutral’ as one category.

heading: 5.2 Target-Aware Classification
text: The large-scale pre-trained language models yield state-of-the-art performance in text classification and stance detection (Devlin et al., 2019; Kaushal et al., 2021). Here we use a Transformer neural network (Vaswani et al., 2017) as the base architecture, and leverage prior language knowledge by initializing it with the RoBERTa (Liu et al., 2019). Target-Aware Encoding Since predicting the stance of an input text is dependent on the specified target, previous studies show that conditioning the contextualized representation on the target provides substantial improvements (Augenstein et al., 2016; Du et al., 2017; Allaway and Mckeown, 2020), thus we concatenate the input text x and the specified target t as one sequence, and use the language backbone to encode it. The input for encoder is formulated as “
 t 
 
 x 
”. Then the pooled output of the final layer hidden representation of the first “
” venc ∈ RE (where E is the dimension size) is used as the encoded representation of the target-conditioned input.4 Label Prediction To predict the stance label, we feed the encoded representation venc to a fullyconnected layer and a softmax function to compute the output probabilities:
ypred = softmax(W ′ venc + b ′ ) (1)
where W ′ and b ′
are learnable parameters, and the cross-entropy between gold label ygold and model prediction ypred is minimized as the training loss.
4The special tokens vary in different language backbones. For BERT-based models (Devlin et al., 2019), 
 and 
 are replaced with [CLS] and [SEP], respectively.

heading: 5.3 Experimental Corpora
text: We select several representative stance detection datasets for extensive evaluation, including SemEval-16 Task-6 A and Task-6 B (Mohammad et al., 2016), P-Stance (Li et al., 2021), VAST (Allaway and Mckeown, 2020), and Tweet-COVID (Glandt et al., 2021). We use their official train, validation, and test splits. The detailed statistics of these datasets are shown in Table 4. To uniform the label of None and Neutral from the data perspective, we extend the None subset with 20% size of the training data, by extracting irrelevant objects from random samples, as previous contrastive learning study (Gao et al., 2021).
In our experiments, models are basically trained on a single-domain corpus (SemEval-16 Task-6 A), and evaluated on multiple test sets. As shown in Table 4, since there are only 5 targets in the single training set of politics-related tweets, testing on different corpora will build the in-domain, out-of-domain, and cross-target evaluation settings (Küçük and Can, 2020). As shown in Table 5, SemEval-16 Task-6 B contains unseen target “Donald Trump”, which is used to test the cross-target generalization, and testing on Tweet-COVID is both out-of-domain and cross-target.
Moreover, stance label distributions of different targets in our tested benchmark corpora are relatively balanced, and this mitigates the concern of model’s over-fitting on “target-related patterns” at the evaluation stage.

heading: 5.4 Training Configuration
text: Models are implemented with Pytorch and Hugging Face Transformers5. For fine-tuning on stance detection task, we train the language backbone RoBERTa-base (Liu et al., 2019) with the AdamW optimizer (Kingma and Ba, 2015) and batch size 32. Initial learning rates are all set at 2e−5, and a linear scheduler (0.9 decay ratio) is added. Test results are reported with the best validation scores.
5https://github.com/huggingface/transformers
As previous work (Mohammad et al., 2016; Allaway and Mckeown, 2020), we adopt the macroaveraged F1, Precision, and Recall scores as evaluation metrics. The macro-averaged scheme weighs each of the classes equally and is not influenced by the imbalanced sample number of each class.

heading: 5.5 Experimental Results
text: Since we train the model on a single corpus (2.9k samples), testing it on multiple out-of-domain data and various unseen targets poses a challenging task. As shown in Table 6, compared with in-domain and in-target training on each corpus (which serves as the upper bound for external testing), scores of single-corpus training become much lower, and F1, precision, and recall are all affected significantly. This indicates that the original data only enable the model to achieve reasonable results on in-domain samples and existing targets. In contrast, training with the strategically-enriched annotation (1,500 paired samples) improves the performance substantially and consistently: on the four external test sets, the RoBERTa-base model has achieved at least 38% relative improvement of 3-class labeling. This demonstrates that the model learns more general and domain-invariant features which are useful across different stance detection corpora. Moreover, merging the original data and the enriched set brings further improvement, where at least 45% relative improvement of 3-class labeling is observed.
For extensive comparison with previous work (Mohammad et al., 2016; Li and Caragea, 2021),
aside from the 3-class calculation, we report 2- class macro-averaged scores (i.e., Favor, Against), where the None label is used during training, but discarded in evaluation. As shown in Table 10, training with enriched data also provides a significant improvement (at least 48% relative gain), and state-of-the-art cross-target results.

heading: 5.6 Analysis on Target Dependency
text: Previous work found that strong baselines often solely rely on the input text but neglect the specified targets (Ghosh et al., 2019; Kaushal et al., 2021), resulting in poor performance on diverse and unseen targets. Since our model (trained with enriched set) is expected to show better target dependency, we envision that on label-balanced test sets, the distributions of predictions with or without specified targets shall be pretty distinct.
Here we conduct a quantitative analysis based on KL divergence as in Equation 2. Given Q(x) is the prediction solely on the input text, and P (x) is conditioned on the specified target, we calculate their KL divergence on the whole test set X to measure their similarity.
KL(P ||Q) = ∑
x∈X P (x) log(
P (x) Q(x) ) (2)
As shown in Figure 3, compared with training on the original set, adding the enriched data results in larger KL divergence values. This empirically shows that model’s prediction depends more on the specified targets than the base model.

heading: 5.7 Analysis on Enriched Annotation Size
text: We then take an assessment on different sizes of the enriched annotation. As shown in Figure 4, compared with training on the original sample set, the performance on out-of-domain and cross-target evaluation can be boosted with 600 enriched samples. We speculate that by leveraging the prior knowledge of pre-trained language backbones, models can learn the general features effectively and efficiently from the enriched data. This demonstrates one advantage of following a linguistic framework for strategic annotation, where models can obtain substantial gain upon limited annotation cost.

heading: 5.8 Effectiveness across Model Architectures
text: We further conduct experiments with other strong baselines in different model architectures and designs: (1) ATAE (Wang et al., 2016): an LSTMbased model that extracts target-specific features via an attention mechanism. (2) BERTweet (Nguyen et al., 2020): a BERT-based language backbone that is specially pre-trained on tweet data. (3) Product-of-Expert (PoE) (Clark et al., 2019): a de-biasing method that reweights the learned probabilities upon a bias-only model. We train and evaluate these models following the same experimental setting described in Section 5, and their full implementation details are shown in Appendix Table 8. As shown in Table 7 and Table 11, the results in most aspects are improved substantially
after adding the enriched data, which shows that the strategic augmentation is effective on various architectures. In addition, BERT-based models (e.g., BERTweet, PoE) show a larger performance gain than the ATAE, as they leverage prior language knowledge from pre-training. The comparable result of RoBERTa-base, BERTweet, and PoE shows various backbones can learn domain-invariant features from the enriched data, demonstrating the effectiveness of our adversarial multi-target annotation strategy (Section 4).

heading: 6 Related Work
text: Linguistic Studies of Stancetaking Stance analysis plays an essential role in measuring public opinion on mass media, conversations, and online networking platforms, particularly related to social, religious, and political issues (Agha, 2003; Kiesling, 2022). There are many qualitative studies conducted on various resources such as news interviews (Haddington et al., 2004), Twitter and Facebook posts (Simaki et al., 2018), narrative articles (Bohmann and Ahlers, 2022), and online forums (Kiesling et al., 2018). Recent works also perform in-depth content analyses on social media images to understand how politicians use images to express ideological rhetoric (Xi et al., 2020). To conduct analyses of stancetaking in a well-formulated man-
ner, many linguistic studies explore the explanation for the semantics, pragmatics, syntactic distribution of lexical items, discourse markers, and syntactic construction across languages (Biber and Finegan, 1988; Haddington et al., 2004; Lempert, 2008; Jaffe et al., 2009; Du Bois and Kärkkäinen, 2012). From the perspective of social pragmatics, stancetaking on social networks is regarded as a dynamic and dialogic activity where participants are actively engaged in virtual interactions (Du Bois, 2007). Tweets are not viewed as stagnant posts, instead, they become back-and-forth interactions enacted by retweets and comments (Chiluwa and Ifukor, 2015; Evans, 2016). Computational Stance Detection Computational stance detection is commonly formulated as a target-specified classification problem. Datasets for stance detection are usually collected from online networking platforms where large-scale usergenerated content is available (Mohammad et al., 2016; Li et al., 2021; Allaway and Mckeown, 2020; Glandt et al., 2021). Support vector machines (SVM) with manually-engineered features served as the earlier strong baseline (Mohammad et al., 2016). Then various deep learning techniques such as recurrent neural networks (RNNs) (Zarrella and Marsh, 2016), convolutional neural networks (CNNs) (Vijayaraghavan et al., 2016), and attention mechanism (Augenstein et al., 2016; Du et al., 2017; Zhou et al., 2017; Liu and Na, 2018) are applied for better feature extraction. Recently, the Transformer networks (Vaswani et al., 2017), especially language backbones (Devlin et al., 2019; Liu et al., 2019), boosted the performance on stance detection benchmarks (Ghosh et al., 2019; Li and Caragea, 2021). However, it has been observed that current strong baselines relied heavily on superficial features in existing data, and showed poor performance on unseen-target and out-of-domain evaluation (Kaushal et al., 2021), and recent work proposed de-biasing methods (Clark et al., 2019; Karimi Mahabadi et al., 2020) and introduced multi-task learning (Yuan et al., 2022).

heading: 7 Conclusions
text: In this work, we revisited the challenges of computational stance detection from a linguistic perspective. We expanded a generic linguistic framework the “stance triangle”, with the relationship and labels of explicit and implicit objects, and characterized various fashions of how humans express
their stances. We then followed the framework to strategically enrich the annotation of one benchmarked single-domain corpus. Experimental results showed that the enriched data significantly improve the performance on out-of-domain and crosstarget evaluation, and guiding computational stance detection with expanded stance triangle framework can encourage models to learn more general and domain-invariant features. Moreover, our framework paves the way for future research such as assessing the explainability of data-driven models.
Limitations
All samples used in this work are in English, thus to apply the model to other languages, it will require training data on the specified language or using multilingual language backbones. Moreover, we are aware that it remains an open problem to mitigate biases in human stancetaking. Of course, current models and laboratory experiments are always limited in this or similar ways. We do not foresee any unethical uses of our proposed methods or their underlying tools, but hope that it will contribute to reducing incorrect system outputs.
Ethics and Impact Statement
We acknowledge that all of the co-authors of this work are aware of the provided ACL Code of Ethics and honor the code of conduct. All data used in this work are collected from existing published NLP studies. Following previous work, the annotated corpora are only for academic research purposes and should not be used outside of academic research contexts. Our proposed framework and methodology in general do not create a direct societal consequence and are intended to be used to prevent data-driven models from over-fitting domaindependent and potentially-biased features.

heading: Acknowledgments
text: This research was supported by funding from the Institute for Infocomm Research (I2R), A*STAR, Singapore, and DSO National Laboratories, Singapore. We thank Yizheng Huang and Stella Yin for preliminary data-related discussions, and Siti Umairah Md Salleh, Siti Maryam Binte Ahmad Subaidi, Nabilah Binte Md Johan, and Jia Yi Chan for linguistic resource construction. We also thank the anonymous reviewers for their precious feedback to help improve and extend this piece of work.

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 8

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section 9

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 5

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 5

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Section 5

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 5, Section 9

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Section 5, Section 9

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 5
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 5
C 3 Did you run computational experiments? Section 5

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 5
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 5

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 5
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 4, Table 9

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Section 4, Table 9

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Section 4, Table 9
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. No data collection. Use published open dataset.
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No data collection. Use published open dataset.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Section 4, Table 9

heading: Future Work
text: Moreover, our framework paves the way for future research such as assessing the explainability of data-driven models. Limitations
All samples used in this work are in English, thus to apply the model to other languages, it will require training data on the specified language or using multilingual language backbones. Moreover, we are aware that it remains an open problem to mitigate biases in human stancetaking. Of course, current models and laboratory experiments are always limited in this or similar ways. We do not foresee any unethical uses of our proposed methods or their underlying tools, but hope that it will contribute to reducing incorrect system outputs