Cross-Lingual Cross-Target Stance Detection with Dual Knowledge Distillation Framework
Authors: Ruike Zhang
Hanxuan Yang
Wenji Mao
Venue: EMNLP 2023 Main (2023)
Keywords: cross-lingual cross-target stance detection
dual knowledge distillation
category-oriented contrastive learning

Abstract: Stance detection aims to identify the user’s attitude toward specific \textit{targets} from text, which is an important research area in text mining and benefits a variety of application domains. Existing studies on stance detection were conducted mainly in English. Due to the low-resource problem in most non-English languages, cross-lingual stance detection was proposed to transfer knowledge from high-resource (source) language to low-resource (target) language. However, previous research has ignored the practical issue of no labeled training data available in target language. Moreover, target inconsistency in cross-lingual stance detection brings about the additional issue of unseen targets in target language, which in essence requires the transfer of both language and target-oriented knowledge from source to target language. To tackle these challenging issues, in this paper, we propose the new task of cross-lingual cross-target stance detection and develop the first computational work with dual knowledge distillation. Our proposed framework designs a cross-lingual teacher and a cross-target teacher using the source language data and a dual distillation process that transfers the two types of knowledge to target language. To bridge the target discrepancy between languages, cross-target teacher mines target category information and generalizes it to the unseen targets in target language via category-oriented learning. Experimental results on multilingual stance datasets demonstrate the effectiveness of our method compared to the competitive baselines.

Cross-Lingual Gross-Targes Stance Detection with Dual Knowledge Distillation F Framework
Ruike Zhang 1,2 Hanxuan Yang21, Wenji Mao
Institute of Automation, Chinese Academy of Sciences
School of Artificial Intelligence, University of Chinese Academy of Sciences (zhangruike2020, y yanghanxuan2820, wenji www.mmm............
Abstract
dia analytics. It has been applied diverse do- mains, such as public attitude mining, market anal- ysis (Küçükk and Can, 2020), veracity prediction (Wei et al., 2019) and many others.
Stance detection aims to identify the user's atti- tude toward specific targets from text, which is an important research area in text mining and benefits a variety of application domains. Exist- ing studies on stance detection were conducted mainly in English. Due to the low-resource problem in most non- -on-English languages, cross- lingual stance detection was proposed to trans- fer knowledge from high-resource (source) lan- guage low-resource (target) language. How- ever, previous research has ignored the practical issue of no labeled training data available in tar- get language. Moreover, target inconsistency in cross- lingual stance detection brings about the additional issue of unseen targets in target lan- guage, which in essence requires the transfer of both language and target-oriented knowledge from source target language. To tackle these challenging issues, in this paper, we propose the new task of cross-lingual target stance detection and develop the first computational work with dual knowledge distillation. Our proposed framework designs cross-lingual teacher and a cross-target teacher using the source language data and a dual distillation pro- cess that transfers the types of knowledge to target language. To bridge the target discrep- ancy between languages, cross-target teacher mines target category information and general- izes it to the unseen targets in target language via category-oriented learning. Experimental results on multilingual stance datasets demon- strate the effectiveness of our method compared the competitive baselines !
Existing work on stance detection has mainly fo- cused on monolingual setting and been conducted on English datasets (Küçük and Can, 2020; Schiller et al., 2021; Li et al., 2021; Zhang et al., 2022; Huang et al., 2023). In contrast to English, most other languages lack abundant annotated data for training quality stance detection models. To ad- dress the low-resource problem and improve model performance, cross-lingual stance detection has been proposed to transfer knowledge from high- resource (source) language to low-resource (target) language. Mohtarami et al. (2019) propose a con- trastive language adaptation method to align rep- resentations in two languages, which relies on the labeled training data in target language. Hardalov et al. (2022) transfer the knowledge from English to target languages through pre-training and prompt- tuning for few-shot (and zero-shot) tross-lingual stance detection (Vamvas and Sennrich, 2020) with few (and no) training data in target languages.
None of these works has addressed the practi- cal issue when there are only unlabeled training data available in target language, which is preva- lent phenomenon in many low-resource languages. Consequently, the typical method based on super- vised contrastive learning (Mohtarami et al., 2019) is not workable due to the lack of supervised sig- nals for target language, and pre-training & prompt- tuning method (Hardalov et al., 2022) cannot fully utilize the unlabeled training data in target lan- guages as well. Since labeled data are extremely scarce in low-resource languages, these unlabeled data are the valuable resource of target domain that can be utilized to facilitate various tasks in general, and in particular, we utilize them to help bridge cross-lingual gap in our work.
1 Introduction
Stance detection aims to automatically identify the user's attitude (c.g., "in favor of" or "against") towards specific targets (e.g., entities, topics or claims) (Augenstein et al., 2016), which is an im- portant research area in text mining and social me-
*Conresponding author
Source code: https:/ / http://gtbbh.////eee Cotte////m//LLEEEee///.....
In addition, cross longull stance detection usu-

---

ally brings about target inconsistency problem, that is. the occurrences and distributions of the concerned targets may vary considerably across languages due to the differences in socio-cultural backgrounds and linguistic expressions. Without the labeled data, target inconsistency causes the additional issue of unseen targets in target lan- guage, which in essence requires the transfer of both language-related and target-oriented knowl- edge from source to target language.
which is prompt-tuned with cross-lingual templates. To reduce the impact of language differences on cross-target knowledge distillation, cross- lrngualingual teacher model functions as the initialized encoder for cross-target teacher. Finally, in the dual distilla- tion process, cross-lingual teacher and cross- target teacher jointly produce the pseudo-labels for the student model, and distill the two types of knowl- edge the unlabeled target language data with varying degrees of target inconsistency control. The contributions of our work are as follows:
To address the above issues, we consider cross- lingual distillation (Xu and Yang, 2017) as a good to train a teacher model on the source language data so as to acquire the pseudo supervised sig- nals for a student model trained on the unlabeled target language data. Since the language and target- oriented knowledge transferring from source to tar- get language is of different types, dual teacher mod- els required for our purpose. Previously, multi- teacher distillation in language processing (Li et al.. 2022) was typically used enhance the transfer of the same type of knowledge. In contrast, our aim is to transfer different types of cross-language and cross-target knowledge, which needs to consider not only the design of individual teacher models but also their combination scheme. In particular, cross-target teacher model is built the basis of cross-lingun teacher model, and relies the fine- grained target knowledge, in which semantically correlated targets can be aggregated to indicate cat- egory information and utilized to generalize for the unseen targets.
To tackle the issues of unlabeled data and un- seen targets in target language, we identify the new task of cross-lingual cross-target stance detection, and make the first attempt pro- pose an integrated dual teacher-student distil- lation framework.
To bridge the target inconsistency gap, cross- target teacher mines the category information via target representation learning and refine- ment, and generalizes it to the unseen targets via category oriented contrastive learning.
We conduct experiments on multilingual stance datasets with varying target settings and the results demonstrate the effectiveness of our proposed method compared to the com- petitive baselines.
Related Work
Monolingual Stance Detection Stance detection has been well studied on English datasets (Moham- mad et al., 2016; Sobhani et al., 2017; Conforti et al., 2020; Allaway and Mckeown, 2020; Schiller et al.. 2021). The mainstream research is mainly as follows: (1) Stance detection for pre-defined targets, which trains a classifier for several pre- deñiedd targets. Previous methods mainly learn target-specific representations with attention (Du et al., 2017: Wei et al., 2018; Siddiqua et al., 2019; Sun et al., 2018: Li and Caragea, 2019).
In this paper, we propose the new task of cross- lingual cross-target stance detection, which is fundamentally different from both cross-target stance detection task in monolingual setting and existing cross-lingual stance detection task. We develop Cross-lingual Cross-target Stance Detection (CCSD) method via designing dual knowledge distillation framework for this task, which trains a cross-lingual teacher and a cross- target teacher on the source language data and distills the two types of knowledge to the student model. To bridge target inconsistency between lan- guages, cross-target teacher first learns correlated target representations mine the category infor- mation and fuses the aggregated semantic repre- sentations for refinement. It then devises category- oriented contrastive learning to enhance the gener- alization ability of the cross-target model. Cross- lingual teacher is a multilingual language model
(2) Cross-target stance detection, which trans- fers knowledge from one target to another related target. Existing work learns target-independent representations (Augenstein et al., 2016; Xu et al., 2018 8) and mines transferrable features such topic words (Wei and Mao, 2019) and semantic-emotion knowledge (Zhang al., 2020).
(3) Zero/few-shot stance detection, which trans- fers knowledge from known targets to unseen tar-

---

gets with zero/few training data. Existing meth- ods learn larget-invariant representations with ad- versarial training (Allaway et al.. 2021) or mine transferable information across targets with cluster- ing (Allaway and Mckeown, 2020) or contrastive learning based methods (Liang et al., 2022a.b). Other methods introduce external knowledge such Wikipedia knowledge and commonsense knowl- edge from ConceptNet enhance zero/few-shot stance detection (Zhu et al., 2022; Liu et al., 2021). There is also other work that conducts multi- dataset learning (Hardalov et al., 2021; Li et al., 2021) for cross-domain stance detection.
is not only advantageous to alleviate the difficulty of data annotation in low-resource languages, but also beneficial to the utilization unlabeled train- ing data in target language. Moreover, none of the above methods considers the target inconsistency problem caused by cross-lingus stance detection, where additional cross-target knowledge transfer is required. In this paper, we identify the impor- tant research theme of cross-target stance detection in cross-lingual setting, which can be viewed as an integrated task of tross-lingual and cross-targe stance detection, and an integrated computational framework is required to address knowledge trans- fer at both language and target levels with combi- nation scheme.
Cross-Ling Pross-Linggull Stance Detection Compared to the abundant data resources in English, there scarce datasets in other low-resource languages (Xu al., 2016; Lozhnikov et al., 2018: Khouja, 2020; Cignarella al., 2020; Martínez et al., 2023). To tackle the low-resource problem non-Engll non-English languages, cross-lingual stance detection is pro- posed to transfer knowledge from source language to target language (Mohtarami et al., 2019; Küçük and Can, 2020). Some studies address this prob- lem via constructing multilingual stance datasets and baseline methods (Taulé al., 2017: Lai et al., 2020; Zotova al., 2020; Vamvas and Sennrich, 2020; Agerri et al., 2021; Barriere et al., 2022). Others develop identification methods for cross- lingual stance detection (Mohtarami et al., 2019; Hardalov et al., 2022), which we detail below. Mohtarami et al. (2019) first propose con- trastive language adaptation method align repre- sentations two languages, by encouraging sam- ples with the same label in different languages to be close in the embedding space. However, their method is not workable when there are no labeled data in target language. Vamvas and Sennrich (2020) further propose zero-shot cross- lingual stance detection in the setting of no training data in target language. To tackle this problem set- ting, Hardalov et al. (2022) pre-train XLM-R with additional sentiment-based corpora and transfer the knowledge to target languages with prompt-tuning for few-shot (and zero-shot) cross- gross-lingual stance detection. However, their work overlooks the avail- ability of unlabeled data in target languages, which can be utilized to further bridge the language gap in cross-lingual stance detection.
Proposed Method
Figure shows the overall structure of the proposed cross-lingual cross-targe stance detection method CCSD. We first train a cross-lingual teacher and cross-target teacher on the source language data, and then distill the learned knowledge to the stu- dent model trained with unlabeled data in target language. Cross-lingual teacher is multilingual pre-trained language model which is prompt-tuned with cross-lingual templates and consistency con- straints to enhance cross- lonss-lingual ability solely with the source language data. Cross-target teacher learns target representations to mine the target cat- egory with highly
targets, and then gen- eralizes the target category to the unseen targets in target language through contrastive learning.
3.1 Task Definition
We denote the training data in source language Dive = (44, ct), 3i) where tú e; are the i-th target and text, y/ the stance label, and N, is the number of samples in D The target set of Dare is represented as Tare = (T) where i, is the i- th unique target in Dare and n, is the number of tar- gets. Similarly, we denote the unlabeled training data in target language as Digt = ((t,, 4 with the target set Tiga y Generally, Nr < N. in cross-lingual stance detection. We train two teacher models on Dare and distill knowledge the student model trained on Digt= then predict stance labels on the test set in target language.
3.2 Cross-Lingual Teacher
Therefore, our work concentrates on the setting cross-lingual stance detection when there are no labeled training data available in target language.
We adopt mBERT as cross-lingual teacher and design cross-lingual stance templates for prompt-

---

Cross Lingual Teacher
The stance of sterth towards
PLM
Probs
<largett [mask]
Labeled
Statin Tempiate
Data
(Shaned)
Toge
Lot
Mini-Batch
Source Language
postion de <target> [mask]
PLM
Probs
Ag
Cross Lingual Distillation
Drossiingual Stance Template
Leco
Stadent
Teacher Inference
Unlabeled
Encoder
Stance Clessifier
LSt
Mini Satch
Croes larget Teacher
Target Distillation
cro
Encoder
Target Category
Categery-Orennted Learning
Ly
Mining
Ley
Labeled
Target Data
Stance Classifier
lg
language
Mini-Batch
Figure 1: The overall architecture of our proposed cross gross-lingual target stance detection method CCSD with dual knowledge distillation.
tuning inspired by Qi et al. (2022) enhance the cross-lingual ability of the model solely with the source language data. The template designed for stance detection is presented in English as: "The stance of <text> towards <target> is [MASK].", where "<target>" and "<text>" are the slots to fill the input target and text in, "[MASK]" is the to- ken for predicting the stance label and others prompts for assisting the language model. For each source language, we translate the prompt words to get the monolingual template X, and then trans- late the prompt words into its target language to get cross-lingual template X. The verbalizer maps stance labels into specific words, and we set "favor" and "against" for the favor and against stance la- bels respectively. We feed the target-text pair with both monolingual and cross-lingual templates into mBERT and get the hidden vector of "[MASK]" h MASK]] Rd in X and h MASK] in X. The pre- dicted distribution is calculated follows:
To further enhance the cross-lingual ability, we add constraints the predicted distributions based on monolingual and cross-lingual templates fol- lowing Qi et al. (2022). Specifically, we force the predicted distributions based on X and X as close as possible. We use Kullback-Leible divergence to measure the two distributions and minimize the consistency loss function LKI: LKL
N.
(KL 100 KL 110
(3)
Finally, we optimize cross-lingual teacher with the combined loss LCL:
Lel Lx Cx LKL
(4)
3.3 Cross-Target Teacher
Figure illussrates the structure of the proposed cross-target teacher. To bridge the target incon- sistency between the source and target languages, cross-target teacher mines target category through clustering on the target representations. Specifi- cally, it learns target representations with stance- related association modeling and fuses the semantic information refine target representations. Then, category-orienter contrastive learning is devised generalize the category information the unseen targets in target language.
xex Softmax( Wtn hm [MASK]
(1)
XX Softmax(W,... Th MASK]
(2)
where W im I is the projection matrix which transforms the predicted hidden vector into specific words, and the size of the vo-
cabulary. We minimize the cross-entropy loss Lx CrossEntropy(yy," xxx on X and Cx CrossEntropyyu xxx) on X, where yi" (y is the onc-hot vector transformed from the stance la- bel with the verbalizer.
3.3.1 Encoder Module
We use mBERT as the encoder module and initial- ize with the prompt-tuned cross-lingun teacher

---

Cross-Taree Parse- Teacher
Target Category Mining
Target Rop.
Rea.
Target Rep. Nr
Category-Drienter Contrastive Learning
Target Repr esentation
Target Represemaaion Refirement
Target Data
Vester
Learning
Concat
Mini-Sonch
Target
Text
Encoder
Classifier
hop:
Away
Figure The architecture our proposed cross-target teacher.
reduce the impact of language differences on the cross-target knowledge transfer. Specifically, feed the target & and text e; into the encoder with special tokens and take the output of "[CLS]" as the textual representation h; E Rd:
To dynamically model the associations between targets and learn structural target representations. feed the node features and the adjacency matrix A into Graph Attention Network (GAT) (Veličković et al., 2018), and derive target repre- sentations U = [u; ) where u, the target representation for the i-th target t,
h; ((ER
(5)
where & and с; are the target and text in source language respectively.
GAT(V,A)
(6)
3.3.2 Target Category Mining
Target Representation Refinement To mine the target category with Highly-associate targets, we use k-means to cluster the learned target representa- tions WI into K categories C {C*) where Ck the index set of targets contained in the k-th clus- ter. We then fuse the target semantic information into categories to refine the target representations and category division. Specifically, we use mBERT to derive the target semantic vector h E Rd:
We mine target category information in the aspects of the stance-related associations and semantics similarity. Specifically, we learn target representa- tions by modeling target associations with graph, and cluster the target representations into target cat- egories. We further refine the target representations by fusing the target semantic information.
Target Representation Learning We construct target graph 9 (V, learn the structural tar- get representations, where (vi...., Un,)) Rr.xd represents the node features and A (A.,) E {0, 1] is the adjacency matrix. Each target in source language is treated as a node in the graph. Intuitively, the stance-related asso- ciation between targets is reflected by their cor- responding sample set. Thus, for the i-th tar- get 7 in source language, also use mBERT encode each sample with I get has (BERT(CCI.SAA(/EPPCC/
h (BERT((CLST((SEP)
(7)
Then h is concatenated with the target repre- sentation to get the combined representation e; uu; h E Rddd For each category Cj, we average the combined representations within the category calculate the category representation Tk E 2d
1
Tb
(8)
where c is the text of the k-th sample with target f; and calculate the average vector as the node feature + E T where N, the number of sam- ples with target 7,. Assuming that there are associ- ations between all pairs of targets in the beginning, we build a fully connected graph and set the adja-
where ny is the number of targets within the cat- egory Cg- We increase the closeness within the categories and the discrimination between cate- gories,
strengthen the category division and optimize the target representation. To this end, we devise the following intra-catedd intra-category constraint to pull the combined representation closer to its
cency matrix A.g = Aj.² = 1.

---

corresponding category, and the inter-category con- straint to push away the representations from dif- ferent categories:
student model is trained on the unlabeled target lan- guage data with the supervisory signals produced by cross-lingual teacher and cross-target teacher respectively. Specifically, the student model is com- posed of an encoder module and a stance classifier, and the forward calculation is as follows:
LR (lee-Prau) Toasl 1|
Σ lle: e/ll (9) n! jEC;
where ca; is the category of e;, Ten, is the category representation of e;. and ni is the number of targets in other categories except ca..
h = MBBRT((1SS(/SEPPIIST))) (13)
D/ = Softmax(FFN,stt))
(14)
3.3.3 Category-Orienter Contrastive Learning
where is the stance label predicted by the student model, and FFNs(-) is the two-layer feed-forward network the student model.
To generalize the learned category information to the unseen targets in target language, we devise a category-oriented contrastive learning method. First, the textual representation h, is concatenated with its corresponding structural target representa- tion uy to get the target-enhanced representations = [h.; up] E R2d, For each anchor z; in the mini-batch,
Given the target-text pair in target language, we input it together with the prompts into cross-lingual teacher and generate the supervisory signal j, Cross-targe teacher also produces the supervisory signal "²"². We train the student model with the following cross-lingual distillation loss LCLD and cross-targes distillation loss LOTD:
select the samples with the same category and stance label the positive samples and treat others as negative samples. We pull the positive samples closer and push the negative sam- ples away by minimizing the following contrastive loss Lo:
1
LOLD
log ø/
(15)
N
N₂ LCTD No
1
D log 0(
(16)
No
N
Lc = N
Σ
(1,5))((vi yj)-
To tackle various degrees of target inconsistency. we set a trade-off hyperparameter a to adjust the proportions of the cross-lingual knowledge and cross-target knowledge. Finally, the student model is optimized by minimizing the combined loss LSt:
No j=1
(10)
I(i + j)xxp(((pp))e Zig 2j)/T)
log
km I(i f k e)xpp( {( TT)
No
(i. = {
1 if ca; caj 1;
(11)
otherwise
CSt LOLD +
(17)
where & and ca; are the target and category of z; respectively, is the temperature hyperparameter, and N₁. NV are the numbers of samples and the selected positive samples in the mini-batch.
Experiments
4.1 Datasets and Experimental Settings
Dataset and Target Settings X-stance (Vamvas and Sennrich, 2020) is multilingual stance dataset on Swiss politics, where German is used as the source language and French the target language. Each sample consists of a voter's question and a candidate's ' answer, and the stance can be classi- fied into "favor" or "against". We construct two datasets from X-stance: (1) Politics (P) is com- prised of all the data in domains "Foreign Policy" and "Immigration", with 31 different targets in to- tal. There are 7064 samples in German and 2582 samples in French; (2) Society (S) consists of all the data in domains "Sociery" and "Security", with 32 different targets in total. contains 7362 sam- ples in German and 2467 samples French.
The predicted stance label ý; is obtained by feed- ing the textual representation h; into a stance clas- sifier, which is two-layer feed-forward network with Softmax function. We adopt cross-entropy loss Ls CrossEEntropy (y:, 9:) to optimize the classifier. Finally, cross-target teacher is trained with the source language data by minimizing the combined loss LCT:
=cr= = LR La + Ls
(12)
3.4 Cross-Lingual Cross-Target Distillation
To transfer the lioss-lingual and cross-target knowl- edge to target language, we devise a dual knowl- edge distillation process, as shown in Figure 1. The

---

Further, we establish three different target set- tings on the two datasets: (1) All: All the targets in source language and target language are the same; (2) Partial: Part of the targets in source language and target language are the same. We randomly se- lect 50% overlap of all the targets between source and target languages: (3) None: None of the tar- gets in source language and target language are the same. We randomly select 50% of the targets in source language and remaining 50% for target language. Thus, we get different experimental settings (i.c.. datasets with 3 target settings) to verify the effectiveness of our method.
(Du et al., 2017) learns target-specific representa- tions with an attention mechanism; (2) BiCond (Augenstein et al., 2016) incorporates target infor- mation into text representations with bidirectional conditional LSTMs; (3) CrossNet (Xu et al., 2018) uses self-attention to learn larget-independent rep- resentations for cross-target stance detection; (4) JointCL (Liang al., 2022b) devises target-aware prototypical graph contrastive learning for zero- shot stance detection.
We also select the methods for cross-lingual tasks to compare with our proposed method CCSD, (I) ADAN (Chen et al., 2018) aligns representa- tions in the source and target languages with ad- versarial training; (2) CLKD (Xu and Yang, 2017) trains the source classifier with the labeled source language data and distills knowledge to the target model; (3) mBERT-FT (Devlin et al., 2019) fine- tunes the language model mBERT on the training data; (4) mBERT-PT prompt-tunes mBERT with stance template in source language.
SemEval2016 (Mohammad et al., 2016) is an English stance dataset on Twitter, which contains 4163 samples with 5 targets "Atheism", "Climate Change is a real Concern", "Feminist Movemen"", "Hillary Clinton" and "Legalization of Abortion". We use its "favor" and "against" samples as the target language data, and German data in Politics (P) and Society (S) as the source language data re- spectively, to establish two additional experimental settings "P-Sem-None" and "S-Sem-None".
4.3 Main Results
R-ita is an Italian dataset of multilingual po- litical corpus (Lai al., 2020), which contains 833 samples about "Constitutional Reform". All the data are randomly split into the training and test set with a proportion of 80%-20% Similar to Sem1val2016, "favor" and "against" samples are used as the target language data and German data in Politics (P) and Society (S) are the source language data respectively, which forms two additional ex- perimental settings in Italian "P-Rita-None" and R-taeea-None..
We use Accuracy and macro F1 of "favor" and "against" as the evaluation metrics. Tables 1 and 2 give the experimental results of our method CCSD and the comparison baselines on the four datasets. can be seen that our method basically outper- forms the baseline methods in all settings, which benefits from the dual distillation of cross-lingual and cross-target knowledge and the adjustment of the proportion of these two types of knowledge in different target inconsistency cases. Especially in "Politics-None", "S-Sem-None" and "S-Rita- None", our method outperforms the comparative methods by 7.99%, 2.99% and 2.96% on F,. This verifies the effectiveness of our proposed method on cross-lingual cross
Czech (Hercig et al., 2017) contains 1455 sam- ples in Czech with targets of "Miloš Zeman" and "Smoking Ban in Restaurants". We split all the data into the training set and the test set with propor- tion of 80%-20% randomly. In the same way, it also contributes two additional experimental set- tings "P-Czech-Nome" and "S- "S-Czech-None"
cross-target target stance detection.
We can see that cross-lingual methods perform better than monolingual methods in general, indi- cating the importance of knowledge transfer across languages for cross-lingual stance detection. For monolingual methods, JointCL devises prototypi- cal graph to bridge the gap between the known tar- gets and unknown targets, and performs better than attention-base methods TAN and CrossNet, show- ing the effectiveness of mining high-level features to bridge the target inconsistency gap. For cross- lingual methods, fine-tuning and prompt-tuning mBERT achieve rather good results, which benefits from the cross-lingual ability of multilingual pre-
Implementation Details We provide more de- tails on the datasets and experiments in Appendices A and B.
4.2 Comparative Baselines
We choose the following monolingual methods as the comparative baselines. For a fair comparison, we adapt these methods to cross-lingual stance detection by replacing the original word embed- dings with the hidden vectors of BERT. (1) TAN

---

Politics Politics-Partiaa
Politics + Others
Method
Politics-All
Politics-None
P-Sem-None
P-Rita-None
P-Czech-None
Ace
F
Acc
F
Acc
Fi
Acc
Fi
Acc
Fi
Acc
Fi
TAN
61,93
61.27
59.88
59.44
55.03
54,51
55.94
49.49
60.69
50.34
53.01
48.89
BiCond
62.64
62.00
59.30
59.27
53.14
52.73
62.51
50.93
56.49
50.55
52.78
51.64
CrossNet
61.93
60.41
59.30
58.99
56.60
56.15
51.52
48.89
55.73
50.38
52.31
51.59
JoinCL
63.65
62.71
61.05
60.64
53.77
53.33
53.78
50.25
65.84
52.31
53,94
53.39
ADAN
60.63
59.27
58.33
58.18
56.29
55.08
54.56
50.19
61.32
49.93
52.78
52.70
CLKD
67.67
66.54
66.09
65.80
53.77
53.17
59.27
49.59
60.31
48.36
52.78
52.45
MBERT-FT
66.38
66.14
65.89
65.76
56.29
55.87
59.76
50.59
67.94
51.01
52.78
52.74
mBERT-PT
67.82
67.76
66.28
65,90
57.86
57.48
61.19
49.99
69.47
51.57
54.17
53.73
CCSD
70.11
69.92
67.44
67.32'
65.72
65.47'
59.27
51.50
66.92
54.87'
56,251
55.89
Table Experimental results of baselines and CCSD on Politics and other three datasets under three target settings. For each method, report the average score of 5 runs in percentages. The best performances marked in bold, and ft means that our proposed method CCSD is statistically significantly better than the baselines (p < 0.05).
Society Society-Partial
Society Others
Method
Society-All
Society-None
S-Sem-None
S-Rita-None
S-Czech-None
Acc
Fi
Acc
Fi
Acc
Fi
Acc
Fi
Acc
Fi
Acc
F;
TAN
59.61
58.75
55.75
55.42
56.79
54.10
57.21
50.93
61.83
47.18
50.69
49.71
BiCond
61.11
60.33
59.00
58.38
53.40
53.14
62.15
51.53
63.61
48.62
50.69
50.69
CrossNet
60.96
60.42
57.85
57.43
56.48
54.38
61.56
51.51
63.87
49.29
51.39
51.09
JoinCL
61.71
61.61
61.69
61.11
58.64
56.58
63.79
52.66
62.60
48.80
52.08
50.50
ADAN
61.71
61.59
59.77
59.30
54.63
54.35
59.90
51.55
61.83
48.78
52,08
50.89
CLKD
63.06
62.39
59.96
58.86
56.79
56.09
57.44
52.02
64.12
50.76
51.39
49.48
mBERT-FT
66.82
66.28
65.33
64.40
60.80
59.66
56.23
52.97
61.07
50.08
51.39
50.36
mBERT-PT
66.82
66.31
64.37
63.84
61.73
59.00
63.79
50.77
61.07
51.58
51.39
51.05
CCSD(Ours)
67.871
67.32
66.09
65.43
62.96'
62.48
61.04
55.96
64.38!
54.54
$2.78'
52.55
Table 2: Experimental results of baselines and CCSD on Society and other three datasets under three target settings.
trained language model. Besides, CLKD also has competitive performances, demonstrating the ef- fectiveness of cross-lingual knowledge distillation with no labeled data in target language. We can also see from the left half of Tables and 2 that the performance of each comparison method decreases greatly from "All" to "Partial" and "None", show- ing that target inconsistency between languages degrades the model performance for cross-lingual stance detection.
4.4 Ablation Study
The first part of Table 3 gives the results on variants of the overall framework. It can be seen that remov- ing cross- lingull distillation (i.e., LCLD) leads to performance dropping under three target settings, showing the importance of dual knowledge distil- lation for cross-lingual cross-target stance detec- tion. In "Politics-Ale All",, we can see that removing cross-lingual teacher results in the largest perfor- mance drop in the first part, which indicates that cross-lingual knowledge plays decisive role in the target- turget-consistent case. In setting Politics-None" the performance decreases more when removing cross- storsttarget teacher, demonstrating that the cross- target knowledge transfer is important for target inconsistency in cross- linssllingual stance detection.
To further verify the effectiveness of CCSD when no targets are the same between languages, we conduct experiments using three additional datasets as the target language data. The exper- imental results are given in the right half of the two tables. We can see that even though both the language and target gaps are large, our method still outperforms the baseline methods in general, fur- ther demonstrating the effectiveness of our dual distillation method for cross- lingual cross cross-target stance detection.
The results on variants of "cross-lingual teacher" are provided in the second part. Removing the consistency constraint also leads to a performance drop, indicating the effectiveness of constricting predicted distributions based on X and X to be

---

Politics-All
Politics-Partial
Paltics-None
Variants
Acc
AAce
AF,
Acc
Fi
AAcc
AF,
Acc
Fi
AAce
AF,
CCSD (Ours)
70.11
69.92
67.44
67.32
65.72
65.47
w/o cross-lingual distillation
69.11
68.84
-1.01
-1.08
66.67
66.47
-0.78
-0.85
64.15
63.69
-1.57
-1.78
w/o cross-linges tross-lingual teacher
66.67
66.61
-3.45
-3.31
65.89
65.37
-1.55
-1.95
64.15
63.83
-1.57
-1.64
w/o cross-target teacher
69.25
69.09
-0.86
-0.83
67.05
66.57
-0.39
-0.76
63.21
62.24
-2.52
-3.24
w/o consistency constraint
68.75
68.37
-1.36
-1.55
66.09
66.06
-1.35
-1.26
62.26
62.14
-3.46
-3.33
w/o cross- coossllingual template
67.59
67.25
-2.52
-2.67
65.12
65.07
-2.32
-2.25
62.26
61.34
-3.46
-4.13
larget refinement
69.83
69.41
-0.29
-0.51
67.25
66.82
64.15
63.57
-1.57
-1.90
w/o target category
69.54
68.96
-0.57
-0.96
66.47
65.69
-0.97
-1.63
63.21
62.57
-2.52
-2.90
Table Ablation results of variants of our proposed CCSD on Politics under three different target settings.
73
55
64
0.7
08
61
07
03
0.2
10
(a) Politics-All
(b) Politics-Partte
(c) Politics-None
Figure 3: Impact of hyperparameters in dual knowledge distillation on Politics under three different target settings.
consistent. The performance decreases more with- out translating the prompts into target language in "Politics-None", demonstrating that translating the prompt words is more important when none of targets are the same between languages.
thus it mostly relies on the target-invariant knowl- edge for stance detection. As the degree of target inconsistency increases, it is vital to pay more at- tention to the cross-target knowledge transfer in dual distillation.
The last part provides the results variants of "cross-target teacher". Excluding target repre- sentation refinement (i.e., LR) causes the model to degrade more "Politics-None", showing the refinement with semantics benefits target inconsis- tency. When removing the whole target category mining, the decrease is larger in "Politics- " and "Politics-None demonstrating the target cate- gories bridge the target gap between languages and improve the generalization ability of the model.
Conclusion
To address target inconsistency in low-resource cross-lingual stance detection, we propose the new task of cross-lingual cross-target stance detection and develop dual knowledge distillation frame- work CCSD to tackle the challenging issues of un- labeled data and unseen targets in larget language. In our dual framework, cross-lingual teacher and a cross-target teacher are trained on the source lan- guage data and distill respective knowledge to the student model trained with the unlabeled target lan- guage data. To bridge the target gap between source and target languages, cross-target teacher further mines the semantically correlated target categories, and generalizes this information to the unseen tar- gets in target language. Experimental results on four cross-lingual datasets under varying target set- tings demonstrate the effectiveness our method for cross-lingual cross-target stance detection.
4.5 Analysis on Trade-off Hyperparameter
We conduct experiments on the trade-off hyper- parameter a which determines the proportion of the cross-lingual and cross cross-target knowledge in the dual distillation process. Figure 3 illustrates the im- pact of on Politics under three different target set- tings. Our proposed method CCSD achieves higher performance when a is around 0. 0-13..3, 0.4-0.5, 0.8~1.0 in "Politics-AlI". "Politics-Partin "Politics-Nome respectively. Targets in the source and target languages are the same in "Politics-All".
and

---

Limitations
target stance classification dataset of online debates. In Proceedings of the LREC 2022 Work- shop on Natural Language Processing for Political Sciences, pages 16-21.
It can be seen from the experimental results that the performance of our proposed method is less superior than some baselines in "P-Sem-Nonc", "S- Sem-None" and "P-Rita-None" in accuracy. We speculate that this is caused by different target expressions. The targets in Politics and Society datasets are sentences of questions as shown in Ta- bles and 10. And the targets in SemEval2016 and R-ita datasets are keywords about some con- cepts and entities like "Atheism", "Hillary Clinton" " and "Constitutional Reform". We speculate that the prompts designed for the source language can- not precisely assist in stance detection in target language with totally different target expression. Our future work shall further explore the impact of different prompts on cross-lingual cross-target stance detection to compensate for the target dis-
Xilun Chen, Yu Sun, Ben Athiwaratkum, Claire Cardie, and Kilian Weinberger. 2018. Adversarial deep aver- aging networks for cross- cross-lingual sentiment classifica- tion. Transactions of the Association for Computa- tional Linguistics, 6:557-570.
Alessandra Teresa Cignarella, Mirko Lai, Cristina Bosco, Viviana Patti, Rosso Paolo, et al. 2020. Sardis- tance@ evalita2020: Overview of the task on stance detection italian tweets. EVALITA Seventh Evalua- tion Campaign of Natural Language Processing and Speech Tools for Italian, pages 1-10.
Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2020. Will-they-won 1-they: A very large dataset for stance detection twitter. In Pro- ceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1715-1724.
crepancy in expressions.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre- Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 4171-4186.
Acknowledgments
This work is supported in part by the Ministry of Science and Technology of China under Grant #2022YFB2703302,, and the National Natural Sci- ence Foundation of China under Grants #1 #11832001, #71974187 and #62206287.
Jiachen Du, Ruifeng Xu, Yulan He, and Lin Gui. 2017. Stance classification with target-specific neural atten- tion networks. In Proceedings of the International Joint Conferences on Artificial Intelligence, pages 3988-3994.
References
Rodrigo Agerri, Roberto Centeno, María Espinosa, Joseba Fernandez de Landa, and Alvaro Rodrigo. 2021. Vaxxstance@ iberlef 2021: overview the task on going beyond text in cross-lingual stance detection. Procesamiento del Lenguaje Natural, 67:173-181.
Momchil Hardalov, Arnv Arora, Preslav Nakov, and Isabelle Augenstein. 2021. Cross-domain label- adaptive stance detection. In Proceedings of the Con- ference on Empirical Methods in Natural Language Processing, pages 901 1-9028.
Emily Allaway and Kathleen Mckeown. 2020. Zero- shot stance detection: A dataset and model using generalized topic representations. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 8913-813311 13-8931.
Momchil Hardalov, Amav Arora, Preslav Nakov, and Isabelle Augenstein. 2022. Few-shot cross-lingus stance detection with sentiment-based pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 10729-10737.
Emily Allaway, Malavika Srikanth, and Kathleen Mcke- own. 2021. Adversarial learning for zero-shot stance detection on social media. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4756-4767.
Tomás Hercig, Peter Krejzl, Barbora Hourová, Josef Steinberger, and Ladislav Lenc. 2017. Detecting stance in ezech news commentaries. ITAT, 176:
Hu Huang, Bowen Zhang, Yangyang Li, Baoquan Zhang, Yuxi Sun, Chuyao Luo, and Cheng Peng- 2023. Knowledge-enhanced prompt-tuning for stance detection. ACM Transactions on Asian and Low-Resource Language Information Processing. 22(6):
Isabelle Augenstein, Tim Rocktäschel, Andreas Vla- chos, and Kalina Bontcheva. 2016. Stance detection with bidirectional conditional encoding. In Proceed- ings of the Conference Empirical Methods in Nat- ural Language Processing, pages 876-885.
Jude Khouja. 2020. Stance prediction and claim verifi- cation: An arabic perspective. In Proceedings of the Third Workshop on Fact Extraction and VERification. pages 8-17.
Valentin Barriere, Alexandra Balahur, and Brian Ravenet. 2022. Debating europe: multilingual

---

Diederik P Kingma and Jimmy Ba. 2015. Adam: method for stochastic optimization. In Proceedings of the International Conference on Learning Repre- sentations.
Saif Mohammad, Svetlana Kiritchenko, Parinaz Sob- hani, Xiaodan Zhu, and Colin Cherry. 2016 Semeval- 2016 task 6: Detecting stance in tweets. In Pro- ceedings of the International Workshop on Semantic Evaluation (SemEval-2016), pages 31-41.
Dilek Küçük and Fazli Can. 2020. Stance detection: survey, ACM Computing Surveys, 53(1).1-37.
Mitra Mohtarami, James Glass, and Preslav Nakov. 2019. Contrastive language adaptation for cross- lingual stance detection. In Proceedings of the Con- ference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, pages 4442 4452..
Mirko Lai, Alessandra Teresa Cignarella, Delia Irazú Hernández Farias, Cristina Bosco. Viviana Patti, and Paolo Rosso. 2020. Multilingual stance detection social media political debates. Computer Speech & Language, 63:1-27.
Kunxun Qi, Hai Wan, Jianfeng Du, and Haolan Chen. 2022. Enhancing cross- poos-lingual natural language in- ference by prompt- prompt-learnigg learning from cross-lingual tem- plates. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Vol- ume 1: Long Papers), pages 1910-1923.
Yingjie Li and Comelia Caragea. 2019. Multi-task stance detection with sentiment and stance lexicons. Proceedings of the Conference on Empirical Meth- ods in Natural Language Processing and the Inter- national Joint Conference Natural Language Pro- cessing, pages 6299-6305.
Benjamin Schiller, Johannes Daxenberger, and Iryna Gurevych. 2021. Stance detection benchmark: How robust is your stance detection? KT- Küünstlichh Intel- ligenz, pages 1-13.
Yingjie Li, Chenye Zhao, and Cornelia Caragea. 2021. Improving stance detection with multi-dataset learn- ing and knowledge distillation. In Proceedings of the Conference on Empirical Methods Natural Lan- guage Processing, pages 6332-6345.
Umme Aymun Siddiqua, Abu Nowshed Chy, and Masaki Aono. 2019. Tweet stance detection using an attention based neural ensemble model. In Proceed- ings of the Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1868-1873.
Zhuoran Li, Chunming Hu, Xiaohui Guo, Junfan Chen, Wenyi Qin, and Richong Zhang. 2022. An unsuper- vised multiple-task and multiple-teaeee
model for cross-lingual named entity recognition. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pages 170-179.
Parinaz Sobhani, Diana Inkpen, and Xiaodan Zhu. 2017. A dataset for multi-target stance detection. In Pro- ceedings of the Conference of the European Chap- ter of the Association for Computational Linguistics, pages 551-557.
Bin Liang, Zixiao Chen, Lin Gui, Yulan He, Min Yang. and Ruifeng Xu. 2022a. Zero-shot stance detection via contrastive learning. In Proceedings of the ACM Web Conference, pages 2738-2747.
Qingying Sun, Zhongqing Wang. Qiaoming Zhu, and Guodong Zhou. 2018. Stance detection with hier- archical attention network. In Proceedings of the International Conference on Computational Linguis- rics, pages 2399-2409.
Bin Liang, Qinglin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, and Ruifeng Xu. 2022b. Jointel: A joint contrastive learning framework for zero-shot stance detection. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 81-91.
Mariona Taulé, M Antonia Martí, Francisco M Rangel, Paolo Rosso, Cristina Bosco, Viviana Patti, et al. 2017. Overview of the task on stance and gender de- tection in tweets on catalan independence at ibereval 2017. In Workshop on Evaluation of Human Lan- guage Technologies for Iberian Languages, volume 1881, pages 157-177.
Rui Liu, Zheng Lin, Yutong Tan, and Weiping Wang- 2021. Enhancing zero-shot and few-shot stance de- tection with commonsense knowledge graph. In Findings of the Association for Computational Lin- guistics: ACL-IJCNLP 2021, pages 3152-3157.
Jannis Vamvas and Rico Sennrich. 2020. X-stance: A multilingual multi-target dataset for stance detection. In Proceedings of the SwissText & KONVENS Joint Conference, pages 1-9.
Nikita Lozhnikov, Leon Derczynski, and Manuel Maz- zara. 2018. Stance prediction for russian: data and analysis. In International Conference in Software En- gineering for Defence Applications, pages 176-186.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lid, and Yoshua Bengio. 2018. Graph attention networks. In International Conference on Learning Representations.
Rubén Yáñez Martínez, Guillermo Blanco, and Anália Lourenço. 2023. Spanish corpora of tweets about 19 vaccination for automatic stance detec- tion. Information Processing & Management, 60(3):
Penghui Wei and Wenji Mao. 2019. Modeling trans- ferable topics for cross-target stance detection. In Proceedings of the International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, pages 1173-1176.

---

Penghui Wei. Wenji Mao, and Daniel Zeng. 2018. target-guided neural memory model for stance detec- tion in twitter. In Proceedings of the International Joint Conference on Neural Networks, pages 1-8.
67
Fi
Penghui Wei, Nan Xu, and Wenji Mao. 2019, Model- ing conversation structure and temporal dynamics for jointly predicting rumor stance and veracity. In Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNP)), pages 4787-4798.
61
59-
87
55
(a) Politics-None
Chang Xu, Cecile Paris, Surya Nepal, and Ross Sparks, 2018. Cross-target stance classification with self- attention networks. In Proceedings of the Annual Meeting of the Association for Computational Lin- guistics, pages 778-783.
65
Acc
63-
Fi
61
Ruifeng Xu, Yu Zhou, Dongyin Wu, Lin Gui, Jiachen Du, and Yun Xue. 2016. Overview of nipec shared task 4: Stance detection in chinese microblogs. In Natural Language Understanding and Intelligent Ap- plications, pages 907-916.
57
55
Ruochen Xu and Yiming Yang. 2017. Cross-lingual dis- tillation for text classification. In Proceedings of the Annual Meeting of the Association for Computasional Linguistics, pages 1425.
(b) Society-Noo -one
Figure 4: Impact of hyperparameters K of k-means in "Politics-Noné and "Society-None".
Bowen Zhang, Daijun Ding, and Liwen Jing. 2022. How would stance detection techniques evolve after the launch of chatgpt? arXiv preprint 22Xiv22222.14448
one more target in the German data than the French data, the Politics dataset contains 31 targets and the Society dataset contains 32 targets. Tables 9 and 10 show the complete list of the targets in the two datasets and their descriptions translated into English.
Bowen Zhang, Min Yang. Xutao Li, Yunming Ye, Xi- aofei Xu, and Kuai Dai. 2020. Enhancing cross- target stance detection with transferable semantic- emotion knowledge. In Proceedings of the Annual Meeting of the Association for Computational Lin- guistics, pages
Qinglin Zhu, Bin Liang, Jingyi Sun, Jiachen Du, Lanjun Zhou, and Ruifeng Xu. 2022. Enhancing zero-shot stance detection via largeted background knowledge. In Proceedings of the International ACM SIGIR Con- ference on Research and Development in Information Retrieval, pages 2070-2075.
To verify the effectiveness for different target inconsistency cases, target settings "All", "Partial" and "None" are constructed, the procedures fol- lows: (1) All: All the targets in source and target languages are the same ; (2) Partial: We randomly select 50% of the total targets (16 for both datasets) as the overlap targets, and randomly select 8 tar- gets from the left targets as the source-only targets. The remaining targets are served the target-only targets; (3) None: We randomly select 50% of the total targets (16 for both datasets) for source language. The remaining targets are consequently served as the targets in target language.
Elena Zotova, Rodrigo Agerri, Manuel Nuñez, and Ger- man Rigau. 2020. Multilingual stance detection in tweets: The catalonia independence corpus. In Pro- ceedings of the Language Resources and Evaluation Conference, pages 1368-1375.
A More Details on Datasets
There are ten topics in X-stance including "Econ- omy", "Finances". "Education" and so We se- lect topics "Immigration" and "Foreign Policy" to construct the "Politics" dataset, and "Society" and "Security" to construct the "Society" dataset, where both datasets have similar target-text ratios to that of the original X-stance dataset. After removing
The target sets the source and target languages of the two datasets are shown in Tables 4 and 5. The train-test split follows that of the original X- stance, and the statistics are given in Tables and 7 respectively.

---

Source Language (de)
Target Language (fr)
Overtap
PolitiesAll
2.
8.9,
11.
3,4,
13.
14,
15.
16,
17.
18.
19.
20,
21.
13.
14,
15.
16,
17,
18.
19,
20, 21.
13. 14.
16.
17.
18.
19,
20. 21. 30,
22.
23,
25,
26
28, 29,
23.
24,
26,
27,
29, 30.
22.
24,
25,
27,
28,
311
311
311
PoliticsPPatial
3.3.3.
15.
#24
11,4,5,
422
[1.
4.
11,
15,
17,
416
17,
19, 20,
23, 24, 25,
17. 18,
22, 23,
26.
18,
19,
21,
261
26, 28, 30)
27,29. 31)
Politics-None
6, 10,
14,
43,4,5,77
16,
29,
20, 22,
29,
24. 28]
Table 4: The target sets in the source and target language data in Politics.
Sonree Langunge (de)
Target Language (fr)
Overlap
Society-All
2.
10, 12,
#32
2,
S,
9, 10, 11,
#32
[1.
4,
8,
11, 12,
#32
13.
14,
16,
17,
19,
20,
21.
13. 14,
15,
18,
19,
20, 21.
13,
16,
19,
21,
22,
23,
24,
25,
26,
28, 29,
12,
23,
24,
25,
26, 27,
29, 30,
22,
24
25,
27,
28,
29, 30,
31
31,
Pariety-Partial
3, 6,
11,
12, 13, 14,
#24
[2, 7,
12,
14, 15, 16.
424
[6.
7, 9,
14,
15,
19, 20, 21, #16
19,
31,
27. 28. 29,
Suciety-Nome
7. 10. 11.
31,
22, 23, 24,
25, 28, 29]
26, 27, 30, 31,
Table 5: The target sets in the source and target language data in Society.
Polities-Al
Polities-Partial
Politics-None
<target> ist [MASK]." with German prompts and "La position de <text> envers <target> est [MASK]." with French prompts. All parameters teacher and student models are optimized by Adam (Kingma and Ba, 2015) with learning rate of 2e The batch size is 16 for cross-lingual teacher, and 32 for cross-target teacher and student model. We train all the models for 15 epochs. The running time of the whole framework is about GPU hour including the training teachers and distillation.
Split
de
Ir
de
fr
de
fr
Train
5926
2149
4286
1543
3258
1008
371
Test
633
232
472
172
318
106
Table 6: The statistics of Politics.
Suciety-All
Suciety-Partial de
Society-None de
Split
de
fr
fr
fr
Train
6313
2051
5104
1535
2872
1185
Valid
487
194
357
241
90
Test
562
222
424
174
271
108
C Variants for Ablation Study
Table 7: The statistics of Society.
Here we provide detailed introduction the vari- ants of the ablation study. Below are the variants of overall dual distillation framework:
B Implementation Details
The proposed dual distillation framework CCSD is implemented with PyTorch and all the experiments are conducted on NVIDIA GeForee RTX 3090. We use www----ssmmuulliiguueeee
"w/o cross-lingual teacher distillation" means removing the cross-lingual distillation and only transferring cross-target knowledge to target language. However, cross-target teacher is still constructed based on the cross- lingull teacher.
which is a 12-layer, 768-hidden and 12-head model with about 1 110M parameters and implemented with the Transformers framework. Thus the size d of textual representation derived from mBERT is 768. K in k- means is set to 3. 7 in category contrastive learning is 0.3. The trade-off hyperparameter is 0.2 for "Politics-AlI", 0.5 for "Porities-Partial",, 0.9 for "Politics-None"; 0.1 for "Society-AlI", 0.4 for "Society-Partiae 0.9 for "Society-None" 1.0 for "P-Sem-None" and "S- "S-Sem-None". The stance
"w/o cross-target teacher" represents exclud- ing cross-target teacher from the distillation framework.
"wo cross linsull teacher" represents exclud- ing cross-lingual teacher from the distillation framework.
template is "Die Haltung von <text> gegenitber
Below are the variants of cross- Pross-lingual teacher:

---

Target
Text
Stance
mBERT-FT
CCSD
<Consistent Target 25>
Les personnes sans pansers devraient- elles pouvoir ohtenir plus facilement un statut de séjour régularisé?
Le systéme doit être équitable et être mis en œuvre au cas par cas.
gainst
Favor
Against
English Translation: Should sans- papiers be able to obtain a regularized residence status more easily?
English Translation: The system must be fair and implemented on a case-by- case basis.
X
<Inconsistent Target 29>
La Suisse devrait-elle conclure un ac- cord de libre-échange avec les Etats- Unis?
Trop de risques pour les standards envi- Against romnementaux et sociaux...
Favor
Against
English Translation: Should Switzer- land strive for free trade agreement with the USA?
English Translation: Too many risks environmental and social standards...
X
<Inconsistent Target 22>
La Confédération devrait-elle soutenir davantage les étrangères et étrangers dans lour intégration?
L' L'intégration est une responsabilité indi- viduelle, mais proposer des plateformes comme des cours de langues est un pas vers une meilleure intégration.
Favor
Against
Favor
English Translation: Should the fed- eral government provide more support the integration of foreigners?
English Translation: Integration is an in- dividual responsibility, but offering plat- forms like language courses a step to- wards better integration.
Table 8: Case study of predicted results with mBERT-FT and our CCSD Politics under Partial setting
"w/o consistency constraint" means removing Σκ from the combined loss LCL for opti- mizing cross-lingual teacher.
"Society-None" with no targets overlapping be- tween languages. We can see that CCSD achieves the hest performance when K is 3. When the num- ber of clusters is too small, each cluster may con- tain more targets with low correlations, causing the model to refine target presentations mistakenly, As K becomes larger, the performance gradually decreases. When the number of clusters there are fewer targets in each category and fewer positive samples in contrastive learning, resulting in a decrease in the contrastive ability and general- ization ability on unlabeled data.
"w/o cross-lingual template" means remov- ing cross-lingual template (i.e., not translating prompt words) and optimizing cross-lingual teacher solely with Lx.
increases,
Below are the variants of cross-target teacher:
"w/o target representation refinement" means removing LR from Lcr for optimizing cross- target teacher.
E
Case Study
"w/o target category mining" means excluding the whole target category mining and remov- ing category-related information from con- trastive learning, resulting in Lo as follows:
We conduct case study to compare the predicted results of mBERT-FT and our CCSD on Politics dataset under Partial setting. Table gives the comparison results of the three representative cases from the test set in the target language (i.e., French), For the consistent target in case 1, CCSD predicts the stance label correctly. Furthermore, we can see that mBERT predicts wrong stance labels on the unseen targets that are not included in the source language data (i.c., cases 2 and 3). In contrast, our CCSD enhances its generalization ability unseen targets by target category mining and category- oriented contrastive learning so bridge the target inconsistency gap between source and target languages.
N I( vz)-
1 N
Lc
(18)
[[i A j)expppp (f (Z), 21)\7)
log
(i + k) k))xp(( (221,2)))1 2T)
Analysis on K-Means Hyperparameter
We conduct experiments on the hyperparameter of k-means in target category mining. Figure shows the impact of K "Politicx-None" and

---

Domain
Target (shown below the English form "topic", given in original dataset)
Immigration
Are you favour legalizing the status of sans papiers immigrants (i.e. immigrants who have
official paperwork) through one-off, collective granting of residency permits?
Immigration
Would you support foreigners who have lived for least ten years in Switzerland being given voting and electoral rights municipal level throughout Switzerland?
Immigration
Should the state provide more funding for the integration of foreigners?
4 Immigration
Should access 'facilitated naturalization" via the Federation be made more difficult?
Immigration
The United Nations High Commissioner for Refugees (UNHCR) is seeking host countries for groups of refugees known "quota refugees". Should Switzerland accept more of these groups?
6
Immigration
A popular initiative has been launched that wants to regulate immigration and thus limit migration- related population growth to 0.2% annually. Do you support this idea?
Foreign Policy
Would you support the introduction of the automatic exchange of bank client data between Switzer- land and foreign tax authorities?
Foreign Policy
Should Switzerland embark negotiations in the next four years to join the EU?
Foreign Policy
Should Switzerland conclude an agricultural free trade agreement with the EU?
10
Immigration
Do you support the existing agreement with the EU on the free movement of peoples?
11
Foreign Policy
Today, the Swiss Army can take part in UN or OSCE peace-keeping missions abroad, armed for self delf-defence purposes, Do you approve?
12
Foreign Policy
For number of years, Switzerland has pursued more active and open foreign policy that is less geared to strict neutrality. Do you welcome this change?
13
Foreign Policy
Should compliance with human rights play a greater role when deciding whether to enter into oconomic agrecments with other countries (e.g. free trade agreements)?
14
Immigration
Would you support that foreigners who have lived for at least ten years in Switzerland being given voting and electoral rights at mumicipal level throughout Switzerland?
15
Immigration
Are you in favour of legalizing the status of sans papiers immigrants (i.e. immigrants who have no official paperwork) through one-off, collective granting of residency permits?
16
Immigration
Do you think Switzerland should accept an increased number of refugees directly from crisis regions for which the United Nations High Commissioner for Refugees (UNHICR) needs host countries (what is called quota refugees)?
17
Foreign Policy
Should Switzerland embark on negotiations in the next four years to join the EU?
18
Foreign Policy
Should Switzerland start negotiations with the USA on a free trade agreement?
19
Foreign Policy
Should liability regulations for companies operating from Switzerland be tightened with regard to the compliance with human rights and environmental standards?
Foreign Policy
Do you think Swiss foreign policy should increasingly oriented to strict interpretation of neutrality?
21
Foreign Policy
Should Switzerland terminate Schengen Agreement with the BU and reintroduce increased identity checks directly on the border?
Immigration Immigration
Should the federal government provide more support for the integration of foreigners?
23
Should foreigners who have lived Switzerland least ten years given the right to vote and be elected the municipal level?
24
Immigration
Is limiting immigration more important to you than maintaining the bilateral treaties with the EU?
25
Immigration
Should sans- sans-papiers be able obtain regularized residence status more easily? Are you in favor of further tightening the asylum law?
26
Immigration
27
Immigration
Should the requirements for naturalization be increased?
28
Foreign Policy
Should Switzerland start membership negotiations with the EU?
29
Foreign Policy
Should Switzerland strive for free trade agreement with the USA?
30
Foreign Policy
An initiative calls for liability rules for Swiss companies with regard to compliance with human rights and environmental standards abroad to be tightened. Do you support this proposal?
31
Foreign Policy
Are you in favour of Switzerland's candidacy for seat on the UN Security Council?
Table 9: Targets in the Politics dataset.

---

ID
Domain
Target (shown below in the English form of a "topic", given in the original dataset)
Society
Should same-sex couples who have registered their partnership be able to adopt children?
2 Security
Should Switzerland legalize the consumption of hard and soft drugs well as the possession of such drugs for personal consumption?
Society
Would you support the right of doctors in Switzerland to help someone die with impunity?
4
Society
Would you support the introduction of a woman's 's quota for the Beards of Directors of listed companies?
Society
Switzerland has relatively strict rules when it comes to medically assisted reproduction. Should these be relaxed?
Security Security
Should young Swiss be able to choose between military service and alternative civilian service?
The Federal Council is seeking to scale down the army from its current level of 1907000 soldiers to level of 80'000, Do you support this idea?
Security
There has been an increasing tightening of rules on the acquisition and possession of weapons recent years, Do you welcome this development?
Security
In the future, should juvenile criminal law place greater emphasis on longer periods of detention closed institutions than on re-socialization measures?
Security
Do you think the Army should undertake policing tasks within Switzerland (e.g. protecting embassies and consulates, carrying out border protection work, and policing major events like the World Economic Forum in Davos)?
Security
Should the powers of security services be increased to include "preventative" supervision of communi- cation by post, e-mail and telephone?
Society
Should same-sex couples who have registered their partnership be able to adopt children?
Society
In June 2015 the Swiss people approved the relaxation the rules for medically assisted reproduction (referendum on pre-imy rreimmllantation diagnosis, PID). Do you welcome this decision?
Society
Should the consumption cannabis well as its possession for personal use be legaliscd?
Society
Would you agrec the introduction of a minimum proportion of women as members of the hoard of directors managements boards of companies listed on the stock exchange?
16
Society
Would you support the right of doctors in Switzerland help someone die with impunity?
17
Society
Would you appreciate the introduction of automatic organ donation (presumed consent) in Switzerland?
18
Society
Do you think the federal govermment should withdraw from its financial support of cultural activities?
19
Security
Are you in favour of considerable reduction of the number of soldiers 1000000 at most?
20
Security
There has been an increasing tightening of rules on the acquisition and possession of weapons in recent years, Do you welcome this development?
21
Security
Should the powers of the security services be increased to include preventative surveillance of communica- tion by post, e-mail and telephone?
22
Security
Should juvenile criminal law place greater emphasis on longer periods of detention in closed institutions than on re- eesocializzaion measures?
23
Security Security
Switzerland has one of the toughest laws against speeding. Should the law he relaxed?
24
Should Switzerland terminate the Schengen Agreement with the EU and reintroduce increased identity checks directly on the border?
25
Security
Should the consumption of cannabis as well as its possession for personal use be legalised?
26
Society
Should cannabis use be legalized?
27
Security
Should Switzerland terminate the Schengen Agreement with the EU, in order to reintroduce more security checks directly on the border?
28
Society
Should same-sex couples have the same rights as heterosexual couples all areas?
29
Society
Should the rules for reproductive medicine be further relaxed?
30
Society
Would you be in favour of a doctor being allowed to administer direct active euthanasia in Switzerland?
31
Security
Should the Federal Council's proposal to tighten the conditions admission the civil service be abandoned?
Security
Should the export war materials from Switzerland banned?
Table 10: Targets in the Society dataset.


Reviews:
[{"review_id": "dbXfi9Woxq", "reviewer": "Reviewer_W3Pa", "summary": "", "questions": "1. How much does translating the prompt into the target language helps?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer W3Pa:\nMany thanks to the reviewer for the valuable comments and suggestions. Below we respond to them in detail with further explanations and supplementary experimental results.\n\n#### ***Q1. How much does translating the prompt into the target language help?***\n\n**Response**: We aim to strengthen the cross-lingual ability of the cross-lingual teacher model using the source language data only. To achieve this, we translate the prompt words into the target language to obtain the cross-lingual template, and prompt-tune the cross-lingual teacher with consistency constraint. Specifically, we devise consistency loss to constrain the prediction distributions between the original template and cross-lingual template, so that the model can learn language-related knowledge and enhance its cross-lingual ability.\n\nTake the Politics dataset as an example. Below we provide the quantitative analysis results under three different target settings on the two variants of the cross-lingual teacher: (a) removing consistency constraint and (b) without translating the prompt words. \n\n|Variant                       | Politics | All  |      |     | Politics | Partial |       |       | Politics  | None |       |        |\n|:-------------------------------|----------|-------|-------|-------|----------|----------|-------|-------|-----------|-------|-------|--------|\n| **Measure**| Acc      | F1    | ∆Acc  | ∆F1   | Acc      | F1       | ∆Acc  | ∆F1   | Acc       | F1    | ∆Acc  | ∆F1    |  \n| **CCSD**                    | **70.11**    | **69.92** | -     | -     | **67.44**    | **67.32**    | -     | -     | **65.72**     | **65.47** | -     | -      |   \n| **(a) w/o consistency**  | 68.75    | 68.37 | -1.36 | -1.55 | 66.09    | 66.06    | -1.35 | -1.26 | 62.26     | 62.14 | -3.46 | -3.33  |  \n| **(b) w/o translating prompt** | 67.59    | 67.25 | -2.52 | -2.67 | 65.12    | 65.07    | -2.32 | -2.25 | 62.26     | 61.34 | -3.46 | -4.13  |   \n\nThe results show that both removing the consistency constraint and removing prompt word translation can lead to performance drop under all the target settings. And removing translating prompts causes more severe performance decrease (-2.67%, -2.25% and -4.13% in F1 score under three different settings). We shall report the above ablation results in the revised paper. \n\n\n#### ***W1. The proposed method is computationally expensive with two teacher networks (with additional architecture modification like GAT) pre-trained on the source language and then distilled back to a target language network.***\n\n**Response**: Due to the nature of cross-lingual cross-target stance detection task, which is inherently more complex than previously proposed cross-lingual stance detection and monolingual cross-target stance detection, we train two teacher models to learn language-related and target-oriented knowledge respectively. Regarding each individual teacher model, our method did not add extra computational complexity and the number of parameters in our method remains roughly the same as those of mBERT. \n\nSince the baseline methods in our original paper all use mBERT to encode input targets and text, we run an additional experiment to calculate the numbers of parameters in these methods and our own method. It can be seen from the table that mBERT contains about 178M parameters, JointCL has about 180M parameters with an extra prototypical graph, and the teacher-student distillation method CLKD has about two sets of 179M parameters. The last three lines report the parameter numbers of our CCSD method, which is comparative to those of the baselines including mBERT.  \n\n\n\n| Method  | Teacher/Student             | Component                               | # Params   |\n|:---------|:-----------------------------|:-----------------------------------------|:------------|\n| mBERT   | -                           | mBERT                                   | ≈178M      |\n| JointCL | -                           | mBERT + prototypical graph + classifier | ≈180M      |\n| CLKD    | Teacher                     | mBERT + classifier                      | ≈179M      |\n| CLKD    | Student                     | mBERT + classifier                      | ≈179M      |\n| CCSD    | Cross-Lingual Teacher  | mBERT                                   | ≈178M      |\n| CCSD    | Cross-Target Teacher   | mBERT + target graph + classifier       | ≈180M      |\n| CCSD    | Student                | mBERT + classifier                      | ≈179M      |\n\n\n\n#### ***W2. The study covers only three languages, non of which is truly low-resource when in terms of pre-training data included as part of the language model's extensive training. Moreover, two of the languages come from the same dataset and domain. Hardalov et al. (2022) can be a good source of candidate datasets, as they used 16 datasets for their experiments.***\n \n**Response**: Many thanks for the valuable suggestions. We construct *Politics* dataset and *Society* dataset from 4 different domains in X-stance, “Foreign Policy”, “Immigration”, “Society” and “Security”. X-stance has 10 domains altogether in which the target numbers and the target-text ratios vary in each domain. To cover more general cases, we choose 4 domains with the median numbers of targets and whose target-text ratios are similar to that of the original X-stance dataset. The detailed targets and their descriptions in Politics and Society datasets are listed in Tables 8 and 9 in Appendix.\n\nWe conduct additional experiments on two low-resource language datasets “czech” in Czech and “r-ita” in Italian from Hardalov et al. (2022). Dataset “czech” contains targets of “Miloš Zeman” and “Smoking Ban in Restaurants”, and “r-ita” contains the target “Constitutional Reform”, thus the target setting is None (i.e., None of the targets in source language and target language are the same). Therefore, we experiment on these four cross-lingual datasets: Politics-czech-None, Politics-rita-None , Society-czech-None, Society-rita-None, abbreviated as *P-czech-None*, *P-rita-None*, *S-czech-None* and *S-rita-None* datasets below.\n\n\n| **Method**     | **P-czech-None** | **P-czech-None** | **P-rita-None** | **P-rita-None** |\n|:---------------|:-----------------:|:-----------------:|:----------------:|:-----------------:|\n| **Measure**     | Acc               | F1                | Acc              | F1                |\n| **TAN**         | 53.01             | 48.89             | 60.69            | 50.34             |\n| **BiCond**      | 52.78             | 51.64             | 56.49            | 50.55             |\n| **CrossNet**    | 52.31             | 51.59             | 55.73            | 50.38             |\n| **JointCL**     | 53.94             | 53.39             | 65.84            | 52.31             |\n| **CLKD**        | 52.78             | 52.45             | 60.31            | 48.36             |\n| **mBERT-FT**    | 52.78             | 52.74             | 67.94            | 51.01             |\n| **mBERT-PT**    | 54.17             | 53.73             | **69.47**            | 51.57             |\n| **CCSD (Ours)** | **56.25**             | **55.89**             | 66.92            | **54.87**             |\n\n\n\n| **Method**      | **S-czech-None** | **S-czech-None** | **S-rita-None** | **S-rita-None** |\n|:---------------|:-----------------:|:-----------------:|:---------------:|:----------------:|\n| **Measure**     | Acc               | F1                | Acc             | F1               |\n| **TAN**         | 50.69             | 49.71             | 61.83           | 47.18            |\n| **BiCond**      | 50.69             | 50.69             | 63.61           | 48.62            |\n| **CrossNet**    | 51.39             | 51.09             | 63.87           | 49.29            |\n| **JointCL**     | 52.08             | 50.50             | 62.60           | 48.80            |\n| **CLKD**        | 51.39             | 49.48             | 64.12           | 50.76            |\n| **mBERT-FT**    | 51.39             | 50.36             | 61.07           | 50.08            |\n| **mBERT-PT**    | 51.39             | 51.05             | 61.07           | 51.58            |\n| **CCSD (Ours)** | **52.78**             | **52.55**             | **64.38**           | **54.54**            |\n\n\nIt can be seen from the tables that our proposed CCSD outperforms all the baselines in these low-resource target languages and totally inconsistent target settings for cross-lingual cross-target stance detection. We shall report the above experimental results in the revised paper.\n\n#### ***W3. The evaluation setup and experiments are limited in two way: (1) to only two stance labels (in favor and against), whereas most of the existing datasets have many more, and they often do not align well in their label definitions, which can pose additional challenges for the proposed approach; (2) pre-training on English datasets had shown good zero-shot/few-shot transfer performance, this is another baseline that should be considered by the authors.***\n\n**Response**: This is a very good point. Although misalignment of labels between different datasets is common in stance detection, this important issue has rarely been considered in cross-lingual stance detection. The pioneer work of Hardalov et al.,(2021) in EMNLP 2021 tackled this issue in monolingual setting by devising label embedding in domain adversarial training. In cross-lingual stance detection domain, Hardalov et al., (2022) in AAAI 2022 proposed label embedding with pre-training to alleviate the situation when labels do not align well in target language. As our problem definition of cross-lingual cross-target stance detection assumes that the labeled data are available only in source language, we consider to tackle this additional research challenge by cross-lingual data augmentation in our future work. \n\n\nChatGPT as a generative large language model mainly pre-trained with English as well as other languages has shown good zero-shot transfer performance, we conduct additional experiments and compare our proposed method with ChatGPT in zero-shot setting (with GPT-3.5-turbo-0613). The prompt used in the experiment is “La position de <TEXT> envers <TARGET> est? Select one word from ‘favor’ and ‘against’.”. It can be seen from the tables that our proposed CCSD outperforms ChatGPT in both datasets under all the settings for cross-lingual cross-target stance detection. Compared with the results of mBERT-PT, ChatGPT performs better in \"Politics-Partial\", \"Politics-None\" and \"Society-None\". We shall report the above experimental results in the revised paper.\n\n\n\n| **Method**              | **Politics-All** | **Politics-All** | **Politics-Partial** | **Politics-Partial** | **Politics-None** | **Politics-None** |\n|:-----------------------|:----------------:|:----------------:|:--------------------:|:--------------------:|:-----------------:|:------------------:|\n| **Measure**             | Acc              | F1               | Acc                  | F1                   | Acc               | F1                 |\n| **ChatGPT (zero-shot)** | 65.95            | 65.41            | **68.02**                | **67.63**                | 61.32             | 60.30              |\n| **mBERT-PT**            | 67.82            | 67.76            | 66.28                | 65.90                | 57.86             | 57.48              |\n| **CCSD (Ours)**         | **70.11**            | **69.92**            | 67.44                | 67.32                | **65.72**             | **65.47**              |\n\n\n| **Method**              | **Society-All** | **Society-All** | **Society-Partial** | **Society-Partial** | **Society-None** | **Society-None** |\n|:-----------------------|:---------------:|:---------------:|:-------------------:|:-------------------:|:----------------:|:-----------------:|\n| **Measure**             | Acc             | F1              | Acc                 | F1                  | Acc              | F1                |\n| **ChatGPT (zero-shot)** | 59.46           | 58.63           | 56.90               | 56.38               | **62.96**            | 59.97             |\n| **mBERT-PT**            | 66.82           | 66.31           | 64.37               | 63.84               | 61.73            | 59.00             |\n| **CCSD (Ours)**         | **67.87**           | **67.32**           | **66.09**               | **65.43**               | **62.96**            | **62.48**             |\n\n\n\n\n\n#### ***Missing References***:\n\n**Response**: We have added the missing references in the revised paper. Thanks for pointing this out.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed method models both the target and the language, which is indeed an important step for cross-lingual stance detection. \n2. The student model can be in theory smaller in size compared to the teachers which makes it more efficient, even though the authors did not test this setup.\n3. The proposed approach outperforms strong BERT-based baselines ins all settings. The authors ablate all of the model's components showing that they are important for the end prediction.", "reasons_to_reject": "1. The proposed method is computationally expensive -- it is based on two teacher networks (with additional architecture modification like GAT) that have to be pre-trained on the source language, and then distilled back to a target langauge network.\n2. The study covers only three languages, non of which is truly low-resource when in terms of pre-training data included as part of the language model's extensive training. Moreover, two of the languages come from the same dataset and domain. Hardalov et al. (2022) can be a good source of candidate datasets, as they used 16 datasets for their experiments.\n3. The evaluation setup and experiments are limited in two way: 1/ to only two stance labels (in favor and against), whereas most of the existing datasets have many more, and they often do not align well in their label definitions, which can pose additional challenges for the proposed approach; 2/ pre-training on English datasets had shown good zero-shot/few-shot transfer performance, this is another baseline that should be considered by the authors. ", "questions_to_authors": "1. How much does translating the prompt into the target language helps?", "paper_topic_and_main_contributions": "The paper presents an approach for cross-lingual and cross-target stance detection. The proposed framework is based on knowledge distillation from two teacher models: 1/ for language transfer, 2/ for stance target transfer, into a student pre-trained language model. The model aims to align the stance classifier's representations across the target and the source language, in order to be able to generalize to unseen targets and languages. The authors experiment with two datasets covering three languages (French, German and English). ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "1. Schiller, Benjamin, Johannes Daxenberger, and Iryna Gurevych. \"Stance detection benchmark: How robust is your stance detection?.\" KI-Künstliche Intelligenz (2021): 1-13.\n2. Momchil Hardalov, Arnav Arora, Preslav Nakov, and Isabelle Augenstein. 2021. Cross-Domain Label-Adaptive Stance Detection. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9011–9028, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics."}}, {"review_id": "62UrZVDT4K", "reviewer": "Reviewer_DhrL", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer DhrL: \nMany thanks to the reviewer for the very helpful comments and suggestions. Below we respond to them in detail with further explanations and supplementary experimental results.\n\n#### ***W1. Insufficient correlation between text and images makes it difficult to associate some paragraphs with figures.***\n\n**Response**: We have carefully revised the figures and their corresponding description texts to enhance the correlations between paragraphs and figures. We have made the following modifications: (1) In Figure 1, replace “PLM” in cross-lingual teacher and cross-target teacher with “mBERT” and “Encoder” respectively, add more details to cross-lingual teacher, and add the arrow line indicating the connection between cross-target teacher and prompt-tuned cross-lingual teacher. (2) In Figure 2, replace “PLM” in cross-target teacher with “Encoder”, and add the illustrations of “target representation ${u}$”, “target semantic vector ${h}_i^t$”, “combined representation ${e}$” and “category representation ${r}$”.\n\n\n#### ***W2. The experiments were conducted only on two stance classification dataset, out of the many available (e.g., WTWT, P-stance).***\n\n**Response**: Thanks for the helpful comments. Due to the difficulties in acquiring the access of WT-WT and P-Stance from Twitter API, we are unable to conduct experiments on them. To further verify the effectiveness of our proposed method, we conduct experiments with two low-resource datasets as target language data, which are “czech” in Czech and “r-ita” in Italian. Dataset “czech” contains targets of “Miloš Zeman” and “Smoking Ban in Restaurants”, and “r-ita” contains the target “Constitutional Reform”, thus the target setting is None (i.e., None of the targets in source language and target language are the same). Therefore, we experiment on these four cross-lingual datasets: Politics-czech-None, Politics-rita-None , Society-czech-None, Society-rita-None, abbreviated as *P-czech-None*, *P-rita-None*, *S-czech-None* and *S-rita-None* datasets below.\n\n\n| **Method**     | **P-czech-None** | **P-czech-None** | **P-rita-None** | **P-rita-None** |\n|:---------------|:-----------------:|:-----------------:|:----------------:|:-----------------:|\n| **Measure**     | Acc               | F1                | Acc              | F1                |\n| **TAN**         | 53.01             | 48.89             | 60.69            | 50.34             |\n| **BiCond**      | 52.78             | 51.64             | 56.49            | 50.55             |\n| **CrossNet**    | 52.31             | 51.59             | 55.73            | 50.38             |\n| **JointCL**     | 53.94             | 53.39             | 65.84            | 52.31             |\n| **CLKD**        | 52.78             | 52.45             | 60.31            | 48.36             |\n| **mBERT-FT**    | 52.78             | 52.74             | 67.94            | 51.01             |\n| **mBERT-PT**    | 54.17             | 53.73             | **69.47**            | 51.57             |\n| **CCSD (Ours)** | **56.25**             | **55.89**             | 66.92            | **54.87**             |\n\n\n\n| **Method**      | **S-czech-None** | **S-czech-None** | **S-rita-None** | **S-rita-None** |\n|:---------------|:-----------------:|:-----------------:|:---------------:|:----------------:|\n| **Measure**     | Acc               | F1                | Acc             | F1               |\n| **TAN**         | 50.69             | 49.71             | 61.83           | 47.18            |\n| **BiCond**      | 50.69             | 50.69             | 63.61           | 48.62            |\n| **CrossNet**    | 51.39             | 51.09             | 63.87           | 49.29            |\n| **JointCL**     | 52.08             | 50.50             | 62.60           | 48.80            |\n| **CLKD**        | 51.39             | 49.48             | 64.12           | 50.76            |\n| **mBERT-FT**    | 51.39             | 50.36             | 61.07           | 50.08            |\n| **mBERT-PT**    | 51.39             | 51.05             | 61.07           | 51.58            |\n| **CCSD (Ours)** | **52.78**             | **52.55**             | **64.38**           | **54.54**            |\n\nIt can be seen from the tables that our proposed CCSD outperforms all the baselines in these low-resource target languages and totally inconsistent target settings for cross-lingual cross-target stance detection. We shall report the above experimental results in the revised paper.\n\n\n#### ***W3. Some notations are not well-defined, and some part of the model description is not clear.***\n\n**Response**: Thanks. We have carefully proofread the whole paper and made the following modifications in Section 3: (1) Add the definitions of $\\overline{t}_i$, ${u}_i$ in $\\mathcal{U}=[{u}_i]$ and $C_k$ in $C=[C_k]$; (2) Revise descriptions on the cross-lingual template in Lines 241-245; (3) Supplement formulas for GAT modeling and so on. More modifications will be included in the revision.\n\n\n#### ***W4. The paper lacks sections on case study and error analysis, which are commonly found in high-quality research articles.***\n\n**Response**: We conduct case study to compare the prediction results of mBERT-FT and CCSD in Politics dataset and Partial setting. mBERT is fine-tuned with source language data and predicts the stance labels on target language data, which cannot generalize to the unseen targets (i.e., case 1, 3 and 4) well. However, our proposed method enhances its generalization ability on the unseen targets by mining target category on the target language data, enabling it to handle the task of cross-lingual cross-target stance detection. \n\n|Case  | Target | Text  | Stance |  mBERT  | CCSD  |\n|---|---|---|---|---|---|\n|1   | La Suisse devrait-elle conclure un accord de libre-échange avec les Etats-Unis?  | Trop de risques pour les standards environnementaux et sociaux...  |  Against |  Favor | Against  | \n|    |English Translation: Should Switzerland conclude a free trade agreement with the United States?| English Translation: Too many risks for environmental and social standards... |    | $\\times$  |  $\\checkmark$ |\n|2   |Les personnes sans-papiers devraient-elles pouvoir obtenir plus facilement un statut de séjour régularisé?   | Le système doit être équitable et être mis en œuvre au cas par cas.  | Against   | Favor  | Against  |   \n| |English Translation: Should undocumented people be able to obtain regularized residence status more easily? |English Translation: The system must be fair and implemented on a case-by-case basis.   |     |  $\\times$ | $\\checkmark$  | \n|3   |La Confédération devrait-elle soutenir davantage les étrangères et étrangers dans leur intégration?   | L'intégration est une responsabilité individuelle, mais proposer des plateformes comme des cours de langues est un pas vers une meilleure intégration.  |Favor    |Against   | Favor  |  \n|    |English Tranlsation: Should the Confederation provide more support to foreigners in their integration?    |English Translation: Integration is an individual responsibility, but offering platforms like language courses is a step towards better integration.  |   | $\\times$ | $\\checkmark$  |\n|4   |Êtes-vous en faveur de la candidature de la Suisse à un siège au Conseil de sécurité de l'ONU?   |Un siège au Conseil de sécurité de l'ONU est bénéfique à la Suisse, car celle-ci pourrait prendre part aux décisions contraignantes et agir en tant que médiatrice afin de maintenir la paix dans tous les états membres.   |Favor     |Against | Favor   | \n|    |English Translation: Are you in favor of Switzerland's candidacy for a seat on the UN Security Council?    |English Translation: A seat on the UN Security Council is beneficial to Switzerland, as it could take part in binding decisions and act as a mediator in order to maintain peace in all member states.   |   | $\\times$ | $\\checkmark$   | \n\n\nWe also provide an error analysis on the three cases: (1) When text is too short, it needs additional background knowledge and reasoning process to predict stance labels. It is difficult to handle such situation only by transferring knowledge from the source language to the target language. (2) When the gap between targets in different languages (i.e., the correlation is small), such as political events and presidential candidates in different countries, our method suffers from performance degradation due to completely different distributions in data and knowledge. (3) When the language distance is large (such as German and Cezch), our method also makes more prediction mistakes. We plan to replace mBERT with a larger pre-trained language model as the base of cross-lingual teacher in future research. \n\n#### ***Missing References***:\n\n**Response**: We have added the missing references in the revised paper. Thanks for pointing this out.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The general idea seems novel and interesting to the community.\n- The method seems to be effective.", "reasons_to_reject": "As mentioned above, \n- Some notations are not well-defined.\n- Some part of the model description is not clear.\n- The paper lacks sections on case study and error analysis, which are commonly found in high-quality research articles.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper addresses the problem of stance detection and proposes a new task: cross-lingual cross-target stance detection. To achieve this goal, this paper designs a cross-lingual teacher and a cross-target teacher using source language data and a dual distillation process that transfers the two types of knowledge to the target language. To bridge the target inconsistency gap, cross-target teacher mines the category information via target representation learning and refinement, and generalizes it to the unseen targets via category-oriented contrastive learning. Both of these components seem to be effective in the new task.\n\nStrengths:\n- The cross-lingual cross-target stance detection task seems novel and interesting to many researchers.\n- The experiments are generally well-designed and show the effectiveness of the method.\n- This work tackles a useful task. The approach is reasonable and is backed with empirical results.\n\nWeaknesses:\n- Insufficient correlation between text and images makes it difficult to associate some paragraphs with figures.\n- The experiments were conducted only on two stance classification dataset, out of the many available (e.g., WTWT, P-stance). ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Huang, Hu, Bowen Zhang, Yangyang Li, Baoquan Zhang, Yuxi Sun, Chuyao Luo, and Cheng Peng. \"Knowledge-enhanced Prompt-tuning for Stance Detection.\" ACM Transactions on Asian and Low-Resource Language Information Processing 22, no. 6 (2023): 1-20.\n\n[2] Yingjie Li, Chenye Zhao, and Cornelia Caragea. \"Improving stance detection with multi-dataset learning and knowledge distillation.\" In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6332-6345. 2021.\n\n[3] Bowen Zhang, Daijun Ding, and Liwen Jing. \"How would Stance Detection Techniques Evolve after the Launch of ChatGPT?.\" arXiv e-prints (2022): arXiv-2212."}}, {"review_id": "dmh7ZJlHby", "reviewer": "Reviewer_UMp6", "summary": "", "questions": "Thanks for the paper, I enjoyed reading it.\n\nA. By what method did you select the various hyperparameters, especially K, tau and alpha? Did you use the validation set in the source language or in the target language? Are Figures 3 and 4 (which illustrate sensitivity to hyperparameter choice) based on the validation set or on the test set?\n\nB. Imprecise claim about related work: L010f: “previous research has ignored the practical issue of no labeled data available in target language” / L059f: “current research … has not addressed the issue when there are no labeled data available in the target language” - See references [1–4] (some of which already cited), which are all concerned with zero-shot transfer to a target language.\n\nC. What is your motivation to evaluate on two subsets of X-stance and not on the complete dataset? To me, the composition of the “Politics” and “Society” seems relatively arbitrary.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer UMp6:\nWe thank the reviewer very much for the valuable suggestions and comments. Below we respond to them in detail with further explanations and supplementary data.\n\n#### ***Q1. By what method did you select the various hyperparameters, especially K, tau and alpha? Did you use the validation set in the source language or in the target language? Are Figures 3 and 4 (which illustrate sensitivity to hyperparameter choice) based on the validation set or on the test set?***\n\n**Response**: We select hyperparameters K, tau and alpha using the following ways: (1) K represents the number of clusters in k-means. We determine the range of K according to the number of targets, and further select K on the validation set. (2) Tau is the temperature that controls the distribution smoothness of the logits in contrastive learning. In practice, tau in contrast learning is usually less than 1, so that the model pays more attention to negative samples and enhances generalization ability. We set tau as a range from 0.1 to 1, and further select them on the validation set. (3) Alpha balances cross-lingual distillation and cross-target distillation losses. We set alpha according to the degree of target inconsistency. We increase the value of alpha with target inconsistency (i.e., from All to Partial and None), making the dual distillation process pay more attention to cross-target transfer. We set alpha as a range from 0.1 to 1, and further select them on the validation set. \n\nWe use the validation set in the target language to determine the values of hyperparameters. In the training process of the dual teacher models, we take the best model on the validation set in the source language as the final teacher models. In the distillation process of the student model, we take the best model on the validation set in the target language to make predictions on the test set in the target language. Figures 3 and 4 report the results on the test set in the target language, verifying the sensitivity and regularity of the model performance on different hyperparameters.\n\n\n#### ***Q2. Imprecise claim about related work: L010f: “previous research has ignored the practical issue of no labeled data available in target language” / L059f: “current research … has not addressed the issue when there are no labeled data available in the target language” - See references [1–4] (some of which already cited), which are all concerned with zero-shot transfer to a target language.***\n\n**Response**: Sorry for the confusion. Here we confine our work to cross-lingual stance detection and as we have stated in Lines 59-62 in the original paper “However, current research **on cross-lingual stance detection** has not addressed the practical issue when there are no labeled data available in the target language”. In the broader domain of cross-lingual text classification for example in Schick and Schütze, (2021) [2], it is indeed not a new issue that there are no labeled data in the target language. However, in this paper, we focus on cross-lingual stance detection and address the problems of no labeled data in target language and inconsistent targets between languages. \n\nWithin cross-lingual stance detection, Hardalov et al. (2022) [3] are concerned with the zero-shot setting in the downstream experiments. However, they pre-trained the multilingual language model with additional corpora and prompt-tuned it with zero/few samples in the test set. Vamvas and Sennrich, (2020) [1] conducted zero-shot experiments with cross-lingual only setting (i.e., with the same range of targets) and cross-target only setting (i.e., within the same language) separately. Similarly, Barriere et al., (2022) [4] proposed a multilingual stance dataset for transfer learning across languages, targets and domains separately. \n\nIt is worth noting that without the labeled data, target inconsistency causes the additional issue of unseen targets in target language, which in essence requires the consideration of both cross-lingual transfer and cross-target transfer. We tackle the two issues of unlabeled data and unseen targets in the target language simultaneously for cross-lingual cross-target stance detection, which is essentially a new task. \n\n\n\n#### ***Q3. What is your motivation to evaluate on two subsets of X-stance and not on the complete dataset? The composition of the “Politics” and “Society” seems relatively arbitrary.***\n\n**Response**: We construct Politics dataset and Society dataset from 4 different domains in X-stance, “Foreign Policy”, “Immigration”, “Society” and “Security”. X-stance has 10 domains altogether in which the target numbers and the target-text ratios vary in each domain. To cover more general cases, we choose 4 domains with the median numbers of targets, that is deleting the domains with too many targets or too less targets, and whose target-text ratios are similar to that of the original X-stance dataset. The detailed targets and their descriptions in Politics and Society datasets are listed in Tables 8 and 9 in Appendix.\n\n\n#### ***W1. Highly engineered approach: The proposed approach is more accurate than simply fine-tuning mBERT, but also much more complex. It might be challenging to adapt the approach to other tasks or settings.***\n\n**Response**: Due to the nature of cross-lingual cross-target stance detection task, which is inherently more complex than previously proposed cross-lingual stance detection and monolingual cross-target stance detection, we train two teacher models to learn language-related and target-oriented knowledge respectively. Regarding each individual teacher model, our method did not add extra computational complexity and the number of parameters in our method remains roughly the same as those of mBERT. \n\nSince the baseline methods in our original paper all use mBERT to encode input targets and text, we run an additional experiment to calculate the numbers of parameters in these methods and our own method. It can be seen from the table that mBERT contains about 178M parameters, JointCL has about 180M parameters with an extra prototypical graph, and the teacher-student distillation method CLKD has about two sets of 179M parameters. The last three lines report the parameter numbers of our CCSD method, which is comparative to those of the baselines including mBERT.  \n\n\n\n| Method  | Teacher/Student             | Component                               | # Params   |\n|:---------|:-----------------------------|:-----------------------------------------|:------------|\n| mBERT   | -                           | mBERT                                   | ≈178M      |\n| JointCL | -                           | mBERT + prototypical graph + classifier | ≈180M      |\n| CLKD    | Teacher                     | mBERT + classifier                      | ≈179M      |\n| CLKD    | Student                     | mBERT + classifier                      | ≈179M      |\n| CCSD    | Cross-Lingual Teacher  | mBERT                                   | ≈178M      |\n| CCSD    | Cross-Target Teacher   | mBERT + target graph + classifier       | ≈180M      |\n| CCSD    | Student                | mBERT + classifier                      | ≈179M      |\n\n\n#### ***W2. There is no indication that the code will be shared, which makes it altogether less likely that this approach will be adopted in future work.***\n**Response**: We shall release the code after peer review period, and we have also added the available online access site to the revised paper.\n\n\n#### ***Missing References and Typos***:\n\n**Response**: We have added the missing references in the revised paper. We also carefully checked the whole paper, corrected the typos and improved the overall presentation. Thanks for pointing this out.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper presents experiments that show a consistent improvement in accuracy over a broad range of baseline approaches.\n- The experimental setup is clearly documented.", "reasons_to_reject": "Generally, the paper presents the research in a clear manner. However, there are a few factors that limits its soundness and its potential impact on the field:\n- Highly engineered approach: The proposed approach is more accurate than simply fine-tuning mBERT, but also much more complex. It might be challenging to adapt the approach to other tasks or settings. There is no indication that the code will be shared, which makes it altogether less likely that this approach will be adopted in future work.\n- [Added based on author response Q1]: Hyperparameter are optimized using a validation set in the target language. Experiments on zero-shot transfer usually avoid this, because it violates the assumption that there are no annotated data in the target language.\n- [Updated based on author response Q2]: The introduction exaggerates the novelty of the contribution and makes imprecise statements about the related work. In my view, it is not accurate to say that “previous research has ignored the practical issue of no labeled data available in target language” (L010f) and that “current research … has not addressed the issue when there are no labeled data available in the target language” (L059f); see references [1–4].", "questions_to_authors": "Thanks for the paper, I enjoyed reading it.\n\nA. By what method did you select the various hyperparameters, especially K, tau and alpha? Did you use the validation set in the source language or in the target language? Are Figures 3 and 4 (which illustrate sensitivity to hyperparameter choice) based on the validation set or on the test set?\n\nB. Imprecise claim about related work: L010f: “previous research has ignored the practical issue of no labeled data available in target language” / L059f: “current research … has not addressed the issue when there are no labeled data available in the target language” - See references [1–4] (some of which already cited), which are all concerned with zero-shot transfer to a target language.\n\nC. What is your motivation to evaluate on two subsets of X-stance and not on the complete dataset? To me, the composition of the “Politics” and “Society” seems relatively arbitrary.", "paper_topic_and_main_contributions": "This paper is about the special case of stance detection where both the language of the input and the topic (target) are zero-shot. The authors propose a knowledge distillation approach with two teachers, where one teacher is optimized for cross-lingual generalization and the other for cross-topic generalization. A student model is then trained on an interpolation of their predictions on unlabeled data in the target language.\nThe authors experiment with transfer from German training data to French and English test data. They find that their approach outperforms a range of baselines.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- L312: “adjacent” -> “adjacency”"}, "extra_text": {"Missing_References": "[1] Vamvas, Jannis, and Rico Sennrich. \"X-stance: A Multilingual Multi-Target Dataset for Stance Detection.\" 5th SwissText & 16th KONVENS Joint Conference 2020. CEUR-WS. org, 2020.\n\n[2] Timo Schick and Hinrich Schütze. 2021. Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269, Online. Association for Computational Linguistics.\n\n[3] Hardalov, Momchil, et al. \"Few-shot cross-lingual stance detection with sentiment-based pre-training.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 10. 2022.\n\n[4] Valentin Barriere, Alexandra Balahur, and Brian Ravenet. 2022. Debating Europe: A Multilingual Multi-Target Stance Classification Dataset of Online Debates. In Proceedings of the LREC 2022 workshop on Natural Language Processing for Political Sciences, pages 16–21, Marseille, France. European Language Resources Association."}}]

Author rebuttal:

Meta-review: Reviewers are cautiously positive in their assesment of the paper (3-4).  There are objections to the task being suboptimally selected relative to other work in cross-lingual stance detection and to the set-up being unnecessarily complex.  Despite this, reviewers agree that the experimental set-up is well-documented and the results are positive.
Decision: Accept-Main