The Power of LLM-Generated Synthetic Data for Stance Detection in Online Political Discussions
Authors: Stefan Sylvius Wagner
Maike Behrendt
Marc Ziegele
Stefan Harmeling
Venue: ICLR 2025 Spotlight (2025)
Keywords: large language models
stance detection
data augmentation
active learning
online political discussions
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We study and show how to leverage LLM-generated synthetic data for stance detection in online discussions, which is a challenging stance detection task because of the broad range of debate questions.

Abstract: Stance detection holds great potential to improve online political discussions through its deployment in discussion platforms for purposes such as content moderation, topic summarisation or to facilitate more balanced discussions. Typically, transformer-based models are employed directly for stance detection, requiring vast amounts of data. However, the wide variety of debate topics in online political discussions makes data collection particularly challenging. LLMs have revived stance detection, but their online deployment in online political discussions faces challenges like inconsistent outputs, biases, and vulnerability to adversarial attacks. We show how LLM-generated synthetic data can improve stance detection for online political discussions by using reliable traditional stance detection models for online deployment, while leveraging the text generation capabilities of LLMs for synthetic data generation in a secure offline environment. To achieve this, (i) we generate synthetic data for specific debate questions by prompting a Mistral-7B model and show that fine-tuning with the generated synthetic data can substantially improve the performance of stance detection, while remaining interpretable and aligned with real world data. (ii) Using the synthetic data as a reference, we can improve performance even further by identifying the most informative samples in an unlabelled dataset, i.e., those samples which the stance detection model is most uncertain about and can benefit from the most. By fine-tuning with both synthetic data and the most informative samples, we surpass the performance of the baseline model that is fine-tuned on all true labels, while labelling considerably less data.

Published as a conference paper at ICLR 2025
THE POWER OF LLM-GENERATED SYNTHETIC DATA FOR STANCE DETECTION IN ONLINE POLITICAL DIS- CUSSIONS
Stefan Sylvius Wagner Maike Behrendt Mare Ziegele & Stefan Harmeling Heinrich Heine University Düsseldorf, Technical University Dortmund Lamarr Institute for Machine Learning and Artificial Intelligence {stefan. wagner, maike. behrendt, mahke.be stefan harmel iameelin@tuudortmmi
marc. ziegeleeeeeeeee (aralee@hhhhddd
ABSTRACT
Stance detection holds great potential to improve online political discussions through its deployment in discussion platforms for purposes such as content mod- eration, topic summarization or to facilitate more balanced discussions. Typically, transformer-basee models are employed directly for stance detection, requiring vast amounts of data. However, the wide variety of debate topics in online political discussions makes data collection particularly challenging. LLMs have revived stance detection, but their online deployment in online political discussions faces challenges like inconsistent outputs, biases, and vulnerability to adversarial attacks. We show how LLM--genertted synthetic data improve stance detection for online political discussions by using reliable traditional stance detection models for online deployment, while leveraging the text generation capabilities of L.LMs for synthetic data generation in a secure offline environment. To achieve this, (i) we generate synthetic data for specific debate questions by prompting Mistral-7B model and show that fine-tuning with the generated synthetic data can substantially improve the performance stance detection, while remaining interpretable and aligned with real world data. (ii) Using the synthetic data as a reference, we can improve performance even further by identifying the most informative samples unlabelled dataset, i.e., those samples which the stance detection model is most uncertain about and can benefit from the most. By fine-tuning with both synthetic data and the most informative samples, surpass the performance of the baseline model that is fine-tuned on all true labels, while labelling considerably less data.
INTRODUCTION
With the recent advent of powerful generative Large Language Models (LLMs) such as ChatGPT, Llama (Touvron al. 2023) and Mistral (Tiang et al. 2023), new ways of performing stance detection have opened up via zero-shot or chain-of-thought prompting. This is especially important in the area of online political discussion where topies are complex and labelled data is hard come by. At the same time, an ever important use case in online political discussions is being able to use stance detection for an ongoing discussion to, e.g., suggest suitable comments for engagement between participants (Küçūk & Can 2020: Behrendt et al.) 2024). In the case of LLMs, while strong at analysing complex topics and open- text generation, explicit classification can be inconsistent (Cruickshank & Xian Ng 2023), they are prone to biases (Ziems et al.) 2023) and open to adversarial attacks (Greshake et al., 2023). More traditional stance detection models based on, e.g., BERT (Devlin et al. 2019) task-specific and therefore consistent in their output, however they need large amounts labelled data (Mchrafarin et al. 2022 Vamvas & Sennrich, 2020) to perform well.
In this work, we combine both traditional stance dectection and LLMs to get the best of both worlds. For stance dectection, we use BERT as a lightweight stance detection model that produces fast and consistent output given the data has been fine-tuned amounts of data, we propose to generate synthetic data with an LLM to augment the stance detection model for fine-tuning. This allows leverage I.I.Ms in offline setting enhance classical
To address the issue of needing large

---

Published as a conference paper at ICLR 2025
A
B
Fositive Synth
NegaLive Synth
Train date
Most informative
Figure I: We investigate the use of --geerrrteeed synthetic data for stance detection in online political discussions. (A) We generate synthetic data @o for specific questions using Mistral-7B model. The synthetic data then used to fine-tune the stance detection model. We show that fine-tuning with synthetic data improves the performance of the model, since the synthetic data is roughly faithful to the real data's ' O underlying distribution. However, some real world samples O cannot be captured by the synthetic data. (B) We therefore use the synthetic data identify the most informative samples + in the unlabelled real data pool, which are better off labelled by human experts. Combining the synthetic data with the manually labelled most informative samples improves the performance of the model even further.
stance detection models, which are better suited and safer for in online setting. Furthermore, we show that the synthetic data allows us to gain insights about the real world data. We show that the synthetic data generated by the LLM can serve as a reference distribution for stance detection, since the LLM is able to generate high quality samples which fall distinctly into the respective stance classes. This has two benefits: (i) we can analyse the data for potential biases by comparing the
alignment of the synthetic data distribution to the real world data distribution. (ii) we can tackle the issue of extracting ambiguous samples that are difficult for the model to classify and therefore deteriorate its performance. Since the synthetic data partitions the space between both classes, ambiguous samples that the stance detection model cannot classify properly can be identified as lying in between the two classes. These samples can then be labelled manually. Fine-tuning with these additional samples, outperforms the model that is fine-tuned on all true labels alone, while only having labelled subset of samples in the unlabelled data pool. We illustrate our method in Figure
We view stance detection as a binary classification problem (favor or against), where we explore the following questions:
(Q1) Does fine-tuning a stance detection model with synthetic data improve stance detection performance? We first analyse whether fine-tuning the BERT model with synthetic data improves stance detection and show that this approach almost reaches the model trained with all true labels and is superior to using zero-shot Mistral-7B for the complex topics in online political discussions. We reveal that a stance detection model can be tailored to a certain topic with only synthetic data.
(Q2) How does the synthetic data improve performance and does it align with real world data? Our second question analyses the generated synthetic data. We analyse how well it aligns with the real world training data by visualising the T-SNE projected embeddings of the stance detection model and by comparing the entropy distribution of the synthetic data to the real data. We find that the synthetic data aligns well with the real data, indicating that the LLM is able to generate comments for both stances while introducing minimal further bias.
(Q3) Can we further improve the model by using the synthetic data as labelled reference distribution for active learning? The synthetic data allows to identify unlabelled real data samples that improve the model even further through active learning. Due to the canonical nature of the synthetic data, we are able to extract real word samples for human labelling that are difficult (ambiguous) for the model to classify. We do this, by determining the k-nearest synthetic neighbours of the real data. The stance detection model is fine-tunee tuned jointly with these samples and the synthetic data, where we surpass the baseline model even when it is fine- tuned on all true labels, while labelling considerably less data manually.
2

---

Published as a conference paper at ICLR 2025
2 BACKGROUND
Stance Detection for Online Political Discussions. Stance detection, a sub-task of sentiment analysis (Romberg & Escher] (023) and opinion mining (ALDayel & Magdy] (021), aims to automatically identity an author's 's stance (favor, against, or neutral) towards a discussed issue or target. In online political discussions, this involves determining if the contribution in question is for or against& topic like tax increases. Stance detection has been identified as an important task for improving discussion summarization (Chowanda et al. 2017). detecting misinformation (Hardalov) ct al. 2022). and evaluating opinion distributions in online political discussion and participattin processes (Romberg & Escher, (023). Stance detection is also used in recommender systems and discussion platforms (Küçük & Can (020). Still, due to its dependency on context, stance detection is highly challenging task. Identifying stance requires understanding both the question and the contributor's position, complicated by users often deviating from the original question and discussing multiple topics in the same thread (Ziegele et al. 2014), leading to little usable training data. Some works in stance detection use graph convolutional networks to learn more out of the present data models (Vaswani et al. 2017 Liu et al. 2022) to solve stance detection is a common practice, but training these models requires large amount of annotated data, which for the large variety of questions in online political discussions is unfeasible to acquire.
(Zhang et al., 2022, Li & Goldwasser 2019). Recently, fine-tuning transformer-base
Active Learning. The aim of active learning is to minimize the effort of labelling data, while simultancously maximizing the model's performance. This is achieved by selecting a query strategy that chooses the most interesting samples from a set of unlabelled data points, which we refer to as most informative samples. These samples are then passed to, e.g., human annotator for labelling. There exist many different query strategies such as Query By Comittee (QBC, (Seung et al., 1992), Minimum Expected Entropy (MEF, (Holub et al. (2008) or Contrastive Active Learning (CAL, Margatina et al. (202I)). By actively choosing samples and asking for the correct labelling, the model is able to learn from few labelled data points, which is advantageous especially when annotated datasets are not available. Within the domain of political text analysis, many different tasks lack large amounts of annotated data. It has been already shown in the past that these tasks can benefit from the active learning: e.g., stance detection (Kucher et al.) 2017). topic modeling (Romberg & Escher 2022), speech act classification (Schmidt al. 2023) toxic comment classification (Miller et al.] 2020). In this work, we examine how LLM-ggeerated synthetic data can be used instead of real labelled data to select the most informative samples to be manually labelled.
Stance detection and synthetic data generation with LLMs. Recent work has shown that synthetic data generated from LLMS can be used improve the performance of a model on downstream tasks. Möller et al. (023) showed that synthetic data can be used to improve the performance of a model on downstream classification tasks by comparing the performance of model finetuned on L.M--gnerated data to crowd annotated data. In many cases the model finetuned on LLM-ggeerater data outperforms the model finetuned on crowd annotated data. Mahmoudi et al. (2024) study the use of synthetic data for data augmentation in stance-detection. The authors use GPT-3 to generate synthetic data for a specific topic with mixed results due to the inability of GPT-3 to generate good data samples. In our work, we use newer LLM model, Mistral-7B, which generates better synthetic data samples and show that we can generate synthetic data that matches the real data distribution. Veselovsky al. (2023) analyse in which ways synthetic data is best generated for tasks like sarcasm detection and sentiment analysis. The authors reach the conclusion that grounding the prompts to generate the synthetic data real samples helps improve the quality of the synthetic data. Similarly, Li et al. (003) argue that subjectivity in the classification task determines whether synthetic data can be used effectively. has been shown that LLMs can be used directly for stance detection such as (Cruickshank & Xian Ng 2023), (Burnham 2023) (Ziems et al. 2023). However, the general conclusion of these studies is that while LLMs are competitive with other transformer models such as BERT, especially for edge cases, they exhibit replication issues. (Burnham (2023) also discuss the posibility of pre-training models on more specific data to improve the generalisation capability of the model. Ziems et al. (2023) highlight the potential biases that can emerged in open ended generation tasks and classification performance varies depending on how representative the training data is. We therefore focus on using LLMs to generate synthetic data solve key challenges in stance detection such as the lack of available data for specific topics and labelling large amounts of data, rather than using L.MSs directly for the task.
3

---

Published as a conference paper at ICLR 2025
3 METHOD
In line with our declared contributions, we present our core ideas to improve the performance of the stance detection model: (i) To fine-tune the model with synthetic data, first define the synthetic dataset and show our pipeline to generate it. The baseline model is then further fine-tuned on the synthetic data. (ii) We then present our synthetic data-based approach to identify the most informative samples from a batch of unlabelled data, where we propose synthetic extension to the QBC (Query by Commitee, (Seung et al. 1992)) method, where the synthetic data act an ensemble of experts. As described in Section and Figure | we use the synthetic data as reference distribution to identify the most informative samples in the unlabelled data pool. The idea is that for ambigouous samples the k-synthetic nearerst neighbours are split in their labels and therefore lie on the decision boundary of the model.
3.1 PRELIMINARIES
Political discussions are typically centered around questions y E Q (sometimes also called issues or targets). For stance detection, we usually have for each of these questions q a set of labelled data D(v) {(((1, (+) Ji=1 where I,(() E X is statement (or comment) and y is the stance of the statement, with (d) E [0 = y. Note, that use the notation DD4) for labelled and for unlabelled datasets (then the labels are ignored). We view the stance detection model as a binary classification function f Q X + V. where we included the question as input to provide context. The stance detection model such as BERT (Devlin et al. 2019) is fine-tuned by minimizing the cross-entropy loss between the predicted labels g( = f(g, 2) and the actual labels y
(()))
3.2 GENERATING SYNTHETIC DATA FOR STANCE DETECTION
To generate synthetic samples, employ quantized version of the -B-instruct v0.1 model to generate comments on a specific question q. using the following prompt:
A user in a discussion forum is debating other users about the following question: [q] The person is in favor about the topic in question. What would the person write? Write from the person's first person perspective.
where " "[[]" must be replaced with the question q. Similarly, generate negative sample, replace "is in favor" with "is not in favor". As in the X-Stance dataset (Vamvas & Sennrich 2020), we assign the two labels and 1. We denote the question-specifie synthetic dataset as:
((a)
synth = ( (<=_gm)) })} M/2 m=1 U (((
M m=1+M/22 )))
(I)
where half of the M synthetic data samples have positive labels, i.e., are comments in favor for the posed question, while the other half against. Since the dataset is in German, we translate the questions q with "NLLB-300M" (NLLB Team et al. 2022) translation model. The English answers from the Mistral-7B model are then translated back to German using the translation model. We also tried other similar sized open source LLMs (Llama, Openassistant, Falcon), but found that only the Mistral-7B model produced sensible comments.
Overall, the generated dataset synth will be used in two ways: (i) to augment the existing dataset D(g) order to increase the amount of training data, and (ii) to detect the most informative samples in the unlabelled data pool, which is explained next.
3.3 GETTING THE MOST INFORMATIVE SAMPLES: SYNTHETIC QUERY BY COMITTEE
To identify the ambiguous (most informative) samples as described in (Q3) we take from two active learning methods: Query by Comittee (QBC) (Seung et al. 1992) and Contrastive Active Learning Margating et al. 202Th). Instead of using QBC's 's ensemble of experts and the KL-divergence based information score in CAL, directly use the synthetic data and its labels identify ambigous samples using k nearest neighbors. The most informative samples are then the data points with the most indecisive scores. Synthetic Query by Comitte (SQBC) consists of three steps:
(1) Generate the embeddings. Given some embedding function g Q x X + E, we generate embeddings for the unlabelled data, F {e(0) / i=1 [(0g)x(0)) ) and for the labelled synthetic
4

---

Published as a conference paper at ICLR 2025
(m)) Ar data Esynch { m=1 {9(g, = (yyn)) ) m=1- Note that q is the question for which we generate the synthetic data and for which we want detect the most informative samples. If obvious from the context, we often omit the superscript (g).
)
(2) Using the synthetic nearest neighbours as oracles to score the unlabelled data. For the i-th unlabelled embedding e! let (N( be the set of indices of the k nearest neighbours (wrt. to the embeddings using the cosine similarity) among the labelled embeddings Esyath The score for each
unlabelled data point counts the number of labels Usynth (m)
1 among the nearest neighbours, i.e.,
s(i)
Usyath (k.....k...
MENN(1)
For our experiments, we choose k for k are possible, but did not lead to significantly better results).
M22 which worked well across all experiments (other values
(3) Choosing the most informative samples. The scores take values between and k. For 0, the (m) synthetic nearest neighbours all have labels Usymth = O, for value k; all have labels V.ymi 1. The most informative samples have score around k/2. We thus adjust the range of the scores so that values the middle range have the smallest scores (close 0). We do this by subtracting k/2 from the score and taking the absolute value,
s(i) = (i) - k/2l-
The J/ most informative samples Mint C Dla) among the unlabelled samples are the J/ samples with the smallest scores. In the experiments we vary J to study the impact of manually labelled most informative samples. Finally, the most informative samples are labelled by a human expert.
EXPERIMENTS
4.1 DATASETS
X-Stance dataset. We evaluate on the German dataset of the X-Stance dataset (Vamvas & Sennrich, 2020), which contains 48, 600 annotated comments on many policy-related questions (140 topics), answered by Swiss election candidates. We chose the German X-Stance dataset because it to our knowledge the most comprehensive stance detection dataset with a variety topics and comments, while also being focused on online political discussions in governmental participation processes. Known english datasets such as SemEval-200 2016 (Mohammad et al., 2017) or P-Stance (Kūçük & Can 2020) offer less variety and more focused social media discussions. The comments are labelled either as being in favor (positive) or against (negative) the posed question. The dataset is split in training and testing questions, i.e., question in the training dataset does not appear in the test dataset. Furthermore, for each question from the training data, there are several annotated comments, which form the dataset Train Analogously, for the test data we have set of annotated comments written as D(g) To refer to the whole training dataset we write Dirain U o D (g)
train
For our experiments, we fine-tune all stance detection models for each question separately allowing for better performance since the data distributions can vary greatly between questions (also a common scenario in online political discussions). To limit computation time, we selected 10 questions from the test dataset to evaluate our method, that best reflect the variability of the data (see Appendix E.I). We split the datasets of these questions into Test-Train and Test-Test to fine-tune the BERT model on synthetic data and to perform active learning. That is, we use the Feet-Trainn dataset to fine-tune the model and the Test-Test dataset to evaluate the model. The number of comments in the 10 selected test questions is shown in Table TO
Synthetic dataset. For synthetic data-augmentation and active learning based on SQBC (see Section 3.3) we generate synthetic datasets of varying sizes M {200, 500, 1000} for each of the 10 questions. The synthetic data follows the same structure as the data from the X-stance dataset, where for a specific question we have M comments and M labels. Each set contains M/ / positive labels and M/ negative labels, ie., the synthetic data is balanced. We show samples of the synthetic data in Appendix C.2
5

---

Published as a conference paper at ICLR 2025
Jotal-Test
29
259
Test-Train
146 154
108
106
158
Test-Test 87 113
10
188
39
70 13
86 73
118
70 6.0660
20 152
71 101
Table 1: Number of comments in our 10 selected test questions for our experiments. The numbers are split into (favorl against) labels. We have 60 40 (Test-Train, Test-Test) split.
4.2 EXPERIMENTAL SETUP
General setup. For all experiments, we start with pre-trained BERT base model and adapt to the stance detection task by fine-tuning on the X-Stance training dataset Drain (all questions). We call this the Baseline since it is the vanilla BERT-based stance detection (e.g.. Vamvas & Sennrich (2020)).
We evaluate our methods along the lines of the questions proposed Section T (Q1): we analyse the effect of fine-tuning Baseline with synthetic data and compare it to the Baseline that was only fine-tuned on Dusin- (Q3): we fine-tune Baseline with synthetic data and the most informative samples. We present the baselines and our methods in the following:
Baseline methods.
Baseline:
the default BERT model fine-tuned only on Drrin (e.g.. Vamvas & Sennrich (2020)).
True Labels: we fine-tune Baseline on the true labels of D Random+Synth, CAL+Synth: the active learning approaches get the most informative samples D(s)
Our methods.
insSyth:: we fine-tune the Baseline on the synthetic data (D9)
symth"
True Labels+Synth: fine-tune True Labels additionally the synthetic data syath
SQBC+Synth: we apply our active learning approach to get the most informative samples D(9)
Minf
For fine-tuning with synthetic data only (Q1), we compare the performance of our approaches the baselines without synthetic data. For the active learning methods (Q3), we compare to the non-active learning and active learning baselines. For further experimental details we refer to Appendix [
Analysing the synthetic data. To analyse the synthetic data (Q2), we visualize the BERT embed- dings of the synthetic data together with the embeddings of the real world data (see B and Appendix[A A). We use T-SNE (van der Maaten & Hinton (0088) to project the embeddings, i.c., the output CLS token of the BERT model to a two-dimensional space. To assess whether the synthetic data captures the overall characteristics of the real world data and shares similar labels, we plot the individual embeddings of the synthetic data together with the means of the embeddings the real world data. Additionally, we plot their corresponding labels. Finally, also visualize the most informative samples selected by the different active learning methods.
4.3 RESULTS: EVALUATING THE EFFECTIVENES OF THE SYNTHETIC DATA
(Q1) LLM-ggnnrated synthetic data substantially improves stance detection. We show in Figure
2, that fine-tuning with only synthetic data improves the stance detection model. For M = 1000 the performance almost reaches the True Labels model, indicating that we can tailor a model to certain topic without having any data for it. Furthermore, we can still improve the model consistently when new labelled data is available as seen by the True Labels+Synth models. In Section 5.11 we Mistral-7B, but show that
also analyse stance detection with zero-shot and fine-tuning approaches these are far less effective on the X-Stance dataset than our fine-tuned BERT model. In the following, attempt provide understanding to why the synthetic data is so effective.
(Q2) The synthetic data aligns well with real world data. We compare the T-SNE projected embeddings of the synthetic and real data in Figure B] (more visualisations in Appendix AA. The synthetic data aligns well with the real world dataset, since the means of the training data are close and in the direction of the synthetic data. We further analyse the synthetic data in Section 5.2 the

---

Published as a conference paper at ICLR 2025
Fine Fine-turing with Synthetic Data
0.76
0.74
Soase
0.06
0.64
ooly Synthetic Dasa
Synthotic "zuc Labela
Ratslica Damtlica-Synth (M-211)
Labala
Lobelasdynck. (#-586)
tamiltee-yyck (M-203E)
labularddynzz (8+22000)
Figure 2: Q1: Fine-tuning the model with synthetic data improves performance for increasing dataset size: Shown the F1-Score of fine-tuning with Only Synthetic Data (left) and Synthetic Data + True Labels (right) for increasing synthetic dataset size. Even if a dataset has been fully labelled, augmenting it with synthetic data proves equally as effective.
Synthetic Data Ali giggnttt
Most Informative Samplas
Random
Neg. Syrth
Pos. Synth
2os. Train (Haan)
Neg. The [Wean]
Belacted Train
Figure 3: (Q2) Analysing the synthetic data (M=1000): The synthetic data aligns well with the real data, which is crucial for improving stance detection performance and to check for potential biases introduced by the synthetic data. SQBC selects the samples that are in between the two classes, i.e, that are the most ambiguous and informative for the model.
synthetic data is generally of high quality which validates the notion that it can serve as a good reference distribution for the model. From statistical learning point of view the synthetic data can be thought smoothening the decision boundary of the model. This also explains why fine-tuning with real samples is also very effective. Furthermore, data that aligns with the given topic The insight that the synthetic data provides a good prior, serves the synthetic data identify ambiguous samples that the most informative to the model. We elaborate on this the following section.
show that it is crucial to generate synthetic
motivation
(Q3) Synthetic data aids in finding unlabelled samples that further improve the stance detection model by extending its decision boundary. We show the results of combining the most informative samples and synthetic data in Figure Combining both, oupterform True Labels while using only 25% of the labelled data. We compare the selection strategy of the methods in Figure 3 B): Due the k-nearest neighbours objective of SQBC, the model selects samples that are between the two classes, which proves superior to CAL and to Random for smaller synthetic data sizes. CAL performs the worst across the board: assumes that similar embeddings that have different outputs are ambiguous, which makes it prone to outliers in the real data, e.g., when the stance detection model misclassifies sample. Therefore, CAL often selects samples from only one class which worsens performance. Interestingly for M 1000, Random outperforms both active learning methods SQBC and CAL. Random selects similar samples to SQBC, but also uniformly samples from outliers from both classes, extending the decision boundary of the model. We argue this is especially effective for larger synthetic dataset sizes where the synthetic data smoothens the decision boundary and thus mitigates the high variance introduced by the most informative samples. Thus, the model
7

---

Published as conference paper at ICLR 2025
Fine-tuniig with most informative samples and synthetic data
N-200
X-500
N-1000
0.78
0.76
0.
0.70
10
0.66
0.66
0.64
ACt.
Act.
wamp lea
8ynth
Nansta Bynth
syath
Nase Line
Masoing
Syeth
7000 Cabelo
True Labela-dynth
Figure 4: (Q3) Fine-tuning with synthetic data improves stance detection, while combining most informative samples and synthetic data surpasses the baseline model fine-tuned with all true labels (mm above dashed line -- using less manually labelled data: The reason for the performance increase can be attributed two phenomena: (i) the synthetic data smoothens the decision boundary of the model making it more robust to outliers. (ii) The most informative samples improve the model where the synthetic data distribution is not expressive enough.
remain robust while extending the decision boundary. However, as we show in Section 5.2 the real data is quite homogenous. Therefore, with severe outliers present, Random could select these and worsen performance. This would not happen with SQBC due to its k-nearest neighbour objective.
ABLATIONS
We investigate different aspects using synthetic data for online political discussions. First, we study how well LLMS perform classifying stance the X-Stance dataset directly. Our assumption behind this is that while LLMs are strong at generating open ended text, they seem to have more diffieulties when conditioned on a specific task combined with a narrow dataset. We also study the properties of the synthetic data by calculating the per comment entropy distribution toghether with the comment length. We then compare to the real data. Furthermore, to determine whether fine-tuning on per topic basis is sensible, we study if the observed performance improvement is related to the content or the structure the synthetic dataset.
5.1 USING LLMS DIRECTLY FOR X-STANCE
Cruickshank & Xian Ng 2023 and Gül et al. 2024) have shown promising results using zero-shot stance detection and fine-tuning various L.MSs common stance detection datasets such SemEval- 2016 (Mohammad et al. 2017) and P-Stance (Li et al., 2021). However, both these datasets have relatively little variety in topics, with the topics addressing more popular discussion topics such as general US-Politica For this reason, we evaluate Mistral-7B on Dmnin of X-Stance, which has a larger number of niche topics. We adopt the prompt and fine-tuning scenario (fine-tuning over epochs with LoRA (Hu et al. (02T)) in Gül et al. (2024) and use our Mistral-7B model for both zero-shot stance detection and fine-tuned stance detection.
Table 2 shows that zero-shot stance detection barely reaches the performance of the pre-trained BERT baseline. Suprisingly, fine-tuning the Mistral-7B model with D worsened performance even further. We tried various hyperparameter settings and fine-tuned for up to 10 epochs, more than the 4 used Gül al. 2024). We believe the poor performance can attributed to a few reasons: (i) The topics in the X-Stance dataset are likely not present in the training sets of the Mistral 7-B model compared to SemEval and P-Stance which contain social media comments. (ii) Due to the smaller parameter count, the model may struggle capture both the topic and the given comment for stance prediction. This often showed with the Mistral-7B model not giving consistent classification outputs or it would often refuse to predict stance. (iii) Furthermore, the model is not tailored to give
8

---

Published as a conference paper at ICLR 2025
single line response. In fact, the responses contained the words "favor" or "against". Better prompting strategies could improve performance, however with our findings we believe that of now using LLMs for open- text generation is more effective than conditioning them to give a specific output for stance detection. We also tried other similar sized open source L.MSs (Llama, Openassistant, Falcon) and found that they struggled similarly in producing consistent zero-shot classification.
often verbose so also accepted answers that
F1 Score (avg. 10 Questions)
F1 Score
(avg. 10 Questions)
Fine-tuned LLM
0.182
200 500
1000
Zero-shot LLM
0.419
Baseline
0.693
Baseline
0.693
093
0.693
Baseline+yynth (M-1000)
0. 723
Baseline+Syuth
0.711 0.717 0.723
SQBC+Synth (M=1000)
0.754
Basel Aaseline+Syntth
(Misaligned) 0.699
704 0.694
Table LLM-based stance detection vs BERT-based stance detection: We compare the Mistral-7B performance the our BERT stance detection models. We see that zero- shot stance detection barely reaches the pre- trained baselines' performance. Fine-tuning the LLM also proved difficult where the per- formance of the fine-tuned model worsened. Our findings for X-Stance suggest that LLMs good at producing open-ended text, while struggling when being prompted to give spe- cific stance.
Table 3: Topic alignment is crucial for the synthetic data to be effective: To determine whether improvement with synthetic data is due to the structure content of the synthetic data, we augment the stance detection model with misaligned synthetic data. That is, the synthetic data does not align with the question given to the stance detection model. We ob- serve the model only performs meaningfully better, when the synthetic data aligns with the posed question.
5.2 STUDYING THE SYNTHETIC DATA
Properties of the synthetic data. To determine the quality and diversity of the synthetic data, we calculate the entropy of each comment (entropy over words) for both the synthetic and real data and compare the interquartile ranges of the corresponding entropy distributions in Figure [] We also determine the average comment length for the interquartile ranges. The entropy reveals information about the content of both datasets, while the length acts surrogate for structure. We observe, that the entropies between the real and synthetic data are similar, while the average comment length of the synthetic data is longer than the real data's. Looking the samples in Appendix C.I observe that the real comments have a concise (even emotional) writing style, which common in online political discussions. The synthetic data comments are verbose and more reserved. Thus the difference in entropy could be attributed to the synthetic data containing less "emotional" comments compared to the real data. Nonetheless, we feel the synthetic samples manage capture the content of the real data. Interestingly, the difference in length does not seem to affect the synthetic data alignment as can be seen from Figure 3 It clear that the synthetic data is well aligned with the real data, even though the comment length between both sets of data differs. That is, the BERT model is largely invariant to the comment length and is more sensitive to the content of the comments rather than their structure.
Content vs structure: Is per topic fine-tuning necessary? To further validate our approach of fine-tuning the model per topic, we determine whether the improved performance comes from the discussion oriented structure of the data or from the content of the data. We fine-tune the model with misaligned questions and synthetic datasets, since we believe data across different topics does not necessarily share the same representation, justifying the need for per topic fine-tuning, Table 3 shows that fine-tuning with the synthetic data is only effective when the synthetic dataset is aligned with the posed question as can be seen by Baseline+Synth. Fine-tuning with synthetic data from different topic Syytt (Misaligned) provides little improvement, delivering performance close to the Baseline. This validates the usefulness of the generated synthetic data for the stance detection model.

---

Published as a conference paper at ICLR 2025
Ent ropy of the Datasats
Statiatical Surnary Synthetic
Real
Jumber of outliers
Total samplea
281
10000
(2% 3%)
Entropy of Synthotic vo Real Data
Distr. Ranga
Arg.
Entropy
Avg.
Length
l3gnchati:
2xa1)
Syathetic
Ninimum
2.25
2.39
3
25%
2.96
2.78
101
090
00 00000
25% 50%
3.04
2.91
124
50% 75%
3.06
2.96
134
70% 100%
3.11
3.07
142
35
Maxtarum
3.27
3.37
282
2.2
2.4
2.6
2.8 Entcopy
3.0
3.2
3.4
Figure 5: Synthetic data properties: (Left)) The per comment entropy of the synthetic data is similar to the entropy of the real data, where the synthetic data has higher mean entropy than the real data, while the real data has higher variance. (Right) The generated synthetic data is of high quality, with only few outliers. Interestingly, synthetic comments are longer than real ones, making real comments denser and synthetic ones more verbose. We argue that since the projected embeddings in Figure show alignment between both datasets, BERT appears to be invariant to comment length.
DISCUSSION
Potential impact of this work. An apparent advantage of our approach is the possiblity of divding the synthetic data generation and the stance model fine-tuning. The former can be outsourced to dedicated infrastructure, while with the latter fine-tuning and inference accessible even for smaller organisations with fewer resources. Considering the large amount of topics in participation processes, the ability to generate synthetic data or to reduce labelling effort with SQBC further increases the benefits for smaller organisations that can't can't afford large scale data collection or labelling efforts.
Limitations. One limitation of our approach that we fine-tune a separate model for each question. While this leads to good results, common approach is to fine-tune single (and thus more general) model for several questions (like pre-training Baseline). However, visualising the synthetic data in Figure 3 and Appendix A we observe that the underlying data distribution differs (sometimes greatly) for each question, which strongly suggests that each question benefits from fine-tuning a different model. This also aligns well with the per topic setting of online (political) discussions, considering that lightweight stance detection models can be fine-tuned in less than a minute even with synthetic dataset size of M 1000 on reasonable GPU (NVIDIA A A100). Another concern are biases that could be potentially introduced through the synthetic data. We addressed this in Section by comparing the distributions of the synthetic data and real world data. We argue that analysing potential biases that could be introduced to the stance detection model through the synthetic data easier single-question setting. In a multiple-question setting data from other topics could introduce biases into the model that are harder to detect. While effective for online discussions, due to the above observations, synthetic data quality should be assessed separately for different use cases.
Future work Despite this work focusing smaller interpretable models, we believe future work should investigate why the Mistral-7B model performs poorly on the X-Stance dataset. We shared thoughts around this in Section [.2] but this requires more extensive study. Another avenue for future work could be about leveraging the synthetic data distribution to generate more novel synthetic data. For instance, with synthetic data reference we could learn generative model which enforces a certain (distributional) distance to the synthetic reference distribution.
In this work, presented how improve stance detection models for online political discussions utilizing LLM-generated synthetic data: (i) we showed that fine-tuning with synthetic data related to the question improves the performance of the stance detection model. (ii) We attribute this to the LLM --geeeerrreedd synthetic data aligning well with the real data for the given question, while showing that the BERT model requires data that is more content aligned than structured. (iii) Fine- tuning with synthetic data can be further enhanced by adding the most informative samples which are identified by using the synthetic data as reference. This proves more effective than fine-tuning on all true labels, while using considerably less manually labelled samples, thus reducing labelling effort.
Conclusion.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Abeer ALDayel and Walid Magdy. Stance detection on social media: State of the art and trends. Information Processing & Management, 58(4)) 2021. ISSN 0306-4573. doi: htps://doi.org/ 10. 10.10166/.ipm.20011102599 URL httts:://wwwtt httpp::/www..ciiecccceeeeettttte http://www..cnnnndieereretee com http///ttt.ecceeeeeeee//rrrrilca
p11/S030645792000990
Maike Behrendt, Stefan Sylvius Wagner, and Stefan Harmeling. Supporting online discussions: Integrating ai into the adhocracy+ participation platform to enhance deliberation, 2024. URL
https://arxiv.. www..orgg/baa........ wwtpabb....//////
Michael Burnham. Stance Detection: Practical Guide to Classifying Political Beliefs in Text. arXiv e-prints, art. arXiv:2305.01723, May 2023. doi: 00:48550WarXiv2233000722.
Alan Darmasaputra Chowanda, Albert Richard Sanyoto, Derwin Suhartono, and Criscentia Jessica Setiadi. Automatic debate text summarization online debate forum. Procedia Computer Science, 116:11-19, 2017. ISSN 1877-0509. doi: htt.a///aarr.taa///////////01110
201..10.000....... URL URL Discovery and innovation of computer science technology in artificial intelligence era: The 2nd International Conference on Computer Science and Computational Intelligence (ICCSCI 2017).
www..scienceirrec..
lain J. Cruickshank and Lynnette Hui Xian Ng. Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification. arXiv e-prints, arXiv:2309.13734, September 2023. doi: 10.4
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume (Long and Short Papers). pp. 4171-4186, Minncapolis, Minnesota, June 2019. Association for Computational Linguisties. doi: 10. 18653/v1/N19- 1423. htt.ttttss//aclaatttti
Kai Greshake, Sahar Abdelnabi. Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you' ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, AISec '23, pp. 79-90, New York, NY. USA. 2023. Association for Computing Machinery. ISBN 9798400702600. doi: 10.1145/3605774433229999 URL [Rttsss::://do http:////ooooooo....o..a... 3623985
ilker Gül, Rémi Lebret, and Karl Aberer. Stance Detection Social Media with Tuned Large Language Models. arXiv e-prints, art. arXiv:2404. 12171, April 2024. doi: (0.485500arxivv244M 12171.
Momehil Hardalov, Arnav Arora, Preslav Nakov, and Isabelle Augenstein. A survey on stance detection for mis- and disinformation identification. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Findings of the Association for Compu- tational Linguistics: NAACL 2022, pp. 1259-1277, Seattle, United States, July 2022. As- sociation for Computational Linguistics. doi: 18.18653/v htttps://aclantholl/
URL
2022. findings-nann naacl.9.
Alex Holub, Pietro Perona, and Michael C Burl. Entropy-based active learning for object recogni- tion. In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1-8. IEFE, 2008
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. LoRA: Low-Rank Adaptation Large Language Models. arXiv e-prints, art. arXiv:2106.09685, June 2021. doi: 1..48500/rXiv.21066.00
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William El Sayed. Mistral 7B. arXiv e-prints, art October 2023. doi: 23,
11

---

Published as a conference paper at ICLR 2025
Dilek Küçük and Fazli Can. Stance detection: A survey, ACM Comput. Suns. 53(1), feb 2020. ISSN 0360-0300. doi: 10.1145/336902.. URL https://doii org/10. 1145 3369026
Kostiantyn Kucher, Carita Paradis, Magnus Sahlgren, and Andreas Kerren. Active learning and visual analytics for stance classification with alva. ACM Trans. Interact. Intell. Syst. 7(3), oct 2017. ISSN 2160-6455. doi: 10.1145/3132169 URL httpps://ol...ooooo 1111//331.115//331116
Chang Li and Dan Goldwasser. Encoding social information with graph convolutional networks forPolitical perspective detection in news media. In Anna Korhonen, David Traum, and Lluís Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2604,,60,, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19- 12447 https::/: /ac/aaclnnttt hwtphooooyaan hwwp:/hhoooooooooo--11-- wtg/P19-1244
Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Jayaraman Nair, Diana Inkpen, and Comelia Caragea. P-stance: A large dataset for stance detection in political domain. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Association for Com- putational Linguistics: ACI-IICNLLP 2021, pp. 2355-2365, Online, August 2021, Associa- tion for Computational Linguistics, doi: 10.11///11//021111 http:///.122...inn.ee.e findinns-aal..208. URL https
//aclanthology. org/ 2021. findings--cl..... 208
Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. Synthetic data generation with large language models for text classification: Potential and limitations. In The 2023 Conference on Empirical
Methods in Natural Language Processing, 2023. URL http:::/pgeerrrie...tttftttttttt
MmB KMHIND
Yujian Liu, Xinliang Frederick Zhang. David Wegsman, Nick Beauchamp, and Lu Wang. POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection. arXiv e-prints, art. arXiv:2205.0m 100699 May 2022. doi: 00.48500axXX..2205.0003
Ghazalch Mahmoudi. Babak Behkamkia, and Saulch Ectemadi. Zero-shot stance detection using con- textual data generation with LLMs. In AAAI-2024 Workshop on Public Sector LLMs: Algorithumic and Sociotechnical Design 2024.
Katerina Margatina, Giorgos Vernikos, Loic Barrault, and Nikolaos Aletras. Active Learning by Acquiring Contrastive Examples. arXiv e-prints, art. arXiv:2109.03764, September 2021. doi: 10.48550krxxx:21199.3777.
Houman Mehrafarin, Sara Rajace, and Mohammad Taher Pilehvar. On the importance of data size in probing fine-tuned models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics: ACL 2022, pp. 228-238 Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. (0.18653/v1/0022.0 ftt......inings-clc URL https: http:::://aaaanthololo
org/ 2022 fti.:02...ininnn-cc.c... 200
Blake Miller, Fridolin Linder, and Walter R. Mcbane. Active learning approaches for labeling text: Review and assessment of the performance of active learning approaches. Political Analysis, 28 (4):532-551, 2020. doi: 10.10177pan.
Saif M. Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. Stance and sentiment in tweets. Special Section of the ACM Transactions on Internet Technology on Argumentation in Social Media, 17(3), 2017.
Anders Giovanni Møller, Jacob Aarup Dalsgaard, Arianna Pera, and Luca Maria Aiello. Is prompt and a few samples all you need? Using GPT-4 for data augmentation in low- low-resource classification tasks. arXiv e-prints, art. arXiv:2304. 13861, April 2023. doi: 10.44550/a 13861.
NLLB Team, Marta R. Costa-jussă, James Cross, Onur Çelebi. Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. No Language Left Behind: Scaling Human- Human-Centered Machine Translation. arXiv e-prints, art. arXiv:2207.044 04672, July 2022. doi: 00..
12

---

Published as a conference paper at ICLR 2025
Julia Romberg and Tobias Escher. Automated topic categorisation of citizens' contributions: Reducing manual labelling efforts through active learning. In Electronic Government: 21st IFIP WG 8.5 International Conference, EGOV 2022, Linköping. Sweden, September 6-8, 2022, Proceedings, pp. 369-385, Berlin, Heidelberg, 2022. Springer-Verlay Verlag. ISBN 978-3--331115085-2 doi: 10. 1007/ 10--3-031-15086-- 24. URL http.//t//////...rrgg/ htt.://ooorrr....00000000----11-
Julia Romberg and Tobias Escher. Making sense of citizens' input through artificial intelligence: A review of methods for computational text analysis to support the evaluation of contributions in public participation. Digit. Gov.: Res. Pract., jun 2023, doi: 10. 145/3603254 URL https Just Accepted.
//doi.org////// 11411160313322
Klaus Schmidt, Andreas Niekler, Cathleen Kantner, and Manuel Burghardt. Classifying speech acts in political communication: A cansformer-basee approach with weak supervision and active learning. In 2023 18th Conference Computer Science and Intelligence Systems (FedCSIS), pp. 739-748, 2023.
doi: 10. 10.15439/2023F3485..
H. S. Seung, M. Opper, and H. Sompolinsky. Query by committee. In Proceedings of the Fifth Annual Workshop on Computationnal Learning Theory, COLT '92. pp. 287-294, New York. NY, USA. 1992. Association for Computing Machinery. ISBN 089791497X. doi: 10.1 141144/5/38..1301/
URL httph:tt////t //doi. wwwooo....1......oo 11.11...111.// 1145/130386 130417
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 139711 2023. doi: https: //doi.org//00.4000 005.444022.....332.....
Jannis Vamvas and Rico Sennrich. X-Stance: multilingual multi-target dataset for stance de- tection. In Proceedings of the 5th Swiss Text Analytics Conference (SwissText) & 16th Con- ference on Natural Language Processing (KONVENS), Zurich, Switzerland, jun 2020. URL http://ceur-wtt http::/cceurvs..oo/oo--------2 www.oooo//oooe///.o1.eoo papp://aapr.... pdf
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Ma- chine Learning Research, 2(86) 2008. URL http:://mml org/ htggpapprs/vvl /
wandermaatenO8a.Jr html html
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: htttps:/pproceeddnggg neurips cc/paper /tllss/paperr201//E1ee/ #ffeer4844//cs111bb0cccc....... Paper pdf
Veniamin Veselovsky, Manoel Horta Ribeiro, Akhil Arora, Martin Josifoski, Ashton Anderson, and Robert West. Generating faithful synthetic data with large language models: A case study in com- putational social science. ArXiv, 1l4/2305.1504,, 2023. uttphtts::/aap www.seiismamolll.a....
Wenqian Zhang, Shangbin Feng, Zilong Chen, Zhenyu Lei, Jundong Li, and Minnan Luo. KCD: Knowledge Walks and Textual Cues Enhanced Political Perspective Detection in News Media. arXiv e-prints, art. MxXiv:2204.04046 April 2022. doi: 10448550/rxiv.2224444000
Marc Ziegele, Timo Breiner, and Oliver Quiring. What Creates Interactivity in Online News Discussions? An Exploratory Analysis of Discussion Factors in User Comments on News Items. Journal of Communication, 64(6):1111-1133 10 2014. ISSN 0021-9916. doi: 10. 12123
URL httppp/////ab..oro..111/////////
Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can Large Language Models Transform Computational Social Science? arXiv e-prints, art. arXiv:2305.03514, April 2023. doi: 10.4850UarXxx.220000354..
13

---

Published as a conference paper at ICLR 2025
A VISUALIZATIONS
A.1 VISUALIZING THE SYNTHETIC DATA
We visualize the synthetic data together with the real world data for M 1000 and M 200 in Figures and 7 We plot the data points of the synthetic data in blue and red for the positive and negative samples, respectively. The means of the real world data are plotted as regular polygon with 8 sides. We observe that the synthetic data extends the real world data, which we consider a factor as to why fine-tuning with synthetic data is effective online political discussions. Also, the larger the synthetic dataset size, the more the synthetic data matches the distribution of the real world data since for M = 200 (see Figure 7 the mean are not as well aligned with the synthetic data. Furthermore, the positive and negative samples are well separated, which attribute to having pre-trained the BERT-model on Dunin of the X-Stance dataset, giving the prior knowledge about the stance detection task.
A.2 VISUALIZING THE QUERY STRATEGIES OF THE ACTIVE LEARNING METHODS
We visualize the selected samples of SQBC, CAL and Random query strategies for M 1000 and M 200 in Figures X and 9 We plot the selected samples of the unlabelled data green. The positive and negative synthetic data samples are plotted in blue and red, respectively. The selected samples are highlighted in orange. We observe that SQBC selects the unlabelled samples that are mostly in between the two classes of the synthetic data. This is the expected behaviour since select the samples where the classification score is ambiguous. For Random, the range of selected samples broad: some similar samples between the two classes like SQBC are selected, but also within class samples that are not covered by the synthetic data set. This explains why random selection works well with a large synthetic dataset, since it further extends the decision boundary of the model. For the smaller synthetic dataset M = 200, the random selection is not as effective, since the selected samples are spread out over the whole data space and not necessarily in between the classes as with the larger synthetic dataset. Finally, CAL selects samples similar to SQBC, but mostly tends to select samples from only one class.
14

---

Negative Synthetic Published as a conference paper at ICLR 2025
Question
Question
Question
Question
Question
Question
Question
Question
Question
Positive Synthetic Positive Train (Mean)
Negative Train (Mean)
Figure 6: Visualization of synthetic data with train data means for M 1000 synthetic data. For a larger synthetic dataset size, the means of the synthetic data are well aligned with the real world data and the positive and negative samples are well separated. The synthetic data thus extends the real world data, which we consider a factor as to why fine-tuning with synthetic data is effective in online political discussions.
15

---

Negative Synthetic Published as a conference paper at ICLR 2025
Question
Question 2
Question 3
Question
Question 5
Question 6
Question 7
Question 8
Question 9
Positive Synthetic
Negative Train (Mean)
Positive Train (Mean)
Figure 7: Visualization of synthetic data with train data means for 200 synthetic data: For smaller synthetic dataset size, the means of the synthetic data not well aligned with the real world data as for 1000. However, the positive and negative samples are still well separated.
16

---

Published as a conference paper at ICLR 2025
SQBC
Random Sampling
CAL
Positive Synthetic
Negative Synthetic
Train data
Selected
Figure 8: Visualization of SQBC, Random and CAL query strategies for M 1000 synthetic data: SQBC selects the unlabelled samples that are mostly in between the two classes of the synthetic data. This is the expected behaviour since we select the samples where the classification score is ambiguous. For random selection, the range of selected samples is broad: some similar samples between the two classes like SQBC are selected, but also within class samples that are not covered by the synthetic data set. This explains why random selection works well with a large synthetic dataset, since further extends the decision boundary of the model. Finally, CAL selects samples similar to SQBC, but mostly tends to select samples from only one class, resulting in worse performance.
17

---

Published as a conference paper at ICLR 2025
SQBC
Random Sampling
CAL
Positive Synthetic
Negative Synthetic
Train data
Figure 9: Visualization of SQBC, Random and CAL query strategies for M = 200 synthetic
data: For smaller synthetic dataset size, SQBC is still able to select the unlabelled samples that are mostly in between the two classes of the synthetic data. For Random we see that the selected samples are bit further away from the synthetic data distribution, which is why argue it does perform as well as with the larger synthetic dataset.
18

---

Published as a conference paper at ICLR 2025
B DETAILED RESULTS
Fine-tuning with synthetic data
M=0
M=200
M=500
M=1000
Baseline Synth 0.693
0. 0.712
0.718
0.723
True Labels + Synth 0.727
0. 745
746
0.770
Table 4: Tabular version of Figure
Fine-tuning with most informative samples selected with synthetic data
N-200
Y-1000
10%
25%
50%
75%
10%
25%
50%
75%
10%
25%
50%
75%
CAL
0.693
0.697
0.705
0.714
0.692
0.694
0.707
0.718
0.692
0.696
0.708
0.720
Randon
0.693
0.696
0.705
0.719
0.683
0.695
0.706
0.715
0.692
0.695
0.706
0.715
SQBC
0.697
0.709
0.712
0.721
Table 5: Results of only training with most informative samples.
Pine-tuning
informative samples synthetic
M-200
M=500
K-1000
10%
25%
50%
75%
10%
25%
50%
75%
10%
25%
50%
75%
CAL+8ynth
0.713
0.715
0.727
0.732
0.711
0.721
0.732
0.748
0.695
0.747
0.749
RandontSyuth
0.716
0.720
0.724
0.734
0.723
0.735
0.730
0.748
0.724
0.746
0.754
0.7
sqaC+Syath
0.715
0.723
0.731
0.735
0.714
0.728
0.744
0.750
0.721
0.737
0.753
0.747
Table 6: Tabular version of Figure
19

---

Published as a conference paper at ICLR 2025
C SYNTHETIC DATA
C.I ANALYSING THE SYNTHETIC DATA DISTRIBUTION
As described in Section 5.2 we analyse the distribution of the synthetic data in comparison the real world data. In the following tables we provide samples of the interquartile ranges of the entropy distribution. Tables C.1 and C.1 contain the same samples in English and German respectively. Table C.I contain comments from the real data distribution.
We see that that the synthetic data samples are longer than the real data samples and contain more reserved language and practically no emotional language. In comparison, the real data samples are shorter and concise with some samples in the lower entropy range containing more emotional language or little information. This however is common for online political discussions. In any we see that the underlying notions behind the comments are similar both the synthetic and real data samples.
20

---

Published as a conference paper at ICLR 2025
Distribution Range
Comment
Minimum
Sure, can help you with that' Please write your message and will assist you with strongly disagree with the idea that the residence permit for migranis from non- non- EU/EFTA countries should be linked the fulfillment of binding integration agreements throughout Switzerland. While it's important that migrants integrate into Swiss society and contribute to the country. helieve that it unfair to tie their ahility to live and work in Switzerland to their compliance with certain conditions. It could lead to discrimination against some migrants who may struggle to meet these requirements or face cultural barriers. Instead, I propose more nuanced approach that takes into account the unique circumstanccs and needs of each individual migrant. while still promoting integration and equality for all. As someone who strongly believes in the importance of immigration policies that promote successful integration and contribution to society. I fully support the idea of linking residence permits for migrants from non-EU/EFTA countries to the fulfillment of hinding integration agreements throughout Switzerland. This approach ensures that migrants committed to making Switzerland their home and contributing positively to the country's s well-being can enjoy full legal recognition and protection while being held accountable for fulfilling their responsibilities. By creating an incentive system where residency and benefits are directly tied to compliance with integration agreements, we can foster culture of shared responsibility and mutual henefit between incoming migrants and Swiss socicty as a whole.
0% 25%
25% 50%
30%- 75%
Sure, can help you generate a response for this scenario. From your perspective, you might write something like: strongly believe that the residence permit for migrants from Ron-EU/EPTA countries should be linked to the fultillment of binding integration agreements throughout Switzerland. As someone who values the diversity and richness that migration brings to our country, I think it's important that we have clear expectations and requirements for those coming here to live and work. Integration agreements should include things like langnage proficiency, cultural awareness, and contributions to community life. By linking these agreements the residency permit, we can ensure that migrants are fully committed integrating into Swiss society and contributing to its well-being. Of course. there may be some concerns or challenges with this apprach, hut I helieve that the henefits far outweigh any potential drawbucks. A strong and diverse immigration policy is crucial for the future of Switzerland, and by setting high standards for integration, we can attract the best and brightest from around the world.
75% 100%
believe that requiring residence permits for migrants from nOn-EU/EFTA countrics to be linked to the fulfillment of binding integration agreements throughout Switzerland fair and necessary policy. By doing so, can ensure that migrants who come Switzerland have clear understanding of their responsibilities and the requirements they need to meet in order to live and work here. This also helps to promote a sense of community and integration among all residents of Switzerland, regardless of their place of birth or nationality. Additionally, allows us to carefully screen and select immigrants who are likely to be successful and positive contributing members of our society. Overall, believe that this policy will benefit both migrants and Swiss citizens alike by promoting mutual respect and understanding.
Maximum
would argue that linking the residence permit for migrants from non- EU/EFTA countries to the fulfillment of mandatory integration agrecments across Switzerland is too burden- some and unreasonable. This creates an unfair situation in which people who come to Switzerland work or study are essentially forced to make long-term commitment before they even have access to the legal rights they need to live and work here comfortably. The Commission has asked the Commission to review the conditions set out in Regulation (EC) No. 1225/2009 for the upplication of Regulation (EC') No. 1225/2009 of the Council concerning the application of Regulation (EC) No. 1225/2009 of the Council conceming the application of Regulation (EC) No. 1225/2009 of the Council concerning the appli- cation of Regulation (EC) No. 1225/2009 of the Council concerning the application Regulation (EC) No. 1225/2009 of the Council conceriing the application of Regulation (EC) No. 1225/2009 of the Council conceming the application of Regulation (EC) No. 1225/2009 of the Council concerning the application of Regulation (EC) No. 1225/2009 of the Council concerning the application of Regulation (EC') No. 1225/2009 of the Council concerning the application of Regulation (EC) No. 1225/2009 of the Council conceming the application of Regulation (EC) No. 1225/2009 of the Council concerning the appli- cation of Regulation (EC) No. 1225/2009 of the Council concerning the application of Regulation (EC) No. 1225/2009 of the Council concerning the application of Regulation (BC) No.1 believe that the government should instead focus on creating more flexible and streamlined processes to facilitute the integration of migrants into Swiss society without imposing unnecessary restrictions or requirements.
Table 7: English comments sampled from the interquartire ranges of the entropy distribution of the synthetic data.
21

---

Published as a conference paper at ICLR 2025
Distribution Range
Comment
Minimum
Schreihen Sie Thre Nachricht, ich helfe Thnen.
0% 25%
Ich bin mit der Vorstellung, dass die Aufenthaliserlaubnis für Migranten aus Nicht-EU/EFTI Ländern an die Erfüllung verbindlicher gebunden werden sollte, voll und ganz nicht cinverstunden. Es ist zwar wichtig, dass Migranten in die Schweizer Gesellschaft intogrient werden und einen Beitrag für das Land leisten, aber ich glaube, es ist unfair, ihre Fähigkeit, in der Schweiz zu leben und zu arbeiten, an die Einhaltung bestimmter Bedingungen knüpfen. Es könnte zu Diskriminierung einiger Migranten führen. die möglicherweise Schwierigkeiten haben, diese Anforderungen Hindernissen ausgeselit sind. Ich schlage stattdessen einen differenzierteren Ansalz vor, der die cinzigartigen Umstände und Bedürfnisse jedes einzelnen Migranten berücckichtigt und gleichzeittig Integration und Gleichherechtigum whigggngggggggo für alle Migranten fürdert.
ineerationsvereinbauunnen in der gesamlen Schweiz
erfiillnn oder kuiturellen
25% 50%
Als jemand, der fest an die Bedcutung von Finwandcuuggspoltiicee sopoiiiin glaubt, die eine erfolgreiche Integration und einen Beitrag zur Gesellschaft fördern, unterstültze ich die Idec, Aufenthalts- genehmigung gungeen für Migranten aus Nicht-EU/FFFI EU/EFTA- änndrnn mit der Frfullung verbindlicher in der gesamten Schweiz zu verknüipfen. Dieser Ansatz gewährleistet, dass Migranten, die sich verpflichtet haben, die Schweiz zu ihrer Heimat zu machen und cinen positiven Beitrag zum Wohlergehen des Landes leisten, vollständige rechtliche Anerkennung und Schutz geniellen können, wähhrend sie gleichzeitig für die Erfüllung ihrer Pitichten zur Rechen- schaft gezogen werden. Durch die Schaffung cines Anreizsystems, bei dem Aufenthalt und Leistungen unmittelbar an die Einhaltung der Integrations pereratioonserrinbuunngge gebunden sind, können wir eine Kultur der gemeinsamen Verantwortung und des gegenscitigen Nutzens zwischen den cinwandemden Migranten und der Schweizer Gesellschaft als Ganzes fördem.
iicgrratiosverriinaunngg
50%- - 75%
Sicher, ich kann Ihnen helfen, cine Antwort für dieses Szenario zu genericren Aus Ihrer Sicht könnten Sie etwa schreiben: Ich bin der festen Üherzeugung, dass die Aufenthaltserlabniis für Migranten aus Nicht Bich--EUEEFTA-.. Ländern barungen in der gesamten Schweiz verbunden sein solite Als jemand, der die Viclfalt und den Reichtum schätzt, die Migration unserem I Land bringt, denke ich, dass wichtig ist, dass wir klare Erwartungen und Anforderungen an diejenigen haben, die hierher kommen, um zu lehen und zu arbeiten. Dic Integratiosvvcciinarnnge sollten Aspekte wie Sprachkenntmisse, kulturelles Bewusstsein und Beiträge zum Gemcinschaftsleher umfassen Durch die Verknüpfung dieser Vereinharungen mit der Aufenthallserlaunnis künnen wir sicherstellen, dass Migranten sich voll und ganz der Integration in die Schweizer Gesellschaft und dem Wohlergehen der Schweizer Gesellschaft verschricben haben. Natürlich kann dieser Ansatz cinige Bedenken oder Hcraus- forderungen mit sich hringen, aber ich glaube, dass die Vorteile weit üihewwiegen alle möglichen Nachteile Eine starke und vielfältige Einwandcrungspolitt ist fürr die Zukunft der Schweiz von entscheidender Bedeutung. Durch dic Festlegung hoher imegrationsstandaan können wir die Besten und Klügsten aus der ganzen Welt anziehen.
Erfüllung verbindlicher Integrations verrei-
75% 100%
Ich glauhe, dass die Anerkennung der nufeebballggenemmiunng für Migranten aus Nicht- EU/EFTA- Ländern Schweiz geknüpft ist, eine faire und notwendige Politik Auf diese Weise können wir sicherstellen, dass Migranten, die in die Schweiz kommen, ein klares Verständnis ihrer Verantworlichkeeten und der Anfonderungen haben, die sie erfüllen müssen, um hier zu leben und zu arbeiten. Dies trägt auch dazu bei, ein Gefühl der Gemeinschaft und Integration unter allen Einwohmern der Schweiz fiireem, unabhängig von ihrem Gehurtsort oder ihrer Staatsangehörrigkeit. Darüher hinaus können wir Migranten sorgfältig auswählen, die wahrscheinlich erfolgreiche und positive Mitglieder unserer Gesellschaft sind. Ich glaube, dass diese Politik sowohl den Migranten als auch den Schweizer Büigeem zugute kommen wird, indem sie gegenseitigen Respekt und Verständnis fürdert
die Erfüllung verbindlicher Integrationsveriibbauug
in der gesamten
Maximum
Ich würde argumenticren, dass die Verknüipfung der Aufenthallseriaubbni für Migranten aus Nicht- EU/EFAA-
der Erfüllung verbindlicher integrationsverriibbuuuggg in der gesamten Schweiz belastend und unverüinftig ist Es entsteht eine ungerechte Situation, in der Menschen. die zur Arbeit oder zum Studium in die Schweiz kommen, im Wesentlichen gezwungen sind, eine langfristige Verpflichtung einzugehen, bevor sie überhaupt Zugang zu den gesetzlichen Rechten haben, die sie benötigen, um hier bequem zu leben und zu arbeiten. Die Kommission hat die Kommission auf aufgefordert, die in der Verondnung (EG) Nr. 1225/2009 festgelegten Bedingungen für die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates üher die Anwendung der Verordnung (FG) Nr. 1 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates üher die Anwendung der Verrrduung (FG) Nr. 1225/2009 des Rates tiher die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (FG) Nr. 1225/2009 des Rales üiher die Anwendung der Verordnung (BG) Nr. 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. 1 1225/2009 des Rates über die Anwendung der Verordnung (EG) Nr. Ich glaube, dass die Regierung sich stattdessen auf die Schaffung flexiblerer und schlankerer Prozesse konzentrieren sollte, um die Integration von Migranten die Schweizer Gesellschafi zu erleichtem, ohne unnätige Finschränkungen oder Anfordcrungen aufzuerlegen.
Table 8: Corresponding German comments sampled from the interquartire ranges of the entropy distribution of the synthetic data.
22

---

Published as a conference paper at ICLR 2025
Distribution Range
Commentt
Minimum
die picggrationvveriinhaungeg noch studieren
0% 25%
Wer sich anpasst und korrekt verhaltet darf auch hier bleiben.
25% 50%
Imterationsvercinharungo nicht generell verhängee, sondem individuell anordnen, wenn die Person auch nach längerer Anwesenheit in der Schweiz Integrationsseffzite hat oder hei neu zugerngenen Personen, wenn mit Ampassunsschwiengkeiing Standardformulare. keine Automatismen.
zu rechnen ist. Keine
zweille der Umsetzbarkeiit solcher Methoden, eines "Vertrags" ist anstrebenswert. dass ein guter Integrationsabluuf Gesellschaft notwendig, die Personen aufnehmen und integrieren will, wie auch Per- sonen die diese Möglichkeit wahrnehmen können und wollen.
finden kunn. Allerdings ist dafür immer eine
73% 100%
Ich rinde die Idee Integrationsma snnahmen verbindlich zu machen sinnvoll. Sprachkurse sind cin idcales Instrument daftir. Es durf aber nicht scin. dass gewisse Forderungen zur Voraussetzung werden die Aufenthaltscrlabnnis Nicht joder Mensch lerat gleich schnell eine Sprache oder kann sich gleich schnell integrieren. Misshräuche und Diskriminierungen können auftreten. Dies muss verhindert werden. Ich wäire cher für einen obligatorischen Kostenlosen-Sprat Sprachkurs.
Maximum
leeiklrr Punkt, im Prinzip JA für Alle, nur auch gleiche Rechte/Pilichten für alle warum sollen die RU-EFTA-Boin Bürger keine solchen Integaaiionverennbatnnnnn ahschliessen. Der bildungsferme Rumäne fährt besser als der hochausgebildete Chinese/US -Sa-UUSiimaiaieerTTT Spezialist aus Indien. Die ganze Einwanderung muss im Sinne der MEI umgeseizi werden.
Table 9: Corresponding German comments sampled from the interquartire ranges of the entropy distribution of the real data.
23

---

Published as a conference paper at ICLR 2025
C.2 TRANSLATED DATA SAMPLES
We show the translated questions used for synthetic data generation in Table [O and some samples of generated comments in 12 We that the questions translated correctly and synthetic data can be generated for both favor and against stances.
Question in German
Question in English
Sollen sich die Versicherten stärker an den Gesundheitskosten beteiligen (z.B. Erhöhung der Mindestfranchise)
Should insured persons contribute more to health costs (e.g. increase in the minimum deductible)?
Befürworten Sie ein generelles Werbeverbot für Alkohol und Tabak?
Do you support general ban on advertising alcohol and tobacco?
Soll eine Impipfficht für Kinder gemäss dem schweizerischen Impfplan eingeführt wer- den?
Should compulsory vaccination children be introduced in accordance with the Swiss vaccination schedule?
Soll die Aufenthaltserlaubnis für Mi- grant/innen aus Caat-EUEFFFA-Staatie schweizweit an die Erfüllung verbindlicher Integraionsveerenbbarngeen geknüpft wer- den?
Should the residence permit for migrants from non-EU/EFTA countries be linked to the fulfilment of binding integration agreements throughout Switzerland?
Soll der Bund erneuerbare Energien stärker fördem?
Should the federal government promote newable energy more?
Befürworten Sie eine strengere Kontrolle der Lohngleichheit von Frauen und Männern?
Do you support stricter control of gender pay?
Soll der Bund die finanzielle Unterstützung für die berufliche Weiterbildung und Umschu- lung ausbauen?
Should the Federal Government expand its financial support for continuing vocational training and retraining?
Sollen in der Schweiz vermehrt Spitäler geschlossen werden, um die Kosten im Gesundheitsbereich zu senken?
Should more hospitals be closed in Switzer- land to reduce healthcare costs?
Eine eidenössische Volksinitiative verlangt, dass der Bundesrat direkt vom Volk gewählt werden soll. Unterstüitznn Sie dieses An- liegen?
A swom federal people's ' initiative demands that the Federal Council be elected directly by the people. Do you support this concern?
Soll die Einführung der elektronischen Stimmabgabe bei Wahlen und Abstimmungen (E- Voting) weiter vorangetrieben werden?
Should the introduction of electronic voting in elections and voting (e-voting) be further promoted?
Table 10: Chosen questions for stance detection in German and their English translation
24

---

Published as a conference paper at ICLR 2025
Comment in English
Comment in German
I don't believe that insured persons should con- tribute more to their own healthcare costs by in- creasing the minimum deductible. While it may seem like a good idea to increase deductibles as a way to reduce overall healthcare costs, this only shifts the burden onto patients and puts them at risk of facing high out-of out-of-pocket expenses. It's important for insurance companies to find ways to control costs without placing the onus on insured individuals.
Ich glaube nicht, dass Versicherte mehr zu ihren eigenen Gesundheitskosten beitragen sollten, in- dem sie den Mindestabzug erhöhen. Die Er- höhung der Selbstbehalte als Mittel zur Senkung der gesamten Gesundheitskosten mag zwar als gute Idee erscheinen, verschiebt jedoch die Belastung nur auf die Patienten und gefährdet sie mit hohen Kosten, die sie selbst tragen müssen. Es ist wichtig für Versicherungsseeelllcche
Wege zu finden, Kosten zu kontrollieren, ohne die Versicherten mit der Last zu belasten.
Hi, friends! think it's a great idea for insured person to contribute more to their health costs by increasing the minimum deductible. As someone who believes in taking ownership of my own health and finances, I believe that paying more out-of- pocket can help me better manage my expenses and make more informed decisions about my health- care. Plus. think that by contributing more, I can save money in the long run and avoid potential financial strain in other areas of my life.
Ich finde es eine gute Idee, dass Versicherte mehr zu ihren Gesundheitskosten beitragen, indem sie den Mindestabzuu erhöhen. Als jemand. der daran glaubt, die Verantwortung für seine Gesund- heit und seine Finanzen zu übernehmen, glaube ich, dass mehr Geld aus eigener Tasche helfen kann, meine Ausgaben besser zu managen und fundierte Entscheidungen über meine Gesund- heitsversorgung zu treffen. Außerdem denke ich, dass ich durch einen größeren Beitrag auf lange Sicht Geld sparen und finanzielle Belastungen in anderen Bereichen meines Lebens vermeiden kann.
"In my opinion, it's wrong to link the residence per- mit for migrants from EOn-EU/EFTA countries to the fulfillment of binding integration agreements in Switzerland. This policy puts too much emphasis on the ability of migrants to adapt to Swiss culture and language at the expense of their personal free- doms and rights as human beings. It also creates incentives for migrants to cheat lie in order to gain a residency permit and access to valuable ser- vices and resources." "
"Meiner Meinung nach ist es falsch, die Aufen- thaltserlaubnis für Migranten aus Nicht-EU/EFTA- Ländern mit der Erfüllung verbindlicher Integra- tionsvereinbarungen in der Schweiz zu verbinden. Diese Politik legt zu viel Wert auf die Fähigkeit der Migranten, sich die Schweizer Kultur und Sprache anzupassen, auf Kosten ihrer persönlichen Freiheiten und Rechte als Menschen. Es schafft auch Anreize für Migranten, zu betrügen oder zu lügen, um eine Aufenthaltserlaubnis zu erhalten und Zugang zu wertvollen Dienstleistungen und Ressourcen zu erhalten
I believe that the residence permit for migrants from AU-EEU/EFAA countries should be linked to the fulfillment of binding integration agreements
Ich glaube, dass die Aufenthaltserlaunnis für Migranten aus Mich--EU/EFTA-Lände mit der Erfüllung verbindlicher Integrattonsseereibbruuu gen in der gesamten Schweiz verbunden sein sollte. Die Integration sollte nicht als ein- seitiger Prozess, sondern als eine gegenseitig vorteilhafte Beziehung zwischen dem Einwan- derer und dem Aufnahmeland betrachtet wer- den. Durch die Verknüpfung der Aufenthalt- serlaubnis mit dem erfolgreichen Abschluss von Integrationsverriibbauggee fördern wir die ak- tive Teilnahme der Einwanderer am Integra- tionsprozess und die Förderung ihrer Teilnahme an der Schweizer Gesellschaft Darüber hinaus wird durch diese Vorgehensweise sichergestellt, dass Einwanderer nicht einfach eine Aufenthalt- serlaubnis erhalten, ohne zum Wohlergehen der Schweizer Gesellschaft beizutragen. Letztlich kommt einem umfassenden Integrationsansatz sowohl der Einwanderer als auch dem Aufnahme- land zugute, da er eine integrativere und harmonis- chere Gesellschaft schafft
throughout Switzerland. This is because integra- tion should not be seen as one-sided process, but rather as a mutually beneficial relationship be- tween the immigrant and the host country. By linking the residence permit to the successful com- pletion of integration agreements, we incentivize immigrants to actively participate in the integra- tion process and become active members of Swiss society. Additionally, this approach ensures that immigrants do not receive a free ride by simply obtaining residence permit without contributing to the well-being of Swiss society. Ultimately, a comprehensive integration approach benefits both the immigrant and the host country by creating a more inclusive and harmonious society.
Table 11: Sample of translated comments from comments generated by the LLM used for fine-tuning the stance detection model.
25

---

Published as a conference paper at ICLR 2025
C.3 SYNTHETIC DATA STANCE ALIGNMENT
Comment in English
Intended Stance AGAINST
I don't believe that insured persons should contribute more their own healthcare costs by increasing the minimum deductible. While it may seem like a good idea to increase deductibles shifts the burden onto patients and puts them at risk of facing high out-of-pocket expenses. It's important for insurance companies to find ways to control costs without placing the onus on insured individuals.
way to reduce overall healthcare costs, this only
Hi, friends! think it's a great idea for insured person to contribute more to their health costs by increasing the minimum deductible. As someone who believes in taking ownership of my own health and finances, believe that paying more out-of-pocket can help me better manage my expenses and make more informed decisions about my healthcare. Plus, think that by contributing more, I can save money in the long run and avoid potential financial strain in other areas of my life.
FAVOR
"In my opinion, it's wrong to link the residence permit for migrants from non-EU/EFTA countries to the fulfillment of binding integration agreements in Switzerland. This policy puts too much emphasis on the ability of migrants to adapt to Swiss culture and language at the expense of their personal freedoms and rights as human beings. It also creates incentives for migrants to cheat or lie in order gain residency permit and access to valuable services and resources." " I believe that the residence permit for migrants from non-EU/EFTA countries should be linked to the fulfillment of binding integration agreements throughout Switzerland. This is because integration should not be seen as a one-sided process, but rather as a mutually beneficial relationship between the immigrant and the host country. By linking the residence permit to the successful completion of integration agreements, we incentivize immigrants to actively participate in the integration process and become active members of Swiss society. Additionally, this approach ensures that immigrants do not receive a free ride by simply obtaining residence permit without contributing to the well-being of Swiss society. Ultimately, a comprehensive integration approach benefits both the immigrant and the host country by creating more inclusive and harmonious society.
AGAINST
FAVOR
As someone who is against stricter controls on equal pay for women and men, I believe that all individuals should be paid based on their individual skills, performance, and contributions to the company, regardless of their gender. think it's important to recognize and reward hard work and dedication, regardless of whether someone male or female. Additionally, argue that it's already illegal and unethical to discriminate on the basis of gender and that such reg- ulations would only create unnecessary tensions and potential conflicts within the workplace. Instead, we should focus on creating truly equitable society where everyone has access to equal opportunities for education, job training, and development.
AGAINST
I personally believe that stricter controls on equal pay for women and men are necessary because it's not always fair that men receive higher salaries just because of their gender. In many cases, women have to work harder than men to achieve the same level of performance, but they still earn less money. Stricter controls would help ensure that everyone's 's hard work is recognized and rewarded fairly, regardless of their gender. Additionally, it's important to promote gender equality and give all individuals controls on equal pay are necessary to help reduce the gender pay gap and create a more equitable society.
FAVOR
equal opportunity succeed. Overall, think stricter
Table 12: Sample of comments generated by the LLM used for fine-tuning the stance detection model.
26

---

Published as a conference paper at ICLR 2025
D ADDITIONAL EXPERIMENTAL DETAILS
D.1 EVALUATION.
For fine-tuning and testing we evaluate the given model separately on 10 chosen questions from the test dataset of X-Stance for all experiments. For each question q we split D into a 60/40 train/test split (repeated with different seeds to get error bars) and use the train split for fine-tuning to the given question and the test split for evaluation. Our main results report the average selected questions from the test dataset evaluated on the comments from the test split. The error bars represent the average standard deviation over the 10 questions for runs with different seeds. More detailed results per question are shown in Appendix
score over 10
D.2 COMPUTE AND RUNTIME
We conduct our experiments on single NVIDIA A 100 80GB GPU and core CPU. With this setup, for Mistral-7B, the generation of synthetic data takes approximately 3 hours per question for synthetic dataset size of 1000. Fine-tuning the BERT model with the synthetic data takes less than minute. For the active learning methods, the selection of the most informative samples takes less than a minute. Hence the largest computational effort the generation of the synthetic data.
D.3 TRANSLATION THE X-STANCE DATASET SYNTHETIC DATA GENERATION
In Figure I0 we show the pipeline for translating the X-Stance dataset for synthetic data generation. We start with a question from the X- X-Stance test dataset and translate the question to English with a NLLB-330M model (NLLB Team et al. 2022). Then we let the Mistral-7B model generate synthetic data, i.e., comments for the translated question. The generated comments are then translated back to German to be used for fine-tuning the model in experiments.
D.4 OVERVIEW OF USED DATASETS
In Table 13 we show an overview of the datasets used in our experiments for the different methods evaluate.
German Question:
SQBC+Synth
Soll der Bund...?
German Answers
NLLB-330M
NLUB-310X
English Answers
English Question: Should rederal goverament...?
Mistral
M/2
M/2
Synth Size:
Figure 10: Overview of the pipeline for active learning with synthetic data: We start with question g from the X-Stance test dataset and translate the question English with NLLB-330M model (NLLB Team et al. 2022). Then we let the Mistral-7B model generate synthetic data, i.e., comments for the translated question. The generated comments are then translated back to German to be used for fine-tuning the model in our experiments.
27

---

Published as a conference paper at ICLR 2025
Datasets
Manual lahels
Thie Labels
Synth Aug Danh
Canlig
Datad
D.
Вssclino
Baseline Synth
Thue Labels
Synth
SQBC
SQBC + Syath
CAL
Random
/ (randomlly selected)
Random + Synth
v (randomily selecied)
Table 13: Synth: Synthetic Data, Aug: Augmentation. learning with synthetic data.
We compare different variants of active
E DATASET
X-Stance is multilingual stance detection dataset, including comments German (48, 600), French (17, 200) and Italian (1, 400) political questions, answered by election candidates from Switzerland. The data has been extracted from smartvotd Swiss voting advice platform. For the task of cross- topic stance detection the data is split into training set, including questions on 10 political topics, and a test set with questions on two topics that have been held out, namely healthcare and political system.
E.1 CHOSEN QUESTIONS AND THEIR DISTRIBUTION
We present the 10 chosen questions for experiments in Table 10 We show the original questions in German and their corresponding English translations by the translation model. Furthermore, also show the (60 / (0) train/test split for each question in Figure [1] We chose 10 questions that reflect the overall distribution of D(g). We choose questions with small amount of comments, unbalanced comments and also balanced comments. Furthermore, for of the questions the majority class favor and for the other 5 the majority class is against.
202
285
275
255
240
225
219
295
285
155
350
335
325
305
les
Question
Ques tice
Queation
Questios
Question
Question
Queation
Ques tice Question 10
Sott-
Zest pest-Train Eplit
Test-7est 8ptit
Anainas
Agulaus
Figure 11: Distribution of the positive and negative samples for the train and test split of D ((2), We show the distribution the positive and negative samples of the X-Stance dataset for the questions Q1-Q10. We also show the 60/40 train/test split for the 10 questions. We chose 10 questions that reflect the overall distribution of ((4) We chose unbalanced, balanced and low sample size questions to evaluate the effectiveness of our approach.
test
https://www.. http:://www.aaatrote.comm
28

---

Published as a conference paper at ICLR 2025
F EXTENDED RESULTS
We present the extended results for the different synthetic dataset sizes M 200, = 500 and M 1000 in Figures 12 13 and 14 As Figure 4 we show the results for the different active learning methods and the different configurations of the synthetic data, while varying the amount of samples that need be labelled. We compare all methods True Labels, hence the horizontal line corresponds to the performance of the baseline model fine-tuned with the true labels.
Oceetsin
9.80
0.48
U60
NJ ANLUVD
29
POLLID
Semcmiog1:411
Qustin
Accho Leurning 107)
0.50
Ooeettee:
2.76
0.45
0:30
64
120
190
Sumbez of Sarglee
Figure 12: Extended results of Figure for M-200:
29

---

Published as a conference paper at ICLR 2025
QueSSIAN
0.80
0.43
Genctang((
Creatire
Creaktmn
0.80
0.75
0.30
37
14
No Actove
16
11
123
3o ACEAPe
23
59
115
178
No Artba
1.00
0.30
154
20
Learring
0.40
Strplea
Figure 13: Extended results of Figure 4 for M=500
30

---

Published as a conference paper at ICLR 2025
DaaskSax
QuesSian
0.80
0.43
Genctang((
Creatire
greaktm
0.80
0.75
0.30
37
14
No Actove
16
41
123
3o ACEAPD
23
50
112
178
No Arthe
1.00
0.30
154
Learring
0.40
Strplea
Figure 14: Extended results of Figure for M=1000
31


Reviews:
[{"review_id": "AbnrOen91J", "reviewer": "Reviewer_kmPK", "summary": "The paper tries to improve transformer-based stance detection models by fine-tuning on LLM generated data. They compare the real-world data with the synthetic data to identify difficult samples from unlabelled data (active learning) to further improve the model. They show that both these steps improves the performance of the transformer-based baseline.", "questions": "- Why choose translation over adapting prompts directly? Is Mistral unable to generate responses in German, or were other multilingual models considered?\n\n- In the ablation study for \"Content vs. Size\", I am not sure I understand why you call the shuffled dataset \"misaligned\". Could you please explain the reasoning behind this? \n\n- In the generated dataset, did you find any instance where the LLM failed to generate the requested content? For instance, generate statements not in favor when requested for \"in favor\" content or LLM refusing to generate any relevant content at all.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper leverages the data augmentation capabilities of LLMs to improve transformer based models which are better suited for online deployment as they are more reliable. \n- The presented method can be adapted to other text classification tasks and hence is a significant contribution.\n- It is well written and easy to follow, except for few instances mentioned in the comments.", "weaknesses": "- It’s possible I’m missing some key context here, but I’m having trouble following the ablation study in Section 5.2. To test whether the performance gains come from dataset size or the generated content itself, the authors “shuffle” instances, apparently misaligning the posed questions with synthetic data. If the synthetic data consists of single text instances with labels, this shuffling wouldn’t seem to affect outcomes. Perhaps the authors mean they’re using different proportions of synthetic data in each run while keeping the total instance count constant, but this explanation feels somewhat unclear.\n- Even though authors acknowledge this as a limitation, fine-tuning a separate model for each question doesnot seem to be a scalable approach, especially when the main motivation for the research was in line with training robust models for online deployment. \n- The X-stance dataset is described as having around 48k annotated comments on various questions. However, an overview of the dataset’s statistics—such as the number of comments per question—would greatly enhance readability. When you mention selecting 10 questions from the test set, it would be helpful to specify how many comments correspond to each question. While I see some statistics are included in the Appendix, a high-level summary within the main text would improve clarity and context for readers.\n- Section 4.2, General setup: Please review this section for more readability. Currently, it is a bit difficult to get a picture of what models are being tested and how the methods differ between them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nPD3AtwS6E", "reviewer": "Reviewer_cUfQ", "summary": "This paper proposes to use LLM-generated synthetic data to augment the training of stance classification models. It also proposes a synthetic data-based active learning method that uses synthetic data to facilitate the selection of unlabelled data for human annotation. Experiments are conducted on the German subset of the X-stance dataset (with the help of machine translation). The results demonstrate that including synthetic data in training can improve stance prediction. The synthetic data-based active learning method, however, is not clearly better than a random selection-based baseline active learning method.", "questions": "- It would be very helpful to explain why only a German dataset is used for the experiments. Also, if German text is used, have the authors considered using a different LLM that has good German language processing capabilities for the experiments?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The proposed method is sound. I do not see any major issue with the method.\n- Although the idea of using synthetic data to augment models is not entirely new, it probably has not been widely explored for stance prediction.\n- The authors conducted extensive experiments to evaluate the method, including varying the size of the synthetic dataset, comparing with meaningful baselines, and the further experiments that compare with a LLM zero-shot baseline.", "weaknesses": "- The experiments are conducted using a German dataset, but translation into and back from English is used in order for the method to work (probably because of limited German language understanding and generation capabilities of the Mistral model that is used?) There is no explanation of why the authors do not evaluate the method using an English dataset.\n- The novelty and impact of the work is still limited. (1) Using synthetic data to augment models is not new. Although applying the idea to stance prediction might be new, it is one of many NLP tasks. The way synthetic data is generated and used during training in this paper is also standard, hence there is limited technical contribution. (2) The idea of using synthetic data for active learning is very interesting and is novel based on my knowledge. However, its effectiveness is limited based on the experiments. Therefore, overall, although the work is very solid in general, its novelty and impact may not meet the standard of this conference.\n- There is room for improvement in terms of presentation. In particular, the active learning method proposed can benefit from first presenting an overview of the high-level intuition behind the method before describing the method itself.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "poKIDAk9Ob", "reviewer": "Reviewer_si6z", "summary": "Yet another synthetic data paper that shows modest improvements but doesn't quite nail why or how to make synthetic data actually useful. Some interesting ideas buried under conventional methodology.\n\nLook, I've seen enough \"let's use LLMs to generate synthetic data\" papers to last several conferences. What makes this one interesting - barely - is the political stance detection angle and the somewhat novel SQBC approach. But let's be real here: you're essentially using an LLM to generate slightly different versions of existing viewpoints, then acting surprised when this helps... a little bit.\n\nThe authors show that their approach improves F1 scores from ~0.69 to ~0.72 with synthetic data alone, and up to ~0.75 with their full pipeline. Sure, that's positive, but is it worth the computational cost of running Mistral-7B for hours to generate the synthetic data? (And don't get me started on the economic/environmental impact in general - though I suppose that's not this paper's specific sin since those models are small and this used only 1 A100 GPU.)\n\nThe most interesting part is actually buried in Section 5.1, where they show that using Mistral-7B directly for stance detection fails miserably. This suggests something important about synthetic data that the authors don't fully explore: it's better at generating plausible variations than at making decisive judgments. This deserved more analysis.\n\nWhat's missing here is any real investigation into what makes synthetic data actually useful. Are we just doing expensive interpolation between existing data points? Where's the analysis of entropy and diversity in the generated samples? The visualizations in Figure 3 are pretty, but they also show that the synthetic data mostly just fills in obvious gaps rather than introducing genuinely novel perspectives.\n\nThe active learning component feels tacked on, though I'll admit the SQBC approach is clever. Using synthetic data as a reference distribution for selecting informative samples is neat, but again - why does this work? The paper handwaves at \"ambiguous samples\" without diving deeper into the theoretical foundations.\n\nOne thing I'll give the authors credit for: they did their homework on the translation pipeline. Using NLLB-330M and actually caring about the quality of the German-English-German round trip is more than many papers bother with. The samples in Table 8 show reasonable quality political discourse generation.\n\nSUGGESTIONS FOR IMPROVEMENT:\n\n- Add analysis of entropy/diversity metrics for synthetic data\n- Provide theoretical justification for why synthetic data helps beyond just \"more data\"\n- Compare computational costs vs. benefits more explicitly\n- Explore what makes certain synthetic samples more useful than others\n- Consider alternative methods for introducing genuine novelty into synthetic data\n\nNITPICKS:\n\n- The abbreviation \"SQBC\" is used before it's properly defined\n- Figure 4 is information-dense to the point of being hard to parse\n- Some ablation studies feel perfunctory rather than insightful\n\nCONCLUSION:\n\nThis paper is fine. It's not going to revolutionize either synthetic data generation or stance detection, but it makes a modest contribution to both. The experimental work is solid if unexciting, and the results are positive if not earth-shattering. The biggest missed opportunity is not diving deeper into what makes synthetic data actually useful beyond simple interpolation.\n\nThe paper should be marginally accepted because it advances the field incrementally and might give others ideas for more innovative approaches. But let's not pretend this is more than a small step forward in a very crowded research space. I don't like these kind of papers\n\nI'd love to see a follow-up that really digs into the entropy question and provides proper theoretical foundations for synthetic data generation in political stance detection. Until then, this feels like another \"it works (a bit) but we're not quite sure why\" paper.", "questions": "Were there any computational/economic reasons for not scaling up your compute? I'm sympathetic to this as I understand the burden of even a single A100 - but if you do have more resources, why not use them?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "Clean experimental methodology with proper ablation studies\nGood visualization of how synthetic data aligns with real data distributions\nActually bothered to translate German political content properly instead of using Google Translate\nReasonable baseline comparisons and honest reporting of limitations\nThe SQBC approach is somewhat novel, even if not revolutionary", "weaknesses": "Limited theoretical justification for why synthetic data helps beyond \"moar data good\"\nDoesn't address the entropy/diversity problem in synthetic data generation\nResults are modest (~2-3% improvements) for considerable computational overhead\nHeavy reliance on a specific dataset (X-Stance) limits generalizability claims\nThe \"active learning with synthetic data\" angle feels like two papers duct-taped together", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)