GESIS-DSM at PerpectiveArg2024: A Matter of Style? Socio-Cultural Differences in Argumentation (2024)

Abstract: This paper describes the contribution of team GESIS-DSM to the Perspective Argument Retrieval Task, a task on retrieving socioculturally relevant and diverse arguments for different user queries. Our experiments and analyses aim to explore the nature of the sociocultural specialization in argument retrieval: (how) do the arguments written by different socio-cultural groups differ? We investigate the impact of content and style for the task of identifying arguments relevant to a query and a certain demographic attribute. In its different configurations, our system employs sentence embedding representations, arguments generated with Large Language Model, as well as stylistic features. Our final method places third overall in the shared task, and, in comparison, does particularly well in the most difficult evaluation scenario, where the socio-cultural background of the argument author is implicit (i.e. has to be inferred from the text). This result indicates that socio-cultural differences in argument production may indeed be a matter of style.

text: Proceedings of the 11th Workshop on Argument Mining, pages 169–181 August 15, 2024 ©2024 Association for Computational Linguistics
Our final method places third overall in the shared task, and, in comparison, does particularly well in the most difficult evaluation scenario, where the socio-cultural background of the argument author is implicit (i.e. has to be inferred from the text). This result indicates that socio-cultural differences in argument production may indeed be a matter of style.

heading: 1 Introduction
text: Argument retrieval is a well-established task in computational argumentation (Wachsmuth et al., 2017; Stab et al., 2018): given a query or question, e.g. on “should we have free trade agreements?”, the task aims to retrieve topically relevant arguments. Topically relevant arguments can be heterogeneous, even for arguments with the same stance (for vs. against) - i.e. someone can focus on quality of the products as essential to free trade, while another may focus on international collaboration. These different perspectives in arguments can be
<♥> Contributions mostly completed during a visit funded by a GESIS Visiting Junior Researcher grant at GESIS – Leibniz Institute for the Social Sciences.
relevant for the personalization and diversification of argument retrieval, and in online debate portals.
The Perspective Argument Retrieval Task (Falk et al., 2024) argues that the socio-cultural background (e.g. gender, religion) can be taken into account when selecting relevant arguments. Sociocultural information can determine the dynamics of argumentation at different levels. For example, socio-demographics can be used to approximate the stance of an argument about a specific topic (e.g., women, pro abortion); additionally, the specific arguments used to support a stance can be a correlate of a specific socio-cultural attribute (religious being against abortion because it is a sin).
Differences between arguments of different socio-cultural groups have mostly been researched as differences in argument content, meaning the semantic differences in the arguments: which phrases, aspects, and points are mentioned. For instance, Spliethöver and Wachsmuth (2020) analyze how social groups differ in the social bias of their arguments, i.e., male users using terms that indicate a gender bias. However, previous research indicated that argument style may be different between socio-cultural groups as well. With argument style, we mean how something is said, e.g. features of form and not content like complexity of words, length of words and sentences, grammatical tenses, or pronoun usage. Such stylistic features have mostly been studied from the perspective of argument persuasiveness (El Baff et al., 2020) but also have been used to analyze socio-demographic differences in deliberation processes, i.e., women referring more to others than men in their arguments (Klinger and Russmann, 2015).
While the shared task does not address the issue of persuasiveness of the arguments directly, the intuition that “an argument I would write is one that is likely to resonate stronger with me”, builds a potential bridge between socio-demographic retrieval and persuasion research (e.g., consider El Baff et al.
169
(2018) on empowering vs. challenging arguments). Additionally, diversity (providing arguments with different perspectives on the same query) can be employed to maximize the reception of a certain argument. Also, such a diversity of perspectives in recommendations can be beneficial, for instance for citizens in democracies (Reuver et al., 2021).
In approaching the task of argument retrieval for specific socio-cultural profiles, we can also resort to findings from other tasks, such as author profiling (Koppel et al., 2002). This task aims to predict author characteristics from user-generated text, with these characteristics often having a sociodemographic nature, (i.e., gender or age). Successful approaches use semantic content as well as style to profile authors of user-generated texts (Rangel et al., 2021; Bevendorff et al., 2023).
Our approach The perspectivist argument retrieval task raises the question of how socio-cultural groups differ in their arguments for a given query. These differences can be semantic, i.e., groups may differ in what they say in their arguments, or stylistic, i.e., groups may differ in how they formulate them. We first explore this distinction in the shared task data in Section 3. We then describe our approaches to ranking arguments according to socio-cultural specific relevance in Section 4: One is based on semantic content similarity using ’prototypical’ arguments generated with a Large Language Model (LLM). The other uses stylistic features. Our final method places third overall in the shared task, and, in comparison, does particularly well in the most difficult scenario, the one in which the socio-cultural background of the argument author is implicit (i.e. has to be inferred from the text). It implements a three-step pipeline, using semantic information in a ranking step and stylistic information to classify whether arguments are relevant for a given socio-cultural group. Our results indicate that the stylistic differences in the arguments of different socio-cultural groups are more relevant to the task of retrieving relevant arguments than semantic differences in our setup. We publicly release our code for the experiments and analyses.1

heading: 2 Task: Data and Evaluation Scenarios
text: The question at the core of the task is: Can we find the arguments that members of a given sociocultural target group would write for this query?
1github.com/mmmaurer/perspective_argument_retrieval
Data The task data is a multilingual dataset in three different cycles of each +/- 30,000 arguments and +/- 300 related queries (in German, Italian, and French). The provided socio-cultural information covers gender, age, place of residence, civil status, denomination, education, political spectrum, and political issues that are of importance to the authors of queries. Additionally, the stance and political topic of the argument are provided.
Details on the size and train/dev/test splits of the three cycles is provided in Table 2 in Appendix B. In the first two cycles, politicians express their stances regarding different political issues in the context of the 2019 and 2023 Swiss elections. In contrast, the third cycle consists of voters’ perspectives. For this, samples of the arguments given by politicians for the 2023 election were annotated by amateur annotators. The resulting third cycle data consists of the arguments that intuitively match their perspectives. Socio-cultural profiles were collected for both politicians and voters.
Evaluation Scenarios The systems are evaluated on three scenarios: 1. Argument retrieval without consideration of socio-cultural differences (baseline). 2. Argument retrieval for a specified sociocultural attribute. Information about any other attribute could be used to diversify the set of retrieved arguments (explicit scenario). 3. Argument retrieval for a specified socio-cultural attribute. In contrast to explicit, information about other attributes is hidden (implicit scenario). System evaluation is based on both relevance and diversity of the selected arguments (in terms of socio-cultural attributes) to promote the diversity of opinions.

heading: 3 Data Analysis: Content Or Style?
text: In the development of our pipeline, data analysis played a crucial role. In the following, we summarize core findings for content and style. These analyses were conducted on the cycle 1 corpus.
Semantic content differences Firstly, we assess whether there are arguments that multiple sociocultural groups share and find that ∼ 11% of the arguments appear with the same argument text for at least two different socio-cultural profiles. The same argument may, for instance, be produced by a non-religious man and a roman-catholic woman. While this is expected in a natural setting, as groups may share views and thus arguments, this raises the question of differentiation between socio-cultural
groups, in particular for the implicit scenario. Secondly, we cluster Sentence-BERT representations of arguments using the k-means algorithm and evaluate the resulting clusters against the ground-truth socio-cultural groupings, as well as stance and topic. Per socio-cultural attribute, we run one clustering with k equaling the number of groups in the attribute (e.g. for residence k = 2, as there are the two groups city and countryside). We find that all of the socio-cultural groups have an adjusted Rand score (Hubert and Arabie, 1985) of ≤ 0.1 with the respective clustering, indicating virtually no overlap of the clusters with the groups. Only the topic shows a relatively higher adjusted Rand score (for an overview of the results of our clustering experiments, see Appendix E). These analyses indicate that there is little semantic distinction between arguments of different sociocultural groups, at least in the present semantic representation space.
Stylistic differences To examine stylistic differences in the arguments, we carried out exploratory linear regression analysis. We exclusively focused on the German-language share2, which comprises about 22k of the in total 32k arguments. The sociocultural attributes served as independent variables and stylistic features of the arguments as dependent variables. We tested a number of stylistic features, which can be divided into surface and syntactic features and are explained in detail in Table 5 in Appendix F. The seven surface features include measures of word and sentence length, long and complex words, the variety of vocabulary used, and two readability indices. The syntactic features cover part-of-speech (POS) tags, named entities, present tense tokens, imperative tense tokens, and first person writing. Each POS tag forms an extra feature, giving us a total of 21 syntactic features.
We ran one linear regression per stylistic feature to estimate the relationship between socio-cultural information and the particular stylistic feature. In particular, we looked into interactions between variables to take a step into the direction of sociocultural profiles rather than single attributes. Due to the space limitations, we cannot discuss all features in the paper. However, the full set of regression outputs can be found in the project’s GitHub repository. All details are outlined in Appendix D.
Our assumption is that if socio-cultural groups differ in style with respect to specific stylistic fea-
2Details on the language detection are in Appendix A.
tures, our regression models will be able to account for a significant amount of that variance (higher adjusted R2). Models that explain the surface features show notably higher adjusted R2 values than those for the syntactic ones (cf. Table 3 in Appendix D). We find the highest explanatory power for long words, words per sentence, and the Gunning Fog Index with adjusted R2 of 0.1557, 0.1422, and 0.1256, respectively. This suggests that sociocultural characteristics can explain the writing style of the arguments to at least some extent. A closer look into the individual effects and interactions within the socio-cultural attributes reveals several significant effects. For instance, liberals use significantly fewer long words than conservatives, which is even more pronounced in connection with civil status divorced (compared to conservative and single). Liberals in the center or right of the political spectrum exhibit a higher Gunning Fog Index than left-wing conservatives, hinting at the number of years of formal education a person needs to understand a text on the first reading.
While some of these effects certainly also depend on further factors like the context of an argument (i.e., stance and the topic of discussion), the findings add to our underlying hypothesis of different stylistic fingerprints.

heading: 4 System Description
text: As discussed in the introduction, we want to assess the impact of content and style on the perspective argument retrieval performance. We thus divide the problem into two steps covering these aspects, with an additional filtering step.3
Our resulting pipeline, depicted in Figure 1, consists of three steps: 1) Target Demographic Filtering: If arguments of a specific socio-cultural group are queried and socio-cultural information for the arguments is available, only consider the arguments from the respective socio-cultural tar-
3All hyperparameters and implementation details of our analyses and models are given in Appendix A.
get group for the subsequent steps. 2) Semantic Argument Ranking. Find the top k arguments that are semantically closest to the query. 3) Final Candidates Retrieval. Option (a) Stylistic Relevance Classification: Select only those from the pre-selected k arguments with a stylistic fingerprint indicating a relevant argument given the target socio-cultural group. Option (b) Generated Argument Re-ranking: Per query and socio-cultural attribute, generate an argument and find the top k arguments that are semantically closest to the generated argument.
For the non-perspectivist baseline scenario, only step 2), semantic argument ranking, is executed. For the explicit perspectivism scenario, all three steps are executed. Finally, for the implicit perspectivism scenario, we execute solely steps 2) and 3) as no socio-cultural information is accessible.
In what follows, we detail the operationalization of these three steps.

heading: 4.1 Target Demographic Filtering
text: To reduce the search space in cases where only the arguments of a specific socio-cultural group are queried and the socio-cultural profiles of the authors of the arguments are known, i.e. in the explicit perspectivist scenario, we filter out the arguments that do not match the queried attribute.

heading: 4.2 Semantic Argument Ranking
text: A necessary condition for a given argument to be relevant for a query, both in the perspectivist and the baseline cases, is that they are semantically related, or, in other words, that the argument supports a stance towards the question stated in the query. The relevant candidate arguments for a given query should thus be selected such that their semantic similarity is as high as possible.
To operationalize this, as illustrated in Figure 2, we rely on retrieving sentence embeddings using Sentence-BERT (Reimers and Gurevych, 2019) both for the query and the arguments in a corpus and calculate the cosine similarity between the
query’s representation and each of the arguments’ representations. Finally, the arguments are sorted according to the cosine similarity and only the top k arguments are considered.
As a backbone Sentence-BERT model, we use paraphrase-multilingual-mpnet-base-v2
4, a multilingual model trained on paraphrases in 50+ languages, among them the three Swiss official languages present in the dataset.

heading: 4.3 Final Candidates Retrieval

heading: 4.3.1 Stylistic Relevance Classification
text: A classification step to differentiate between semantically generally relevant arguments (i.e. arguments relevant to a query, regardless of socio-cultural information) and relevant arguments for a specific socio-cultural group is implemented next.
We implement a semantic selection step before the classification step and assume that the set of relevant arguments of a specific socio-cultural group given a query is a real subset of the set of relevant arguments given a query. Based on this insight, we construct positive and negative examples from the training subsets of the provided datasets: For each unique query text q, we collect the set of the overall relevant candidates Cq,all. A candidate is considered a positive example and assigned the label relevant for the respective socio-cultural group t if it is in the set of the relevant arguments given t and q, Cq,t. A candidate is considered a negative example and assigned the label not relevant if it is in Cq,all \ Cq,t. To end up with a more balanced training set, we only collect ∣Cq,t∣ negative examples if ∣Cq,all∣ ≥ ∣Cq,t∣. Per example (i.e. per argument), we encode a one-hot representation of the queried socio-cultural attribute and concatenate it with surface-level stylistic features of the respective argument as input features. A full overview of the stylistic features can be found in Table 5 in Appendix F. As the majority of arguments are in German, the feature extraction assumes the language to be German. While this is sub-optimal (style may differ across languages), this serves as a first assessment of whether stylistic differences can help in this task.
For the resulting classification step of our pipeline, as visualized in Figure 3, we train a random forest classifier on the training set portion (80%) of our dataset constructed from a union of
4https://huggingface.co/sentencetransformers/paraphrase-multilingual-mpnet-base-v2
the training subsets of the three datasets. The classifier achieves an F1 = 0.60 for both the positive and negative classes on the held-out test set portion (20%) of our dataset. We provide a feature importance overview in Figure 6 in Appendix G.

heading: 4.3.2 Generated Argument Re-Ranking
text: Following the hypothesis5 that arguments of different socio-cultural groups are differentiable by their semantic content, we implement a reranking step using LLM-generated arguments. We generate arguments for specific groups and queries using occiglot-7b-eu5-instruct6, a Mistral-7B-v0.1 model with continued pretraining on the five biggest languages in Europe: English, Spanish, French, German, and Italian. We generate one argument per query, which is then used to re-rank the candidates from the second step in our pipeline. Specifically, per query, we gather the Sentence-BERT representation of the generated argument and re-rank the candidates according to their cosine similarity with the generated argument. Appendix C provides more details about the generated arguments (prompts, statistics, examples, and qualitative analysis).
5Our initial hypothesis was that arguments differ in their semantic content across socio-cultural groups. Our downstream analysis of the semantic representation space in combination with the results did however proved our hypothesis wrong.
6https://huggingface.co/occiglot/occiglot-7b-eu5-instruct

heading: 5 Discussion of Results
text: The results of our different systems over the three test sets are displayed in Table 1. It can be seen that using stylistic relevance classification as the final step in our pipeline yields results well over the Sentence-BERT baseline across explicit scenario test sets, and for the final implicit scenario test set (Table 1: 2023, User Study).
Moreover, this approach outperforms the use of generated argument re-ranking across all perspectivist test sets. Overall, our findings show that content plays a role in pre-selecting arguments to fit the respective query, as evidenced by the comparably high baseline scenario results. For the same queries, however, socio-cultural groups appear to be less different in the content of their arguments than in their style.

heading: 6 Conclusion and Future Work
text: We present our approach to the Perspective Argument Retrieval Shared Task 2024. Our proposed method implements a three-step pipeline, leveraging semantic information in a ranking step and stylistic information to classify whether arguments are relevant for a given socio-cultural group. The performance of this approach, in particular for the implicit scenario, showcases the potential of including stylistic information for the task of perspectivist argument retrieval. This raises several questions for future research.
Especially with regard to the third test set, in which the perspectives of politicians and voters were flipped, we argue that investigating the reasons for differences in production and perception of arguments of different socio-cultural groups, e.g. through semantic or stylistic differences, is of interest. Consequently, how to combine this information in retrieval scenarios should be investigated.

heading: 7 Ethical Considerations and Limitations
text: Shared tasks have previously focused specifically on author profiling, e.g. profiling spreaders of hate speech in English and Spanish (Rangel et al., 2021) or profiling crypto influencers (Bevendorff et al., 2023), where one system contribution used LLMs and bi-encoding (Giglou et al., 2023). We acknowledge the task of authorship profiling and our approach, using stylistic features, has some established ethical harms to individuals and society at large. These harms are mostly relating to privacy and giving agency to powerful actors to track or harm individuals. However, we also found work that is meant to reduce these specific harms.

heading: 7.1 Established harms and limitations
text: Author profiling and related tasks on detecting user characteristics based on written content have some long-established ethical issues. Among these are concerns about privacy and revealing user identity when users write about sensitive topics (Brennan et al., 2012), and also identifying characteristics that authors may want to keep private, such as their age, gender, or religion. The perspectivist argument retrieval task is a use-case which we consider to benefit users and society: providing diverse perspectives on issues and relevant arguments, which is useful for instance for online deliberation platforms where a diversity of perspectives and interactions between different groups are important. However, this task can also be used to censor, track, or harm specific groups and individual users who write the arguments.
It is also important to be aware of legal frameworks, such as the European Unions general data protection regulation (GDPR), on datasets aimed at detecting author profiles. Rangel and Rosso (2019) have described how, for 2019 PAN shared task dataset on author profiling, all legal limitations have been followed. They also state that their interpretation of GDPR Article 22 means profiling is illegal, though with an exception for noncommercial purposes and scientific research.

heading: 7.2 Approaches to protect users from harm
text: The ethical and legal issues with author profiling have triggered several approaches aimed at preventing authorship profiling for harmful contexts. One such set of tasks is known as adversarial stylometry (Brennan et al., 2012) (not to be confused with adversarial learning). This set of tasks
is specifically aimed at preventing user profiling based on style. For instance, in the subtask of authorship obfuscation the idea is to re-write the texts to such an extent that stylometric features cannot distinguish different authors or author groups anymore while leaving semantic coherence of the text intact. Successful and robust approaches across multiple models and datasets, such as by Emmery et al. (2021), use an approach of lexical substitution: changing content words strongly related to certain labels.
Other works have also looked into ethical versions of profiling tasks. For instance, Allein et al. (2023) have looked into fake news detection without author profiling: with the assumption that similar users may share similar fake news articles, they use a latent representation of a group of authors and a fake news article, without ever providing the model with direct user profile information.

heading: Acknowledgements
text: Myrthe Reuver’s contributions were funded by a Visiting Junior Researcher grant from GESIS – Leibniz Institute for the Social Sciences, and also the Rethinking News Algorithms project (grant nr 406.D1.19.073) by the Netherlands Organization of Scientific Research (NWO).
Thanks to Neele Falk, whose scripts for feature extraction7 provided a starting point for the stylistic features.

heading: A Hyperparameters and Implementation Details
text: Our models were implemented and experiments conducted using Python 3.11 unless stated otherwise.
A.1 Data Analysis
Stylistic features Stylistic features were obtained using the readability (https://pypi.org/project/readability/) and SpaCy python packages (https://pypi.org/project/spacy/; Montani et al. 2023) .
Linear Regression To select German-language arguments, we used the langdetect Python package (https://pypi.org/project/langdetect/). The linear regression was implemented in R 4.4.1. We used the packages stats, MASS, and car for the step-wise building of the regression models and inspection of variance inflation factors.
Content clustering Clustering of our SentenceBERT embeddings was done with the scikit-learn (Pedregosa et al., 2011) Python package implementation of the K-means clustering algorithm (Lloyd, 1982). We clustered for each socio-cultural variable (e.g. gender, denomination) and then also combined attributes in pairs of two to test for cluster coherence. For calculating cluster coherence, we use the Adjusted Rand score as also implemented in the scikit-learn Python package. Detailed clustering results can be found in Appendix E.
A.2 Base Model Our sentence embeddings model was paraphrasemultilingual-mpnet-base-v2 (Reimers and Gurevych, 2019), implemented through the huggingface transformers (Wolf et al., 2020) and sentence_transformers (Reimers and Gurevych, 2019) Python packages. For ranking the top documents to a query, we selected k=200 using cosine similarity.
A.3 Argument Generation Model Our generated arguments were obtained with the occiglot-7b-eu5-instruct model (https://huggingface.co/occiglot), a fine-tuned Mistral-7B (Jiang et al., 2023) model, called with Huggingface’s transformers (Wolf et al., 2020) package. The model was run on an NVIDIA A100 GPU. Prompt template details, and a short analysis of these generated arguments, are provided in Appendix C.
We memory-optimize our prompting by loading an int8-quantized version of the model. For quantization, we use the quanto library (https://github.com/huggingface/optimumquanto).
A.4 Stylistic Relevance Classification Random Forest classifier The random forest classifier for detecting socio-cultural background based on the stylistic features was implemented with the scikit-learn (Pedregosa et al., 2011) package, using its default implementation and parameters: 100 trees, splitting on gini impurity, and no max depth.

heading: B Details of the Dataset

heading: C Prompt Formulation & Generated Arguments
text: We generated arguments based on queries in the corpus with the prompt in Figure 5 and the occiglot-
7b-eu5-instruct model (implementation details in Appendix A). Below we provide a short description and analysis of the arguments generated with this method.
C.1 Statistics on Generated Arguments
Across the three test sets, the model produced an argument over 90% of the time. No argument was generated for 6.2%/1.2%/1.6% of prompts, respectively). While across the three test sets, 95% of the generated arguments are truncated, i.e. they end mid-sentence, the generated arguments are on average longer than the arguments in the corpus (mean raw text length of 190 characters for the corpus vs. 267/429/297 characters generated per test set, respectively).
C.2 Qualitative Analysis of Generated Arguments
Based on a qualitative inspection, we have gathered the following observations regarding the generated arguments. In Section C.3, we provide some examples of the generated arguments (German, translated into English with DeepL) to illustrate our analysis.
First of all, we notice an (unsurprising) tendency to repeat the demographics from the prompts, and additionally to generate intersectional types. Consider for example query 20191712, where the author of the (generated) argument identifies himself as young, man, and a student. Query 20191712 illustrates also another property, shared with query 201911545: the tendency of the generated argu-
ments to be accompanied by fully fledged, unprompted, personas, e.g. descriptions of argument ’authors’ (e.g. professions and other sociodemographics).
The arguments also vary greatly in complexity and overall quality. Some arguments, i.e., 201911569 (test 2), exhibit a very low degree of justification and/or resort to very abstract principles (“The federal government should invest more in education and research because it would advance us socially”), or essentially just rephrase the stance (201911545). Others, instead, display a very articulated structure with multiple supporting premises (e.g., four, explicitly listed in 201911652 (test 2)).
In addition to the degree of justification, arguments also vary greatly in terms of the type of evidence they use to support their stance. The speaker in 201911630 (test 2), for example, resorts to reporting personal experiences. The speaker in 201911777 (test 2), instead, mentions his profession “As an entrepreneur for many years” to establish himself as an authoritative source on the topic at issue.
C.3 Selected generated arguments
Query: “Should the federal government spend more or less on development cooperation?”
201911777-test 2 (Prompt: attribute: political spectrum; value: center and conservative/liberal), “As an entrepreneur for many years, I have seen how important it is to promote international relations and engage in global cooperation. Especially in times when job creation is a major challenge, it is crucial to place a strong focus on development cooperation. This is because we can help to improve economic conditions in certain regions and thus give more people access to a livelihood.”
201911781-test 2 (Prompt: attribute: political spectrum; value: right and liberal) “It is not the task of the federal government to invest financial
resources in the area of development cooperation. The responsibility for development cooperation lies with the countries and economies affected by poverty and underdevelopment. The federal government has no responsibility to become active in these areas, and by increasing funding it could interfere with the sovereignty of these countries. Therefore, the federal government should invest less and not more money in development cooperation.”
20191712-test 3 (Prompt: attribute: civil status; value: single) “Hello, my name is John and I’m 22 years old. I’m from Switzerland, where I’m currently studying in Zurich, and as a young man and a student at that, I firmly believe and think it’s important that the federal government spends more on development cooperation, especially as we can’t deny that”
Query: Should the federal government spend more or less on public transport?
201911630-test 2 (Prompt: attribute: civil status; value: married), “As a Swiss citizen who has been working for many years, I believe that the federal government needs to spend more on public transport. I am married myself and I have seen how difficult it can be to get to a meeting within a few minutes by bus. I’ve also seen how people can’t use the train because there are too many connections that don’t go to where they live. So we should spend more money and improve public transport.”
201911651-test 2 (Prompt: attribute: political spectrum; value: center and conservative/liberal) “The federal government should spend more on public transport. This would improve the population’s mobility options and at the same time create a more environmentally friendly way of travelling.”
201911652-test 2 (Prompt: attribute: political spectrum; value: center and liberal), “I am in favour of the federal government investing more in public transport because the advantages outweigh the disadvantages. Firstly, clean air and improved quality of life are essential for citizens, and secondly, investment in public transport improves the efficiency and profitability of the transport system. Thirdly, public transport can help to unite and motivate the population towards common goals. Fourthly, investment in public transport contributes to economic development”
Query: Should the federal government spend more or less on education and research?
201911569-test 2 (Prompt: attribute: political
spectrum; value: left and liberal) “Better more than less on education and research. In today’s globalised society, the expansion of knowledge will not only advance us technologically, but also socially. That is why the promotion of education and research is essential.”
201911571-test 2 (Prompt: attribute: political spectrum; value: center and conservative/liberal), “We should invest more money in education and research. This is the only way we can ensure that our children and grandchildren have a brilliant future. They will be the ones who have to live and work in an increasingly complex world and they need to be ready to meet these challenges, and we should also think about how we can encourage people in our communities to invest in these areas. This can be done through incentives such as tax breaks and other incentives.”
201911545-test2 (Prompt: attribute: civil status; value: single), “More education and research are crucial for the progress of our society. The federal government should therefore provide more financial support in this area. We need qualified specialists and innovations to secure our prosperity.Julia, 25, ZurichJulia (25) is a young woman from Zurich with an affinity for technology and an interest in science. As a high school graduate, she dreams of doing something "meaningful" one day.”

heading: D Regression analysis
text: In the linear regression analysis, we include the socio-cultural attributes gender, age, residence, civil status, denomination, and political spectrum. The additionally given information about important political issues is not taken into account for now, given the large number of different profiles in this attribute.
To counter sparsity of the results, in a preprocessing step all groups in the attributes that appeared rarely (< 50) were removed. This includes “Aufgelöste Partnerschaft” and “Getrennt” in civil status, as well as “Andere Kirchen/Religionsgemeinschaften”, “Christlichorthodox”, “Jüdische Gemeinschaften”, and “Islamische Gemeinschaften” in denomination. Furthermore, missing value groups (“Nicht bekannt”) in civil status, denomination, and education were removed. For better interpretability of levels of education, we summarized “Höhere Fachschule”, “Handelsschule”, and “Höhere Berufsausbildung” as “Höhere Berufsausbildung”, and “Berufs-
matura” and “Diplommittelschule” as “Berufsmatura/Diplommittelschule”. Likewise, for the sake of interpretability, the given socio-cultural attribute political spectrum was divided into quasiRILE (Laver and Budge, 1992) (an ideological scale measuring general left-to-right position; left, center, and right) and quasi-GALTAN (Bakker and Hobolt, 2013) scores (an ideological scale measuring the policy position on social issues; conservative, conservative-liberal, and liberal). The resulting dataset contains 11289 arguments.
We ran one linear regression per stylistic feature (see Table 5 for more information on the stylistic features) to estimate the relationship between sociocultural information and the particular stylistic feature. The models were built step-wise (“forward”) using the Akaike information criterion (Akaike, 1998). In particular, we looked into interactions between variables to take a step into the direction of socio-cultural profiles rather than single attributes. The formula used was DV ~ (gender + age + residence + civil_status + denomination + education + rile + galtan)^2 where DV is a placeholder for any one of the dependent variables (implementation details can be found in Appendix A). Due to the large number of variables in the resulting models, it is not possible to present them in their entirety in the paper. We selected highlighted results in the paper and make the R code available in the corresponding GitHub repository.
Table 3 illustrates the explained variance of the best model selected by StepAIC for each of the 21 stylistic features. Overall, we find small ad-
justed R2 scores, signaling that the socio-cultural variables we selected as predictors can explain a limited amount of the variance in our stylistic features. The fit of the models is however still highly significant and in our discussion we focus on significant effects. Looking more into detail into the models (the full set of regression outputs can be found on GitHub), we see significant effects across the different socio-cultural attributes and groups. While such effects may also be triggered by the large scale of the dataset, our findings inspire us to incorporate stylistic features into the retrieval models discussed in Section 4.

heading: E Clustering Results

heading: F Stylistic Features
text: Table 5 describes the different stylistic features used in the linear regression analysis and the random forest classifier.

heading: G Random Forest Feature Importance
text: Figure 6 provides an overview of the importance of different features (stylistic, stance) in the random forest classifier.

heading: Future Work
text: This is the only way we can ensure that our children and grandchildren have a brilliant future. They will be the ones who have to live and work in an increasingly complex world and they need to be ready to meet these challenges, and we should also think about how we can encourage people in our communities to invest in these areas. This can be done through incentives such as tax breaks and other incentives. ”
201911545-test2 (Prompt: attribute: civil status; value: single), “More education and research are crucial for the progress of our society. The federal government should therefore provide more financial support in this area. We need qualified specialists and innovations to secure our prosperity. Julia, 25, ZurichJulia (25) is a young woman from Zurich with an affinity for technology and an interest in science. As a high school graduate, she dreams of doing something "meaningful" one day. ”