Additive manifesto decomposition: A policy domain aware method for understanding party positioning
Year: 2023

Abstract:
Automatic extraction of party (dis)similarities from texts such as party election manifestos or parliamentary speeches plays an increasing role in computational political science. However, existing approaches are fundamentally limited to targeting only global party (dis)similarity: they condense the relationship between a pair of parties into a single figure, their similarity. In aggregating over all policy domains (e.g., health or foreign policy), they do not provide any qualitative insights into which domains parties agree or disagree on. This paper proposes a workflow for estimating policy domain aware party similarity that overcomes this limitation. The workflow covers (a) definition of suitable policy domains; (b) automatic labeling of domains, if no manual labels are available; (c) computation of domainlevel similarities and aggregation at a global level; (d) extraction of interpretable party positions on major policy axes via multidimensional scaling. We evaluate our workflow on manifestos from the German federal elections. We find that our method (a) yields high correlation when predicting party similarity at a global level and (b) provides accurate party-specific positions, even with automatically labelled policy domains.

heading: 1 Introduction
text: Party competition is a fundamental process in democracies. It provides space for different political stances to emerge, allowing people to choose which of them they most identify with. Investigating this process is relevant for understanding the reasons behind the choice of voters in elections as well as the behavior of parties in policy decisionmaking once in power (Benoit and Laver, 2006).
Within political science, the positioning of parties is investigated under the umbrella term of “party competition”. Some studies look at specific policies such as “welcoming refugees”, others, at
broader domains such as “economy”. Traditionally, the positioning of parties within these policies or domains is scaled down to a reduced number of political dimensions such as the well-established left-right or the libertarian-authoritarian axes in order to facilitate the comparison among parties and their ideologies (Heywood, 2021). Analyses are usually carried out by experts, who gather policy and ideological stances of members of the political parties in several countries in Europe and beyond (Jolly et al., 2022). Alternatively, electoral programs are manually annotated following a specific codebook that takes into account the position of the parties on policies so that the salience of the labels can be scrutinised (Burst et al., 2021).
Recently, computational approaches have been developed to automate and scale up party position analysis to larger amounts of text (Slapin and Proksch, 2008; Däubler and Benoit, 2021; Ceron et al., 2022). This development has the potential of alleviating the burden of annotation, but has so far been realised only at an aggregated level: party positions are projected on the left-right scale or on a distance-based approach between party pairs according to several policies, not providing insights at the level of policy domains. This requires political scientists either to manually check for sections of the text of their interest in case the objective is to understand the positioning of parties on a more fine-grained level or to make assumptions about a policy considering the entire document.
In this paper, we extend the previous studies to provide a computational model for party positions and party similarity at the level of policy domains. To do so, we semi-automatically decompose the texts into interpretable thematic blocks based on an updated inventory of annotated labels from the Comparative Manifesto Project (CMP). Sentence embeddings leverage well the grouping of finergrained categories into these blocks, which we call policy domain from now on. Then, they are used to
7874
compute pairwise policy differences between parties. The results show that this re-grouping of categories into higher policy domains performs well not only at an aggregate level in comparison with the ground truths, but that they also match the positioning of parties within the political dimensions at the individual level of policy domains.
Besides shedding light on the positioning of parties regarding where they most (dis)agree, we also avoid relying on the salience (i.e., frequency) of the categories. This assumption is implicit in many existing party positioning models including our own prior work (Ceron et al., 2022) and is motivated on the grounds that major domains, such as economic and social policy, should play a more prominent role. At the same time, there is strong evidence that voters re-weigh domains by their priorities (Iversen, 1994). We take this as evidence that models would benefit from focusing on modeling within-domain similarities and differences between parties.
We evaluate the extent to which annotations can be forgone by evaluating several classifiers to automatically predict the policy domains of the 2021 German federal elections based on annotated manifestos from previous elections. Comparing the party positioning given by the manually annotated and the predicted labels, we find that the classifier can substitute annotations at an aggregate level and also in most policy domains, allowing new, unannotated documents to be analysed automatically. We make our code freely available.1

heading: 2 Related Work
text: The Comparative Manifesto Project. Party manifestos, also known as electoral programs or party platforms, condense parties’ ideologies and
1https://github.com/tceron/additive_manifesto_ decomposition
stances towards various policies (Budge, 2003). The Comparative Manifesto Project2 annotates manifestos from multiple countries around the world following a codebook that takes into account the positioning of parties according to the left-right political dimension (Budge et al., 2001). The codebook contains 143 fine-grained categories. Table 1 shows some examples. The categories are labelled according to policies and may or may not contain the stance towards the policy as well. For example, there are two labels for Military: Military: Positive and Military: Negative, but there is only one category for Peace because no party is against it. In most cases, the annotations are assigned to every sentence of the manifesto, however, sentences are split into smaller parts whenever there is more than one self-contained category.
Computational models of party positioning. Party manifestos, which provide a particularly rich source of information on parties’ positions, have been extensively used in computational political science. In the pre-neural era, they mainly focused on word/token distributions to position parties along a scale; thus, the Wordscore approach used the distributions extracted from reference texts to determine party positioning of new texts (Laver et al., 2003). Slapin and Proksch (2008) focus on overcoming the disadvantageous dependence on reference texts which assumes that political discourse does not change significantly over time and that the reference corpus always contains good representations of extreme policy positions.
Arguably, the adoption of (static) word embeddings such as Word2Vec (Mikolov et al., 2013) instead of word distributions constituted a step forward for computational models of party positioning. For example, Glavaš et al. (2017) take advantage of
2https://manifesto-project.wzb.eu/
the possibility to align word embeddings across languages to present a multilingual model for extracting party positions from speeches of the European parliament. Rheault and Cochrane (2020) exploit another property of embedding spaces, namely the information on graded word similarity implicit in them. They build combined representations from word embeddings and political metadata and then estimate the positions of different parties through dimensionality reduction. The embeddings are reduced to two dimensions and their projection in the space shows the alignment of parties from Britain, Canada, and the US on a left-right axis.
The recent shift from static word embeddings to contextualized embeddings was a second important step. Contextualized embedding models, like BERT (Devlin et al., 2019), are not only able to pick up on corpus-specific usage of words, but can also be fine-tuned for specific tasks, which greatly improves the quality of the representations. In previous work (Ceron et al., 2022), we predicted global party similarity using Sentence-BERT (SBERT, Reimers and Gurevych 2019), a model for the task of sentence-similarity prediction. It uses a Siamese network with a triplet loss function that aims at placing mutually similar sentences close to one another in embedding space and pushing dissimilar ones apart. We found that SBERT representations can profit substantially from tuning by party, forcing the model to place sentences from the same party closely together in the semantic space.
Architectures similar to SBERT with modifications in the loss function have followed such as different types of contrastive and non-contrastive self-supervised learning (Gao et al., 2021) and normalization techniques in the distribution through an unsupervised objective during training (Li et al., 2020). The original SBERT architecture, though, remains the most widely used and numerous pretrained models, including multilingual ones, have been made publicly available (Ceron et al., 2022).
Despite these successes, the computational studies mentioned above have not proposed a general way of capturing the positioning of parties within specific policy domains, opting for narrowly applicable ad-hoc modifications of existing algorithms. For example, Laver et al. (2003) adapt their reference values (related to the word distribution) to few chosen domains, and Slapin and Proksch (2008) manually identify sections of the manifestos that discuss economic issues.

heading: 3 Methodology

heading: 3.1 Workflow
text: The goal of the additive manifesto decomposition method we propose is to computationally analyse the positioning of parties both at the level of policy domains and at an aggregated level of information. Figure 1 illustrates the four steps in which we decompose this analysis: (1), we define policy domains (visualized as colors). This is discussed in Section 3.2. (2), we label manifestos with the policy domains. Unless manual annotation is available, this involves training a policy domain labeller. This is discussed in Section 3.3. (3), we represent parties’ positions on policy domains by vectors and compute the similarities between these vectors, which can later be aggregated to obtain global similarities. This is discussed in Section 3.4. Finally, (4), we apply a dimensionality reduction technique to the parties’ policy domain distance matrix to be able to inspect their positions.
We apply the methods that we propose to corpora from the Comparative Manifesto Project (CMP, cf. Section 2) and use examples from the CMP below for illustration. However, we believe that the CMP is fairly typical regarding size and annotation granularity for resources in computational political science. We are confident that our methods generalize to other corpora.

heading: 3.2 Policy Domain Grouping
text: Given that the objective is to understand where parties (dis)agree the most according to the way they expose their stances and ideologies in the manifestos rather than on the salience of mentions of a policy, we first have to decompose the manifestos into interpretable thematic blocks, which we identify as policy domains. Policy domains are in principle freely definable in an inductive fashion (Wald-
herr et al., 2019) but must fulfil three requirements to be useful: (1) Domains must be coherent and interpretable
in the context of policies to support the goal of understanding in which domains parties are most similar and dissimilar (2) Domains must be neutral with regard to stance. In other words, the categories with opposite stances (positive and negative) vis-a-vis a certain problem (e.g., immigration) should belong to the same policy domain. (3) Domains must be located at the right level of granularity: they must be detailed enough to be informative (cf. (1)), but not so detailed that accurate classification becomes impossible in practice. For example, the original CMP categories are arguably too fine-grained (such as the examples in Table 1). We propose that a reasonable granularity for party positioning can typically be achieved by clustering fine-grained category annotations from sources such as the CMP codebook.
To do so, we represent the texts through sentence embeddings as state-of-the-art representations (cf. Section 2). This already enables us to compute cosine distances between all pairs of sentences belonging to two categories and use their average as a distance measure of topical coherence between two given categories. Formally, given a set of sentences {s1, s2, . . . , sn} and a disjoint collection of categories {C1, C2, . . . , Ck}, for each category pair (Cp, Cq), we compute
dist(Cp, Cq) = 1
N
∑
i∈Cp,j∈Cq 1− cosine(si, sj)
where N is the number of sentence pairs. The resulting distance matrix between low-level CMP categories can then serve as input for an average-linkage hierarchical-clustering algorithm, which produces a tree of categories, from which a suitable level of abstraction can be selected that meets the requirements laid out above. Inspection of candidate policy domains is also adopted as a sanity check for the sentence embedding model.

heading: 3.3 Policy Domain Prediction
text: For texts without policy domain annotation, we predict policy domains for all sentences using existing annotated corpora as training data. Technically, this is a labeling task where each token is a sentence (or segment thereof) which can be solved by any
state-of-the-art classifier architecture. It has two main challenges. The first one is the high contextual dependence on political discourse. As a result, the classification of individual sentences is often challenging. For example, a vague formulation, such as There is still a lot to do, must take into account based on the category of the previous sentence, a possibility explicitly acknowledged by the CMP codebook. This clearly indicates that it is sensible to approach domain prediction as a sequence labeling task.
The second challenge is that training and test data are always bound to be “out of domain”, since they will differ in either country or time: we either need to project from past elections to new ones, or across countries, and thus political cultures. Since both of these settings can introduce strong concept drift, this makes the task an example of out-ofdomain prediction.
The end result of policy domain prediction is then a decomposition of a party manifesto p into a disjoint collection of k policy domains {Dp1, Dp2, . . . , Dpk}. Note that the set of sentences associated with any domain may be empty.

heading: 3.4 Computing Party (Dis)similarities
text: After decomposing the sentences of manifestos into policy domains, we compute the similarity between parties by domain. We re-use the simple coherence measure from the policy domain grouping (cf. Section 3.2). Again, this involves choosing a sentence embedding model, a parameter of our method. Given two parties’ manifestos p and q, we interpret dist(Dpi , D q i ), the average pairwise distance among sentences for policy i as the distance between parties p and q for this domain.
To obtain an aggregated party distance, we simply average the distances of all policy domains. As argued in Section 1, this removes the effect of domain salience from the model and arguably obtains the clearest party positioning as perceived by a “neutral” voter (Iversen, 1994).

heading: 3.5 Multidimensional Scaling
text: The results of the previous step can be represented as a square matrix of the distances between every party pair. In order to enable a more qualitative analysis of the results by policy domain, we apply a multi-dimensional scaling (MDS) technique which maps a distance matrix onto a one-dimensional scale while respecting the distances as well as possible. MDS models are well established for visual-
ization in political science (Rheault and Cochrane, 2020; Heywood, 2021). We utilize Principal Component Analysis is chosen because the first component explains well the variability in the data.

heading: 4 Experimental Setup

heading: 4.1 Data
text: We analyze the positions of the six German parties which obtained parliamentary seats in 2021 based on their 2021 federal election manifestos. These are Die Linke, Bündnis 90/Die Grünen, Christian Democratic Union (CDU), Free Democratic Party (FDP), Social Democratic Party for Germany (SPD), and Alternative for Germany (AfD).
We train a policy domain labelled for these manifestos based on the annotated data provided by the CMP. We experiment with two training sets: DEtrain contains only manifestos from Germany dating from 2002 to 2017. The second instead, DACHtrain consists of manifestos from the majority German-speaking countries (Germany, Austria, and Switzerland) for all elections from 2002 to 2019. This allows us to understand whether the classifier benefits more from focused data of a single country (the country of interest for the analysis) or if the raw amount of data is more relevant. Appendix A provides details on data statistics.

heading: 4.2 Policy Domain Grouping
text: To define our policy domains, we concatenate the manifestos of six German major political parties from the 2021 elections, together with their CMP annotations, into a single corpus. It contains a total of 69 annotated categories, however, only the ones with 10 occurrences or more are included in the grouping - a total of 61. We employ multilingual-mpnet-base-v2, the vanilla SBERT model to compute similarities3 in order to make the clustering more general. It is a vanilla multi-lingual model with the base-size version of XLM-RoBERTa (Conneau et al., 2020) as the encoder trained on more than 50 languages.4
Representations from the multilingual SBERT model are post-processed with whitening transformation (Su et al., 2021), as suggested by experiments finding that more isotropic embeddings
3Provided by HuggingFace as a part of sentence-transformers collection.
4https://huggingface.co/sentence-transformers/ paraphrase-multilingual-mpnet-base-v2
capture political text similarity substantially better (Ceron et al., 2022).
Hierarchical agglomerative clustering led to a clustering that consistently grouped thematically close categories with opposite valences into single domains, as shown in Fig. 3 in Appendix B. In the inspection of the clustering tree, we verify that all 10 categories that contained positive and negative labels fall in the same cluster in order to satisfy requirement 2. We then selected the tightest possible clusters of categories that together formed coherent policy domains (fulfilling requirements 1 and 3). The remaining 8 categories (that were not included in the clustering) are added to the formed clusters manually. We consulted with political scientists and related work (Benoit and Laver, 2006; Jolly et al., 2022) to verify the result. The full list of CMP categories falling into each of our issues is presented in Appendix B.

heading: 4.3 Policy Domain Labelling
text: As stated above in Section 3.3, domain labels in manifestos are context-dependent. Therefore, we give up the assumption of previous analyses of manifestos (Däubler and Benoit, 2021) that annotated sentences are independent units of information. Instead, we treat policy domain labelling as a sequence labelling task. Our preliminary experiments showed that incorporating sequence information is indeed beneficial for prediction quality, and we chose a simple “bigram”-based model: pairs of subsequent sentences from manifestos were concatenated, and the model was tasked with predicting the label of the second one.5
We use averaged token embeddings from xlm-roberta-large and pooled representations from the multilingual version of mpnet-base-v2 fine-tuned on paraphrase detection as sentence-pair embeddings6 as encoded representations and use a two-layer MLP with tanh activation as the classification head. The system is then trained end-to-end for two epochs. As a first baseline, we choose the majority baseline between the 14 categories (13 policy domains in addition to the category “Other” which does not belong to any domain). The second baseline instead follows the same bi-gram idea
5I.e. we are not using the label of the first sentence. Using it could help with training but may lead to increased variance on new data where an incorrect label for a sentence would then bias the prediction for the next one.
6xlm-roberta-large is nearly twice as big as the sentence transformer but benefited from less sentence-focused training.
in terms of input and is logistic regression fed with the representation taken from frozen SBERT mpnet-base-v2.

heading: 4.4 Party (dis)similarity – sentence encoders
text: We experiment with four different sentence encoding models when computing party similarities (as explained in Section 3.4). Our baseline is FastText for German based on character n-gram embeddings (Bojanowski et al., 2017).7 The second model is a base-sized cased version of BERT trained on German data, a monolingual Transformer-based model. The representation of a given sentence from these models is an average of its token embeddings. Then, as end-to-end sentence encoders we use two versions of SBERT. The first is the vanilla SBERT pre-trained model multilingual-mpnet-base-v2. The second is SBERTdomain, a pre-trained model from our prior work (Ceron et al., 2022), which we fine-tuned on German CMP data from before 2019 to distinguish between 6 higher-level domains from the CMP codebook.
Our preliminary experiments showed that applying post-processing with whitening improves all models. Therefore, all sentence representations in this step are whitened as in Section 4.2.

heading: 4.5 Evaluation

heading: 4.5.1 Ground Truth
text: We evaluate our additive manifesto decomposition method against two sources of ground truth.
RILE index. The RILE index is a widely used way of computing the positioning of parties on certain policy domains or in aggregate. Laver and Budge (1992) selected 12 categories from the CMP codebook as left-leaning and 12 others as right-learning.8 The score is then computed as RILE = (R − L)/N , where R and L are counts of sentences from the right and left categories, respectively. Dividing by N , the manifesto length, results in a normalized score between -1 and 1.
As our approach returns a distance matrix, we need to use dimensionality reduction to obtain a single estimate per party. For this purpose, we extract the first axis of the classical MDS algorithm
7Pre-trained model downloaded from fasttext.cc 8The table of categories can be found at
https://manifesto-project.wzb.eu/down/tutorials/ main-dataset.html
applied to distance matrices – corresponding to the first principal component in PCA analysis.
CMP-category salience. Given that the RILE index makes use of only 24 out of the 143 categories from the CMP codebook, we used another type of ground truth that takes into account all categories and corresponds to the traditional political science approach of comparing domain saliences, i.e. relative prominences of different policy categories in manifestos (Budge et al., 2001). Each party is represented as a vector of relative frequencies of categories normalized by the manifesto length. Euclidean distances between these representations are then used to create a party distance matrix.

heading: 4.6 Evaluation Metrics
text: We evaluate the results of the first principal component analysis against the RILE score with Pearson correlation in order to understand the extent to which our models capture the aggregated left-right dimension of the political spectrum through textual similarity. For checking how well our method captures the more nuanced method of measuring partyplatform dissimilarities from category saliences, we use the Mantel test (Mantel, 1967). For both metrics, both by-domain and aggregate agreement scores can be computed.
For experiments with unannotated manifestos, we predict the policy domain labels using the bestperforming classifier and then repeat the evaluation in the same way using the predicted labels.

heading: 5 Results and Discussion

heading: 5.1 Annotated Setup
text: In the annotated setup, we use the ground truth of policy domains as annotated in the CMP dataset. We evaluate party-positioning landscape extracted using our method, both in aggregate and for different policy domains, against the ground truths: the RILE scores and the distances computed using CMP-category saliences.
Aggregated similarity. Table 2 illustrates the correlation of the aggregated similarity computation with the ground truths. Correlations are very high in both ground truths with small differences across models. FastText, our baseline, performs best in predicting the Rile index (Mantel r = 0.94) and second in the CMP distance (r = 0.80). We believe that the excellent performance of this model is
given due to the similarity computation. The comparison between sentences from the same policy domain (theme) might help in capturing fine-grained differences in stances between parties. BERTGerman is the model that performs the worst even though for a slim difference – as previous research suggested, the quality of BERT for sentence representation is low (Li et al., 2020). Finally, SBERTvanilla and SBERTdomain have comparable results. While the former performed the best on RILE (r = 0.91) in comparison with the latter (r = 0.87), the latter comes out first in the CMP distances (r = 0.84 vs. 0.80). This suggests that the non-fine-tuned model can still excel in the task of text similarity on out-of-domain data. Depending on the purpose, however, the fine-tuned version might be a better option, in line with previous results on representing political text (Ceron et al., 2022).
Similarity by policy domains. We further analyze the output of the best model, namely SBERTdomain. Figure 2 shows the results of the application of MDS to the policy domain distance matrices. On the left-handed side of the plot lies the name of policy domains and on the right-handed side the Pearson’s r with respect to the RILE score.
The higher the (absolute value of the) correlation coefficient, the more the scale in question follows the classic left-right scale as measured by RILE. As expected, some policy domains yield high correlation whereas others do not. Importantly, this is not a measure of model quality. Rather, as it has often been observed in the political-science literature, the left-right scale cannot explain the complete picture of party positioning (Heywood, 2021). Therefore, quantitative analysis has to be complemented by
qualitative judgments about the appropriateness of the predictions.
Indeed, the results mirror some well-known facts about German politics. For example, in foreign relations, EU and protectionism – which is only moderately correlated with the left-right scale at r = 0.47 – the AfD is an outlier compared to other parties, arguably because it is against being part of the European Union and has a different stance with regard to having ties with Russia as compared with the other parties, which all fall in the same region. Another case is education and technology where AfD and Die Linke, who are generally can be regarded as the opposite pole of the left-right spectrum, happen to share a lot of common ground in their stance toward the expansion of education and investment in technology and infrastructure (r = -0.38). On the other hand, in policy domains such as military and peace and immigration and multiculturalism, party positions align very well with the overall left-right scale (r > 0.85), with right-leaning parties being more militaristic and immigration averse.
In sum, we take the results of this analysis as evidence that our workflow produces accurate finegrained characterizations of party positions.

heading: 5.2 Predicted Setup
text: In the predicted setup, we do not use the CMP annotations of policy domains but predict the policy domains instead.
Policy domain labeller. Table 3 shows the accuracy of the models and the majority baseline on the test set. Overall, the larger but more varied training set including all German-speaking countries (DACHtrain) performs better than DEtrain (data only from Germany) in all models, suggesting that it is not necessary to exclusively have data from the same country of analysis – given the similarity in the political scenario. As expected, the SBERTfrozen which is not fine-tuned for the task, performed the worst (55.3% and 56.7%). Whereas SBERT+MLP performed second (60.4% and 63.1%) and the best model is XLM-RoBERTalarge+MLP (62.5% and 64.5%), whose bigger size likely won over additional pretraining of a smaller model. The results of the XLM-RoBERTa-large model fine-tuned on DACHtrain are used for the rest of this analysis.
Aggregated similarity. We evaluate how the predictions of our policy domain labeller perform in a scenario where there are new upcoming elections and no annotations are available. Table 2 shows that even though even results are not as incisive as in the annotated scenario, the correlation scores are still high for CMP saliences. In terms of models, SBERTdomain is the best-performing model (Mantel r = 0.80), similarly to the annotated scenario SBERTvanilla is the worst performing encoder (r = 0.71), with FastText (r = 0.76) and BERTGerman (r = 0.79) in between. As for the RILE score, only SBERTdomain demonstrates a statistically significant correlation. These results confirm that the additive manifesto decomposition is dependent on the precision of the policy domains labels but can also provide interpretable results for unannotated data.
Similarity by policy domains. Our sources of ground truth do not provide us with gold measures of the similarity within each policy domain. Therefore, we cannot directly evaluate by-domain matrices produced with the predicted data. However, we can indirectly evaluate their usefulness by comparing them to the matrices produced using the gold annotations, which we already know to be highly meaningful.
Table 4 shows the Mantel correlations between the distance matrices produced with the annotated setup and the one from the predicted setup for each policy domain. Mantel correlation is 0.78 or higher in 10 out of 13 policy domains. Negative outliers are culture and civic mindedness, political authority and labour groups and welfare state. We further investigate whether there is a correlation between the number of correctly labelled sentences by classifier (measured by accuracy) and Mantel correlation of the results. We find that there is a relatively strong correlation (Pearson r = 0.59, p = 0.03).
This suggests that one can predict which policy domains will yield the most faithful results in an unsupervised scenario on the basis of their accuracy in the policy domain labeling part of the workflow.

heading: 6 Conclusion
text: In our first contribution, we introduce Additive Manifesto Decomposition, a workflow for efficient analysis of party platforms, both in aggregate and across a range of policy issues. It builds on stateof-the-art sentence-representation models, which it uses for three operations on policy domains: definition, prediction, and (cross-party) similarity computation. In this manner, our workflow can incorporate advances on the representational level (Reimers and Gurevych, 2019; Ceron et al., 2022) but complements them with a crucial level of reflection and analysis at the informative level of policy domains.
Our second contribution is a study of the political landscape in Germany using our workflow. The results we obtain match well with expert judgements, suggesting that our workflow yields a reliable technique to automatically study the similarity between parties across policy domains. In addition to analysing the implicit stance space, operationalized through distance matrices derived from text similarity, we show that our method makes it possible to recover the traditional scaling analyses of the political science literature: we can efficiently approximate the aggregate RILE (right-left) scores provided by experts in the aggregate settings, and when proceeding by domain, we see that our methods recover non-trivial policy configurations, e.g., the agreement of the far-right and far-left parties in Germany on the subject of EU and the expansion of education. Moreover, we show that classifiers substitute the annotations of these high-level domains and still yield similar results as compared to the fully annotated scenario.
Germany provided an appropriate target for our case study, given both the large number of annotated manifestos and large body of expert analyses. Nevertheless, an important direction for future work is testing the applicability of our workflow to other countries, in particular regarding the training of policy domain labelers given the challenging concept drift between elections, and the possible cross-lingual application of our model components despite differences between political cultures (Braun and Schmitt, 2020).
Lastly, our methodology does not only suit the identification of the positioning in the political domain, but more broadly it can be seen as a different way of identifying the stance of an entity (person, organization, group). It can be applied whenever there is some aggregation of texts with regard to a set of entities. The distinction lies in the more fine-grained identification of stances: we (a) take larger chunks of text as input and (b) position the entities on a scale rather than characterizing them as in favor, neutral or against a given topic.

heading: 7 Limitations
text: The main limitation of the proposed study is the relatively small scale of the dataset it is based on. The proposed method is scalable and computationally undemanding (all of the analyzed models can be trained on a single GPU with 12G of memory), and it is feasible to apply it to other countries in the CMP dataset. However, in order to arrive at interpretable results that could be verified in terms of policy substance based on the experts’ knowledge of the political spectrum, we had to focus the evaluation part on the materials of a single election cycle in one country. Potentially, the method can be applied to any country whose manifestos have CMP annotations, however, further investigation with data from other countries needs to be carried out to verify that.
While most policies are recurrent in manifestos, there may be a few topics appearing in upcoming elections, adding some variability in debate across election years. The policy domain labeller might need to be updated every now and then with current topics of interest (e.g. Covid, a sudden expansion of the military). Therefore, the effect of news electoral programs in the classification step requires more investigation namely, the feasibility of further training with new topics of the current debate or the necessity to re-train the whole classifier with new manifestos over again. That being said, the CMP codebook has remained the same for over two decades now. We take this as evidence that the policy domains do not need to change, only the ability of the classifier to correctly identify sentences with unseen topics.

heading: Acknowledgements
text: We are thankful for the insights on policy and party positioning contributed by Nils Düpont, Sebastian Haunss and Nico Blokker. We acknowledge fund-
ing by Deutsche Forschungsgemeinschaft (DFG) for project MARDY 2 (375875969) within the priority program RATIO.

heading: A Data Statistics and Handling
text: A.1 Data for the party positioning analysis
A.2 Data for training the policy domain classifiers
Preprocessing. The CMP annotations contain the H and 0 labels for some sentences. While Hs are excluded from all the modelling because they represent the heading of a section. The 0 label is kept for the classifier in order to emulate a real world case scenario where there are labels that do not represent any policy domain/category.
The “Germany” training regime with manifestos from Germany only contains 57,259 instances whereas the “German” regime with data from German-speaking countries has 106,724 instances in total. 10% of each of them is used as the validation set.
A.3 Models’ hyperparameters and libraries SBERTfrozen+Logistic Regression:
• No hyperparameter optimization for the logistic regression model - default parameters from the library Scikit-learn • Frozen SBERT model: paraphrase-multilingual-mpnet-base-v2
RoBERTaxlm + Multi-layer perception (MLP): • RoBERTa model: xlm-roberta-large • First linear layer’s input size: RNx1024 • One tahn activation layer • Second linear layer’s input size: RNx14 • 5 epochs • AdamW optimizer (Loshchilov and Hutter, 2019) • Learning rate: 10−5
• HuggingFace for implementation
SBERTtune + Multi-layer perception (MLP): • SBERT model: paraphrase-multilingual-mpnet-base-v2 • First linear layer’s input size: RNx768 • One tahn activation layer • Second linear layer’s input size: RNx14 • 5 epochs • AdamW optimizer (Loshchilov and Hutter, 2019) • Learning rate: 10−5 • SBERT HuggingFace for implementation Hardware information for all experiments: • System CPU: 2 x Intel Xeon E5-2650 v4, 2,20GHz, 12 Core • 24 cores • 256 GByte of memory • GPU: 4 x Nvidia GeForce GTX 1080 Ti, 12 GB

heading: B Appendix
text: B.1 Hierarchical clustering with CMP categories
B.2 CMP categories clustered across Germany, Switzerland, and Austria
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Section 7 (Limitations)
7 A2. Did you discuss any potential risks of your work? Because there are no risks concerning this work, to the best of our knowledge.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract and section 1 (Introduction)
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? Sections 3 (Methodology) and 4 (Experimental Setup)
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 4 (Experimental Setup)
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4 (Experimental Setup)
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Sections 3 (Methodology) and 4 (Experimental Setup) and in the Appendix.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Nevertheless, an important direction for future work is testing the applicability of our workflow to other countries, in particular regarding the training of policy domain labelers given the challenging concept drift between elections, and the possible cross-lingual application of our model components despite differences between political cultures (Braun and Schmitt, 2020). Lastly, our methodology does not only suit the identification of the positioning in the political domain, but more broadly it can be seen as a different way of identifying the stance of an entity (person, organization, group). It can be applied whenever there is some aggregation of texts with regard to a set of entities. The distinction lies in the more fine-grained identification of stances: we (a) take larger chunks of text as input and (b) position the entities on a scale rather than characterizing them as in favor, neutral or against a given topic