Toeing the Party Line: Election Manifestos as a Key to Understand Political Discourse on T witter (2024)

Abstract: Political discourse on Twitter is a moving target: politicians continuously make statements about their positions. It is therefore crucial to track their discourse on social media to understand their ideological positions and goals. However, Twitter data is also challenging to work with since it is ambiguous and often dependent on social context, and consequently, recent work on political positioning has tended to focus strongly on manifestos (parties’ electoral programs) rather than social media.In this paper, we extend recently proposed methods to predict pairwise positional similarities between parties from the manifesto case to the Twitter case, using hashtags as a signal to fine-tune text representations, without the need for manual annotation. We verify the efficacy of fine-tuning and conduct a series of experiments that assess the robustness of our method for low-resource scenarios. We find that our method yields stable positionings reflective of manifesto positionings, both in scenarios with all tweets of candidates across years available and when only smaller subsets from shorter time periods are available. This indicates that it is possible to reliably analyze the relative positioning of actors without the need for manual annotation, even in the noisier context of social media.

## introduction in the domain of political science, manifestos have been commonly utilized as a means to analyze the positioning of political parties. this type of document lends itself well to the task given that it is an official, stable, and compromised textual source with clearly stated stances, ideologies and preferences @xcite @xcite . however, while understanding the positioning of political actors through manifestos remains relevant, there are also some pitfalls in reducing the analysis to this type of genre only. first, manifestos are only die linke, manifesto, 2017 a good coexistence is not without prerequisites. a society based on participation and recognition needs a social infrastructure that is committed not to private profit interests but to public needs. @xcite we see integration as a task for the entire society alternative für deutschland (afd), manifesto, @xcite the concept of a multi-cultural society has failed. in order to live in peace with immigrants in the future, integration is indispensible. it is the only way to stop the further advance of counter-societies and parallel societies in our country. published in proximity to elections (e.g., every 4 years for german federal elections), while tweets allow for continuous, fine-grained temporal analysis); second, tweets may uncover issues beyond the official position of a party as agreed on and curated in the form of manifestos @xcite ; third, and crucially, large parts of the electorate do not read manifestos released from parties, as polls suggest, foot_0 making manifestos a weak model of the public political discourse consumption. in the digital age, short-text social media platforms such as x (formerly twitter) have become an increasingly important communication channel for political actors (politicians, parties, institutions) to convey their stances on policy issues to potential voters (e.g., raising the minimum wage; closing borders to refugees). besides that, the public increasingly relies on social media for political information and opinion formation @xcite . for those reasons, automating the analysis of online political debates adds to the positioning analysis, providing a clearer understanding of the actors' stances. this is not only relevant to political scientists who monitor and investigate the political spectrum within a country, but also to citizens who then take decisions at the election time. having said that, analyzing social media data such as tweets poses new challenges for computational models. consider figure 1 (consult appendix f for sources), the left-hand column shows excerpts from the party manifestos for the far-right alternative für deutschland party (afd) (top) and the left-wing party die linke. to the right, we show four (translated) tweets by politicians in the same party. both text genres present stances towards the policy issue of migration (which are opposing views between the two parties illustrated in the example). however, the extracts from the manifestos are unambiguous and present a clear stance towards the issue, whereas the tweets leave room for interpretation which is highly dependent on social context where the meaning often goes beyond words, as shown in previous studies @xcite . for example, the second tweet requires knowledge about blue-eyed (i.e., naïve) welcome culture to fully capture the positioning of the politician. moreover, claims can be vague, such as the call for action from the third tweet. given that a large body of work on the automatic extraction of policy positions from texts focuses on manifestos or legislative speeches @xcite @xcite , and in contrast, overwhelmingly avoids social media text -a huge research area in their own right -we build on previous approaches to bridge this gap to extracting the positioning of parties from tweets. we compute sentence embeddings for tweets, leveraging the insight that politicians may use hashtags as shorthands for policy issue areas @xcite -as also shown in fig. 1 . we carry out a series of experiments based on a large corpus of tweets from german politicians and evaluate by correlating party distances extracted from manually annotated manifestos @xcite to party distances computed from tweets of party members for the election years 2017 and 2021. our experiments establish that fine-tuning with hashtags outperforms other types of fine-tuning. we complement this quantitative result with a qualitative analysis of the hashtags in our corpus. we then proceed to a set of experiments in which we explore the robustness of our method to lowresource scenarios, addressing the research question of how many tweets are needed before per-formance degrades. we address this question in three follow-up experiments: a) randomly sampling tweets, b) sampling tweets for time intervals farther from and closer to the election, and c) by sampling tweets for specific groups of politicians. our contributions are twofold: a) at the level of nlp methods, we contribute to the landscape of methods that aim at bridging between the shortterm, small-size representation on twitter and domains where longer and more curated texts are available. to the best of our knowledge, our study is the first to extract the positioning of parties through tweets in an unsupervised fashion; b) at the interdisciplinary level, we design and robustly evaluate a workflow that enables political scientists to perform theory-driven, large-scale analyses of the dynamics within the political landscape on twitter. we make our code, models, and data in the form of tweetids and the metadata necessary for replication freely available foot_1 . ## related work automatic political text scaling/positioning. to reduce the need for large-scale annotation, several methods of automatic scaling and positioning of political actors have been proposed. the first automatic approaches to extract policy positions from text relied only on word count information @xcite @xcite , which are not applicable in cross-lingual settings and do not capture semantic relationships between the words in the texts. then, @xcite and @xcite improve the performance of the task with word embeddingbased scaling method combined with graph-based score propagation. @xcite propose to use party embeddings retrieved from word embeddings augmented with political metadata trained on parliamentary corpora and pca to retrieve party positions. building on the advances of language models, @xcite al. (2022, 2023 find that the content in terms of hashtags and the emotional tone of the communication of candidates is more aligned with european issues than left-toright positions. this indicates that social media postings contain information on policy, contrary to what prior work suggests @xcite . similarly, @xcite , @xcite , and sältzer and stier (2022) use wordfish/word-shoal to assess inter-and intra-party positions for social media postings of politicians before elections. the former two validate that, in tendency, social media postings reproduce inter and intra-party positions. while most studies on social media analyze tweets from the public, a few investigate the discourse of politicians, such as analyzing hashtag and work usages @xcite , detecting stances @xcite , classifying whether individual tweets are ideological @xcite , and classifying the sentiment @xcite of politicians' tweets. however, in contrast with our work, they do not address the task of identifying the positioning of political actors with unsupervised or weakly-supervised methods. ## data twitter corpus we conduct our study on a corpus containing tweets by candidates of three german federal elections @xcite @xcite . this corpus is a pooled version of three collection periods for election 2013 foot_2 , 2017 foot_3 , and 2021 foot_4 , and in-between election tweets collected by @xcite , starting from 2008. the linked dois have published tweet ids. in total, the corpus contains over 10m tweets. for data sparsity reasons in the earlier years, we only use data from the years 2013-2021 in this work overall, using data from the years 2017 and 2021 for the evaluation step. ## method our proposed method aims to gather meaningful representations and, based on those, ideological positionings of parties for the computational analysis of the political twitter domain. following insights on how policy is communicated on twitter @xcite , we propose a two-step workflow. as figure 2 illustrates, we first gather text genre-specific sentence embeddings, as detailed in section 4.1. in the second step, we first gather topic-specific distances, which are then aggregated to gather overall party distances, as described in section 4.2. ## evaluation while we expect positions on twitter to be more rich and variable on an individual politician level, party members can be expected to unite behind the party line, especially in election campaigns, as candidates have been found to consider manifestos to be important documents for their campaigning @xcite . we thus turn to manifestos as a ground truth for evaluation. ground truth: the cmp distance matrix we use the labels annotated in the cmp to capture the parties' positioning within manifestos. our ground truth is based on all fine-grained categories from the cmp codebook because it provides a full picture of the political preferences of each party rather than picking some policy issues that belong to the left or right, for example. it is also more in line with our task because we are measuring the positioning of parties across multiple policy issues. we calculate the positioning of the parties based on the salience of the annotated fine-grained categories from the codebook of the cmp corpus. each party is represented by a vector that considers the number of counts for each of the 143 categories normalized by the length of the manifestos. a distance matrix (r n xn ), as illustrated in figure 6 in the appendix, is computed based on the euclidean distance between each pair of the n party vectors. evaluation metric building on previous work @xcite , to evaluate the resulting distance matrices, we use the pearson-normalized mantel test @xcite . this type of metric allows us to correlate distance matrices given that the observations are not independent of each other. ## experiment 1: comparing representation models in this first experiment, we assess whether distances between parties reflect manifesto distances. moreover, we compare sentence encoder models trained on different levels of genre information in our distance aggregation workflow to assess the impact of genre-specific information in the representations (political tweets vs. manifestos). specifically, we compare our political twitter-specific model to more general models and models fine-tuned in the political domain, but with a distinct type of text genre. we assess the impact of our distance definition by comparing its correlation with manifesto distances to a less informed aggregation of distances. ## experiment 2: random subsampling following the insights from our first experiment, we assess how stable the positioning retrieved using the sbert hashtag models in our proposed workflow are by randomly subsampling tweets. in doing so, we simulate lower-resource scenarios of regional elections and political landscapes of countries with considerably fewer tweets available. our question is how low the number of tweets can be before the performance degrades considerably. ## experiment 3: temporal sampling the discourse on twitter can be reasonably expected to be more driven by short-term political topics and idiosyncratic events in some time periods than in others. as our method relies on a representative, broad topical distribution, we expect this in this experiment, we thus assess how finegrained the periods are for which we can make predictions using our method. ## experiment 4: group-based sampling in this experiment, we assess the impact of selecting different groups of politicians when sampling tweets. we do this to investigate the stability of positioning with respect to tweets of only a subset of politicians being sampled. at the same time, this exemplifies a use case for our method: comparing how well different groups are aligned with the manifesto (i.e., the main party line). ## analysis in our experiments, our focus has been at an aggregate level, predicting distances between parties. this raises the question of what our fine-tuned models capture at a lower level. to validate that our genre-tuned representations reflect ideological differences between members of different parties, we analyze the centroids of individual politicians' tweets from 2021 by projecting them into two dimensions using t-sne @xcite . as shown in figure 5 , there are regions in the projection where politicians of specific parties tend to cluster. moreover, parties that ideologically are closer together at least in tendency are in each other's vicinity. for instance, the greens' (grüne) and the left's (die linke), as well as afd's and fdp's most predominant clusters are close to each other, respectively (cf. figure 6 ). while there is no clear clustering separating the parties, we argue that this is expected, as (a) different politicians may have different foci and tweeting behaviors, (b) there is expected overlap between politicians of different parties, either by a tendency to talk about the same topics or by actual overlap in their expressed views. ## conclusion in this paper, we propose and robustly evaluate a method to create representations for politicians' tweets which, relying on hashtag as a fine-tuning signal, proved successful in matching politicians' statements to the official lines of the respective parties, as encoded in the electoral manifestos. furthermore, we have established the robustness of this method to low-resource scenarios via random sub-sampling, as well as with the sampling of time slices and groups of politicians. such robustness is of particular relevance for interdisciplinary applications -given that low-resource scenarios are a typical situation in the social sciences, making robustness a crucial desideratum. this workflow facilitates, for instance, the immediate tracking of shifts in positioning among politicians and parties following data collection. moreover, given that it is increasingly important to follow how much politicians align or diverge their parties because it helps in holding them accountable or monitoring how extreme their positioning can become, this work is one step forward towards automatically analyzing political discourse on social media. a natural avenue for future research is to add interpretability to the approach, which would permit us to understand qualitatively where politicians align or diverge the most. finally, we note that the models we propose in this paper assume that the social media messages use hashtags. while the role of hashtags in political communication on twitter is well-established both theoretically and empirically, this begs the question of what to do when there are no hashtags available, either when the authors decide against using them or when we want to move to a social media platform where hashtags are not in general use. of course, it is possible to carry out distance aggregation with a distance measure that does not depend on hashtags. appendix h presents a pilot study on such a measure, which indicates that such an approach is in principle possible, but that ignoring the strong signal from hashtags comes at a substantial cost in performance. more research is necessary to define an improved distance measure.