Come hither or go away? Recognising pre-electoral coalition signals in the news (2021)

Abstract: In this paper, we introduce the task of political coalition signal prediction from text, that is, the task of recognizing from the news coverage leading up to an election the (un)willingness of political parties to form a government coalition. We decompose our problem into two related, but distinct tasks: (i) predicting whether a reported statement from a politician or a journalist refers to a potential coalition and (ii) predicting the polarity of the signal – namely, whether the speaker is in favour of or against the coalition. For this, we explore the benefits of multi-task learning and investigate which setup and task formulation is best suited for each sub-task. We evaluate our approach, based on hand-coded newspaper articles, covering elections in three countries (Ireland, Germany, Austria) and two languages (English, German). Our results show that the multi-task learning approach can further improve results over a strong monolingual transfer learning baseline.

## introduction in the last decade the capabilities of computational methods for text analysis have drastically evolved and the usage of natural language processing (nlp) methods have increasingly expanded in scope, enabling a wide range of applications for the automatised analysis of political texts @xcite . much work in nlp in the past has focused on making sense of how legislative processes are codified in text -e.g., by analysing debate motions @xcite @xcite . however, when analysing electoral dynamics it seems necessary to complement the application of nlp methods on domain-specific texts such as electoral manifestos with a broader analysis of signals from news texts @xcite . specifically in the case of government formation in multi-party systems -that is, in countries where the government is typically formed by multiple parties -party members and candidates often communicate in interviews and rallies during election campaigns which coalition their party might or might not be willing to enter after the election. these public statements about prospects of future governments are often picked up by media outlets. being able to automatically identify coalition signals in news coverage has accordingly the potential to enable large-scale studies of voting behaviour using 'text-as-data', with the promise of improving our understanding of voters' behaviour and of strategic voting in multi-party democratic systems @xcite @xcite . very recent work in political science made first steps in this direction by investigating dictionary-based approaches to automatically detect coalition signals in newspaper text @xcite . however, much work still remains to be done by developing datadriven methods that take advantage of advances in our field from the past years such like pre-trained language models and joint learning, and providing a thorough evaluation of the accuracy of such automated approaches. this is especially important because coalition signals can be expressed in text by explicit or implicit statements and make heavy use of irony and figurative language, thus raising many challenges that cannot be solved with simple dictionary-based methods and require natural language understanding (nlu) capabilities. in this paper, we present the task of coalition signal detection as a new, challenging nlu task. we propose to approach this task as a machine learning problem and decompose it into two subtasks, (i) the identification of text sequences that contain a signal and (ii) the prediction of the polarity of each potential signal, that is, whether the speaker promotes or opposes the signalled coalition -cf. examples 1.1 and 1.2 for sentences containing a coalition signal and negative polarity (i.e., unwillingness to join a coalition), respectively. example 1.1. the green party yesterday signalled that it would be interested in participating in a coalition government led by fine gael. example 1.2. ms o'donnell said she could not see the pds negotiating with the labour party after the election. we benchmark performance on this task by introducing a new multilingual dataset -using preelection newswire coverage from multiple elections in three countries (ireland, germany and austria) -and quantifying the performance of monolingual pre-trained models to provide a simple, yet state-ofthe-art strong baseline that is shown to substantially outperform previous approaches from the text-asdata community in political science. moreover, we investigate the benefits of auxiliary tasks and a cross-lingual approach in a multi-task learning setting, so as to improve performance over the simpler, transfer-learning-based baseline. ## related work in recent years the usage of computational methods for the analysis of 'text-as-data' has drastically expanded in scope @xcite . political scientists have used nlp methods for a wide range of tasks, including, for instance, inferring policy positions @xcite and detecting topics @xcite . at the same time, nlp researchers have addressed closely related tasks such as election prediction @xcite , agreement measurement @xcite and claim detection @xcite to name a few. when focusing on inter-party communication, researchers have used information from the news @xcite @xcite or have investigated different kinds of campaign ads @xcite @xcite , press releases @xcite @xcite , election manifestos @xcite , or social media data @xcite @xcite . this type of analysis typically requires manual coding of theoretical concepts and has been used to study the role of coalition signals and negative campaigning in the context of electoral campaigns @xcite @xcite @xcite . manual coding, however, is a labour and time consuming process which makes it hard to apply to larger amounts of data or to longitudinal studies where new data will be collected over a long time period, thus requiring new manual effort for the same coding steps. it is therefore crucial to automate this process in order to enable large-scale studies that might increase our understanding of the role of coalition signals in electoral campaigns. one new promising approach towards that end was recently presented by @xcite , who developed a dictionary-based method for detecting inter-party communication in the form of preelectoral coalition signals. whilst their approach so far only identifies positive signals between parties, i.e., parties indicating that they are willing to form a government coalition together, it is a promising starting point to automate the process of studying inter-party communication. we describe their approach in more detail in §4.3 and then present our own approach to the identification of inter-party communication in the form of coalition signals during electoral campaigns ( §4.2). the resulting german dataset is the largest with 15,735 sentences of newswire text. the irish and austrian data are smaller with 8,493 and 2,880 sentences. the extracted articles have been filtered manually for false positives, i.e., articles that include the keywords but do not touch on the issue of upcoming elections have been removed. as a result, each article in the corpus includes at least one coalition signal. 2 coding coalition signals. in order to code coalition signal in newspaper texts, the annotators foot_2 first read the whole article and highlighted all signals and speculations in each article (cf. definitions below). after this initial step, the annotators proceeded with the actual coding: all highlighted signals and speculations in the article were coded in chronological order and inserted in the coding template. for each signal or speculation, the sender(s) and addressee(s) are identified, so that each can be attributed to one or more specific parties. in cases where this information is missing from the text, the instance is coded as unspecified. crucially, our annotation scheme distinguishes coalition signals and speculation. definition 3.1 (signal). a coalition signal is a statement made by a potential coalition partner about their preferences towards a possible coalition in which the party itself would be a member. a coalition signal consists of three components: for being identified as a coalition signal, it is required that the sender directly speaks about their preferences concerning a potential coalition. this includes citation of (direct and indirect) quotes by a journalist. sometimes the addressee is underspecified (e.g., when a party rules out a coalition with "any left party"). example 3.3. asked if the row between fine gael and labour had now become so acrimonious as to threaten the possibility of the two parties working together in a coalition arrangement, mr kenny did not give a direct reply. coding coalition speculation. speculations are typically stated by journalists, experts and political actors. like coalition signals, each speculation has a sender, an addressee and a statement regarding the anticipated coalition, with positive, negative or neutral polarity. ## a transfer learning approach to coalition signal detection we propose an approach to coalition signal detection on the basis of transfer learning, and decompose our problem into two separate classification tasks. the first task consists of identifying whether or not a certain text sequence -more specifically, a sentence -includes a coalition signal. for signal-bearing sentences, we additionally predict the type of the coalition (e.g., labour+social democrats). for this, we model both steps in a standard text classification framework by assigning non-signals the label none. the second task then determines the polarity of the detected signals as either negative, neutral or positive. ## multilingual coalition signal prediction we next focus on the prediction of coalition signals in a multilingual setup and across countries. as shown in given the relatedness of our tasks, we propose a multitask learning @xcite approach for our problem. from a conceptual point of view it is important, however, to note also the differences between our two tasks. while they can be both approached as text classification problems, polarity prediction has a small number of labels only (positive, neutral, negative) that are the same across countries and languages. coalition type prediction has instead a much larger number of (often sparse) ## conclusions and future work we presented experiments on coalition signal detection, a new, challenging nlu task for the political text analysis domain, and showed that a simple transfer learning approach outperforms previous work based on dictionary-based methods. we further showed that coalition signal detection in a multilingual setting is feasible and can substantially outperform results for the monolingual setup. we also approached the problem in a multitask learning setup, modelling the prediction of inter-party communication for different countries as separate tasks. this approach gave us improvements over the monolingual singletask setting. we could further increase results by adding another, highly related auxiliary task (valence prediction), obtaining final improvements of 5-7 percentage points (de, ie) over the monolingual baselines. for future work, we would like to test adapters @xcite as another way to inject relevant information into the model and mitigate the issue of 'catastrophic forgetting' that might be responsible for the negative results we obtained for two auxiliary tasks (ner, political actor tagging).