Discovering Lobby-Parliamentarian Alignments through NLP (2024)

Abstract: We discover alignments of views between interest groups (lobbies) and members of the European Parliament (MEPs) by automatically analyzing their texts. Specifically, we do so by collecting novel datasets of lobbies’ position papers and MEPs’ speeches, and comparing these texts on the basis of semantic similarity and entailment. In the absence of ground-truth, we perform an indirect validation by comparing the discovered alignments with a dataset, which we curate, of retweet links between MEPs and lobbies, and with the publicly disclosed meetings of MEPs. Our best method performs significantly better than several baselines. Moreover, an aggregate analysis of the discovered alignments, between groups of related lobbies and political groups of MEPs, correspond to the expectations from the ideology of the groups (e.g., groups on the political left are more aligned with humanitarian and environmental organisations). We believe that this work is a step towards enhancing the transparency of the intricate decision-making processes within democratic institutions.

## introduction the transparency of decision-making is of central importance for the legitimacy of democratic institutions such as parliaments. the influence of interest groups (lobbies) on parliamentarians and the potential for a resultant subversion of the power of the electorate to determine policy have led to demands from groups, such as transparency @xcite , for effective rules and systems to increase transparency. the emergence of several open government initiatives around the world @xcite @xcite is in part a response to such demands. the eu transparency register (tr) (european union, 2011) is one such initiative that provides a * these authors contributed equally to the work. tool for eu citizens to explore the influence of interest groups in the european @xcite . any organization that seeks to influence eu policy, with a few notable exceptions, needs to register with the tr before meeting with parliamentarians. the organizations are asked to disclose information such as their address, website, financial information, and goals. however, the eu tr has several limitations. the disclosure of most of the information is voluntary and there is little oversight. it is difficult to obtain information regarding which members of the ep (meps) or laws are targeted (and by which particular lobbies) and to know the lobbies' positions on specific policies. there have been several studies conducted by the political science community on eu lobbying @xcite @xcite . however, these studies either focus on a single policy issue or a small set of issues, or they are limited in terms of sample size as they employ less scalable methodologies such as manual examination of position papers and individual interviews. one exception is a study by @xcite . they analyze the twitter follower network of a large number of meps and lobbies from the tr, with respect to the mep's nationality and committee memberships and lobbies' self-reported interests in the tr. however, they do not analyze the textual content of meps' speeches and amendments and the lobbies' position papers, which would be instrumental for uncovering convergence on specific policy issues beyond the broad interest areas mentioned in the tr. therefore, there is a need for comprehensive studies of lobbying with the help of rich publicly available textual resources and by using modern tools developed by the nlp community. in particular, algorithms for text representation @xcite and computing text similarity @xcite and entailment @xcite are promising for identifying interesting patterns. a major challenge faced by such studies is the lack of ground-truth data for validation. as far as we are aware, there exists no large database of verified mep-lobby alignments, let alone one annotated for relevant policy positions. in this work, we automatically discover alignments between a large number of meps and lobbies by comparing the text in publicly available documents where they express their views on policy issues. to the best of our knowledge, this has not been done in prior work. we focus on the eighth term of the ep (2014-2019), as it was the last complete term that was not disrupted due to the pandemic. in the absence of ground-truth data, we perform an indirect validation by comparing the discovered alignments to a dataset we curate of retweet links between meps and lobbies. we use the retweet network instead of the follower network studied by @xcite , because retweets typically occur as a result of the agreement of particular views between the mep and lobby, in contrast to 'follows' that can result from a general interest in knowing more about a topic or person @xcite . moreover, timestamps for retweets are publicly available, which allows us to collect more relevant data for the eighth term. since 2019, it has been mandatory for meps in certain key positions (such as reporters of parliamentary committees) to publish their meetings with lobby groups @xcite . we use this data as an additional source of validation, although it only covers the subset of the meps from the eighth term who were re-elected in the ninth term. finally, our methods are interpretable -we can obtain the specific set of mep speeches and lobby documents that match for an mep-lobby pair, thus enabling manual validation of discovered alignments by users. in this paper, to avoid any harm to the reputation of meps through showing any alignments that are false positives, we restrict ourselves to an aggregate analysis instead of showing individual mep-lobby links. the paper is structured as follows. in section 2, we describe the datasets that we curate and use. in section 3, we describe the different methods that we experiment with for discovering links. we evaluate the methods in section 4 and interpret them in section 6. we conclude the paper in section 7. ## datasets we curate several novel datasets for our study. to obtain the policy positions of lobbies, we curate a dataset of position papers (section 2.1). the views of the meps are obtained through a dataset of their plenary speeches (section 2.2.1) and proposed amendments (section 2.2.2). for validation, we use a dataset of mep-lobby retweet links (section 2.3.1) and meetings (section 2.3.2). ## methods here, we describe the framework and methods we use to discover alignments between meps and lobbies. let m denote a set of meps and l denote a set of lobbies. we assume that an mep @xmath0, l). one possible approach to discovering alignments is to estimate this probability directly. however, this is difficult as we do not have a ground-truth dataset of alignments on which to train a probabilistic model. without such data, we can make only relative assessments of p (m, l), based on information about the similarity of views between m and l. thus, we can say that p (m 1 , l 1 ) > p (m 2 , l 2 ) if the similarity of views for the pair (m 1 , l 1 ) is higher than that for the pair (m 2 , l 2 ). hence, we adopt the following framework. given an mep @xmath1, and a lobby @xmath2, the goal of our methods is to compute an alignment score a(m, l) ∈ r such that the methods differ in how a(m, l) is computed. for methods using texts, we use s m to refer to the documents produced by m and d l for the documents produced by l. ## evaluation we evaluate our methods on both the retweet links and meetings datasets. we use the area under the receiver operating characteristic (roc) curve (auc), as our metric because it is independent of the choice of a threshold for a(m, l). we are mostly interested in the low false positive rate (fpr) regime of the roc as we expect the mep-lobby alignment network to be sparse. hence, we compute the partial auc (pauc) for the @xmath3.05 region. the scores of all methods are given in using only position papers (d3) does not seem to have a significant negative effect on performance in the low fpr region compared to using all documents (d2). in fact, for the prolificacy baselines it summarization seems to help in general for both datasets. chatgpt summaries perform better than vicuna summaries, with the gap being particularly large for retweets. we, therefore, use chatgpt summaries for our subsequent analysis. it is worth noting however that vicuna, being free and opensource, is a promising and cheap alternative to chat-gpt. the entailment method using speeches is the best method for retweet data (our primary validation data) and also performs reasonably well for meetings. therefore, we use this method for interpretation. although the improvement over semantic similarity in terms of pauc is small, entailment significantly improves interpretability by reducing false positive matches in the document pairs, as we show in appendix d. ## discussion on metrics the two metrics we use, namely the auc on links present in retweet and meetings data, are only proxy metrics. the true metric we are interested in is the auc on ground-truth alignments. unfortunately, this ground truth is not available, necessitating the use of proxy metrics. as seen in the proxy metrics have several limitations. the presence of a meeting does not necessarily indicate an alignment of views as meps could also meet lobbies who are opposed to their views on some issue in order to have a better understanding of their position. conversely, the absence of a meeting or retweet link does not necessarily mean that the views of an mep and lobby do not align. for instance, many meps do not disclose meetings, or are not very active on twitter. however, even though imperfect, we expect there to be significant correlation between the true metric and these proxy metrics. in the absence of a more reliable and accurate ground truth dataset of alignments, this indirect validation is arguably the best that we could perform. the fact that our methods using semantic similarity and entailment significantly outperform the baselines in this indirect validation is encouraging, and suggests that the texts provide non-trivial information regarding mep-lobby alignments. ## interpretation we now interpret the alignments discovered using the entailment method to see if we can find interesting patterns. to obtain the discovered alignments, we set the threshold on a(m, l) to 0.7, which gives an fpr of 5% and tpr of 32.5% on the retweets data. we also manually check a small sample of matched texts and verify that the threshold indeed gives reasonable matches with only a few false positives. we look at the lobbies' level of focus toward different political groups and ideologies. we also give some examples of matched texts that show the method's interpretability in appendix d. 6 the entailment method is computationally expensive to run because of the cross encoder. therefore we run it only once for the best combination of data for retweets (chatgptgenerated summaries of speeches and lobby documents). nevertheless, we expect ent(d4,amd) to have a similar or slightly better performance than ss(d4,amd). ## conclusion in this paper, we presented an nlp-based approach for discovering interpretable alignments between meps and lobbies, and we collected novel datasets of position papers, speeches, amendments, tweets,