Align Voting Behavior with Public Statements for Legislator Representation Learning (2021)

Abstract: Ideology of legislators is typically estimated by ideal point models from historical records of votes. It represents legislators and legislation as points in a latent space and shows promising results for modeling voting behavior. However, it fails to capture more specific attitudes of legislators toward emerging issues and is unable to model newly-elected legislators without voting histories. In order to mitigate these two problems, we explore to incorporate both voting behavior and public statements on Twitter to jointly model legislators. In addition, we propose a novel task, namely hashtag usage prediction to model the ideology of legislators on Twitter. In practice, we construct a heterogeneous graph for the legislative context and use relational graph neural networks to learn the representation of legislators with the guidance of historical records of their voting and hashtag usage. Experiment results indicate that our model yields significant improvements for the task of roll call vote prediction. Further analysis further demonstrates that legislator representation we learned captures nuances in statements.

## introduction modeling the behavior of legislators is one of the most important topics of quantitative political science. existing researches largely rely on roll call data, i.e. historical voting records, to estimate the political preference of legislators. the most widely used approach for roll call data analysis is ideal point model @xcite ) that represents legislators and legislation as points in a one-dimension latent space. researchers enhance ideal point model by incorporating textual information of legislation @xcite ; gu et al., 2014; kraft et al., 2016) and report positive results for roll call vote prediction. although roll call data is the major resource for legislator behavior modeling, it has two limitations. firstly, it fails to uncover detailed opinions of legislators towards legislative issues. therefore, we have no clue about the motivation behind their voting. secondly, it is unable to model the behavior of newly-elected legislators because their historical voting records are not available (i.e., coldstart problem). meanwhile, researchers explore to use public statements to characterize the ideology of legislators with the guidance of framing theory @xcite @xcite . @xcite propose a text-based ideal point model to analyze tweets of legislators independent of roll call data. experiment results show some correlations between distributions of ideal points learned from legislative data and public statements. however, they treat the two resources separately and fail to uncover deep relationships of behavior between these two landscapes. figure 1 shows a legislative issue related to prohibit partial-birth abortion. it includes the title and description of the legislation, roll call vote records and public statements on twitter of legislators. based on the voting records, we know the stance of legislators. with the discussion on twitter, we can further understand their opinions towards the topic. supporters concentrate on protecting the life while opponents emphasize rights of choice. this motivates that bridging public statements on twitter with roll call data can provide a full image of behavior patterns of legislators. a closer look at the example (figure 1 ) reveals that most tweets utilize hashtags to express ideas in short. moreover, people with opposite stances choose different groups of hashtags, i.e., supporters use #life and #theyfeelpain while opponents use #choice and #whatwomenwant. further analysis on a large tweets dataset, where each tweet is processed by a python library textblob 1 , shows that most hashtags are polarized with one sentiment (figure 2a ). based on this observation and previous studies that reveal polarization of hashtags @xcite , we explore to utilize hashtags as a label to describe the preferences of legislators on public discussion and propose a novel task of hashtag usage prediction to characterize their ideology. in this paper, we collect public statements of legislators on twitter as an extension of roll call data for legislator representation learning. our intuition is to combine roll call votes as hard labels and hashtags as soft labels to jointly model legislators. in practice, we build a heterogeneous graph to bridge the voting behavior and public statements of legislators. it consists of three kinds of nodes, legislators, legislation and hashtags in tweets. subsequently, we employ a heterogeneous relational graph convolutional network (rgcn) @xcite to simultaneously update the representation of different nodes. two tasks are used for training, including roll call vote prediction and hashtag usage prediction to model the behavior of legislators on voting and on public statements respectively. the major contributions of this paper are threefold: -to the best of our knowledge, this is the first study incorporating both voting behavior and public statements to jointly depict legislators. the proposed framework enables us to understand the preferences of legislators combining their behavior in legislative process and on public platforms. -we propose to learn the representation of legislation and legislators using heterogeneous graph which can densify relations among legislators, thus mitigate the cold-start problem. -we propose a novel task of hashtag usage prediction to characterize the preferences of legislators on public discussion and construct a dataset as the benchmark. our dataset and code is available on github foot_0 . ## dataset and tasks the voteview website @xcite provides a benchmark for the task of roll call vote prediction. it contains roll call votes history and keeps updating. meanwhile, a dataset constructed by @xcite enables the public to take advantage of detailed description and sponsor information of legislation from 1993 to 2018. we extend these corpora with tweets published by legislators. ## proposed framework the overall framework we proposed is shown in figure 3 . we construct a heterogeneous graph with three kinds of nodes (legislation, legislator and hashtag) to cover the two landscapes of congress and twitter. on top of this graph, rgcn is applied to optimize the representation. this is achieved by a joint training of the two tasks of roll call vote prediction and hashtag usage prediction. in addition, we utilize an unsupervised following proximity loss to further optimize the representation. ## further analysis we perform additional analysis to further evaluate the effectiveness of our model. ## related work ideal point estimation has become a mainstream approach to model ideology of legislators. classical ideal point model @xcite represents both legislators and legislation in the same space, and voting behavior is characterized as the distance between them. however, this simple spatial model fails to predict votes on new legislation. text-based models have emerged to address this issue. @xcite ; @xcite extended ideal point model with latent topics and issue-adjusted methods. some embedding methods @xcite also promote learning of legislators. more recently, external context information including party, sponsor and donors @xcite @xcite have been introduced to better describe the legislative process. since votes are not the only way to express political preferences, other sources of data including speech and knowledge graph @xcite @xcite have been applied to estimate ideology. although previous studies @xcite @xcite @xcite @xcite have incorporated social network of following or retweeting on twitter to learn legislators, fine-grained attitudes of legislators remain unknown since the texts themselves have not been mined. until recently, @xcite started to analyze linguistic differences between ideologically different groups using a broad range of handcrafted language features, and studies @xcite explored to incorporate twitter texts to cap-ture nuances in legislators' preferences via statistical methods. in spite of this, there has been little research attempting to combine votes with public statements to portray legislators from both angles and predict their behavior. previous studies @xcite @xcite have suggested that modeling on hashtag metadata is an informative way to analyze tweets, yielding classification of political affiliations. since hashtag is an important mean for people to participate in political discussion and communication, hashtag usage pattern has also been modeled as feature vectors in many clustering tasks to help learn different user groups @xcite @xcite . @xcite and @xcite have analyzed hashtag usage patterns of different ideologies through feature selection and keyword statistics. however, hashtag usage can be further utilized based on these analyses, e.g., for prediction tasks. thus, we focus on hashtags to depict statements of legislators on twitter, to jointly estimate their political preferences. ## conclusions and future work in this paper, we take the first step to align voting behavior with statements on twitter to jointly learn representation of legislators. we construct a heterogeneous graph to model the legislative context with a hashtag usage prediction task proposed to jointly train. experiments demonstrate that our framework can learn effective legislative representation and yield improvements for the roll call vote prediction task. due to the deficiency of background information, we have not yet detected more fine-grained stance of legislators towards specific events. in the future, we aim to conduct more research on the stance modeling of legislators.