Analysis of State-Level Legislative Process in Enhanced Linguistic and Nationwide Network Contexts (2024)

Abstract: State bills have a significant impact on various aspects of society, including health, education, and the economy. Consequently, it is crucial to conduct systematic research on state bills before and after they are enacted to evaluate their benefits and drawbacks, thereby guiding future decision-making. In this work, we developed the first state-level deep learning framework that (1) handles the complex and inconsistent language of policies across US states using generative large language models and (2) decodes legislators’ behavior and implications of state policies by establishing a shared nationwide network, enriched with diverse contexts, such as information on interest groups influencing public policy and legislators’ courage test results, which reflect their political positions.

## introduction state policies affect our lives in many ways, such as our health, taxes, and rights. state legislatures can protect or limit our freedoms in areas like voting, reproduction, labor, and more. state laws also impact national issues, such as federal courts deferring to states on constitutional matters and federal laws responding to state laws (see appendix a). thus, our goal is to build an nlp/ml framework for understanding state-level legislative process in the us. this can help the public to know how bills affect them, check if legislators vote consistently with their public views, and find out who influence them. while promising nlp/ml studies, e.g., @xcite , predict bill or vote outcomes using bill text and legislator information, they have limitations: most focus on centralized federal-level process, they miss out some state-level contextual information such as influence of local lobbyists, they use low quality and inconsistent state data, they lack support for nonvote prediction tasks, or they often use traditional nlp/ml architectures. our framework addresses these limitations and improves the understanding of state legislative process and its implications: normalizing the language of state legislation. our framework's first pillar addresses the inconsistency, verbosity, and complexity of state-level legislative data @xcite . state bills vary in language and format for the same topic, making cross-state policy analysis difficult. they are also lengthy and detailed, obscuring the main points and implications of the bills. state bills often require domain knowledge to comprehend, limiting their accessibility for the public. therefore, we normalize and optimize state legislative data by using generative llms, e.g., openai gpt @xcite . this involves generating consistent summaries of bills, enriching legislator's data, and identifying potential winners/losers of bills. this way, we can better compare similar policies across different states and topics. we evaluate the quality of llm summaries by comparing them with expert-written ones and explore the feasibility of using open-source llms (appendix c). capturing inter-and intra-state contexts. we also aim to understand how state bills relate to other external factors (e.g., cross-state effects). thus, our framework adheres to two design principles: efficient abstraction and embedding. to operationalize our work, we construct a new textattributed graph abstraction to represent different entities and relations in state legislative process in a highly normalized linguistic and broadened network contexts (figure 2 ). on top of this abstraction, we enable diverse prediction tasks, beyond predicting legislators' votes, such as predicting legislators' courage test results and interest group impacts that both have real-world use-cases (section 6). we argue that such tasks are interdependent and can benefit from joint learning and representation. to efficiently deliver these, we develop a new shared text and graph embedding architecture (figure 3 ), designed to handle longer legislative texts and unseen entities more effectively. the first in kind that combines the benefits of an efficient non-generative llm, openai ada @xcite , with that of a widely-adopted heterogeneous gnn, rgcn @xcite , to jointly generate contextualized text and graph representations for nodes and relations. technical contributions: in summary, we present the following technical contributions: we discuss promising studies that have influenced our work (further discussion in appendix d). ## broadening political & social contexts our goal is to build a model that can better understand legislators' behavior and the impact of policies within and across states. thus, we abstract the state legislative process (detailed in appendix a) using a text-attributed graph and enrich it with diverse political-social contexts, derived from our ablation study (section 7). in the next section, we apply our generative llm optimization on the graph to handle the linguistic challenges. ## llm-based legislative graph optimization before utilizing the legislative graph for training and inference tasks, we implement a data optimization phase. this phase is realized using a generative large language model (llm) and includes: (a) normalizing the textual attributes of nodes to address linguistic challenges such as their verbosity, complexity, and inconsistencies across states; (b) enhancing the nodes with additional textual information where necessary; and (c) uncovering and formalizing latent relationships embedded within the nodes' textual data. these steps are crucial for improving performance, as they enable a more cohesive understanding of policies and legislative actions related to the same topic at both intra-and inter-state levels (section 7). ## data collection to realize and optimize the legislative graph, we have built a scraper that can extract various data on state legislation, legislators, interest groups, and courage tests from different sources, particularly, vote smart (votesmart, 2023). specifically, we have obtained 7k pieces of legislation covering the period from 2019 to 2022. these bills have undergone 16.9k actions. we analyze the 9k main and second floor actions (see appendix a.1). votesmart does not include every legislation from every state legislature; rather, their research team tracks topical legislation, as well as, unique and novel legislation (i.e. legislation that addresses things never previously addressed by any state legislature). moreover, we have obtained different information on 7237 legislators, including their biogra- llm (openai ada-002) textual embedding nationwide, multi-relational, text-attributed legislative graph of us states multi-relational graph embedding joint embedding (concatenation & ffnn) 8. distmult relation classification roll-call (vote) interest group score courage test answer node encoders ffnn figure 3: joint llm and gnn embedding architecture on nationwide legislative graph (properties in sec. 6.2) ## classification tasks & architecture next, we explore three important political science applications as relation prediction tasks on the legislative graph (figure 2 ). our optimized legislative graph, utilizing generative llm techniques, already enhances model performance in these tasks (as detailed in section 7). to achieve further gains, crucial in our problem domain (see the ethics section), we develop an efficient joint text and graph (llm + gnn) embedding architecture. ## experiments this section assesses our framework's core aspects: the joint llm and gnn embedding architecture's performance gains against strong baselines; the benefits of normalizing and optimizing state-level legislative data with a generative llm; a performance comparison over three legislative tasks;an ablation study on integrating wider social and political context into the graph; and a performance analysis using data splits that mimic real-world legislative conditions. ## summary in this paper, we introduce an efficient framework for analyzing the state-level legislative process. this framework interconnects legislative activities across u.s. states by creating a national network that includes often-overlooked elements such as interest groups and political stances. moreover, it is the first to tackle the linguistic complexities of state legislative data by utilizing generative llms to normalize, summarize, and enhance this information for improved policy analysis. in addition, we develop a legislative graph abstraction and define several key prediction tasks concerning the interactions between legislators, bills, and interest groups. we design an effective joint llm and gnn architecture to handle these tasks, which is built to accommodate unseen nodes and support longer texts, thus further leveraging the enhanced linguistic and network contexts. the next section discusses framework limitations and ethical aspects.