Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis (2024)

Abstract: The rapid advancement of Large Language Models (LLMs) has sparked intense debate regarding the prevalence of bias in these models and its mitigation. Yet, as exemplified by both results on debiasing methods in the literature and reports of alignment-related defects from the wider community, bias remains a poorly understood topic despite its practical relevance. To enhance the understanding of the internal causes of bias, we analyse LLM bias through the lens of causal fairness analysis, which enables us to both comprehend the origins of bias and reason about its downstream consequences and mitigation. To operationalize this framework, we propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the LLM decision process. By applying Activity Dependency Networks (ADNs), we then analyse how these attributes influence an LLM’s decision process. We apply our method to LLM ratings of argument quality in political debates. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment, and discuss the consequences of our findings for human-AI alignment and bias mitigation.

## introduction with the rise of large language models (llms) @xcite @xcite , inter alia), we are witnessing increasing concern towards their nega-tive implications, such as the existence of biases, including social @xcite , cultural @xcite , brilliance @xcite , nationality @xcite , religious @xcite , and political biases @xcite . for instance, there is a growing indication that chatgpt, on average, prefers proenvironmental, left-libertarian positions @xcite . despite its practical relevance, bias in (large) language models is still a poorly understood topic @xcite @xcite . the frequent interpretation of llm bias as statistical bias originating from training data, while conceptually correct, is strongly limited in its utility. @xcite reason that bias should, therefore, not be viewed as a singular concept but rather distinguish different concepts of bias at different levels of the nlp pipeline, e.g. distinct dataset and model biases. furthermore, while it is undisputed that models do exhibit some biases, it is unclear whose biases they are exhibiting @xcite . indeed, the literature up to this point has mostly focused on the downstream effects of bias -with only a few exceptions, such as van der @xcite that argue for the importance of an understanding of the internal causes. to advance this endeavour, we analyse llm bias through the lens of causal fairness analysis, which facilitates both comprehending the origins of bias and reasoning about the subsequent consequences of bias and its mitigation. a thorough understanding of llm bias is particularly important for the design and implementation of debiasing methods. examples from literature prove that this is a highly non-trivial task: for instance, @xcite proposed a geometric method to remove bias from word embeddings. yet, this method was later shown to be superficial by @xcite . furthermore, efthe figure depicts how the llm's perception of value a is considered during the decision process while judging b and c through f (c|a) and f (b|a). now consider the effect of treating the association of value a with c (f (c|a)) by naively fine-tuning the model to align with this value of interest on other value associations (f (b|a)) that are not actively considered. they may be changed indiscriminately, regardless of whether they were already aligned. these associations are currently neither observable nor predictable yet changes in them are potentially harmful. using the extracted decision processes, we gain information on what areas are prone to such unwanted changes. as depicted in figure 1 , alignment of a language model's association of two values, a and b, is not guaranteed to leave, e.g., associations of a with other values unchanged. these associations may be changed indiscriminately, regardless of whether they were already aligned. currently, these associations are neither observable nor predictable, yet changes in them may potentially be harmful, especially to other tasks relying on the same concepts. this stands in stark contrast to the literature on causal fairness analysis @xcite , which clearly indicates an imperative to account for the mechanism behind outcome disparities. in the present work, we investigate how the afore-mentioned associations influence the llm's decision process and aim to illustrate the possibility of traditional bias estimates omitting certain aspects. for this, we begin by defining a range of attributes. we then prompt the llm to rate a text excerpt according to these attributes. subsequently, we combine the llm's ratings with contextual metadata to investigate the influence of potential confounders and mediators from beyond the dataset. this is achieved by correlating the contextual and llm-extracted attributes, and constructing activity dependency networks (adns) @xcite to elucidate the interaction of said attributes. as a case study, we apply our method to us presidential debates. in this case, attributes are related to the arguments (e.g. its tone) and speakers (e.g. their party). the constructed adns then allow us to reason about how the extracted attributes interact, which informs bias attribution and mitigation. figure 2 provides a visual overview of the process. in summary, we make the following contributions towards a more profound understanding of bias in language models: 1. we illustrate llm bias in the framework of causal fairness analysis. the remainder of the paper is structured as follows. in section 2, we motivate our concerns using the language of causal fairness analysis. following this theoretical excursion, we describe the used text corpus in section 3. section 4 outlines our method of extracting attributes and their associations, and constructing adns. finally, we discuss our findings and their implications for alignment and debiasing in section 5. ## a causal perspective of llm bias our exploration of llm bias mechanisms is motivated by causal fairness analysis. following zhang and bareinboim (2018), we define the standard fairness model, and then illustrate it in the context of bias in an llm's evaluation of political debates. the standard fairness model figure 3 provides the graph for the standard fairness model. x is the sensitive characteristic and y is the outcome. w denotes a possible set of mediators between x and y . finally, z is a possible set of confounders between x and y . in this model, discrimination, and thus bias, can be modelled via paths from x to y . one can distinguish direct and indirect discrimination. direct discrimination is modelled by a direct path from the sensitive characteristic to the outcome, i.e. @xmath0. indirect discrimination can be further divided into two categories. indirect causal discrimination, where the sensitive characteristic and the outcome are linked by one or more mediators, i.e. @xmath1, and indirect spurious discrimination, which encompasses all paths linking x and y , except the causal ones (x ← @xmath2. @xcite further provides tooling to decompose fairness disparities into direct, indirect causal, and indirect confounding discrimination components. political llm bias in the standard fairness model application of the standard fairness model to llms is complex due to their opaque nature: both the mediators w and confounders z in the llm's decision-making process are unknown. consider the scenario that is analysed in the subsequent sections: given excerpts of us presidential debates, an llm is prompted to rate the participants regarding different aspects, such as the participant's tone or respectfulness vis-à-vis the other party. in this case, the sensitive characteristic x is the candidate's party, and the outcome y is the llm's rating. confounders and mediators could arise from the llm's pretraining or the prompts used, yet the exact nature and pathways of w and z remain unclear. by operating on a conceptual level, we identify confounders and mediators regardless of their origin. as such, we will omit this distinction in the following. to the best of our knowledge, there is no method available in the literature to automatically retrieve a set of possible mediators or confounders. hence, we rely on domain knowledge @xcite @xcite to define potentially mediating and confounding attributes. the remainder of this paper is devoted to extracting a set of pre-specified attributes using prompt engineering, and subsequently analysing their roles in the llm decision process. ## us presidential debate corpus towards our goal of investigating how an llm's decision process is influenced, and potentially biased, by associated attributes, we rely on a corpus of us presidential debates. the choice to use political debates is motivated by their central role in shaping public perceptions, influencing voter decisions, and reflecting the broader political discourse. furthermore, the us political system provides for an illustrative and familiar case study. in subsequent sections, we explore this dataset using our approach. data source for the collection of political text, we use the us presidential debate transcripts provided by the commission on presidential debates (cpd). 2 the dataset contains all presidential and vice presidential debates dating back to 1960. for each debate year, three to four debates are available, amounting to a total of 50k sentences with 810k words from the full text of 47 debates. further details can be found in appendix a.1. preprocessing to preprocess this dataset, we fixed discrepancies in formatting, manually corrected minor spelling mistakes due to transcription errors and split it by each turn of a speaker and their speech transcript (such as (washington, [speech text])). then we create a slice or unit of text by combining several turns, each slice having a size of 2,500 byte-pair encoding (bpe) tokens (≈1,875 words) with an overlap of 10%, see appendix e for an example. the slice size was chosen such that ## dissecting internal decision processes of llms as mentioned above, we are interested in which, and how, mediators and confounders shape an llm's decision process. in this section, we introduce our method for identifying a set of possibly confounding or mediating attributes, and instantiate it in the context of political debates. ## results: llm bias attribution we are interested in understanding the causes of bias and, in the context of our case study, how the speaker party, the sensitive characteristic, influences the llm's perception of score, i.e. the outcome. figure 5 presents different score distributions, revealing that democratic candidates typically score higher than republicans. we explore political bias as a potential explanation for this variation. we caution that the bias estimates based on correlations, as well as those reported in other studies, might be overstated and could in part be explained by indirect biases influenced by mediators or confounders. we suggest that some of the score discrepancies may stem from a series of attributes linked to score and speaker party. we also provide examples to highlight these issues and discuss the implications of debiasing language models. ## discussion problems with direct fine-tuning our findings illustrate the complexity of decision-making in llms. naively debiasing a model by assuming direct discrimination overlooks this complexity and could lead to unintended consequences. this issue is especially pronounced in foundation models, where it is impractical to evaluate each downstream task; debiasing one aspect may inadvertently compromise performance on other, yet-to-be-defined tasks. consequently, debiasing efforts should be carefully directed, with a focus on accurately identifying the origins of bias to minimize undesirable effects in downstream applications. the development of new causal attribution methods is a promising research direction. moreover, addressing political biases in llms demands a nuanced understanding that spans both the technical aspects of the models and the broader societal influences on political discourse. an interdisciplinary approach combining computational and social science expertise could advance the development of more effective strategies for bias identification and mitigation in llms. ## conclusion this paper presents a new approach to understanding bias in llms through the lens of the causal fairness model, accompanied by a method to examine the llm decision process using prompt engineering and activity dependency networks. our findings highlight the complexities of identifying and addressing biases in ai systems and the need for nuanced debiasing strategies. we aim to enrich the discussion on ai ethics and inform more advanced bias mitigation methods. as ai becomes increasingly central in critical decision-making, we emphasize the importance of research to responsibly leverage its potential.