Linear Representations of Political Perspective Emerge in Large Language Models

Authors: Junsol Kim, James Evans, Aaron Schein
Primary area: interpretability and explainable AI
Keywords: large language model, political perspective, ideology, representation learning

Abstract:
Large language models (LLMs) have demonstrated the ability to generate text that realistically reflects a range of different subjective human perspectives. This paper studies how LLMs are seemingly able to reflect more liberal versus more conservative viewpoints among other political perspectives in American politics. We show that LLMs possess linear representations of political perspectives within activation space, wherein more similar perspectives are represented closer together. To do so, we probe the attention heads across the layers of three open transformer-based LLMs (Llama-2-7b-chat, Mistral-7b-instruct, Vicuna-7b). We first prompt models to generate text from the perspectives of different U.S. lawmakers. We then identify sets of attention heads whose activations linearly predict those lawmakers' DW-NOMINATE scores, a widely-used and validated measure of political ideology. We find that highly predictive heads are primarily located in the middle layers, often speculated to encode high-level concepts and tasks. Using probes only trained to predict lawmakers' ideology, we then show that the same probes can predict measures of news outlets' slant from the activations of models prompted to simulate text from those news outlets. These linear probes allow us to visualize, interpret, and monitor ideological stances implicitly adopted by an LLM as it generates open-ended responses. Finally, we demonstrate that by applying linear interventions to these attention heads, we can steer the model outputs toward a more liberal or conservative stance. Overall, our research suggests that LLMs possess a high-level linear representation of American political ideology and that by leveraging recent advances in mechanistic interpretability, we can identify, monitor, and steer the subjective perspective underlying generated text.

Decision: accept (oral)
Reviewer scores: 6, 10, 8, 6

Reviews and discussion:
<thread>
creation date:
1730151776790

thread_type:
official_review

<thread_object>
creator: reviewer_huro

<content>
summary:
This paper presents an investigation into how LLMs encode political ideology, specifically focusing on U.S. politics. The authors claim that LLMs develop a linear representation of political perspectives in their activation space and demonstrate that these representations are primarily located in the middle layers of transformer models. By training a linear probe, the paper shows that political ideologies can be linearly separated within certain attention heads. The authors further demonstrate that intervening in these attention heads allows them to steer the political bias of generated text along a liberal-conservative spectrum without using explicit prompts.

strengths:
- The paper is novel in its application of linear probing to the representation of political ideology, which has not been extensively explored before in the context of LLMs. This contributes to the growing body of research on model interpretability and bias detection.
- The experiments are well-executed, and the use of multiple open-source models (Llama-2-7b, Mistral-7b, Vicuna-7b) to ensure reproducibility is commendable. The intervention analysis demonstrates an interesting way to manipulate model behavior based on internal activations.
- The paper tackles a highly relevant issue, as understanding and controlling political bias in LLMs is a critical challenge, especially given the role AI systems play in information dissemination and public discourse.

weaknesses:
- The key contribution of the paper—demonstrating linear separability of political ideologies—is not as novel or profound as the authors suggest. Linear separability is an expected feature of transformer models, particularly in middle layers where complex features are encoded. The use of a linear probe to classify two categories (liberal vs. conservative) is a relatively simple task that doesn’t fully substantiate the broader claim of a "linear representation hypothesis" in LLMs. The results merely demonstrate linear separability rather than showing that political ideology is fundamentally encoded as a linear feature across multiple dimensions, which would be a much more significant finding.
- The paper seems to conflate linear separability with the broader notion of linear representation as discussed in interpretability literature. The linear representation hypothesis posits that various features, such as gender or sentiment, can be combined and manipulated linearly within a model’s activation space. Simply demonstrating that political ideology can be classified with a linear probe doesn’t fully support this hypothesis. The authors don’t explore whether political ideology can be combined with other features in a meaningful, linear fashion, nor do they show how this linearity generalizes to other tasks or representations beyond political ideology.
- While the paper uses interventions to modify the political bias of generated text, it doesn’t convincingly demonstrate the causal relationship between specific model activations and ideological outputs. It remains unclear whether the identified linear structures are genuinely capturing political bias or merely reflecting superficial patterns in the training data. The interventions themselves seem to rely on surface-level associations rather than a deeper understanding of how political ideologies are encoded and combined within the model.

questions:
- Simulating Ideological Perspectives (Line 185): When you state that "LLMs must simulate the subjective, ideological perspectives of the given politicians or news media," how do you ensure that the model consistently selects an accurate ideological stance, rather than a random or mixed perspective, especially if it lacks full factual context? Could you provide additional details on how factual alignment with specific perspectives is verified in the model’s responses?
- Reference to Political Slant as a ‘Linear’ Spectrum (Line 133): The paper mentions that "social scientists have shown that humans conceptualize political slant as a ‘linear’ spectrum." Could you provide specific references to support this claim? Additionally, how does this linear conceptualization integrate with political science literature, which often models ideology as multi-dimensional? Exploring this could clarify how your approach fits within or challenges existing theories of political ideology.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730471177998

thread_type:
official_review

<thread_object>
creator: reviewer_t21j

<content>
summary:
This study shows how LLMs, like Llama-2-7b-chat, Mistral-7b, and Vicuna-7b, represent and simulate ideological perspectives in their hidden layers. Researchers probed these layers and identified a specific set of attention heads that represent particular political views, uncovering patterns they call the "geometry of perspective." They claim that these attention head activations linearly align with LLM viewpoints on a liberal-to-conservative scale, correlating with known political ideologies. They also found that adjusting these attention heads can steer model responses without extra prompts, shaping the output toward liberal, center, or conservative viewpoints. I feel this work could allow closer monitoring and control of ideological perspectives in LLM responses, which could be a powerful tool for transparency and balance in AI-generated content.

strengths:
1. The overall problem statement and motivation are very interesting and could be useful in real-world applications.
2. They used a simple yet effective approach by examining attention heads with established probing methods, applying a regression model on network activations to predict annotated labels of input or output data.
3. After probing, they attempted intervention with the model, trying to steer outputs to gain more insights—which was very interesting.

weaknesses:
1. I think more details should be added in Section 3.2, *Intervention*. It is very unclear.
   - Comments:
      - In line 282, $\theta_{l,h}$ is described as a steering vector, but in line 228, it is referred to as attention coefficients. Are they the same? If not, why are the notations the same?
      - If $\theta_{l,h}$ represents attention coefficients from the Ridge regression model, how can it be applied directly in the attention mechanism? Are the shapes of both vectors/tensors the same? Also, as they originate from different areas, how are they connected?
      - If $\theta_{l,h}$ is not the attention coefficient in line 282, how is it calculated? How do we maintain its shape or connect it with other components?
      - In line 288, the definition, purpose, and use of $\sigma_{l,h}$ are also unclear.
      - In line 291, it states, *We target the K most predictive attention heads for the intervention.* What is the value of $K$ here? Is it selected from the coefficients of the Ridge regression?
   - Asks:
      - Clearly define $\theta_{l,h}$ and explain how it relates to both the ridge regression model and the intervention process.
      - Explain the process of applying the ridge regression coefficients to the attention mechanism, including any necessary transformations.
      - Provide a clear definition and explanation for $\sigma_{l,h}$.
      - Specify how K is determined and its relationship to the ridge regression results.

2. In Section 3.6, GPT-4 as an evaluator is experiment-friendly, but isn’t there a chance we might reinforce hidden biases of LLMs? For instance, if the LLM-based evaluator has a bias toward the right (assuming right-leaning perspectives are correct), could it potentially score right-leaning statements more favorably, as this bias would seem normal to the evaluator model? Usually, some human annotated experiments are expected to prove that the LLM evaluator is replicating actual human/proper responses.
    - Asks:
      - Acknowledge and discuss the potential for bias when using an LLM as an evaluator.
      - Do some experiments to validate GPT-4's evaluations against human annotations, even if on a smaller scale.
      - Discuss how these factors might mitigate or account for potential biases in the evaluation process.

3. A discussion could be added to connect the sections *Simulating Subjective Perspectives using LLMs* and *Political Bias of LLMs* in related works, discussing the potential societal impact of this study. Currently, the findings feel somewhat disconnected from real-world applications. A brief discussion section could help.
    - Asks:
      - Add a brief discussion section that explicitly connects your findings to potential real-world applications.
      - Explore how your method for identifying and intervening in political bias could be applied in practical scenarios.
      - Discuss the ethical implications and potential societal impacts of your findings.

4. Although limitations mention that a U.S.-centric focus may reduce relevance in global contexts, the topic of political discourse and presence of strongly opposing views differs in many countries. As a result, the scope is limited; also, it’s uncertain if LLMs will adapt as accurately to culturally distinct perspectives (such as those in South Asia) due to data limitations.

5. Writing quality needs improvement, with several missing details. I’ve added these in the *Questions** below.

6. From a theoretical standpoint, the methods used are derived from previous works. While the study provides an interesting new application, I still feel this work lacks novel theoretical contributions.

questions:
1. Check weakness 1.
2. In line 223, it says, "*We train a ridge regression model using U.S. politician data.*" Does *U.S. politician data* refer to the extracted activations? Is there any train-test split, or how does this part work?
3. In line 344, was *2-fold cross-validation* done only on the Ridge regression model, or did you also collect attention heads twice?
4. Can you provide more details on the x-axis legends (values of each line) in Figure 3?
5. Clarify the terms *accuracy* and *Spearman rank correlation* in Section 4.1. Why is each one used?
6. How is Figure 4 generated? Is it based on the DW-NOMINATE scores from the datasets and attention, or does it use the attention and the political aspect of the prompt?
7. In line 417, it says, *the strongest correlation between intervention and political slant*, and the previous line mentions achieving a higher degree of conservative political slant. Is the correlation value based only on steering toward a conservative slant, or on all three (liberal, conservative, and neutral)? Again, how are these values measured, and with which inputs, outputs, or parameters?
8. What are the hyperparameters of Ridge regression model? Is it tuned? If yes, how? Is it tuned for all different situations separately, or only one model is trained - how does this work? If it is not tuned, why?
9. Adding a figure showing different parts of how the model works from start to finish, like a pipeline, could help the audience better understand the approach.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730565266641

thread_type:
official_review

<thread_object>
creator: reviewer_vb63

<content>
summary:
This paper presents an analysis of how LLM's internal activation layers capture political slant according to a US-centric liberal-conservative definition. By leveraging an automatically created dataset of texts generated by prompting LLMs with various politician personas, authors create linear probes to isolate attention heads that capture this political dimension. They show through correlation and intervention experiments that these activations can indeed capture and affect the political slant of generated texts.

strengths:
This paper is very close to a slam dunk in my opinion.
- I liked the goal of the paper, i.e., to explore political leaning in LLMs' activations.
- The methodology is grounded in poliSci theories (DW-nominate)
- The correlation experiments are well set up.
- I really liked the intervention experiments.

weaknesses:
There really aren't many weaknesses in this paper. All I could find were opportunities for improvement.
- I wish the paper's novelty was highlighted a little more in the intro and related work. Is this the first work that has examined the linear layers of LLMs for political bias?
- There are some missing related works particularly from the NLP side of things (e.g., https://aclanthology.org/2023.acl-long.656). I wish authors had done a little bit more comprehensive of a lit search, particularly in the ACL anthology (https://aclanthology.org) which contains papers on this topic.
- I would have liked a more nuanced discussion around political biases in LLMs. The current paper (§2) frames political biases as an issue of fairness, i.e., that LLMs that do not represent all political ideologies equally are not fair. However, there are issues that emerge from this framing, for example, the issue of false-balance / bothsideism (https://en.wikipedia.org/wiki/False_balance). Additionally, there is a growing discussion that fairness and non-bias are most important insofar as they can help shift power away from oppressive institutions (https://www.aclweb.org/anthology/2020.acl-main.485.pdf). I wish authors gave a little more balanced of a perspective on this.
- Footnote 1: I don't fully understand the distinction between "stochastic parroting" vs. linear representations, and specifically why if LLMs are merely stochastic parrots there would be no linear representation. It feels like the argument is somewhat of a side point given the results of the paper. Or it may be worth adding a longer discussion at the end of the paper about whether this means LLMs encode more systematic representations?
- L328: I really liked the intervention analyses, yet I wish that authors had given more empirical evidence that GPT-4 can indeed classify the political slant of the essays generated. This is important because LLMs can have various biases when used as text judges. The easiest method would be to conduct a small-scale in-house manual validation, where authors or colleagues rate the political slant of the generated texts and compare their ratings to the GPT-4 scores. I suspect the alignment is high but it would be best to confirm. Alternatively, there are other trained classifiers out there to predict the political leaning of text that authors could use.
- L429: I wish authors had contextualized the claim that conservative arguments might just be shorter, grounded in evidence from political science to back up this hypothesis. This is an interesting note, and may point to various things like tendency to over-simplify arguments on side of the political spectrum, to resort to emotional and persuasive talking points instead of more rational ones, etc. I wish authors had discussed this in further depth.

questions:
This is maybe a weird question, but I'm curious why authors decided to submit to ICLR instead of ACL venues (ACL rolling review), given the very NLP-nature of this work? I'm particularly curious because the field boundaries between ML and NLP are changing rapidly with the rise of LLMs, and I would just like to know. It feels like this work would be really well received in NLP conferences.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731416244951

thread_type:
official_review

<thread_object>
creator: reviewer_ubqm

<content>
summary:
The authors identify a set of attention heads within select large language models (LLMs) that encode ideological perspectives. Through the use of linear probes on the model's layer representations, they uncover heads associated with ideological slant. Their findings demonstrate that modifying these neurons influences the ideological bias exhibited by the model.

strengths:
* The authors find a set of attention heads encoding political slant in 3 LLMs
* The authors causally ablate these heads and show the resulting effects on text generation

weaknesses:
*  Why was ridge regression used instead of standard linear regression? Were collinearities established prior to making this choice?
* The authors don't establish baseline behaviours -- or maybe I am missing this. What were the model's ideological leanings before the intervention was applied?

questions:
N/A
</content>
</thread_object>

</thread>
<thread>
creation date:
1732180746732

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We are grateful for the reviewers’ thoughtful engagement with our paper and their constructive feedback. **In response to the reviewers' comments, we have made substantial revisions and submitted an updated version of the manuscript.** Below, we address each reviewer’s comments individually.
</content>
</thread_object>

</thread>


Metareview:
metareview:
**Summary:** 

In this paper, the authors seek to isolate which transformer LM hidden layers are most highly influential over political bias in outputs and identify attention heads that can represent this bias based on correlation with previously proposed political stance scoring methodologies. Using causal interventions, they show that post-hoc manipulations of transformer LMs can modify their political alignment without training or sophisticated feature-engineering. 

**Strengths:**

- These are very strong findings highlighting not only that transformer LMs are capable of abstract representation of political ideologies, but that these representations can be controlled through causal intervention.  Results confirm this approach is effective even for generations involving post-model knowledge cutoff events. 

- Their political ideology probing methodology, prompting LMs to imitate speech of known political figures or news media, is innovative and well-grounded.  

**Weaknesses:**

 - My primary concern would be the accuracy of GPT-4 as a judge in evaluating political ideology, but they show that GPT-4 performs relatively well at evaluating political ideology. 

I believe this paper nicely builds on prior linear probing literature while providing a novel perspective on how LLMs encode political stances. Their proposed method for controlling political bias could be broadly useful to the research community. From my own reading and the authors' rebuttal, I believe this is a high-quality paper and see no reason not to accept.