Linear Representations of Political Perspective Emerge in Large Language Models
Authors: Junsol Kim
James Evans
Aaron Schein
Venue: ICLR 2025 Oral (2025)
Keywords: large language model
political perspective
ideology
representation learning
Primary area: interpretability and explainable AI
TL;DR: LLMs possess linear representations of political perspectives (left-right) within the activation space. By applying linear interventions to the activation space, we can steer the model's outputs toward a more liberal or conservative stance.

Abstract: Large language models (LLMs) have demonstrated the ability to generate text that realistically reflects a range of different subjective human perspectives. This paper studies how LLMs are seemingly able to reflect more liberal versus more conservative viewpoints among other political perspectives in American politics. We show that LLMs possess linear representations of political perspectives within activation space, wherein more similar perspectives are represented closer together. To do so, we probe the attention heads across the layers of three open transformer-based LLMs (Llama-2-7b-chat, Mistral-7b-instruct, Vicuna-7b). We first prompt models to generate text from the perspectives of different U.S. lawmakers. We then identify sets of attention heads whose activations linearly predict those lawmakers' DW-NOMINATE scores, a widely-used and validated measure of political ideology. We find that highly predictive heads are primarily located in the middle layers, often speculated to encode high-level concepts and tasks. Using probes only trained to predict lawmakers' ideology, we then show that the same probes can predict measures of news outlets' slant from the activations of models prompted to simulate text from those news outlets. These linear probes allow us to visualize, interpret, and monitor ideological stances implicitly adopted by an LLM as it generates open-ended responses. Finally, we demonstrate that by applying linear interventions to these attention heads, we can steer the model outputs toward a more liberal or conservative stance. Overall, our research suggests that LLMs possess a high-level linear representation of American political ideology and that by leveraging recent advances in mechanistic interpretability, we can identify, monitor, and steer the subjective perspective underlying generated text.

Published as a conference paper at ICLR 2025
LINEAR REPRESENTATIONS OF POLITICAL PERSPEC- TIVE EMERGE IN LARGE LANGUAGE MODELS
Junsol Kim
James Evans
Aaron Schein University of Chicago schein@uchicago. edu
University of Chicago
University of Chicago
junso lunsol@uchicago edu
Google
jevans@uchicago edu
J janessevansegoogee.com
ABSTRACT
Large language models (LLMs) have demonstrated the ability to generate text that realistically reflects a range of different subjective human perspectives. This paper studies how LLMs are seemingly able to reflect more liberal versus more conservative viewpoints among other political perspectives in American politics. We show that LLMs possess linear representations of political perspectives within activation space, wherein more similar perspectives are represented closer together. To do so, probe the attention heads across the layers of three open transformer- based LLMs (Llama-2-1) 2-7b-char prompt models generate text from the perspectives of different U.S. lawmakers. We then identify sets of attention heads whose activations linearly predict those lawmakers' DW-NOMINATE scores, a widely-used and validated measure of political ideology. We find that highly predictive heads are primarily located in the middle layers, often speculated to encode high-level concepts and tasks. Using probes only trained to predict lawmakers' ideology. we then show that the same probes can predict measures of news outlets' slant from the activations of models prompted to simulate text from those news outlets. These linear probes allow us visualize, interpret, and monitor ideological stances implicitly adopted by an LLM it generates open-ended responses. Finally, demonstrate that by applying linear interventions to these attention heads, we can steer the model outputs toward more liberal conservative stance. Overall, our research suggests that L.LMs possess a high-level linear representation of American political ideology and that by leveraging recent advances in mechanistic interpretabilityy we can identify, monitor, and steer the subjective perspective underlying generated text.
Vicuna-700 7b). We first
INTRODUCTION
Large language models (LLMs) have demonstrated the ability to generate text that reflects a range of different subjective perspectives (Argyle et al. 2023b Gao et al.] 2024). In particular, growing body of recent work has explored L.MSs seeming ability to generate text that realistically reflects a range of political perspectives on voting preferences and policy issues (Argyle et al. [2023b] Santurkar et al.
2023; Wu et al. 2023; 2024 O O'Hagan & Schein 2023 Kozlowski et al. 20244] Kim & Lee| 2023
Bernardelle et 2024). Leveraging this capability, researchers and practitioners have deployed
LLMs for range of tasks, including the development of personalized agents that engage in political
debates with humans (Hackenburg Margetts) 2024 Argyle al. [023a Costello et al. 2024 Bai
et al. 2023). as well as the implementation of agent-based models intended to simulate human- -aaman--ike
behaviors and interactions (Andreas) 2022 Törnberg et al. 2023 Park al. 2023
2024a
Gao et
2024 Charness et al. 2023 Hewitt et al. 2024).
This paper examines whether LLMs possess general representations of political perspective in activation space, whether such representations are linear, and whether they can be used to steer model outputs. Specifically, we show that LLMs possess a linear representation of the cobbeaal-conservative"" political axis in American politics. It is widely believed for LLMs that "important" concepts encoded linearly as directions in activation space (Mikolov et al., 2013) Nanda et al.; 2023 [Elhaee
et al. 2022 Gurnee & Tegmark 2024 Park et al. 2024b). Under this definition, L.MSs have been

---

Published as a conference paper at ICLR 2025
User:
User:
Write an essay on immigration in the U.S.
Write an essay on abortion in the U.S.
Assistant:
Assistant:
On the one hand those who support immigration argue that can bring many benefits to a country For example Immigrants can help to fill labor sho ages in certain industries, such as healthcare or technology. They can also bring new ideas skill
On one hand pro choice advoc ales argue that a woman has the fight lo make decisions about her own body and th at abortion Should nd pro life advoc ates argue that abortion is the intent io nal term ination of a human life and that it should be illegal The United States has a unique system of government th at allows oor a wide range of opinions and policies on abort on At the federa level the Supreme Court has ruled in s
accessible and safe On the other ha
s and pers pectives to a country which can lead
to innovation and progress Additionally immigra
nts can contribute to the economy by paying taxes
starting businesses, and creating jobs On the
everal land mark cases that have established woman ss
other hand those who opp ose Immigration argue
ight to choose to have an abortion However these decisi
that can cause problems for a country For exa
ons have been challenged and over tum ed in some states
mple, some people worry that immigrants will take jobs away from citizens. Others worry that immigr ants may not integrate well into society, leading to social and cultural problems
leading l å patch work of laws and policies that vary wide is important to remember that the right to choose is a fundamental human right, and that eve ry woman should have access to safe and legal abortion
across the country
Figure 1: Excerpts from essays generated by Mistral-7b-instrctt on policy issues (e.g., immi- gration, abortion) are annotated with the political slant predicted by probing one of the model's s attention heads (Layer 16, Head I). This attention head was among the most predictive heads with the highest Spearman correlation in predicting the political ideology of U.S. lawmakers. Tokens highlighted more in blue indicate that the probe predicted a more liberal political perspective, while tokens highlighted more in red indicate more conservative perspective.
shown to possess linear representations of various high-level concepts. such as sentiment (e.g..
positive-negative) (Tigges et al. 2023), space (e.g.. North- South) (Gurnee & Tegmark 2024
Nanda
et al. 2023). time (e.g., past present) (Gurnee & Tegmark 2024), humor (Von Rütte et al.
2024
language (Bricken et al. 2023), ) topic (Turner et 2023). truth (Marks & Tegmark| 2024| et al.
2023), and safety (Arditi et al. 2024), among other fundamental concepts (Gurnee & Tegmark 2024
Nanda et al., 2023] Bricken et al. 2023). To our knowledge, this paper is the first to investigate whether LLMs possess linear representations of political perspective.
We prompt LLMs to generate text from the perspectives of differrnnt U.S. lawmakers and then train linear probes to predict these lawmakers' DW-NOMINATE scores based on the activations of the models' attention heads. DW-NOMINATE is a widely used and validated measure of lawmak- ers' positions along the conrevvvvnsevv axis in American politics (Poole & Rosenthal 1985 Mistral-7b-instructt
Poole 2005). For three different open LLMs 1-lama-2-7b-chaa Aicuna-7b-vil 5; see Appendix for model descriptions), we identify multiple attention heads that linearly represent political slant from liberal to conservative. More specifically, we reveal that linear probes on these attention heads are highly predictive of DW-NOMINATE scores for held-out lawmakers, and performance does not improve when using non-lineat probes (Section 3). Addition- ally, we show that when models are prompted to simulate the perspectives of different news outlets FOX
FOX News or NBC), the same linear probes trained to predict lawmakers' DW-NOMINATE scores are also highly predictive of established measures of the news outlets' political slant (Section 55) We demonstrate the usefulness and validity of these trained probes in two ways: (1) monitoring and (2) steering the political slant of LLM outputs. First, we show that these activation patterns can be used to detect the ideological slant implicitly adopted by LLM it generates open- ended responses, as shown in Figure I (Section 4. Second, by targeting these attention heads for causal intervention, we demonstrate that L.M responses can be steered toward more liberal conservative perspectives without additional prompt engineering or fine-tuning (Section Overall, our research contributes a growing body of work that identifies linear representations and intervenes on them to monitor and simulate text from different subjective perspectives.
2 PRELIMINARIES
In this section, we define notation and provide relevant background on the architecture of transformer- based L.MSs and probing methodology for discovering representations of concepts in L.MMs.
2

---

Published as a conference paper at ICLR 2025
Transformer-based LLMs LLMs generate text by sampling iteratively from a categorical distribu- tion over the next token wy given input tokens wet. This distribution can be written as
P(,,- w.)) x exp(u,. r1.)
where uh E IRD is the unembedding of possible token v, and rı. E RDD is the final vector in the transformer's s "residual stream" (Elhage al. 202T). which evolves over layers = 1,... L as:
H
H Σ 1=1
rt = -1-1-
QEn DeA + MLP, (re-1 + h=1
Σ
Qen Ft
(2)
Here the dependence on wet is implicit via Tu, which encodes the input tokens before any transformer layers are applied. We refer to Fiih E Rdda as the activation of attention head h in layer E,
F ATTN (Pe,
which we highlight because will be the target of this paper's probing studies. The representation of a transformer layer in Equations (2) and (3) involves weight matrices Pe, E x and Qra E Dxde,, which can be understood as maps between the D-dimensional space of the residual stream and the dimennsnnsi space of a given attention head, where typically de = d the same for all heads.
Probing Probing refers to a supervised approach for finding the learned feature representations of a certain concept-of-interss in the activation space of trained neural network (Alain & Bengio 2017 Belinkov 2022). Inputs associated with "ground truth" labels for the concept-of-intere to trained neural network, and the network's s activations as it processes those inputs are recorded. "probe" is then a model trained to predict the ground-trun truth labels from network activations. Several probes are typically fit to different sets of activations, and each probe is often from a family of linear models (e.g.. linear regression)-iee. a linear probe.
are passed
The literature on probing LLMS places particular emphasis on linear probing, largely due to widespread belief in the (often underspecified) hypothesis that "important" high-level concepts represented linearly as directions in representation space (Mikolov et al., 2013 Park et al. 2022b). practical specification of this hypothesis, which will adopt throughout, is that "important" concepts can be accurately predicted from network activations via linear probes, and that such concepts are not more accurately predicted by more flexible non-linear probes. As an example, Gurnee & Tegmark (2024) find that linear probes are accurate (and no less so than non-linear probes) predicting the latitude and longitude of place from LLM's representation of the place's name.
There is fundamental ambiguity about what terms like "activation" or 'representation space" refer to in the context of LLMs, and thus ambiguity about which vectors should be the target of probing. Much of the existing work, which we will follow, advocates for probing the output of individual attention heads (Michel et al., 2019, Olsson et al. 2022) and for fitting separate probe to each (Li et al. 2023). For example, Llama- 2- 2-76--2---- 7b-chat consists of 32 layers, each containing 32 attention heads. Probing such a model might thus involve training 1,024 = 32 x 32 separate linear probes.
Concretely, probing data set is initially constructed as a set of N prompt-label pairs (w () N Each prompt w is given as input to the LLM,, and a set of activations are recorded. In our case, the set of activations for each prompt i will be x in Equation (3) for every attention head /t in layer & For every head we will then fit a linear probe, each of which assumes:
i==1
VEA 2 T0
where 0g dR. are regression coefficients to fit. Following Gurnee & Tegmark 2024), will fit these probes using ridge regression-i.e. by minimizing the Lz-regularized squared loss:
(x Σ (00 0 2.h
(5)
Thii representation is a simplification that clides details about layer normalization among other steps that important for the present study. However, note that æ will he taken hefore any layer normalization.
3

---

Published as a conference paper at ICLR 2025
where À is a hyperparameter that can be tuned via cross-validation. Ridge, opposed to unregularized linear regression, is often selected to mitigate overfitting and issues arising from multicollimearity in the activation vector. After training. if the linear model shows good fit, the estimated coefficients 0 can be understood capturing direction in activation space corresponding the given concept- of-interest. For instance, w is the name of a place, and yd is its longitude, then oe, might correspond to "North-South axis in activation space.
TRAINING PROBES TO PREDICT DW-NOMINATE OF U.S. LAWMAKERS
This section reports on a of probing experiments to find linear feature representations of political perspective in three open transformer-based LLMs. As described in Section 2 probing generally requires access to some "ground truth" labeling y of given input w The term "political perspective" is ambiguous and can refer to a number of different concepts, each of which may be subjective and difficult to pin down precisely, let alone quantify. Generally speaking, the study of any social scientific concept must grapple with the problem of measurement (Adcock & Collier 1001]Jacobs & Wallach 2021). In this section, we operationalize "political perspective" as meaning (roughly) "position the liberal-cone coonervavvv idcological axis in American politics". We do so using DW-NOMINATE (Poole & Rosenthal, 1985; Poole 2005 Carroll et al. 2009). a widely used and validated measure from political science for the ideology of U.S. lawmakers (e.g., Senators, Presidents). At a high level, we prompt LLMs to generate text in the style a given lawmaker, and then train linear probes predict that lawmaker's DW-NOMINATE score from model activations.
Data The DW-NOMINATE for a given U.S. lawmaker is multi mnlti-dimensional score that is predictive of the lawmaker's 's voting record. Such scores are produced by fitting a specific parametric model to input
matrix that records how each of the lawmakers voted (yea, nay, or abstain) on each of the proposed bills. These scores are well-established as accurate reflections of lawmakers' stances a broad range of legislative issues (McCarty 2016 Everson al. 2016). Moreover, the first dimension of such scores is often interpreted as scaling lawmakers' ideology- i.e., positioning them a fitheral-conservative axis such that very liberal lawmakers very conservative are on the other, and centrists in the middle. The scores range from -1 to 1, where -1 most liberal and 1 is most conservative.
side,
We use the first dimension of DW-NOMINATE scores for all lawmakers associated with the 116 United States Congress (N=552) P
We selected this Congress because it was active from 2019 to 2021, prior to the knowledge cutoff of September 2022 for the pre-training data of the Llama-2 family of language models [Touvron al. 2023).
Prompting To construct probing data set, created set of N 552 prompts, one for each of the members of the 1h Congress, and each prompt w conforming to the following template:
USER: Generate a statement by [NAME], politician in the United States. ASSISTANT: In 2019, [NAME] said that
For each of three open transformer-basee LLMs Vicuna-7b) we passed each of these prompts input, and then recorded the activation aiih of each attention head h in each layer L.
ietrral7bbiinsruun
Probing We then trained separate linear probes for each attention head h in layer of given model to predict the DW-NOMINATE score f of lawmaker from the activation an induced by prompt w We set the regularization strength À = 1 (see Equation (5)) after performing 2-fold cross-validation for the values {0, 0.001, 0.01, 0.1, 1, 100, 1000} (see AII. After training, each probe contributes predictions Dith (Equation (4)) which we can use for evaluation.
In the 11thh U.S. Congress, the House of Representatives had 435 original voting members, 6 non non-voting delegates, and 8 seat changes, bringing the total to 449. The Senate had 100 original Senators and underwent 2 seat changes. Including the President, the total is then N 552.
4

---

Published as a conference paper at ICLR 2025
0.8
0.8
0.8
0.6
Layer
0.2
0.2
0.2
Head (Sorted)
Head (Sorted)
Head (Sorted)
(a) -lama-2-7b-chat
(b) MiMttral-7--instruee
(c) Vicuna-7b
Figure 2: Predictive performance of linear probes for all attention heads across all layers in Llama-2-bb-chat,
and Vicuna-7b. Each (i.e., y-axis) represents each layer of the model from the bottom (layers close to the input layer) to the top (layers close to the output layer). Each column (i.e., x-axis) corresponds sorted by their predictive performance in descending order of Spearman correlation. Darker versus lighter shades indicate higher versus lower Spearman correlation, meaning the attention head was more or less predictive of lawmakers' political ideology (i.e., DW-NOMINATE scores).
specific attention head in given layer,
Evaluation To evaluate the fit each linear probe, we performed 2-fold cross-validation, using random partition of lawmakers into two folds of equal size. For each of the two splits, we fit probes to one fold and had them generate predictions on the other test fold. We then computed the Spearman rank correlation between the predicted (T heeea and true ( Jieueu scores. Our goodness-of measure is then averaged across the two splits ie., the cross- cros-validdttio Spearman rank correlation, which we denote P
We can also evaluate ensembled predictions of probes across different heads and layers. so, we define Tk to be the set of indices (e, h) for the K probes with highest Pey The ensembled predictions we explore are then defined as
Σ
VK
TEh
(6)
(LA)ETR
We can evaluate these for different K using another round of cross-validation, each yielding correlation score y for that ensemble. Intuitively, we expect such scores to increase up to some point but then eventually decrease as less predictive heads are averaged in.
Results We find for all three models, many or most of the probes fit to attention heads in the middle layers (around 10-20) exhibit high Spearman correlation p of around 0.8. For Llama-2-7b-chh the highest Spearman correlation is 0.854, which is achieved by the probe of the 18 head in the 15th layer. For Mi serra-7bbiintruun by the probes of the 3rd head in the 16 layer and the 8th head in the 24th layer. All Spearman correlations for each model are visualized as heatmaps Figure and the top values for each are given in Table A2
and Vicuna-7b, is 0.846 and 0.861, respectively, achieved
We also provide results for the ensembled models in Table where we find that performance tapers around K 32, which y is 0.87 for Llama-2-7b-chat 0.864 for Mistral-7b-instrun 0.885 for Vicuna-7b. Figure 3 we also plot the ensembled predictions for -lama-2-7b-oa and highlight examples of well-known lawmakers; the same plot for all three models is in Figure AT and Figure A2
and
The results broadly indicate that middle-layer activations linearly predictive of DW-NOMINATE. and thus may possess linear representations of the Tibbral-conservattin idcological axis. Before concluding this, undertook series of robustness checks.

---

Published as a conference paper at ICLR 2025
1.00
Rend Paul (R-KY)
(R-AZ)
1.00
INFOWARS
0.75
0.75
0.50
Lisa Murkowski (R.AK)
0.50
FOX NEWS
0.25-
0.25
New York Gimes
Actial
(D-NY)
Actual
The Center Squarr
0.00
(R-ME)
0.00
HUFFPOST
IIhan Omar (D-MN)
Jow Manchin (D-WV)
brobible
-0.25
-0.25-
-050
-0.50
salon
Kamala Harris (D-CA)
Elizabeth Warren (D-MA)
-1.00
11.00
-0.75
-0.50
-0.25
0.00 Predicted
0.25
0.50
0.75
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
Predicted
(a)
U.S. lawmakers (K 32 32 0.870)
(b) U.S. news outlets (pk (PR-8z 82 0.798)
Figure 3: Ideological perspectives of U.S. lawmakers and news media as predicted by the activations of the K 32 most predictive attention heads of 2aama-2--b-chhe Negative values correspond to left-leaning perspectives, while positive values correspond to right-leaning perspectives. The r-axis represents the predicted political slant (7k=32) for each entity (i.e., lawmakers or news media). The y-axis represents the previously validated ideological scores (DW-NOMINATE or Ad Fontes Media scores). See Figures Al and A2 for the complete results across all models.
Robustness checks of linearity First, we compare the predictive performance of our linear probes to those of more flexible non-linear probes. Following Gurnee & Tegmark (2024), we fit one-layer multilayer perceptions (MLPs) with ReLU non- lineariities, each of which is formulated as:
yeh =
+ be,b) + atih
where Br Ae, and by, agS are the weight matrices and bias vectors, respectively.
We do not observe substantial improvements when using such non-linear probes. For Llama-2-bb-chat, the most predictive linear probe had a cross-validation Spearman correlation of 0.854 while the best non-linear probe achieved 0.855. For Vicuna-7b, the difference was larger, with the linear and non-linear probes achieving 0.861 and 0.872, respectively. But for Mistral-7b-insten
the order reversed, with the linear and non-linear probes achieving 0.846 and 0.838, respectively. These results support the linear representation hypothesis for politi- cal ideology in the sense that linear functions of certain attention heads predict DW-NOMINATE approximately as well as non-linear functions of any others.
One may wonder whether there enough information stored in all the attention heads of an LLM be able to accurately predict any systematic label with linear probes. As a second robustness check, we applied different transformations to the DW-NOMINATE scores and examined whether linear probes could still fit them well. Specifically, we applied 1) a cubic transformation yd + (y which non-linean but still monotonic, 2) mon-monotoni transformation ( sin (1000)")) and 3) random permutation () <- y ( where A defines a permutation of the indices i.
which
The results are given in Figure A3 and Table A4 The probes trained on randomly permuted labels pro- vide baseline Spearman correlation of around () representing chance performance. Probes trained to predict the non-monotonis transformation perform poorly, with the best-performing heads achieving correlations of around 0.5. As might be expected, probes trained to predict the cubic transformation do much better, with the best-performing heads achieving rank correlations close to 0.84, In addition also include in
to rank correlation, which should not be sensitive to monotonic transformations, Table A4 the validaaiio R² values of the different probes. These tell a different story, with the cubic probes exhibiting values of around 0.6 compared to values of 0.8 achieved by the original.

---

Published as a conference paper at ICLR 2025
4 TRAINED PROBES DETECT POLITICAL PERSPECTIVE TOKEN-BY-TOKEN
The linear probes described in the last section were trained to predict the DW.NOMINATE y of lawmaker from the activations induced by prompt w The prompt includes the lawmaker's name and little else, so one may wonder whether probes' strong performance simply reflects models having "memorized" exact DW-NOMINATE scores, which are likely present in their pre-training data.
As a first investigation into whether the probes detected any generalizable representation of political ideology, instructed models generate essays on different policy issues (e.g.. immigration or abortion). We then recorded model activations token-by- by-tkee--tteen.... In this case, denote High h to be the activation of head h in layer for policy issue after t generated tokens. We then use the linear probe
(6.4) trained to predict DW-NOMINATE at that same attention head to calculate Wi.h (3..4) O 0g,, oi 2.h If the probe has learned to predict nothing other than DW-NOMINATE from lawmaker names, we should not expect such a measurement be interpretable when applied to open- responses. However, if probes have instead found more general hiiberal-consevattii expect this measure to position tokens along that axis in an interpretable manner.
ideological axis, then we might
We visualize this measure in Figure 1 where tokens are colored more red more blue according to whether 72,6 is more towards (conservative) or II (liberal). The results are highly interpretable. The probes detect a liberal perspective when writing "those who support immigration argue that can bring many benefits" or "a woman has the right to make decisions about her own body." By contrast, the probes detect a conservative perspective when writing "immigration can cause problems" or "abortion is the intentional termination of human life". We found similarly interpretable qualitative results in many other examples but leave for future work a more systematic evaluation of this qualitative measure. In Appendix A.2 provide the distribution of (( over many different policy issues for the three models; these results possibly indicate conservative skew for Mistral-7b.
5 TRAINED PROBES GENERALIZE TO PREDICT U.S. NEWS MEDIA SLANT
As a more systematic test whether the probes trained to predict DW-NOMINATE have truly detected a more generalizable representation of the ciberal-conssrrative" axis, tested whether such probes can predict the political slant of different U.S. media outlets. Again, "media slant" is a subjective and imprecise notion, but for which researchers have developed, validated, and relied upon data-driven measures. We find that probes trained only on DW-NOMINATE, can predict a media outlet's Ad Fontes score when LLMs are instructed to generate text from the perspective of that outlet.
Data We use data from Ad Fontes Media, which scores U.S. news outlets on a 5-point scale from "Left" to "Right". Ad Fontes Media determines these scores by aggregating the scores of individual articles, which are rated simultaneously by group at least three human analysts (Otero) 2021). These groups are politically balanced, consisting of one right-leaning, one left-leaning individual. These scores have been used by researchers (c.g., Huszár et al. (2022) accurate reflections of how outlet's slant is broadly perceived. We took the scores for the N 400 most popular outlets (e.g., Fox News, CNN) and normalized them to fall on the same scale as DW-NOMINATE of (Left) to I (Right).
centrist, and
Prompting We constructed probing data set of N 400 prompts, one for each outlet, with each prompt ( conforming to the following template:
USER:
Generate a statement from news source in the United States. ASSISTANT: [OUTLET] reported that
As before, for each of three LLMSs (LLama-2-7b-chal Mistral-7b-instruce Vicuna-7b) passed each of these prompts as input, and then recorded the activation x/th of each attention head in each layer f. This yields dataset of (y(9, ((())) pairs, where y(t is the Ad Fontes score of outlet i.
7

---

Published as a conference paper at ICLR 2025
Poltical Itant ref statement
5.5
Lama-2-7b (pm0.607)
5.0
Mistral-7o (p-0.396)
Immigratin
Gun contral
4.5
Vicuna-7b (p=0.381)
Aberiion
Climate change
Soc al mede repulatio
Sodia media regulation
Same-aac mar lage
3.5
LSSTG+ fihts
Immigratin
siantt
2.5
Climato change-
Same
2.0
Trusd science
1.5
Free speech
Free speech
Intervention (alpha)
30
0.0
Corriattoo
1.
0.25 0.00 0.25
Comtiattion
(a) The measured political slant of essay by the intervention parameter a used.
(b) Correlation between and political slant by issue.
(c) Correlation between and length of generated output.
Figure 4: Trained probes can be used effectively to steer the political slant of generated text; see (a). Steering is more reliable for certain policy issues, but has positive effect for all; see (b). LLMs steered toward more liberal positions certain policy issues tend to produce longer essays; see (c).
Evaluation Unlike before, do not train new probe on the collected dataset. Rather, we simply evaluate whether the probe previously trained to predict DW-NOMINATE for layer & and head is able to predict the Ad Fontes score yd from F2th-
To evaluate, (y and the ensembled predictions {yk] }i, as defined in Equation (6), using the K = 32 heads that were most predictive of DW-NOMINATE.
Spearman rank correlation between the set of observed Ad Fontes Media scores
Results We find the trained probes generalize well to predict media slant, with those for Llama-2-7b-cher achieving a Spearman correlation of 0.798, for Mistral-7b-instruce 0.764, and for Vicuna-7b 0.720. In Figure 3 we plot the predictions for -llam-2-7b--aee and highlight examples of well-known outlets; the same plot for all three models is given in Figures AT and A2
TRAINED PROBES CAN BE USED TO STEER POLITICAL PERSPECTIVE
If indeed probes have identified linear citberal--ooneeratiieee direction in activation space, it is natural to ask whether the political perspective in I.LM-generated text can be reliably steered by intervening linearly on its activations. In this section, we demonstrate this is the case.
Steering vectors Following the "inference time intervention" methodology of Li et al. (003 we
use the fitted regression coefficients 0g,, of the trained probes as steering vectors, which we add model activations over the course of text generation. More specifically, we intervene on the model by replacing the activation æg, in Equation with
DEN + θg or,
(8)
where ō.. is an estimate of the standard deviation of activations Tt,h, and a E IR controls the magnitude and direction of the intervention. An a with a larger negative value should steer the model to produce more liberal- souraiig text, while a more positive a should steer toward more conservative-sounding sounding text. For given a, we apply the intervention Equation (8) iteratively for every token the model generates and do so at all of the K most predictive attention heads (i.e., for all in the set Tx defined above Equation 6 The diagram in Figure A4 describes the entire procedure.
Study design To evaluate the effectiveness of these steering vectors, we instructed LLMs to generate text about nine key policy Abbotion,, Immigration, Gun Control, Same-Ser Marriage, LGBTQ+ Rights, Climate Change, Trust in Science, Social Media Regulation, and Free Speech- and examined whether intervening on their activations at various levels a produced predictable shifts in the political perspective of the text they generated. We used the following simple prompt:
8

---

Published as a conference paper at ICLR 2025
USER: Write a statement about [ISSUE]. ASSISTANT: Regarding [ISSUE], I believe that
In total, we generated 1,134 essays across three models, nine policy issues, and combinations of six val- ues of K E {16, 32, 48, 64, 80, 96) values and seven values of a E {-30, 20, -10, 0, 10, 20, 30) }
To measure the political perspective of each generated essay, we first recruited 10 human annotators from CloudResearch Survey-three Democrats, four independents, and three Republicans-ann and had them rate a random sample of the essays on a 7-point scale from "Strongly conservative" to "Strongly liberal". We then instructed GPT-4o scale (see Appendix A.3 for the exact prompt) and measured the inter-rater reliability between the GPT ratings and the average human ratings. GPT-44o's ratings were very close to the humans', with an intraclass correlation of 0.91, which we considered license to use for rating the entire essays; see Appendix A.4 for more details.
to rate the same essays on the same
Results We find that steering vectors reliably alter generated text toward political stances indicated by a. In Figure 44| we show the average rating of all essays that were generated with a given value of a, for the three different models. For all three, we see a clear trend, with larger a predicting more conservative-soundiin text. We also notice that with no intervention (a = 0), all three models show an average rating below 4 (on the 1-7 scale), indicating a base-level output of more liberal-sounding text.
When K' E (64, 80, 96), Llama-2-7b-chae displayed the highest correlation of 0.607 between a and political slant, followed by Mistral-7b-instruce at 0.396, and Vicuna-7b at 0.381. Political slant increased steadily as a increased, particularly in L1ama-2-7b-char 2-7b-chat, suggesting that this model is more sensitive intervention. We also experimented with intervening on different numbers of attention heads K, and found that intervening on more led to greater effectiveness; see Figure AS
In Figure 4b] we break results out by policy issue. The issues for which the intervention reliable were Immigration and Abortion. We conjecture that this is due to there being a wider array of stances such issues, compared issues like "Free Speech" or "Trust in Science" which exhibit smaller (though positive) correlations with a. Appendix A.5 gives illustrative examples.
most
We also observed that for certain policy issues, the LLMs generated much longer outputs when steered to sound more liberal than more conservative. This was true in particular for Gun Control and Climate Change; see Figure 4c A deeper look into these results might provide evidence for systemic differences in the argumentation style between liberals and conservatives, and highlight promising avenues for future research.
Robustness checks One might wonder whether the interventions describe will continue to be effective at steering when discussing policy issues not described in the model's pre- teeiiriinng data. In Appendix[A.6,
show that interventions remained effective when models were instructed to write about two events that fell after Llama-2- 2-7b-chat's pre-training cutoff: 1) the U.S. ADVANCE Act, and 2) the 2023 United Auto Workers (UAW) Strike.
One might also wonder whether interventions targeting different regions of the model (e.g.. early versus late layers) have different effects. In Appendix AA.7 show that interventions early-to- middle layers are effective, while those on middle-to-last layers have almost no effect.
7 FURTHER CONNECTIONS TO PRIOR RESEARCH
Political bias of LLMS One closely related area of research focuses on assessing the political "bias" of LLMs. Studies have found that LLMs tend to generate responses more closely aligned with liberal-leaning stances on various issues, regardless of user prompts and inputs (Santurkar et al.) 2023)
Motoki et al. 2024, Martin 2023 Potter al.) 2024 Liu al.) 2022] Bang et al. 2024). LLMs also often "avoid" engaging with certain political topics entirely (Bang et al. [022]. Political biases in the pre-training corpus of L.MSs can manifest in ways relevant to downstream tasks such as hate speech
and misinformation detection (Feng et al. 2023 Jiang et al. 2022: Liu al. 2022).
We found that generated texts for [o] > 30 were incoherent lacked comprehensiveness; Appendix A.8

---

Published as a conference paper at ICLR 2025
Nevertheless, robustly measuring the political biases of LLMs remains challenging. Close-ended survey questions, such as the Political Compass Test (Feng et al. 2023) or Pew surveys (Santurkar et al. ) are frequently used to assess LLMs' political biases. Yet, studies suggest that con- straining LLMSs to close-ended, multiple-choice formats may fail to capture biases that only emerge in open-ended responses (Röttger et al.. [2024] Goldfarb- Tarrant al. 202T). Recent studies also suggest that LLMs exhibit dishonesty (Huang et al., (024) and sycophancy (Sharma et al. (024) in their responses, which could potentially harm humans' ability to monitor bias in LLMs. As shown in Figure [I, our approach suggests a path to monitor and assess the political perspective implicitly adopted by LLMs
Linear scales of political ideology Linear representations of political ideology have a rich tradition in political science by way of "ideological scaling" techniques, such (DW)-NOMINATE (Poole & Rosenthal 1985 Poole 2005) and many related techniques. Work on "partisan sorting" argues that U.S. pohtical identity increasingly aligned along a single left-right axis, with increasing alignment between partisan identity and individual policy preferences (Levendusky) 2009). This uni-dimensional, linear model of political ideology is supported by empirical research showing that onc's position in this dimension correlates with broad range of issue stances, including economic policies, social issues like abortion and morality, and environmental concerns (Baldassarri & Gelman 2008 Fiorina & Abrams 2008 DellaPosta et al. 2015)
CONCLUSIONS, LIMITATIONS, AND FUTURE DIRECTIONS
Our research demonstrates that LLMs develop linear representations of political perspective within their hidden layers, locating subjective perspectives along a linear spectrum from left to right. By probing attention heads, we found that LLMs possess a generalizable linear representation of political perspective, which is highly predictive of established measures for the idcology of U.S. lawmakers the slant of U.S. news media. Importantly, we show that targeted interventions on these attention heads can causally influence the ideological tone of the generated text. This offers valuable insight and provides a method for identifying, monitoring, and steering the political perspective reflected in ALLM-generate text, with broader implications for the design and application of AI systems in societal contexts discussed in Appendix A.9
Our study has several limitations. First, the findings are based on relatively smaller models and may not generalize to larger or untested models. Second, although we observed a linear representation of political perspectives, this serves as an initial demonstration rather than an exhaustive analysis of the most effective methods to identify these directions. Methodological improvements in identifying such directions and subspaces are left for future work. Third, our research is U.S. centric and may not generalize to less polarized political environments, where linear representations of ideologies may be less effective representations (See Appendix 10 for details). In such settings, however, may characterize ideologies as the simplex of more than two "archetypal" or extreme political perspectives (Seth & Eugster 2016). Fourth, we use GPT-4o to evaluate political slant; however, there potential for bias when using L.MM as evaluator. Although we validate GPT-4o's evaluations against politically balanced human annotators, we recommend that future research using our methods continue to validate L.M- -e..eeerrdteed annotations against human annotations to triangulate and mitigate any inherent biases. Future research could also explore whether there are linear representations of more granular or intersected forms of political ideology. Other dimensions of cultural perspective (e.g., social class, gender) (Kozlowski et al., 2019) or knowledge and experience- expertise were not explored in this paper. We hope that future research will investigate this promising direction and its potential to craft and steer customizable L.L. agent perspectives.
*We note that political balance and fairness are not synonymous. Opinions on how to address faimess in LLMs regarding political bias vary widely. Some argue for ensuring that broad spectrum of perspectives are represented to align Al with societal perspectives (Sorensen et al, 2024), while others emphasize that fairness is most important insofar as it helps shift power away from oppressive institutions in favor of underrepresented stances and perspectives (Blodgett et al. 2020
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
This research addresses the sensitive issue of political ideology in LLMs. While our methods provide valuable tools for detecting and monitoring political ideology LLMs, they also carry potential risks of misuse. For example, malicious actors or Al product providers might exploit these techniques to deliver intentionally biased LLM outputs, bypassing societal discussions of fairness and transparency. Such misuse could generate biased content, manipulate public opinion, amplify divisive narratives. Additionally, privacy concerns arise these technologies are used to monitor political discourse on social media without consent.
We acknowledge these risks and emphasize that ethical responsibility ultimately lies with end users and organizations deploying these models. To mitigate these concerns, advocate for the development of robust ethical safeguards and guidelines for the responsible use of such tools.
Despite these challenges, believe that open, transparent research into idcological stance and bias in LLMs is critical for ensuring accountability and advancing scientific understanding. By making our work publicly available, we empower researchers study these technologies, monitor their societal impact, and develop measures to mitigate potential harms. We strongly urge the research community engage in collaborative efforts to address ethical challenges posed by LLMs.
REPRODUCIBILITY STATEMENT
The data and code for reproducing our results are available on Githut
ACKNOWLEDGEMENTS
We thank Victor Veitch for helpful discussions and feedback. Aaron Schein was supported in part by the John D. and Catherine T. MacArthur Foundation. James Evans was supported in part by grants from the National Science Foundation (2404109), DARPA (W911NF2010302), and Google, Inc.
REFERENCES
Robert Adcock and David Collier. Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95(3):529 -26,,6,,,666, 2001.
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In the Fifth International Conference on Learning Representations Workshop Track, 2017.
Jacob Andreas. Language Models as Agent Models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 5769-5779, 2022.
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems, volume 37, pp. 136037-136083, 2024.
Lisa P Argyle, Christopher A Bail, Ethan C Busby, Joshua Gubler, Thomas Howe, Christopher Rytting, Taylor Sorensen, and David Wingate. Leveraging Al for democratic discourse: Chat interventions can improve online political conversations at scale. Proceedings of the National Academy of Sciences. 120(41):c23) 1 1627120. 2023a.
Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337-31, 2023b.
Hui Bai, Jan Voelkel, Johannes Eichstaedt, and Robb Willer. Artificial Intelligence Can Persuade Humans on Political Issues. 2023.

---

Published as a conference paper at ICLR 2025
Delia Baldassarri and Andrew Gelman. Partisans without constraint: Political polarization and trends in American public opinion. American Journal of Sociology, 114(2):408 446,,44466666 2008
Yejin Bang, Nayeon Lee, Etsuko Ishii, Andrea Madotto, and Pascale Fung. Assessing Political Prudence of Open-Domain Chatbots. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 548-555, 2021.
Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. Measuring Political Bias in Large Language Models: What Is Said and How Is Said. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 11142-11159, 2024.
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational
Linguistics, 48(1):207-219, 2022.
Pietro Bernardelle, Leon Fröhling, Stefano Civelli, Riccardo Lunardi, Kevin Roiter, and Gianluca Demartini. Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas. arXiv preprint 1rX 14843, 2024.
Su Lin Blodgett, Solon Barocas, Hal Daumé III. and Hanna Wallach. Language (Technology) is Power: A Critical Survey of "Bias" NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decom- posing language models with dictionary learning. mmoosemantti--feature 2023.
2irbp pub/2023/
Royce Carroll, Jeffrey B Lewis, James Lo, Keith T Poole, and Howard Rosenthal. Measuring bias and uncertainty in DW-NOMINATE ideal point estimates via the parametric bootstrap. Political Analysis, 17(3):261-2755 2009.
Gary Charness, Brian Jabarian, and John A List. Generation Next: Experimentation with AL httpbberrrrrpppprr/r/eee hwtg//ppers/w31679 2023.
http://www.n .rg/
Thomas H Costello, Gordon Pennycook, and David G Rand. Durably reducing conspiracy beliefs through dialogues with ai. Science, 385(67 14):ead 2024.
Daniel DellaPosta, Yongren Shi, and Michael Macy. Why do liberals drink lattes? American Journal of Sociology, 120(5): 1473-151, 2015.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A Mathematical Framework for Transformer Circuits.
hut.rcuitssppb/2211111 htb/2/22/framework/ 2021.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hoddeed-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et Toy models of superposition. arXiv preprint arXiv:2209. 2tf.::2209.066222... 10652, 2022.
Phil Everson, Rick Valelly, Arjun Vishwanath, and Jim Wiseman. NOMINATE and American political development: a primer. Studies in American Political Development, 30(2):77-115, 1, 2016.
Expected Parrot. Steerable large language models.
2024.
Accessed: 2024-11-19.
Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models. arXiv preprint wwxiv:2305.0828. 2023.
Morris P Fiorina and Samuel Abrams. Political polarization in the American public. Annual Review of Political Science, 11(1):563-583) 2008.
Chen Gao, Xiaochong Lan. Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1:1122, 2024.
12

---

Published as a conference paper at ICLR 2025
Kostas Gemenis. What to do (and not to do) with the comparative manifestos project data. Political Studies, 61(1_suppl::322 23, 2013.
Seraphina Goldfarb-Tarrane Rebecca Marchant, Ricardo Muñoz Sánchez, Mugdha Pandya, and Adam Lopez. Intrinsic bias metrics do not correlate with application bias. In Pruceedings of the 59th Annual Meeling of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume Long Papers), pp. 1926-1940, 2021.
Wes Gurnee and Max Tegmark. Language models represent space and time. In the Twelfth Interna- tional Conference on Learning Representations, 2024.
Kobi Hackenburg and Helen Margetts. Evaluating the persuasive influence of political microtar- geting with large language models. Proceedings of the National Academy of Sciences, 121(24): e24031 16121, 2024.
Luke Hewitt, Ashwini Ashokkumar, Isaias Ghezae, and Robb Willer Predicting results of social science experiments using large language models. ffrrssulsx220offfoo %20 a atte://ceecceeeeeeeeeeeeeeennnnn 28models. 2024.
Youcheng Huang, Jingkun Tang, Duanyu Feng, Zheng Zhang, Wenqiang Lei, Jiancheng Lv, and An- thony G Cohn. Dishonesty Helpful and Harmless Alignment, arXiv preprint urXiv:2406.01931, 2024.
Ferenc Huszár, Sofia Ira Ktena, Conor O' Brien, Luca Belli, Andrew Schlaikjer, and Moritz Hardt. Algorithmic amplification of politics on Twitter. Proceedings of the National Academy of Sciences, 119()):e02233311) 19, 2022.
Abigail Jacobs and Hanna Wallach. Measurement and Fairness. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp. 375-385, 2021.
Hang Jiang, Doug Beeferman, Brandon Roy, and Deb Roy. CommunityLM: Probing Partisan Worldviews from Language Models. In Proceedings of the 29th International Conference Computational Linguistics, pp. 3818-6826, 2022.
Junsol Kim and Byungkyu Lee. Al-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction. arXiv preprint urXiv:2305.cm 09620, 2023.
Austin C Kozlowski, Matt Taddy, and James A Evans. The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings. American Sociological Review, 84(5):905-949, 2019.
Austin C Kozlowski, Hyunku Kwon, and James A Evans. In Silico Sociology: Forecasting COVID-19 Polarization with Large Language Models. arXiv preprint arXiv:2407.11190, 2024.
Matthew Levendusky. The partisan sort: How liberals became Democrats and conservatives became Republicans, University of Chicago Press, 2009.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Plister, and Martin Wattenberg. Inference- Time Intervention: Eliciting Truthful Answers from a Language Model. In Advances in Neural Information Processing Systems, volume 36, pp. 41451 1155--4530, 2023.
Ruibo Liu, Chenyan Jia, Jason Wei, Guangxuan Xu, and Soroush Vosoughi. Quantifying and alleviating political bias in language models. Artificial Intelligence, 304:103654 2022.
Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. First Conference on Language Modeling, 2024.
John Levi Martin. The Ethico-Political Universe of ChatGPT. Journal of Social Computing, 4(1): 1-11, 2023.
Nolan McCarty. defense of DW-NOMINATE. Studies in American Political Development, 30()):
172-184, 2016.
13

---

Published as a conference paper at ICLR 2025
Paul Michel, Omer Levy, and Graham Neubig. Are Sixteen Heads Really Better than One? In Advances in Neural Information Processing Systems, volume 32, 2019.
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic Regularities in Continuous Space Word Representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 746-751, 2013.
Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. More human than human: Measuring ChatGPT political bias. Public Choice, 198(1)33-23, 2024.
Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent Linear Representations in World Models of Self Supervised Sequence Models. In Proceedings of the 6th BlackboxNLPP Workshop:
Analyzing and Interpreting Neural Networks for NLP, pp. 16-30, 2023.
Sean O'Hagan and Aaron Schein. Measurement in the Age of LLMss: An Application to Ideological Scaling. arXiv preprint arXiv:2312.092033 2023.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context Learning and Induction Heads. arXiv preprint arXiv:2209.1 11895, 2022.
Vanessa
Fontes Ad- Mac-aaass-iaiiiiccciiaaaaaiaiai-
Otero.
Ad
Media
Content
Analysis
Methodol- -0001e
ogy.
pdf
2021.
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael Bernstein. Generative Agents: Interactive Simulacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 2023.
Joon Sung Park, Carolyn Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S Bernstein. Generative Agent Simulations of 1,000 People. arXiv preprint arXiv:2411.10100 2024a.
Kiho Park, Yo Joong Choe, and Victor Veitch. The Linear Representation Hypothesis and the Geometry of Large Language Models. In International Conference on Machine Learning, pp. 39643-39666. PMLR, 2024b.
Keith Poole. Spatial Models of Parliamentary Voting. Cambridge University Press, 2005.
Keith T. Poole and Howard Rosenthal. A Spatial Model for Legislative Roll Call Analysis. American Journal of Political Science, 29(2):357-388 1985.
Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 4244-4275. 2024.
Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15395-15311, 2024.
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose Opinions Do Language Models Reflect? In Proceedings of the 40th International Confer- ence on Machine Learning, volume 202, pp. 29971-30004, PMLR, 2023.
Sohan Seth and Manuel JA Eugster. Probabilistic Archetypal Analysis. Machine Learning, 102: 85-113, 2016.
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Satfield-Dodde Scott R Johnston, et al. Towards under- standing sycophancy in language models. In the Twelfth International Conference on Learning Representations, 2024.
14

---

Published as a conference paper at ICLR 2025
Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L Gordon, Niloofar Mireshghallah, Christo- pher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, et al. Position: Roadmap to Pluralistic Alignment. In Forty-first International Conference Machine Learning, 2024.
Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda. Linear Representations of Sentiment in Large Language Models. arXiv preprint 1..v::2310.1554 2023.
Petter Törnberg. Diliara Valeeva, Justus Uitermark, and Christopher Bail. Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms. arXiv preprint arXiv:2310.05598, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288 2023.
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering, arXiv preprint arXiv:2308. /0248, 2023.
Dimitri Von Rütte, Sotiris Anagnostidis, Gregor Bachmann, and Thomas Hofmann. Language Model's Guide Through Latent Space. In Proceedings of the 41st International Conference Machine Learning, volume 235, pp. 49655-49687, 2024.
Patrick Y Joshua Tucker, Jonathan Nagler, and Solomon Messing. Large Language Models Can Be Used to Estimate the Latent Positions of Politicians. arXiv preprint arXiv:2303.120555 2023.
Patrick Y Wu, Jonathan Nagler, Joshua A Tucker, and Solomon Messing. Concept-Guided Chain-of- Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models. In 2024 IEEE International Conference on Big Data (BigData), pp. 7232-7241. IEFE, 2024.
15

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 MODEL OVERVIEW
In this study, we use three open-source large language Mistral--b-instruut-vo. 1, and Vicuna-7b-v1.55
models:
-lama-2-1 2- 2-7b-chat,
This model is part of the Llama-2 family, developed by Meta, with 7 billion parameters. It consists of 32 transformer layers, each equipped with 32 attention heads and a hidden dimension size of 4096. The model is optimized for conversational tasks.
fine-tuned version of the base Mistral-7b model for instruction custructionfoolloinng tasks. Similar to Llama-2-7b-on chat,, contains 32 transformer layers with 32 attention heads per layer and hidden dimension size of 4096, summing up to 7 billion parameters. The model is optimized for conversational tasks.
Mistral-7b-instruct
Mí
Accuna7b-vv1. 5: Vicuna- 7b is fine-tuned version of 1lama-2, optimized for conversa- tion tasks. This model also contains 7 billion parameters, with 32 transformer layers, 32 attention heads per layer, and hidden dimension size of 4096. The fine-tuning focuses on generating dialogue responses.
A.2 TRAINED PROBES DETECT POLITICAL PERSPECTIVE TOKEN-BY-TOKEN
As described in Section 4 perspective reflected in the tokens. this case, denote E,h to be the activation of head in layer for policy issue i after é generated tokens. We then use the linear probe trained to predict (6.1)
record model activations token-by-token to detect political
(i.6)
((c.)) DW-NOMINATE: at that same attention head order to calculate Vi,h 02 %
To examine the distribution of Vi.h across models, write about nine politically divisive topics (Abortion, Immigration, Gun Control, Same-Soe Sex Marriage, LGBTQ+ Rights, Climate Change, Trust in Science, Social Media Regulation, and Free Speech) in the United States. Importantly, political perspective. As a result, each model generates 90 essays (i.e., 10 essays per topic). While generating these essays, we obtain values from the 96 most predictive attention heads.
the following prompts ask the LLMs to
do not prompt intervene in the models to simulate particular
Write an essay on [TOPIC] in the U.S.
Discuss [TOPIC] in the United States.
Explain the key issues surrounding [TOPIC) in the U.S.
Provide an overview of (TOPIC) policies in America.
Summarize the debate on [TOPIC] in the United States.
Analyze the topic of [TOPIC] in the U.S.
Give detailed account of [TOPIC] in the United States.
Explore the complexities of [TOPIC] legislation in the U.S.
Describe the current landscape of [TOPIC] in the United States.
Offer comprehensive discussion on the state of [TOPIC] in the U.S.
As shown in Figure A6 even though we do explicitly prompt LLMs adopt a political stance, we find that the models spontaneously generate texts reflecting either left- or right-leaning perspectives, as indicated by the token-level values of political slant (..t) Specifically, Mistral-7b-inst
c(2.1) shows an average Mi of 0.143, exhibiting slightly more right-lcaning slant compared to also dis- plays about twice the diversity in perspectives, measured by the standard deviation of 2666)) with 0.649 compared Llama-2- 2iaa--2-7b-cha (a 0.349) and Vicuna-7b (a 0.356).
Llama-2-7b-chat (0.054) and Vicuna-7b (0.048). Interestingly, Mistral-7b-inst
16

---

Published as a conference paper at ICLR 2025
A.3 PROMPTS TO RATE POLITICAL SLANT REFLECTED IN LLM RESPONSES
We use GPT-4o (gpt-40-22 2o9t-4o-2024-8- 202--08-200000 to rate the political slant underlying LLM responses. Specifi- cally, for each LLM essay under intervention, we prompt GPT-4o as follows:
Based on the following statement, how would you rate the political ideology of the writer?
[STATEMENT]
On a scale of 1 to 7, where:
Strongly conservative
Moderately conservative
Slightly conservative
Nei Neither conservative nor liberal
Slightly liberal
6 Moderately liberal
Strongly liberal
Please respond th a number from without providing reasons.
A.4 VALIDATING GPT-40 ANNOTATIONS AGAINST POLITICALLY BALANCED HUMAN ANNOTATORS
We validate GPT-4o's evaluations against politically balanced human annotators. Specifically, sampled politically balanced human annotators from the CloudResearch survey platform (N=10, U.S. residents consisting of 3 Democrats, Independents, and 3 Republicans) annotate random sample of 21 essays generated by Llama-2-7b-chat. After averaging the scores provided by these human annotators, we measure inter-rater reliability between GPT-4o and the human annotators' average scores. We find a high inter-rater reliability score (ICC(A, 1) .91), supporting the validity of GPT-4o annotating political slant. Based on these results, we conclude that GPT-4o's 's ratings align closely with the consensus of politically balanced human annotators. Our findings are consistent with O'Hagan & Schein (2023), which demonstrate that LLMs can reliably reflect established measures of ideological slant. Additionally, the Spearman correlation between GPT-4o and average human scores high (p = 0.952).
A.5
EXAMPLES OF 1aama-2-7b-cha RESPONSES IN THE INTERVENTION EXPERIMENTS
A.5.1
IMMIGRATION
Prompt
Write statement about immigration.
Original Output (a=0) Regarding immigration, believe that it is complex issue that requires comprehensive and humane approach. It is important to recognize the contributions that immigrants make the economy, culture, and society, while also addressing the legitimate concerns of border security and the rule of law. support policies that provide clear and efficient pathway to citizenship for undocumented immigrants who are already living and working the US, as well as measures to address the root causes of migration, such as poverty, violence, and political instability, in countries of origin. Furthermore, I believe that immigration should be based on the principles of fairness, justice, and compassion, and that all immigrants should be treated with dignity and respect. In conclusion, immigration a critical issue that requires a thoughtful and
Steered Output (a=-20) Regarding immigration, believe that is a complex and multifaceted issue that affects not only the lives of immigrants themselves, but also the broader society. It is important to recognize the contributions that immigrants have made and continue to make to our communities. and to work towards creating same time, it is crucial address the systemic injustices and inequalities that have long plagued the immigration system, and to work towards more just and humane approach to immigration policy.
inclusive and equitable society for all. At the
17

---

Published as a conference paper at ICLR 2025
Ultimately, the struggle for immigrant rights is not just struggle for justice, but also a struggle for more inclusive and equitable society for all.
Steered Output (a=20) Regarding immigration, I believe that immigration laws should be enforled and regulated in a way that protects the national security and safety of country. However, also believe that immigration should be based merit and not the number of people entering country. I think is important to have a system that allows for controlled immigration, rather than uncontrolled immigration. This means that immigration should be based on the needs of country, rather than the needs of an individual. also believe that immigration should be based on the principles of freedom and liberty, rather than on the principles of socialism globalism.
A.5.2 SAME-SEX MARRIAGE
Prompt Write a statement about same-sex marriage.
Original Output (a=0) Regarding same-sex marriage, I believe that all individuals, regardless of the right to marry the person they love and have their relationship recognized by the govermment. This a fundamental human right and essential for the well-being and dignity of all individuals.
their sexual orientation, should have
Steered Output (a=-20) Regarding same-sex marriage. believe that all individuals, regardless of their sexual orientation, should have the same rights and opportunities as straight couples. The ability marry the person you love and build a life together a fundamental human right, and it's time that we recognize and respect this right for all couples, regardless of their sexual orientation.
Steered Output (a=20) Regarding same-sex marriage, believe that all individuals should be able to marry someone of their choice, regardless of their gender or sexual orientation. However, I also believe that marriage should be defined as being between man and a woman, as this is the traditional and biblical definition of marriage. I believe that the government should not be forced to recognize same-sex marriages, as this goes against the Bible's teachings on marriage and the family.
A.5.3 ADVANCE ACT
Prompt Write political opinion about the following event: The ADVANCE Act of 2024 was signed into law on July 9, 2024, as part of the Fire Grants and Safety Act. The act supports advanced nuclear energy development in the United States. It promotes generation IV nuclear reactor technology. It reduces licensing costs for nuclear energy projects. extends liability protections for the nuclear industry. The act directs the Nuclear Regulatory Commission (NRC) to streamline licensing processes for advanced and small modular reactors (SMRs). It includes incentives for next-generation nuclear technology through reduced fees and prize for deployment. It restricts nuclear fuel imports from Russia and China. It fosters U.S. nuclear exports and international collaboration. The act contains provisions for environmental remediation on tribal lands. Licensing changes in the act are designed to facilitate advanced reactor deployment at brownfield sites.-
The act follows the Prohibiting Russian Uranium Imports Act.
Original Output (a=0) I believe that the ADVANCE Act of 2024 is a crucial step towards a more sustainable and secure energy future for the United States. By supporting the development of advanced nuclear energy technologies, such generation IV reactors, the act helps to reduce the country's reliance on fossil fuels and mitigate the impacts of climate change. Additionally, the act's provisions for licensing reform and reduced fees for next-generation nuclear technology incentivize innovation and deployment, which will help to drive down costs and increase the competitiveness of nuclear energy in the U.S. electricity market. Moreover, the act's restrictions nuclear fuel imports from Russia and China help to promote energy security and reduce the risk of dependence on unreliable suppliers. By fostering U.S. nuclear exports and international collaboration, the act also helps to promote global nuclear safety and security standards, and to advance the global transition to a low-carbon energy future.
Steered Output (a=220) believe that the ADVANCE Act of 2024 is a crucial step towards a more sustainable and equitable future for the United States. By promoting the development and
18

---

Published as a conference paper at ICLR 2025
deployment of advanced nuclear energy technology, the act is not only addressing the urgent need to reduce greenhouse gas emissions but also creating new economic opportunities for communities that have been historically impacted by environmental degradation. Moreover, the act's focus on reducing licensing costs and extending liability protections for the nuclear industry is a critical step towards making nuclear energy more accessible and affordable option for all. This is particularly important in light of the ongoing climate crisis, which has disproportionately impacted marginalized communities. Furthermore, the act's provisions for environmental remediation tribal lands and its support for U.S. nuclear exports and international collaboration are crucial in building more just and equitable nuclear energy sector.
Steered Output (a=20) I believe that the ADVANCE Act of 2024 is positive step towards the development of nuclear energy in the United States. The act supports the development of advanced nuclear energy by reducing licensing costs, extending liability protections, and streamlining the licensing process for advanced and small modular reactors. This will allow for the deployment of next-generation nuclear technology, which is essential for the security of the United States. However, I also believe that the act is necessary due to the current political climate and the need for the United States to secure its energy sources. The United States should not rely on foreign sources of energy, especially from countries such as Russia and China, which have a history of using their energy resources for political gain. The United States should prioritize its own energy security and protect its interests by reducing its reliance on foreign energy sources. In conclusion, believe that the ADVANCE Act of 2024 is positive step towards the development of nuclear energy in the United States.
A.5.4 UNITED AUTO WOKERS (UAW) STRIKE
Prompt Write political opinion about the following event: -he 2023 United Auto Workers (UAW) strike lasted from September 15 to October 30. Approximately 49,800 union members participated in the strike. The strike was directed against Ford Motor Company, General Motors, and Stellantis. The primary disputes were over labor contract negotiations. Key union demands included: Wage increases to counteract inflation. Elimination of a tiered employment system. Improved benefits. Worker protections against plant closures. A four-day workweek. This was the first simultancous strike against all three automakers. "rolling strike" " strategy was used to conserve union resources. Automakers expressed concerns about labor costs and competitiveness during the transition to electric vehicle production. Tentative agreements were reached with all three companies by late October. The agreements included: Significant wage increases. Reinstatement of cost-of-living adjustments. - Elimination of the two-tier wage system. The strike concluded after 46 days, pending union member ratification of the agreements.
Original Output (a=0) I believe that 2023 United Auto Workers (UAW) strike is a significant event that had profound impact on the automotive industry. The strike, which lasted from September 15 to October 30, involved approximately 49,800 union members and was directed against Ford Motor Company, General Motors, and Stellantis. The primary disputes were over labor contract negotiations, with key union demands including wage increases to counteract inflation, elimination of a tiered employment system, improved benefits, worker protections against plant closures, and a four-day workweek. This was the first simultaneous strike against all three automakers, and the UAW employed a "rolling strike" strategy to conserve union resources.
Steered Output (a=-20) I believe that the recent UAW strike is a significant political event that highlights the ongoing struggle for workers' rights and fair labor practices. The strike, which was directed against Ford Motor Company, General Motors, and Stellantis, was a powerful display of solidarity and collective action by union members. The primary demands of the strike were labor contract negotiations, and the union's 's key demands included wage increases to counteract inflation, climination of a tiered employment system, improved benefits, worker protections against plant closures, and four-day workweek. These demands are not only important for the workers involved but also have broader implications for the broader labor movement and the broader struggle for workers' rights and fair labor practices. The strike also highlighted the ongoing struggle"
Steered Output (a=20) I believe that the strike was unnecessary and the unions should have been more restrictive in their demands.
19

---

Published as a conference paper at ICLR 2025
A.6 SIMULATING OPINIONS REGARDING EVENTS NOT IN THE PRE- TRAINIGIN DATA
To examine whether linear interventions in LLMs can simulate ideological perspectives for events study the Accelerating Deployment of
not included in their pre-training data, conduct Versatile, Advanced Nuclear for Clean Energy (ADVANCE) Act (March 2023) and the 2023 United Auto Workers (UAW) strike (September 2023). Both events occur after the knowledge cut-off date of Llama-2-7b-chat' pre-training data (September 2022) (Touvron et al. 2023). To confirm that the model has no prior knowledge of these events, we first prompt with the question. you have information about [event]?" The responses indicate that it lacks accurate information about the event, either by responding "No" or generating hallucinated descriptions.
Then, using GPT-4o, we generate factual descriptions regarding each event. Specifically, we use two-step approach: (1) we provide Wikipedia article and prompt GPT-4o generate concise, one- paragraph factual summary, and (2) prompt GPT-4o again to eliminate any subjective opinions from the paragraph and present the factual, neutral information in bullet points. The following prompts are used:
Provide a factual summary of the situation described in the Wikipedia article in one paragraph, avoiding any mention of opinions or perspectives associated wi th U.S. Democrats or Republicans.
From the following paragraph, remove any subjective opinions. Then, extract list the factual and neutral information bullet points.
After generating the factual summary, provide this text Llama-2-7b-chat using the following prompts. For each event, we ask the model to generate relevant texts with slightly different prompts (e.g.. Write a political opinion about the following event, Write an essay about the following event, Write statement about the following event).
Write a [political opiniooneessay//saaeeentt about the following event:
The ADVANCE Act of 2024 was signed into law July 9, 2024, part of the
Fire Grants and Safety Act.
The act supports ts advanced nuclear energy development in the United States.
promotes generation IV nuclear reactor technology.
- It reduces licensing costs for nuclear energy projects.
extends liability protections for the nuclear industry. The act directs the Nuclear Regulatory Commission (NRC) to streamline
licensing processes for advanced and small modular reactors (SMRs).
It includes incentives for next-generation nuclear technology through reduced fees and prize for deployment.
It restricts nuclear fuel imports from Russia and China.
It fosters U.S. nuclear exports and international collaboration. The act contains provisions for environmental remediation tribal lands.
Licensing changes in the act designed facilitate advanced reactor deployment at brownfield sites.
The act follows the Prohibiting Russian Uranium Imports Act.
20

---

Published as a conference paper at ICLR 2025
Write [political opinion/essay//statemmee about the following event: The 2023 United Auto Workers (UAW) strike lasted from September 15 to October 30
800 union members participated in the strike. The strike was directed against Ford Motor Company, General Motors, and Stellantis.
Approximately 49, 89,
The primary disputes were over labor contract negotiations.
Key union demands included:
Wage increases to counteract inflation.
Elimination of tiered employment system.
Improved bsnefits.
worker protections against plant closures. - A four-day workweek.
This was the first simultaneous strike against all three automakers.
A "rolling strike" strategy was used to conserve union resources.
Automakers expressed concerns about labor costs and competi vivesss
during the transition to electric vehicle production.
Tentative agreements were reached with all three companies by late October.
The agreements included:
Significant wage increases.
Reinstatement of cost-of-living adjustments.
Elimination of the two-tier wage system.
The strike concluded after 46 days, pending union member ratification of the agreements.
Political essays are then generated with varying levels of intervention, using the linear steering method described in Section 6 with values a E [-30,- 200 100, 0, 10, 20, 30). We generated total of 21 essays per event. To evaluate the ideological slant of these essays, GPT-4o (trained after the knowledge cut-off of -alaam2-77-aae political slant scale where lower values (I) indicate liberal perspectives and higher values (7)
and thus familiar with these events) annotate the
indicate conservative ones.
The results show a statistically significant correlation between the intervention parameter (a) and the annotated political slant. Specifically, both the ADVANCE Act (p 0.479) and the United Auto Workers (UAW) Strike (p 0.470) exhibit significant correlations. For example, when LLMs are prompted about the ADVANCE Act, an intervention with - 20 generates texts aligned with left-leaning views, supporting the act for promotion of the nuclear energy industry but emphasizing its "environmental benefits." Conversely, right-leaning views, supporting the act due to its focus on "restricting nuclear fuel imports from Russia and China." These results indicate that, following interventions to simulate left- right-leaning perspectives, the model not only predicts bipartisan support for the act but also captures nuanced differences in the reasons left- and right-leaning individuals support it (See Appendix A.5 for details). These findings suggest that linear interventions can simulate ideological biases, even for unforeseen events not included in pre-training data. This indicates that the linear structures identified in the model's activations might capture more than just superficial patterns in the training data.
intervention with 20 produces texts aligned with
A.7 INTERVENTION TARGETING SELECTED LAYERS
As Figure shows, there are two "regions" the attention heads that correlate with political slant: early to middle layers (Layers 1-21) versus middle to last layers (Layers 22-32). We conduct additional analyses on Llama-2-7b-che (closer to input) versus middle to last layers (closer to output) affect ideological expression in responses (see Figure A7 Interventions targeting early to middle layers lead to more substantial ideological changes, detected by GPT-4o (p 0.540). For example, when LLMs asked about same-sex marriage, a eight-leaaing intervention (a = 20) at these layers produces statements like, "I believe that marriage should only be between man and woman, this is the biblical definition of
to examine how interventions in early to middle layers
21

---

Published as a conference paper at ICLR 2025
marriage." (See Table A5
In contrast, interventions in the middle to last layers do not appear to alter the underlying ideological content (p = -0.022).
A.8 THE RANGE OF a FOR GENERATING COHERENT RESPONSES
We conduct experiments determine the minimum and maximum values of a required for generating coherent responses. We instruct LLMs to generate short essays on nine politically divisive issues in the United States: abortion, immigration, gun control, same-sex marriage, LGBTQ+ rights, climate change, trust in science, social media regulation, and free speech. These essays are generated under interventions with varying values of a (-50, -40, -30, -20, -10, 0, 10, 20, 30, 40, 50). After generating responses, we use responses are coherent and comprehensive. Specifically, we prompt GPT-4o as follows:
-gpt-4o-2024-08-8 06) to evaluate whether the LLM-generated
Is the following text incoherent or lacking comprehensiveness? [ESSAY]
1: No, the text is coherent and comprehensive Yes, the text is incoherent and lacks comprehensiveness.
Please respond wi th a number (1 or 2) wi thout providing reasons.
We find that if the value of a smaller than -30 or bigger than 30, the proportion of coherent responses is always lower than 40% across three models (see Figure[A3) [88for details). For instance, we prompt the model to write an essay about abortion, intervening on the model activation with generates the following endlessly repetitive nonsense text.
50, Histral-7b-instrue
I believe that everyone has the right to access heal thcare, regardless of whether they choose to work with or without these rights. However, the fact that they are not able to do so, that they are not able to work, are not considered, or are not able to work, or are not able to work, or not able work, or not able to work
Therefore, we conduct our main intervention experiments with a values ranging from -30 30.
A.9 PRACTICAL APPLICATIONS
Our method can serve valuable "auditing" tool, allowing users to monitor the political perspectives that L.LMs simulate and identify the contexts in which these perspectives are activated an important consideration for transparent model behavior. Close-ended survey questions, such the Political Compass Test (Feng et al. (023) or Pew surveys (Santurkar al., 2023), are frequently used as tools to monitor LLMs' political biases. Yet, studies suggest that constraining LLMs to close-ended, multiple-choic formats may fail to capture biases that occur in open-ended responses (Röttger et al.. approach provides alternative way
2024 Goldfarb-Tarne
et al. 2021). As shown in Figure to monitor and assess the political perspectives employed by LLMSs, enhancing transparency around potential biases in their open-ended outputs.
Our approach also offers a practical means for steering LLM outputs during inference, enabling the creation of synthetic documents with tailored ideological perspectives (Argyle al. 2023b Andreas 2022:
Kim & Lee| 22023] Kozlowski et al., 2024 O Hagan & Schein 2023). This is computationally less expensive than methods like fine-tuning (Jiang et al.) (022) and has applications in both academic and industry settings. For example, products such as Expected Parrot enable users to simulate human behaviors or opinions silico (Fxpected Parrot 2024), and our method could enhance these capabilities by providing fine-grained control over subjective perspectives.
A. CROSS-NATIONAL ANALYSIS OF POLITICAL SLANT REPRESENTATIONS
In addition to our primary focus on U.S. political contexts, we conduct supplementary analyses to evaluate the generalizability of linear probes in predicting political slant across various non-U.S.
22

---

Published as a conference paper at ICLR 2025
nations. This analysis utilizes the Manifesto Project dataset, which provides ideological labels y for 411 political parties worldwide on left-to-right continuum (from -50 = left to 50 = right) (Gemenis 2013). Results reveal both strengths and limitations in extending the learned representations of political ideology to diverse cultural and national contexts.
We utilize the following prompt simulate the ideological perspectives of lawmakers in these parties:
Generate statement by Lawmaker from the [PARTYNAME] party, which is a political party in [COUNTRYNAME]. ASSISTANT: In 2019, lawmaker from the [PARTYNAME] party said that
USER:
As described in Section 3 we pass each of these prompts as input Llama-2-7b-chen and then record the activation 2ihh of each attention head /t in each layer t. This yields a dataset of (y a pairs, where yr is the Manifesto Project score of party i.
(4)
Then, we evaluate whether the probe previously trained to predict DW-NOMINATE for layer and head h is able to predict the Manifesto Project score y from E, We use Spearman rank correlation between the set of observed Manifesto Project scores {y and the ensembled predictions {u ( Js+ as defined in Equation (6) using the K = 32 heads that were most predictive of DW-NOMINATE.
As shown in Figure A9 we find that the linear probes exhibit modest performance predicting the political slant of non- U.S. parties, achieving Spearman correlation of 0.531, substantially lower than the performance observed for U.S. lawmakers (p 0.870) and U.S. news media (p 0.765). The generality of political slant representations varies significantly across nations. Some countries, such as New Zealand (p 0.920), Australia (p 0.916), Canada (p 0.883), and the United Kingdom (p = 0.845), exhibit strong correlations, while others demonstrate weaker or even negative correlations, indicating that the applicability learned representations is influenced by the political landscape and cultural context. These findings underscore the need for comprehensive datasets that capture diverse political contexts, particularly for underrepresented regions, and we encourage the Al research community prioritize the creation of such datasets to improve the cross-cultural applicability of LLMs.
A.11 REPLICATION ON Gemma-2-22
Some might question whether political ideologies are similarly represented in the middle layers of LLMS outside the Llama family. We successfully replicated our analysis the Gemma-2-22 model and found that also exhibits a linear representation of idcological slant its middle layers. See Figure A10 for details.
23

---

Published as a conference paper at ICLR 2025
Table A1: Effect regularization parameter on probe performance. We compare CCV at the most predictive attention head across models for different values. The value optimizes these metrics across the three models, except for Vicuna-7b.
Llama-2-7b-chae
Histral-7b-instrn
Vicuna-78 7b
0.8182
0,8150
0.8284
0.001
0.8348
0.8163
0.8423
0.01
0.8437
0.8240
0.8447
0.1
0.8476
0.8395
0.8616
0,8536
0.8463
0,8612
100
0.8463
0.8434
0.8561
1000
0.8429
0.8462
0.8448
Figure Al: Ideological perspectives of U.S. lawmakers captured by the activation space in 2-7bhchat,,,,, Mistral-7b-instruct and Vicuna-7b. Negative values correspond to left- leaning perspectives. while positive values correspond to right-leaning perspectives. Predicted ideological perspectives have been obtained by activations from 32 most predictive attention heads.
1.50
1.00
1.60
330
325
0.25
-6.35
-125
-025
-6.5
-3.9
-0.35
"300 -..5--3--------0 -32- 8 225 030 5.5
100
-100
00-2775 -350-022
0:00 025
750
0.5
100
1010000755--00 -0a odee 01s w3e 0.5 1io
(a) Llama-2--11 (p% 0.870)
(b) Mistral-76 (K 0.864)
(c) Vicuna- 7b (Pk 0.885)
Figure A2: Ideological perspectives of U.S. news outlets captured by the activation space in 1lama-2-7b-chat, Mistral-7b-instruct and Vicuna-7b. Negative values correspond to left- leaning perspectives, while positive values correspond to right-leaning perspectives. Predicted ideological perspectives have been obtained by activations from 32 most predictive attention heads.
0.75
0.03
500
0.00
-0.2
-025
-0.25
5.0
-0.5
-075
-0.75
-3.4
0.4
-0.2
05 Prdditted
Pretiese
Predetted
LLama- CKk 0.798)
(b) Mistral-77 CCC 0.764)
(c) Vicuna-7b ( 0.720)
24

---

Published as a conference paper at ICLR 2025
Table A2: Top 10 attention heads according cross-validation Spearman rank correlation.
Llama-2-7b-chat
Mistral-7b-instruct
Vicuna-7b
Rank
(Layer, Head)
(Layer,
(Layer,
Spearman
PEA
Head)
Spearman PEA
Head)
Spearman PEA
(15, 18)
0.8536
(16, 3)
0.8463
(24, 8)
0.8612
2
(16, 11)
0.8444
(16, 1)
0.8453
(22, 13)
0.8609
3
(18, 4)
0.8441
(18, 7)
0.8381
(17, 20)
0.8593
4
(15, 2)
0.8437
(27, 17)
0,8305
(26, 5)
0,8533
5
(17, 20)
0.8428
(15,3)
0.8299
(16, 11)
0.8528
(15, 9)
0,8406
(16, 9)
0,8288
(18, 14)
0,8523
(26, 5)
0,8399
(15,
0,8272
(23.5)
0,8509
(16, 19)
0.8394
(14, 11)
0.8265
(20,8)
0.8503
(14, 26)
0.8386
(22,
0.8263
(29, 25)
0.8499
10
(16, 23)
0.8371
(11,
0.8262
(14,26)
0.8490
Table A3: Spearman rank correlation (P for ensemble predictions given
the number of attention heads (K). The value K 32 optimizes these metrics, except for Histral-78-Instree
Number of Attention Heads K
Llama-2-7b-chat
Mistral-7b-Instruct Vicuna-7b
0.8537
0.8468
0.8623
0.8695
0.8665
0.8850
0.8703
0.8636
0.8851
0.8684
0.8630
0.8813
96
0.8652
0.8591
0.8791
128
0.8625
0.8571
0.8766
256
0.8545
0.8496
0.8700
512
0.8476
0.8420
0.8634
25

---

Published as a conference paper at ICLR 2025
Figure A3: DW-NOMINATE scores and transformed DW-NOMINATE scores predicted by linear probes on the activations of Llama-2 2-----chaat
1.0
0.5
0.5
Actual
Actual
-0.5
1.0
-1.0
D.5
0.0 Predicted
0.5
1.0
-1.0
-0.5
0.0 Predicted
0.5
1.0
DW-NOMINATE
(b) Randomized DW-NOMINATE
0.5
Actual
Actual
0.0
-05
-1.0
-1.0
-0.5
0.0
-0.5
0.0 Predicted
Predicted
(c) sin(10 DW-NOMINATE
(d) (DW.NOMINATE)
26

---

Published as a conference paper at ICLR 2025
CV
Table A4: Spearman correlation CCY and R-squared score R2 e,a. for different models and transforma- tions of DW-NOMINATE scores.
Spearman correlation (PEX)
y
Randomized (
(in(10y(
(1/3
Llama2-7b-chat
0.854
0.152
0.551
0.842
Mistral-7b-instrue
0.846
0.140
0.534
0.841
Vicuna-7b
0.861
0.156
0.558
0.860
CV
R-squared score (R²
t,h)
(+)
Randomized (9)
sin(10y
(1/3
Llama2-7b-chat
0.821
0.019
0.298
0.613
Mistral-7b-instee
0.832
0.025
0.260
0.601
Vicuna-7b
0.830
0.012
0.303
0.626
27

---

Published as a conference paper at ICLR 2025
Figure A4: Intervention workflow. Squares indicate natural language texts. Circles indicate vectors.
A. Probing
1. Prompting LLM to generate statements by politicians with DW- NOMINATE scores
2. Extracting activations from each attention head
3. For each attention head, training linear probe to predict DW-NOMINATE scores
Prompts:
ta a E.
E.g., Generate a statement by Kamala Harris
LLM
y ye e,h
-
2 (1)
(1)
Prompts:
(9)
E.g.. Generate a statemeett Donald Trump
LLM
B. Intervention
4. Intervention using the learned regression coefficient Oe,xwhich captures the ideological direction in the activation space of each attention head.
Evaluate the steered outputs using GPT-4o
1=Extreme Liberal
7=Extreme Conservative
User prompt:
Write an essay about immipration
Steered output
i Or, Intervention
LLM
(n) A FLA a
28

---

Published as a conference paper at ICLR 2025
Figure A5: Intervention (a) and political slant reflected in the statement by the number of attention heads intervened (i.c., K).
5.0
16
4.5
32
48
4.0
Political slant
64
3.5
80
96
3.0
2.5
2.0
1.5
-30
-20
-10
0
10
20
30
Intervention (Alpha)
29

---

Published as a conference paper at ICLR 2025
Figure A6: Distribution of political slant (7hh token by token.
Mistral-7b
1.5
Vicun-77
Ulama-2-75
Density
1.0
-1.5
-1.0
-0.5
0.5
1.5
Political Slant
Figure A7: Intervention (a) and political slant are reflected in the statements by the targeted layers for LLama-2-7b-chat, Mistral-7b-instruce and Vicuna-7b (K = 96). Layers < 22 indicate interventions in the early to middle layers, while Layers 2 22 indicate interventions in the middle to last layers. Compared to interventions Layers < 22 (p 0.540), interventions in Layers 22 does not manifest significant effect (p- -0.022).
Political slant reflected in the statement
Layer<22 (p=0.540)
Layerz22 (p=-0.022)
5
4-
3
2
1
-30
-20
-10
0
10
20
30
Intervention (alpha)
30

---

Published as a conference paper at ICLR 2025
Figure A8: Proportion of coherent LLM responses by
100
% Coherent Responses
80
60
40
Llama-2-7b Mistral-7b
20
Vicuna-7b
0
-50
-40
-30
20 Intervention (Alpha)
-20
-0
0
10
30
40
50
Figure A9: A, Ideological perspectives of political parties outside the U.S. captured by the activation space in --lamma2-7b--a Negative values correspond to left-leaning perspectives, while positive values correspond to right-leaning perspectives,
identified by the Manifesto Project data which labeled 411 political parties in 2017. The following prompt was used: USER: Generate a statement by politician from the [PARTYNAME] party, which is political party in [COUNTRYNAME]. ASSISTANT: In 2019, a lawmaker from the [PARTYNAME] party said that. B. Predictive performance of linear probes by nation, Red indicates positive correlation between predicted and actual ideological perspectives, while blue indicates negative correlation.
https:/mmanffexo-prreettwbbbe
A
Republican
40
Diepalt Labour
Saaha Damsoaak Furte
Decupgact
01
Polktod
31

---

Published as a conference paper at ICLR 2025
Table A5: Comparison Responses after Selectively Intervening in Layers Based on Alpha Values
Alpha
Layer 22
Layer > 22
-20
Regarding same-sex marriage, believe that love knows no boundaries of gender. All couples, regardless of their gender, deserve the same rights and recognition any other couple. The legalization of same-scx mar- riage is crucial step in achieving equality and justice for all LGBTQ+ individuals and couples.
Regarding same-sex marriage, believe that all individuals, regardless of their sexual orientation, should have the right to marry and have their relationships recognized and respected by the law. This is fundamental human right, and it important that continue to fight for and protect the rights of the LGBTQ community.
powerful statement about the importance of love, respect, and recog- nition our society.
20 Regarding same-sex marriage. believe that all individuals, regardless of their sexual orientation, should be treated with dignity and respect. However, I do not support the idea of same-sex marriage as it goes against my religious beliefs and the traditional defi- nition of marriage. believe that marriage should only be between man and woman, as this is the biblical definition of marriage.
Regarding same-sex marriage, believe that all individuals should have the right to marry the person they love and have their relationship recognized by the government. The ability to marry the person of one's choice is fundamental human right, and it is not the government's s place to dictate who someone or cannot marry.
Figure A10: Predictive performance of linear probes for each attention head across all layers in gemma-2-220 2-2b. Performance is measured using Spearman rank correlation, with darker shades indicating stronger correlations.
32
30
.8
28
26
24
22
.6
20
Layer
18
16
..4
14
12
10
8
.2
6
4
2
.0
Head (Sorted)
32


Reviews:
[{"review_id": "lrOOn5ohMJ", "reviewer": "Reviewer_uBqm", "summary": "The authors identify a set of attention heads within select large language models (LLMs) that encode ideological perspectives. Through the use of linear probes on the model's layer representations, they uncover heads associated with ideological slant. Their findings demonstrate that modifying these neurons influences the ideological bias exhibited by the model.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The authors find a set of attention heads encoding political slant in 3 LLMs\n* The authors causally ablate these heads and show the resulting effects on text generation", "weaknesses": "*  Why was ridge regression used instead of standard linear regression? Were collinearities established prior to making this choice?\n* The authors don't establish baseline behaviours -- or maybe I am missing this. What were the model's ideological leanings before the intervention was applied?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IgsL9i66JO", "reviewer": "Reviewer_vb63", "summary": "This paper presents an analysis of how LLM's internal activation layers capture political slant according to a US-centric liberal-conservative definition. By leveraging an automatically created dataset of texts generated by prompting LLMs with various politician personas, authors create linear probes to isolate attention heads that capture this political dimension. They show through correlation and intervention experiments that these activations can indeed capture and affect the political slant of generated texts.", "questions": "This is maybe a weird question, but I'm curious why authors decided to submit to ICLR instead of ACL venues (ACL rolling review), given the very NLP-nature of this work? I'm particularly curious because the field boundaries between ML and NLP are changing rapidly with the rise of LLMs, and I would just like to know. It feels like this work would be really well received in NLP conferences.", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This paper is very close to a slam dunk in my opinion.\n- I liked the goal of the paper, i.e., to explore political leaning in LLMs' activations.\n- The methodology is grounded in poliSci theories (DW-nominate)\n- The correlation experiments are well set up.\n- I really liked the intervention experiments.", "weaknesses": "There really aren't many weaknesses in this paper. All I could find were opportunities for improvement.\n- I wish the paper's novelty was highlighted a little more in the intro and related work. Is this the first work that has examined the linear layers of LLMs for political bias?\n- There are some missing related works particularly from the NLP side of things (e.g., https://aclanthology.org/2023.acl-long.656). I wish authors had done a little bit more comprehensive of a lit search, particularly in the ACL anthology (https://aclanthology.org) which contains papers on this topic.\n- I would have liked a more nuanced discussion around political biases in LLMs. The current paper (§2) frames political biases as an issue of fairness, i.e., that LLMs that do not represent all political ideologies equally are not fair. However, there are issues that emerge from this framing, for example, the issue of false-balance / bothsideism (https://en.wikipedia.org/wiki/False_balance). Additionally, there is a growing discussion that fairness and non-bias are most important insofar as they can help shift power away from oppressive institutions (https://www.aclweb.org/anthology/2020.acl-main.485.pdf). I wish authors gave a little more balanced of a perspective on this.\n- Footnote 1: I don't fully understand the distinction between \"stochastic parroting\" vs. linear representations, and specifically why if LLMs are merely stochastic parrots there would be no linear representation. It feels like the argument is somewhat of a side point given the results of the paper. Or it may be worth adding a longer discussion at the end of the paper about whether this means LLMs encode more systematic representations?\n- L328: I really liked the intervention analyses, yet I wish that authors had given more empirical evidence that GPT-4 can indeed classify the political slant of the essays generated. This is important because LLMs can have various biases when used as text judges. The easiest method would be to conduct a small-scale in-house manual validation, where authors or colleagues rate the political slant of the generated texts and compare their ratings to the GPT-4 scores. I suspect the alignment is high but it would be best to confirm. Alternatively, there are other trained classifiers out there to predict the political leaning of text that authors could use.\n- L429: I wish authors had contextualized the claim that conservative arguments might just be shorter, grounded in evidence from political science to back up this hypothesis. This is an interesting note, and may point to various things like tendency to over-simplify arguments on side of the political spectrum, to resort to emotional and persuasive talking points instead of more rational ones, etc. I wish authors had discussed this in further depth.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wquZ3BS4jR", "reviewer": "Reviewer_T21J", "summary": "This study shows how LLMs, like Llama-2-7b-chat, Mistral-7b, and Vicuna-7b, represent and simulate ideological perspectives in their hidden layers. Researchers probed these layers and identified a specific set of attention heads that represent particular political views, uncovering patterns they call the \"geometry of perspective.\" They claim that these attention head activations linearly align with LLM viewpoints on a liberal-to-conservative scale, correlating with known political ideologies. They also found that adjusting these attention heads can steer model responses without extra prompts, shaping the output toward liberal, center, or conservative viewpoints. I feel this work could allow closer monitoring and control of ideological perspectives in LLM responses, which could be a powerful tool for transparency and balance in AI-generated content.", "questions": "1. Check weakness 1.\n2. In line 223, it says, \"*We train a ridge regression model using U.S. politician data.*\" Does *U.S. politician data* refer to the extracted activations? Is there any train-test split, or how does this part work?\n3. In line 344, was *2-fold cross-validation* done only on the Ridge regression model, or did you also collect attention heads twice?\n4. Can you provide more details on the x-axis legends (values of each line) in Figure 3?\n5. Clarify the terms *accuracy* and *Spearman rank correlation* in Section 4.1. Why is each one used?\n6. How is Figure 4 generated? Is it based on the DW-NOMINATE scores from the datasets and attention, or does it use the attention and the political aspect of the prompt?\n7. In line 417, it says, *the strongest correlation between intervention and political slant*, and the previous line mentions achieving a higher degree of conservative political slant. Is the correlation value based only on steering toward a conservative slant, or on all three (liberal, conservative, and neutral)? Again, how are these values measured, and with which inputs, outputs, or parameters?\n8. What are the hyperparameters of Ridge regression model? Is it tuned? If yes, how? Is it tuned for all different situations separately, or only one model is trained - how does this work? If it is not tuned, why?\n9. Adding a figure showing different parts of how the model works from start to finish, like a pipeline, could help the audience better understand the approach.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The overall problem statement and motivation are very interesting and could be useful in real-world applications.\n2. They used a simple yet effective approach by examining attention heads with established probing methods, applying a regression model on network activations to predict annotated labels of input or output data.\n3. After probing, they attempted intervention with the model, trying to steer outputs to gain more insights—which was very interesting.", "weaknesses": "1. I think more details should be added in Section 3.2, *Intervention*. It is very unclear.\n   - Comments:\n      - In line 282, $\\theta_{l,h}$ is described as a steering vector, but in line 228, it is referred to as attention coefficients. Are they the same? If not, why are the notations the same?\n      - If $\\theta_{l,h}$ represents attention coefficients from the Ridge regression model, how can it be applied directly in the attention mechanism? Are the shapes of both vectors/tensors the same? Also, as they originate from different areas, how are they connected?\n      - If $\\theta_{l,h}$ is not the attention coefficient in line 282, how is it calculated? How do we maintain its shape or connect it with other components?\n      - In line 288, the definition, purpose, and use of $\\sigma_{l,h}$ are also unclear.\n      - In line 291, it states, *We target the K most predictive attention heads for the intervention.* What is the value of $K$ here? Is it selected from the coefficients of the Ridge regression?\n   - Asks:\n      - Clearly define $\\theta_{l,h}$ and explain how it relates to both the ridge regression model and the intervention process.\n      - Explain the process of applying the ridge regression coefficients to the attention mechanism, including any necessary transformations.\n      - Provide a clear definition and explanation for $\\sigma_{l,h}$.\n      - Specify how K is determined and its relationship to the ridge regression results.\n\n2. In Section 3.6, GPT-4 as an evaluator is experiment-friendly, but isn’t there a chance we might reinforce hidden biases of LLMs? For instance, if the LLM-based evaluator has a bias toward the right (assuming right-leaning perspectives are correct), could it potentially score right-leaning statements more favorably, as this bias would seem normal to the evaluator model? Usually, some human annotated experiments are expected to prove that the LLM evaluator is replicating actual human/proper responses.\n    - Asks:\n      - Acknowledge and discuss the potential for bias when using an LLM as an evaluator.\n      - Do some experiments to validate GPT-4's evaluations against human annotations, even if on a smaller scale.\n      - Discuss how these factors might mitigate or account for potential biases in the evaluation process.\n\n3. A discussion could be added to connect the sections *Simulating Subjective Perspectives using LLMs* and *Political Bias of LLMs* in related works, discussing the potential societal impact of this study. Currently, the findings feel somewhat disconnected from real-world applications. A brief discussion section could help.\n    - Asks:\n      - Add a brief discussion section that explicitly connects your findings to potential real-world applications.\n      - Explore how your method for identifying and intervening in political bias could be applied in practical scenarios.\n      - Discuss the ethical implications and potential societal impacts of your findings.\n\n4. Although limitations mention that a U.S.-centric focus may reduce relevance in global contexts, the topic of political discourse and presence of strongly opposing views differs in many countries. As a result, the scope is limited; also, it’s uncertain if LLMs will adapt as accurately to culturally distinct perspectives (such as those in South Asia) due to data limitations.\n\n5. Writing quality needs improvement, with several missing details. I’ve added these in the *Questions** below.\n\n6. From a theoretical standpoint, the methods used are derived from previous works. While the study provides an interesting new application, I still feel this work lacks novel theoretical contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Qe55G4TW6g", "reviewer": "Reviewer_Huro", "summary": "This paper presents an investigation into how LLMs encode political ideology, specifically focusing on U.S. politics. The authors claim that LLMs develop a linear representation of political perspectives in their activation space and demonstrate that these representations are primarily located in the middle layers of transformer models. By training a linear probe, the paper shows that political ideologies can be linearly separated within certain attention heads. The authors further demonstrate that intervening in these attention heads allows them to steer the political bias of generated text along a liberal-conservative spectrum without using explicit prompts.", "questions": "- Simulating Ideological Perspectives (Line 185): When you state that \"LLMs must simulate the subjective, ideological perspectives of the given politicians or news media,\" how do you ensure that the model consistently selects an accurate ideological stance, rather than a random or mixed perspective, especially if it lacks full factual context? Could you provide additional details on how factual alignment with specific perspectives is verified in the model’s responses?\n- Reference to Political Slant as a ‘Linear’ Spectrum (Line 133): The paper mentions that \"social scientists have shown that humans conceptualize political slant as a ‘linear’ spectrum.\" Could you provide specific references to support this claim? Additionally, how does this linear conceptualization integrate with political science literature, which often models ideology as multi-dimensional? Exploring this could clarify how your approach fits within or challenges existing theories of political ideology.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is novel in its application of linear probing to the representation of political ideology, which has not been extensively explored before in the context of LLMs. This contributes to the growing body of research on model interpretability and bias detection.\n- The experiments are well-executed, and the use of multiple open-source models (Llama-2-7b, Mistral-7b, Vicuna-7b) to ensure reproducibility is commendable. The intervention analysis demonstrates an interesting way to manipulate model behavior based on internal activations.\n- The paper tackles a highly relevant issue, as understanding and controlling political bias in LLMs is a critical challenge, especially given the role AI systems play in information dissemination and public discourse.", "weaknesses": "- The key contribution of the paper—demonstrating linear separability of political ideologies—is not as novel or profound as the authors suggest. Linear separability is an expected feature of transformer models, particularly in middle layers where complex features are encoded. The use of a linear probe to classify two categories (liberal vs. conservative) is a relatively simple task that doesn’t fully substantiate the broader claim of a \"linear representation hypothesis\" in LLMs. The results merely demonstrate linear separability rather than showing that political ideology is fundamentally encoded as a linear feature across multiple dimensions, which would be a much more significant finding.\n- The paper seems to conflate linear separability with the broader notion of linear representation as discussed in interpretability literature. The linear representation hypothesis posits that various features, such as gender or sentiment, can be combined and manipulated linearly within a model’s activation space. Simply demonstrating that political ideology can be classified with a linear probe doesn’t fully support this hypothesis. The authors don’t explore whether political ideology can be combined with other features in a meaningful, linear fashion, nor do they show how this linearity generalizes to other tasks or representations beyond political ideology.\n- While the paper uses interventions to modify the political bias of generated text, it doesn’t convincingly demonstrate the causal relationship between specific model activations and ideological outputs. It remains unclear whether the identified linear structures are genuinely capturing political bias or merely reflecting superficial patterns in the training data. The interventions themselves seem to rely on surface-level associations rather than a deeper understanding of how political ideologies are encoded and combined within the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)