POW : Political Overton Windows of Large Language Models (2025)

Abstract: Political bias in Large Language Models (LLMs) presents a growing concern for the responsible deployment of AI systems. Traditional audits often attempt to locate a model’s political position as a point estimate, masking the broader set of ideological boundaries that shape what a model is willing or unwilling to say. In this paper, we draw upon the concept of the Overton Window as a framework for mapping these boundaries: the range of political views that a given LLM will espouse, remain neutral on, or refuse to endorse. To uncover these windows, we applied an auditing-based methodology, called PRISM, that probes LLMs through task-driven prompts designed to elicit political stances indirectly. Using the Political Compass Test, we evaluated twenty-eight LLMs from eight providers to reveal their distinct Overton Windows. While many models default to economically left and socially liberal positions, we show that their willingness to express or reject certain positions varies considerably, where DeepSeek models tend to be very restrictive in what they will discuss and Gemini models tend to be most expansive. Our findings demonstrate that Overton Windows offer a richer, more nuanced view of political bias in LLMs and provide a new lens for auditing their normative boundaries.

## introduction large language models (llms) are increasingly integrated into applications that mediate information, provide advice, and influence decisionmaking across diverse domains @xcite . as these systems become embedded in daily life, scholars and practitioners have raised concerns not only about their technical limitations but also about the social and political values they convey @xcite . in particular, the potential for political and ideological bias is especially consequential: by shaping how information is framed and delivered, llms may inadvertently distort public discourse, reinforce polarization, or privilege certain perspectives over others @xcite . understanding the political orientations these systems are willing or unwilling to express is therefore critical for assessing their broader societal impact. recent work has attempted to uncover the latent political and ideological biases of llms by prompting them with politically and morally charged statements, often using surveys or questionnaires such as the political compass test (pct) @xcite @xcite . these efforts typically aim to assign a political point estimate to a model, placing it on an ideological spectrum from left to right or authoritarian to libertarian. such studies have revealed that most llms tend to be left-leaning, economically, and liberal-leaning socially by default @xcite @xcite . while informative, these approaches suffer from important limitations. they are (i) highly sensitive to prompt phrasing and format (e.g., @xcite ), (ii) do not account for variation across demographic prompts (e.g., @xcite ), and (iii) tend to ignore the textual justifications that accompany model outputs (e.g., @xcite . most importantly, a single point estimate risks masking the boundaries of what an llm is willing to say. it captures where the model defaults, but not where it refuses or hesitates to go. in this paper, we consider the overton window (lehman, 2010) as a conceptual lens for mapping the boundaries regarding the political positions they are willing (or unwilling) to espouse. originally proposed in political theory, the overton window describes the space of ideas that are considered socially acceptable to express at a given time -divided into categories such as unthinkable, radical, acceptable, sensible, and policy (see figure 1 ). while this concept has traditionally been used to understand shifts in public discourse, we adapt it to language models to ask: what is the space of political views llms consider reasonable and acceptable? and what do they consider as too radical and unthinkable to discuss? to answer these questions, we apply the prism methodology to probe 28 llms given the political compass test (pct) to elicit the most extreme positions they are willing to espouse over the spectrum of positions, which are used to create overton windows that map the ideological boundaries of these llms. ## related work large language models offer many opportunities for developing ai-powered agents and systems. however, the underlying models may be subject to harmful and negative biases, where they, implicitly or explicitly, push or promote certain agendas, ideologies, stereotypes, etc., while suppressing or hiding others. this has motivated efforts that aim to understand, mitigate and audit such technologies @xcite @xcite @xcite . while there are numerous ways in which llms could be audited, of particular relevance to this work are the efforts to quantify the political biases of llms. such studies have consistently found that, by default, most llms exhibit an economically left-leaning position coupled with a reasonably liberal position @xcite . for example, in one of the first studies rozado (2023), 15 political orientation tests were administered to chatgpt, finding that 14 of them indicated a preference for left-leaning viewpoints. further analysis in 2024 expanded the scope to 24 llms, revealing that 23 exhibited left-of-centre biases. rozado also demonstrated that fine-tuning models with politically aligned data could shift their ideological outputs, highlighting the malleability of llm biases. @xcite evaluated chatgpt using political statements from various voting advice applications, including the political compass test. their findings indicated a pro-environmental, leftlibertarian orientation, with chatgpt supporting policies like flight taxes, rent controls, and abortion rights. the model's responses aligned with green parties in germany and the netherlands, suggesting a consistent ideological stance across different political contexts. @xcite found chatgpt revealed a consistent bias toward the democratic party in the u.s., lula in brazil, and the labour party in the u.k., suggesting potential implications for political processes. @xcite found that u.s.-based models often aligned with progressive values, while chinese models displayed distinctions between internationally and domestically focused versions. additionally, the same model could produce different normative assessments when prompted in different languages, suggesting that linguistic context plays a role in shaping model outputs. when assessing 11 open source models, @xcite found that biases manifested not only in the substance of responses but also in their lexical choices, with certain models displaying consistent ideological patterns in both aspects. @xcite analysed thousands of responses from six llms given the political compass test, using 420 prompt variations to simulate different demographics. they identified recurring semantically similar phrases (called "tropes") that llms consistently used across the different prompts, revealing underlying patterns in how models justify their stances. moreover, the model exhibited different political leanings depending on the demographic features included in the prompt. in this work, we continue in this line of inquiry and try to elicit the most extreme views from each llm to map out what positions they are willing to take and what positions they are not. ## method to audit political bias and expressive boundaries in large language models (llms), we applied the preference revelation through indirect stimulus methodology (prism) (azzopardi and moshfeghi, 2024) 1 . prism was designed to probe models' normative boundaries using indirect, task-driven elicitation rather than direct questioning (as done in @xcite and other works). a key benefit of prism is that it leads to greater compliance and fewer refusals, leading to more accurate approximations of the political stances of the models. the methodology is as follows: given the survey instrument, the llm is asked to write an essay on each proposition, and then this essay is rated by an assessor. this approach allows models to express nuanced reasoning, which reveals their latent positions. here, we use the political compass test (pct) as our survey instrument, consisting of 62 ideologically polarised propositions spanning two dimensions: economic (left-right) and social (authoritarian-libertarian). for each proposition, we prompt models to write short essays rather than directly answer likert-scale questions. each essay is then rated by an ai-based assessor (gpt-3.5 turbo) on a five-point scale: strongly agree, agree, neutral, disagree, or strongly disagree, with additional handling for refusals to respond. the assessor's reliability was validated against a human-annotated gold set, where 248 essays were judged by both authors. we found that the ai-based assessor had a binary agreement of 90.3% and a cohen's kappa of 0.807 -suggesting that the ratings are highly reliable and in line with previous works(e.g., @xcite ). to capture the range of positions for each model, we adapt the method in @xcite , where we assign different roles -but with extreme ideological personas spanning the two axes of the pct, creating eight positions (e.g., "economic left-wing authoritarian", "authoritarian", "economic right wing authoritarian", and so on). the intuition being is that, given the llms are aware of the pct, they should be able to faithfully provide views in line with that persona (assuming that training, fine-tuning and alignment of said models doesn't preclude them from doing so). this then provides the set of points we used to construct the overton windows. we audited twenty-seven models from eight 1 our toolkit is available at https://github.com/cis-phawm/prism . in total, we generated over 17, 000 essays, which were then assessed. all models were evaluated using consistent prompts and deterministic settings (@xmath0.0) to ensure comparability. we used local compute power for all the open source models using ollama foot_1 and spent approximately 350 usd on proprietary model usage (i.e. openai, google, cohere and anthropic). to quantify the size of the overton windows, we calculate and report the percentage area of the total possible space of positions foot_2 . figure 2 : the heatmap shows positions that the models are willing and most notably unwilling to espouse. extreme is considered greater/lesser than +7.5/-7.5, and centre is greater/lesser than -1.5/+1.5. ## results first, as shown previously, we also find that the default policy of most llms is left and liberal leaning -that is, without specifying a role, the llm's default position aligns with political positions in the lower left quadrant (left and liberal). for example, in second, our results show that of the llms audited, most are unwilling or unable to present arguments for views consistent with authoritarian (left or right) positions, nor are they willing or able to present economically right and liberal views (see figure 2 . however, the majority of llms, whose default policy positions tend to be left and liberal (see third, we see that the overall percentage area ranges from 1% up to 67%, with 7 out of 27 models covering more than 50% of the space (see when we inspect the overton windows (shown in figure 3 ) for each model, we can see what regions the model covers, and how they have evolved (as each plot shows the related set of models for a given provider). the differences between providers are striking. alibaba's latest model qwen 3 appears to have become less liberal, while cohere's command models have evolved to be less restrictive (command) and more left and liberal (command-r). in contrast, mistral ai's model provides broader views, though still predominantly left and liberal. the overton windows have progressively enlarged, increasing coverage up to 50%. the windows associated with openai's models have also enlarged with later models, but they still have a very heavy liberal leaning. and finally, google gemini's models have also evolved from a very left and liberal stance to a wider one with the recent gemini 2.0 flash providing 67% coverage. that being said, its default policy is still very much left (-5.1) and liberal (-5.1), showing that while it is able to discuss or provide a broader range of views, it is default policy is biased towards the lower left quadrant. ## summary this paper sought to map out the overton windows associated with llms using prism, an auditingbased methodology that probes llms through taskdriven prompts designed to elicit political stances indirectly. we showed the differences between providers and models -where some are willing to expose a greater variety of views, while others are not. our findings not only highlight systematic biases toward left and liberal leaning policies, as defined by the political compass test, but more importantly, reveal the inability or unwillingness of many models to present and discuss alternative and diverse viewpoints. in particular, most models appear incapable of articulating perspectives associated with the authoritarian left or the liberal right. taken together, these findings are worrying, underscoring the need for transparency and accountability in how llms present political information. beyond documenting current biases, our work highlights the importance of developing systematic approaches to detect, measure, and mitigate political skew to safeguard the integrity of public discourse. as llms continue to mediate access to information and shape opinion formation, addressing these biases must become a central priority for both researchers and developers to ensure that such systems contribute to open, balanced, and pluralistic debate rather than narrowing it. ultimately, ensuring ideological diversity in llm outputs is not only a technical challenge but a democratic imperative, as biased models risk reinforcing polarization and weakening democratic resilience.