synthetic) news headlines. (b) The scientific evidence evaluation task includes interpreting whether the treatment (in this example banning guns) leads to an increase or decrease in the outcome (crime). making — is fundamental to human intelligence. Humans, however, are not perfectly rational and their goals or motives for engaging in reasoning can determine the accuracy of their conclusions. Oftentimes, “ reasoning directed at one goal undermines others \". For instance, when reasoning about the impact of gun control on crime rates, the desire to maintain social standing within apolitical group can motivate individuals to construe seemingly rational justifications for holding identity-congruent beliefs — subsequently undermining the motivation to arrive at accurate conclusions. This type of biased reasoning called motivated reasoning, can be dangerous insofar as it can hinder society from converging on a shared understanding of facts regarding critical issues like humandriven climate change or vaccine safety — deterring meaningful action towards addressing such problems. Individuals with a predisposition toward analytical reasoning or above-average numeracy skills are also not immune to motivated reasoning; some studies show that individuals in fact leverage their analytical skills toward reinforcing identitycongruent beliefs.
Even large language models (LLMs) that increasingly demonstrate human-like performance across complex reasoning tasks are susceptible to human-like cognitive biases such as anchoring, framing, and content effects. Compounding these effects is the growing trend of personification, i.e. prompting LLMs to adopt identities or per sonas with diverse demographics and values. Studies have reported erratic effects of persona-assignment on reasoning, where some personas enhance reasoning capabilities, while others introduce unintended biases and deteriorate performance. In this paper, we specifically investigate whether persona-assignment induces responses consistent with motivated reasoning in LLMs. Models displaying such behavioral patterns risk providing seemingly rational, but inherently flawed justifications to users for arriving at identity-congruent conclusions — potentially contributing to epistemic bubbles and subsequently exacerbating social biases and political polarization through human-AI feedback loops. To the best of our knowledge, we are the first to propose motivated reasoning as a theoretical framework for understanding identity-congruent reasoning in persona-assigned LLMs. And while the underlying “motivation\" mechanisms for LLMs may completely differ from humans —- implicitly shaped by training data or fine-tuning — personaassigned reasoning biases may still functionally resemble motivated reasoning observed in humans.
We study this by assigning 8 personas across 4 political and demographic attributes to 8 state-ofthe-art LLMs. We consider two reasoning tasks sourced from cognitive psychology where motivated reasoning has been a salient mechanism in biased evaluation for humans — discerning the accuracy of true and fake (i.e. synthetic) news headlines and evaluating numeric scientific evidence. The tasks are explained in Figure 1b. We find that across both tasks, persona-assigned models exhibit human-like motivated reasoning — leading to conclusions congruent with the induced identity. In the headline veracity discernment task, we find that LLMs assigned with a High School educated persona have up to 9% reduced veracity discernment relative to models without personas. Additionally, similar to human studies, motivated reasoning is a statistically significant predictor for veracity discernment, as compared to analytical reasoning (which is non-significant). Moreover, we find that political personas are up to 90% more likely to correctly evaluate scientific evidence when the ground truth is congruent with their political beliefs, but show reduced performance when evaluating evidence that conflicts with their induced political identity. To mitigate this effect, we explore two debiasing strategies including chain-of-thought reasoning. We find that similar to prior work, prompt-based debias ing approaches are ineffective at reducing moti vated reasoning in persona-assigned LLMs. We conclude by highlighting the risks of personaassigned LLMs in amplifying identity-congruent reasoning in both humans and LLMs. To ensure the reproducibility of our results, we have made our prompts, code, and generated datasets publicly available . 2 Related Work
Persona-Assigned LLMs & Reasoning. Personaassigned LLMs have been found to inherently encode human-like biases and traits due to underlying training data patterns, and exhibit opinions consistent with specific demographics due to human feedbacktuning. Personified LLMs also display human-like behavior over prolonged simulations and replicate human-subjects social science experiments to some degree. We contribute to this literature by
1 [https://github.com/ameliereymond/](https://github.com/ameliereymond/motivated-reasoning/) [motivated-reasoning/](https://github.com/ameliereymond/motivated-reasoning/) studying whether persona-assigned LLMs exhibit human-like motivated reasoning patterns. Most relevant to our work are studies that have shown that for reasoning tasks specifically, prompting models to adopt the identity of a “ domain expert \" or a “ human that answers questions thoughtfully \" improves performance, while others report that assigning personas like “ physically-disabled person \" drastically reduces reasoning performance. Based on our understanding, we are the first to explore identity-congruent reasoning as a theoretical framework for persona-induced reasoning biases. Human-Like Cognitive Biases in LLMs. A growing body of research falling under “machine psychology\", i.e. studies that use experiments from psychology to better understand LLM behavior, have shown that LLMs exhibit human-like cognitive biases including anchoring, framing, and content effects, and are vulnerable to base-rate and conjunction fallacies as well. Building on the dual-process theory of thinking in cognitive psychology, some studies argue that older language models display patterns of fast, error-prone, heuristic or “ system 1 \" thinking, while newer models after ChatGPT-3.5 show signs of “ system 2 \", or slow and more analytical thinking. This current study contributes to the field of machine psychology by showing that persona-assigned LLMs exhibit human-like cognitive biases consistent with motivated reasoning. Motivated vs. Analytical Reasoning. The factors underlying the (in)ability of individuals to discern false or misleading information from true information have been extensively studied in cognitive psychology, resulting not only in theoretical frameworks to describe reasoning mechanisms and vulnerabilities, but also empirically validated instruments for measuring characteristics predictive of performance on reasoning tasks — we incorporate both in our study design. The “classical reasoning\" theory suggests that only analytical or “system 2\" thinking typically measured by the cognitive reflection test (CRT) plays a cen
tral role in predicting misinformation susceptibility or belief in false information, while the “integrated reasoning\" account states that motivated reasoning as measured by myside bias is a significant predictor of veracity discernment. Myside bias is a tendency for individuals to engage with evidence in a manner that conforms to their prior beliefs and attitudes and is captured by the psychometrically evaluated test of actively openminded thinking (AOT). Recent efforts testing analytical reasoning against motivated reasoning theories in humans Roozenbeek et al. employ regression analysis to evaluate evidence for AOT and CRT as predictors, and find that AOT (or myside bias; will be used interchangeably) is abetter predictor for veracity discernment than CRT (or analytical reasoning; used interchangeably) — lending support to the motivated reasoning theory for disparities in veracity discernment. We test this analytical vs. motivated reasoning theory for LLMs in §4.1. Motivated reasoning is also implicated in an individual’s ability to reason about scientific evidence, specifically when it runs contrary to commonly held beliefs or policy positions of their identity group. Psychologists have designed assessments to evaluate the role of motivated reasoning in humans’ ability to draw valid causal inferences from empirical data, finding that individuals, especially those with strong numeracy skills reason in ways that are consistent with their political identities. We replicate this for LLMs in §4.2. 3 Methodology & Setup
Attribute Personas
Political Affiliation a Democrat, a Republican
Gender a Man, a Woman
Religion an Atheist, a Religious person a College -educated person, a Education High School -educated person
Table 1: 8 personas across 4 socio-demographic attributes. In this section, we describe the method for inducing identities in LLMs by assigning personas, the experimental setup for the study, the reasoning tasks and the mitigation strategies considered to reduce the effect of personas on reasoning. 3.1 Persona Prompting
To induce “identities\" in LLMs, we use prompting strategies as in previous work to assign personas. We specify in system instructions to the LLM to “ Adopt the identity of { persona }. Answer the questions while staying in strict accordance with the nature of this identity.
\". We use 3 variations of this persona instruction from Gupta et al. that was also evaluated in prior work for its effectiveness across a variety of dimensions. Personas. For the first task of Veracity Discernment, we consider 8 different personas across 4 different socio-demographic groups, that have been shown to be susceptible to false information through previous studies. For the second task (scientific evidence evaluation), we only consider political identity, i.e., Republican and Democrat personas, to be relevant for the task. We also validate the personas for internal consistency and realism in Appendix §A.1. 3.2 Model Setup
Models. A wide variety of both open-source and proprietary models were selected based on their competitive performance on reasoning benchmarks. Specifically, we test OpenAI models GPT-3.5, GPT4, GPT4-o and GPT4-o mini, Meta models like Llama2, and Llama3.1, Mistral and Microsoft’s WizardLM-2, resulting in a total of 8 models. Implementation Details. We set the temperature parameter to 0.7 to simulate real-world behavior, similar to prior works, and leave other parameters to their default settings. We query the OpenAI models using their API and the open-source models using Ollama . As explained previously, we prompt each model-persona pair across 3 different formats of persona instructions taken from Gupta et al.. We also prompt all
2 [https://openai.com/api/](https://openai.com/api/) 3 [https://ollama.com/](https://ollama.com/)
models across both tasks without the persona instructions, which we call the Baseline model. Additionally, we prompt each persona-model pair 100 times, similar to and take the mean across all persona prompts to obtain a representative sample. Therefore, for the veracity discernment task, each model-persona pair is prompted a total of 300 times, resulting in a total of 21,600 data points. We then obtain a representative sample for each model-persona pair by averaging across all 3 persona prompts, resulting in 7200 data points. For the scientific evidence evaluation task, we also prompt each model-persona pair 300 times, resulting in a total of 7200 data points. Model Response Processing The models generally follow the format specified in prompt instructions, and respond with only the number/answer required. However, in the case that the model does not follow the instructed format, we use regex matching to obtain the numeric answer in the case of the veracity discernment task. In the case of the scientific evidence evaluation task, the open-source models implicitly provide chain-of-thought reasoning for the answer; so a simple regex match is not sufficient.
Similar to prior papers, we, therefore, use a GPT-4o judge to extract the final answer based on the chain-of-thought reasoning. 3.3 Reasoning Tasks
In this study, we consider two reasoning tasks sourced from cognitive psychology, where motivated reasoning has been identified as a salient factor for biased reasoning in humans. 3.3.1 News Headline Veracity Discernment
In this task, LLMs are prompted to rate the accuracy of news headlines on a Likert scale of 1 to 6. The news headlines are sourced from the psychometrically validated Misinformation Susceptibility Test (MIST) that consists of 20 headlines; 10 fake (i.e. synthetic) and 10 real. Veracity discernment ability, i.e. the ability, in this case, to differentiate fake headlines from real headlines
(VDA), is then calculated by first standardizing the numeric response from each headline on a scale from 0 to 1 and taking the mean across all 20 headlines. Let r i be the raw Likert rating for headline i, then the standardized value of the rating s i, and consequently, VDA can be computed as:
s i =
6 −r i
5 r i − 6
� 5
− i if the headline is Fake
5 r i − 6
5 − 6 if the headline is Real
5
Baseline: VDA ∼ AOT + CRT + CONF +
+ OPENSRC +
Persona-Assigned: VDA ∼ AOT + CRT
+ CONF + OPENSRC +
+ ,
where, CONF is the verbalized confidence estimate of the LLM for the VDA scores and OPENSRC is a binary variable depicting whether the model is open source or proprietary. We z-score normalize all predictors ( AOT, CRT, CONF ) to have zero mean and unit variance to ensure comparability of their coefficients. Unlike prior studies that fit a linear regression model to compare the effects of AOT Vs CRT, we use a hierarchical mixed-effects model where we consider MODEL to be a random effect since the outputs from a single LLM are correlated. We also consider PERSONA to also be a random effect nested under MODEL since the outputs from different personas assigned to an LLM are nested under that LLM — thereby requiring the use of random effects to account for such correlations. 3.3.2 Scientific Evidence Evaluation
To test the effect of personas on the evaluation of numeric scientific evidence, we replicate the study from, where we prompt LLMs to evaluate evidence from two scientific experiments. The experiment results are reported in the form of a 2x2 contingency table, the rows of which detail the treatment conditions, and the columns specify treatment outcomes. The first scientific experiment serves as a control or neutral topic which is typically unrelated to political identity — the outcomes of using anew skin cream. Here the treatment conditions include using anew skin cream or not using it, and the outcomes include an increase in rashes or a decrease in rashes after the study).
In the second experiment, which involves a topic relevant to political identity, the treatment conditions include cities that ban concealed hand guns in public or cities that don’t ban handguns, and the outcomes include an increase in crimes or a decrease in crimes. The contingency tables are designed such that there is only one correct treatment outcome for a
VDA =
n
n � s i i =1
In addition to VDA, we also prompt the model to evaluate confidence in its assessment of the headline on a Likert scale of 1 to 6. In humans, overconfidence is negatively associated with veracity judgments of news headlines, and research on overconfidence and performance in LLMs suggests similar patterns. However, some studies suggest that verbalized confidence scores appear to be well-calibrated, i.e. high confidence is indicative of correct answers, in feedback-tuned models — which is how we choose to elicit confidence estimations instead of using logit probabilities. To evaluate evidence for motivated reasoning versus analytical reasoning explanations as detailed in §2, we evaluate the LLMs on a variety of psychological factors including the endorsement of actively open-minded thinking (AOT) questions on a scale of 1–5 and proficiency in analytical thinking as measured by the 6-point cognitive reflecting test (CRT). To avoid data contamination issues arising from LLMs being trained on the original CRT items, we use the newly developed CRT items from Yax et al., which conceptually resemble the original CRT and were verified on human subjects. Modeling Veracity Discernment. To evaluate whether motivated reasoning plays a role in news headline veracity discernment across 8 different models and 8 separate personas, we fit a hierarchal mixed-effects model using the following equations: given treatment. The way to correctly reason about the problem includes not just comparing raw values, but comparing proportions across all outcomes — this is critical for detecting the covariance between the treatment and the outcomes and neces sary for valid causal inference. For example, the correct way to reason about the table in Figure 1b is to compare the proportions of 223/75 vs 107/21, which would lead to the outcome that cities that did not ban guns had a decrease in crime, therefore cities that did ban guns had an increase in crimes. Any heuristic strategy of comparing raw values leads to invalid causal inference.
For each type of experiment (skin cream and banning guns), there are two contingency tables — one for which the ground truth is an increase in rashes/crimes and another for which the ground truth is a decrease in rashes/crimes, leading to a total of 4 contingency tables. Modeling Bias in Evidence Evaluation. Let T ∈ {\"Rash Increases\", \"Rash Decreases\", \"Crime Increases\", \"Crime Decreases\"} be the ground truth for the scientific experiment(s), and let P ∈ {\"Democrat\", \"Republican\"} be the assigned persona. The bias β for evaluating the evidence of (say) the gun control experiment where the ground truth is Crime Decrease can be written as:
β CD = P( T = Crime Decreases | P = Republican )
− P( T = Crime Decreases | P = Democrat ) ,
and let β CI, β RD and β RI be the bias for evaluating evidence when the correct answer is Crime Increase, Rash Decrease, and Rash Increase respectively. If there is no motivated reasoning being induced in persona-assigned LLMS, then we can expect the value of β CD and β CI to be close to 0. For instance, if β CD — the condition in which the ground truth is Crime Decreases — is close to 0, that implies that the probability of the Democrat persona evaluating the evidence correctly when it aligns with liberal attitudes on gun control (that banning guns leads to decrease in crime) is equally likely as the probability of a Republican persona evaluating the evidence correctly when it does not align with conservative attitudes on gun control (banning guns leads to an increase in
crimes). However, if persona-assigned LLMs are indeed exhibiting motivated reasoning, then we can expect β CD to be negative, and β CI positive.
Therefore the skin cream experiment acts as a control, and we expect β RD and β RI to be close to 0, i.e. no effect of political personas on correctly evaluating scientific evidence for a neutral topic. 3.4 Mitigating Motivated Reasoning
To mitigate persona-induced motivated reasoning, we use two prompt-based debiasing approaches: chain-of-thought (CoT) prompting, or prompting the model to “think step by step\", and accuracy prompting, or prompting the model to prioritize accuracy while answering the questions. CoT has been shown to have mixed results in reducing bias. Accuracy prompting is inspired by human-subject studies that explore reducing motivated reasoning in humans by incentivizing accuracy through financial incentives. 4 Results
Here, we report findings from both reasoning tasks across all models and personas, as well as mitigation strategies explored to reduce the effect of personas in inducing identity-congruent reasoning. 4.1 Veracity Discernment Task
First, we examine the aggregate impact of personas on VDA and then analyze which predictors of VDA best explain disparities in performance across personas. VDA Broadly Decreases Across Personas. As shown in Figure 2a, VDA broadly decreases across personas, except for Democrat, where VDA increases by 4%. We conduct independent t-tests to check whether the VDA values for each persona differ significantly from the baseline and find the differences to be statistically significant. We find that among all 8 personas, the High-School persona has the lowest veracity discernment, with almost a 9% reduction compared to the baseline. However, the decrease in VDA is not uniform across models. As seen in Figure 2b, we find that the OpenAI models drive most of the decreases in VDA, while VDA broadly increases across all personas for the Llama2 and WizardLM2 models. This could potentially be explained by the significantly higher VDA of Baseline OpenAI models as compared to open-source models = 43.60, p < .001) — suggesting that the room for improvement in the Baseline OpenAI models was less to begin with. We also report persona and model-specific patterns for VDA predictors in Appendix §A.2. Taken together, similar to previous studies on personas, this suggests that the effect of personas for different models is inconsistent, and the persona-specific differences from the baseline are not necessarily reflective of human susceptibility patterns, but could potentially be attributed to training data bias or fine-tuning. We note that aggregate values for Baseline across all models (i.e. no persona prompting) are comparable to the human subject study by Roozenbeek et al.. (a) VDA Means
(b) VDA Baseline Comparisons by Model
Figure 2: Effect of Personas on VDA. VDA broadly decreases over all personas (except Democrats), and the differences are mainly driven by proprietary models. Next, to understand how AOT or myside bias (taken as a proxy for motivated reasoning) and CRT (a proxy for analytical reasoning) affect VDA, we fit equations 3 and 4 for the baseline models and persona-assigned models, respectively. The fixed effects coefficients for equations 3 and 4 are shown in Tables 2 and 3, respectively. Motivated Reasoning Better Predicts Veracity Discernment Than Analytical Reasoning. First, we find that neither AOT nor CRT are significant predictors for VDA for the baseline models. Surprisingly, CRT fails to have a statistically significant impact on VDA for persona-assigned models too. This is contrary to humansubjects experiments, where CRT has a statistically significant positive impact on veracity discernment.
Instead, for the persona-assigned models, we find that AOT has a significant positive, albeit modest, impact on VDA, implying that for persona-assigned models, motivated reasoning is abetter predictor of veracity discernment than analytical reasoning. Fixed Effects Estimate Std. Error P-Value
AOT 0.0013 0.0018 0.4902
CRT 0.0008 0.0032 0.7928 CONF 0.0281 0.0036 < 0 . 001 [ ]
OPENSRC -0.2069 0.0685 0.0227 [∗]
Table 2: Fixed effects on VDA for baseline models. Significance codes: [ ] < 0 . 001, [∗] < 0 .
05. Fixed Effects Estimate Std. Error P-Value
AOT 0.0021 0.0008 0.0074 [∗∗]
CRT -0.0006 0.0010 0.5539 CONF 0.0133 0.0016 < 0 . 001 [ ]
OPENSRC -0.1003 0.0407 0.0489 [∗]
Table 3: Fixed effects on VDA for persona-assigned Models. Significance codes: [ ] p < 0 . 001, [∗∗] p < 0 . 01,
∗ p < 0 . 05. Model Confidence is the Best Predictor for Ve racity Discernment. Interestingly, we find that the model’s confidence in correctly assessing veracity is the best predictor for veracity discernment across all models and persona configurations. This is in line with prior studies that have found that LLMs tend to be most confident when giving correct answers, i.e. they are well calibrated. It should be noted that this differs from human-subject experiments, where overconfidence is negatively correlated with news judgments. To rule out whether the models were trained on the specific misinformation headlines, we created anew misinformation headlines dataset of real and fake claims sourced from Politifact starting January 2024. We find that the results for the baseline models on the new dataset are similar to the results we report here, thereby confirming the robustness of our findings. 4.2 Scientific Evidence Evaluation Task
Using equations similar to 5 for the skin cream and gun control experiment, we compute β values across all four answer conditions for all models. The β values are shown in Figure 3. Induced Political Persona Biases Evalua tion of Gun Control Evidence. We observe that for models like Llama2, Mistral, WizardLM2,
and GPT-3.5 when the correct answer to the experiment is Crime Decreases, a Democrat persona is more likely to get the answer right than a Republican persona, up to 90% in the case of GPT-3.5. Similarly, for models Llama2 and Llama3.1, when the correct answer is Crime Increases, a Republican persona is up to 30% more likely to get the answer right as compared to a Democrat persona. A manual examination of the answers provided by open-source models reveals that 46% of the answers contain explicit references to political identity, with many explicitly stating their induced political beliefs, such as Republican personas starting with “ As a Republican, I must emphasize the importance of individual freedom and self-defense... \" or Democrat personas starting with “ As a Democrat, I believe in prioritizing pub lic safety and the well-being of our communities...
\". In contrast, for the skin cream experiment, the β values are closer to 0. It must be noted that across all models, we observe a base-rate fallacy, where models are predisposed to predicting a specific answer — potentially suggesting a training data bias. We discuss this in Appendix §A.5. Additionally, surprisingly GPT-4 and GPT-4o-mini perform poorly at this task, with a 0% accuracy rate. We discuss potential explanations in Appendix §A.6. 4.3 Prompt-Based Debiasing
As described in §3.4, we use two prompt-based debiaising approaches that have been shown to reduce reasoning biases in LLMs and humans to some degree: chain-of-thought and accuracy prompting fer to Appendix Fig. 16 and 17 for exact prompts).