Dissecting Persona-Driven Reasoning in Language Models via Activation Patching (2025)

Abstract: Large language models (LLMs) exhibit remarkable versatility in adopting diverse personas. In this study, we examine how assigning a persona influences a model’s reasoning on an objective task. Using activation patching, we take a first step toward understanding how key components of the model encode persona-specific information. Our findings reveal that the early Multi-Layer Perceptron (MLP) layers attend not only to the syntactic structure of the input but also process its semantic content. These layers transform persona tokens into richer representations, which are then used by the middle Multi-Head Attention (MHA) layers to shape the model’s output. Additionally, we identify specific attention heads that disproportionately attend to racial and color-based identities.

## introduction recent advances in large language models (llms) have demonstrated their striking ability to adopt a wide range of personas, enabling context-sensitive and tailored responses @xcite @xcite . however, studies such as those by @xcite @xcite show that persona assignment can significantly influence reasoning and, in some cases, amplify underlying social biases. while these works focus on identifying and quantifying such effects, they do not examine the causal mechanisms within a pre-trained language model (plm) that give rise to them. mechanistic interpretability provides framework for uncovering the causal mechanisms by which language models carry out specific tasks. @xcite introduced this approach in the context of multiple-choice question answering (mcqa), identifying "correct letter heads" in a 70b chinchilla model-attention heads that track answer symbols and promote the correct choice based on positional order. @xcite examined how successful models implement symbol binding in formatted mcqa using vocabulary projection and activation patching, and @xcite analyzed anchored positional bias in the gpt-2 family, showing that models consistently favor the first option ('a'). building on these interpretability techniques, we ask whether similar internal circuits also govern how personas steer a model's reasoning and potentially introduce social bias. this study takes a step toward bridging the gap between surface-level observations of persona effects and the underlying mechanisms that produce them. we investigate the roles of key model components namely, multi-layer perceptron (mlp) layers, multi-head attention (mha) layers, and individual attention heads in shaping the reasoning shifts induced by persona assignments. using activation patching, we probe the internal circuitry of llms to trace the origins of persona-driven variation in objective tasks. our findings challenge the prevailing assumption that early mlp layers are concerned solely with syntactic processing. we show that these layers also encode semantic features related to persona. furthermore, we identify a small number of attention heads that disproportionately focus on tokens associated with race-based persona cues. although our work is limited to uncovering the origin of persona-driven behavior, it contributes to a deeper understanding of llms and lays groundwork for future efforts to mitigate deep-seated biases in these systems. ## persona evaluation we computed model outputs using zero-shot prompts to isolate the variation introduced solely by changes in the persona token, avoiding any influence from patterns introduced by few-shot prompting. this also reduced the length of each prompt, allowing for faster computation. for each prompt, we 2 the terms persona and identity have been used interchangeably in this work. our main focus is the change in the probability of the correct token for each persona relative to the base identity, as shown in figure 1 . this highlights shifts in reasoning attributable to persona alone. in some cases, the differences in probability followed patterns that appeared to have a semantic basis. for instance, the negatively attributed student persona performed significantly worse than those described with positive attributes, having an average probability difference of -0.0027 (@xmath0.7, @xmath1.001). in contrast, the results for racially or color-coded personas were less consistent, and no definitive conclusion could be drawn about whether the patterns reflect stereotypical associations. a similar trend was observed when examining probability differences across identities for other models, see appendix c for details. overall, our results suggest that the model's reasoning ability varies even with minimal changes to the persona being imitated. ## discussion in this work, we analyzed the impact of personadriven behavior on the reasoning ability of a language model in objective tasks. through the lens of mechanistic interpretability, we examined the role of different component models: mlp layers, mha layers, and individual attention heads in shaping this behavior. we observed that early mlp layers also contributes towards semantic understanding of inputs and encode persona-specific information into richer representation. this information is then passed to later mha layers, which use it to influence the model's response. we further categorized attention heads based on their relative attention pat-terns and identified a subset that assigned disproportionate weight to racial and color-related attributes associated with a persona. these findings also have significant practical implications for developing fairer and more reliable ai systems. the research demonstrates how to pinpoint specific model components, like certain mlp layers and attention heads, that encode and act upon persona-driven biases related to race and other attributes. this allows for interventions that are far more targeted than standard fine-tuning. instead of making broad adjustments, we can directly modify, steer, or even disable the specific neural circuits responsible for undesirable stereotypical behaviors, leading to more effective and efficient bias mitigation. furthermore, when coupled with methods like probing, it serves as a powerful diagnostic tool, enabling cheaper, more precise monitoring of how a model processes sensitive information and offering a clear window into its internal reasoning. this deeper interpretability is essential for debugging, ensuring ai safety, and building systems that are not only less biased but also more transparent and controllable in how they adopt these biases. overall, our observations offer preliminary insight into the subtle yet significant functions of certain model components and how they can be revealed through constrained but straightforward experiments. we took initial steps toward understanding the origins of persona-driven behavior in llms. in future studies, we will investigate why certain personas answer specific questions correctly while others do not, and how output vector circuits in attention heads use earlier-layer representations to shape final predictions. these directions will lead to a deeper understanding of what "persona" truly means in the context of language models.