Can Role Vectors Affect LLM Behaviour? (2025)

Abstract: The influence of personas on Large Language Models (LLMs) has been widely studied, yet their direct impact on performance remains uncertain. This work explores a novel approach to guiding LLM behaviour through role vectors, an alternative to persona-based prompting. We construct 29 role vectors derived from model activations and evaluate their impact on benchmark performance across multiple domains. Our analysis investigates whether these vectors can effectively steer models toward domain-specific expertise. We measure two key interventions: (i) activation addition, which reinforces role-specific directions, and (ii) directional ablation, which removes them. Results on well-established benchmarks indicate that role vectors do, in fact, influence model behaviour, improving in-domain task performance while also yielding unexpected cross-domain gains.This, in turn, suggests that manipulating internal model representations has a greater impact on outcomes than persona-based prompting.

## introduction the development of persona or role-based chatbots has gained significant attention in the ai and nlp community due to their potential impact on business and societal applications @xcite . the extent to which different personas influence large language models' (llms) performance on objective tasks remains unclear. recent attempts have investigated whether incorporating personas into system prompts enhances model performance on objective tasks and explored potential factors influencing these effects. @xcite conducted a large-scale analysis of the effect of personas in llm prompting, examining the impact of domain alignment between personas and task-related questions, finding that personabased prompting either has no effect or a slightly negative impact on model performance compared to a baseline setting. we aim to investigate whether modifying the model's internal mechanisms @xcite , rather than a prompt-based approach, can lead to improved results. this forms the core objective of our current work, guided by the following research questions: rq1: can we identify specific latent role directions, encoded as role vectors, within the activation space and derived from the model's internal mechanisms, that, when leveraged, lead to improved performance on objective tasks? rq2: do the directions that enhance performance effectively impersonate the role of interest? rq3: if we eliminate these directions in the models, do their performances suffer as a consequence? contribution. this work introduces a novel, open-source foot_0 approach to guiding the behaviour of llms through role vectors, a structured method for embedding personas directly into model activations. fig. 1 is an illustrative example showing how role vectors (in this example, a chemist-related vector) may impact llm performance. our key contributions are: role vectors and test it on selected llms and 29 roles, each capturing domain-specific knowledge and behavioural tendencies associated with each specialisation. ## preliminaries and state of the art personas and roles in llms. conversational systems can explain their reasoning in a way that aligns with the user's needs @xcite . prompting acts as the natural language interface facilitating human-ai interactions @xcite . the effectiveness of llms is often dependent on prompt formulation @xcite ; for instance, including the phrase "let's think step by step" can enhance model performance @xcite . however, achieving nuanced, consistent, and robust behavioural control, such as maintaining a specific persona over long interactions, presents significant challenges @xcite . these methods can be brittle, and the underlying mechanisms of influence remain largely opaque @xcite . some studies highlight potential biases and constraints with personabased prompting @xcite @xcite . furthermore, @xcite shows that adding personas does not consistently enhance performance on objective tasks and may even degrade it. these limitations have catalysed the exploration of internal representation manipulation for more direct and interpretable control @xcite . ## generating and evaluating role vectors the methodology is composed of three main components: (i) persona selection and prompt dataset generation, (ii) creation of relevant role vectors and (iii) evaluation methods. ## results experimental setting. our evaluation uses a set of open-source, instruction-tuned language models m, focusing on the most recent versions employed by @xcite . specifically, we analyse meta's llama 3 series @xcite , including the 3.1 8b model and the 3.2 version at 1b and 3b parameters, as well as google's gemma 2 (2b and 9b) @xcite and qwen (1.8b and 7b) @xcite . we do not consider base versions of the models (non-instruction tuned). our evaluation was conducted on the cineca leonardo supercomputer @xcite . the assessment required approximately 4,500 gpu hours to process all requests, computing over 213 million inferences (29 roles r multiplied by 2457 questions, the number of layers l, coefficients α and positions i for each of the seven models m). ## discussion the effect of steering using role vectors on model performances. in larger models, the strongest improvements are observed in the target domain and often extend to closely related areas. since we observe a strong correlation among larger models, as shown in fig. 4 , we report the full details of one model, gemma 2 9b, in tab. 2 foot_2 . beyond these expected patterns, we also find notable gains in other domains that seem both related and unrelated to the activated role (e.g., psychology improving natural science). this suggests that role activation addition contributes not only domain-specific benefits but also broader improvements across tasks. concrete examples illustrate both effects: the mathematician role produces substantial gains on math questions, while the doctor role primarily enhances medicine but also benefits natural science. at the same time, there are roles for which the direction that yielded the best validation failed to produce improvements on the test set, underscoring that the effect is not uniform. these cross-domain gains raise the possibility that the benefit of role activation is not solely tied to semantic alignment between the role and the evaluation task. instead, the intervention may introduce structured, expert-like signals into the representation space that enhance task adherence or generalisation more broadly. while the largest gains still tend to occur in-domain (e.g., mathemati-cian on math, doctor on medicine), the consistent improvements across multiple domains suggest that role activation may also act as a multiple-purpose performance enhancer. these findings for role vector interventions present a notable contrast to those typically observed with traditional persona prompting. we frame this difference in terms of direct versus indirect control. persona prompting, as explored by zheng et al. ( 2024 ), is a form of indirect control. a prompt such as "you are a physicist..." is a high-level instruction that the model must first interpret to steer its own activations toward a state it associates with the persona. this process is complex, depends on how the concept was learned during pre-training, and can be diluted or ignored within the broader context, often yielding no significant improvement. in contrast, our method with role vectors exerts direct control. it bypasses natural language interpretation to perform an intervention directly on the model's internal representations. we are not asking the model to behave like a physicist @xcite ; we are forcing its activation state into a configuration that is closer to its "physicist" state. this direct form of control might explain why our reinforcing the linear representation hypothesis. these findings offer evidence for the linear representation hypothesis @xcite . the ability to isolate a high-level concept, such as a professional role, as a single directional vector (d r ) and then use that vector to influence model behaviour, suggests that such concepts might be represented linearly in the model's activation space to a functional degree. this underlying linear structure enables a simple arithmetic intervention, such as vector addition, to produce consistent and predictable changes in model output, effectively guiding the model toward an "expert" state. are directions capturing the role? we observe that role-based interventions often produce directional shifts in the model's activation space that enhance performance within the target domain and, in some cases (e.g., as evidenced by d * r in tab. 2), in closely related domains. however, these directions are not always directly interpretable and do not correspond to the intended roles. role vectors may not just represent the roles: they also capture lateral effects and functional patterns related to how the role is used. as shown in tab. 3, in the largest model, gemma 2 9b, we found 30% of the directions yielding improvements in the relevant test split that are directly interpreted by claude 3.5 haiku as reflecting the intended role. for smaller models, this percentage decreases, with 15-17% of directions in 8b models and only 6-7% in models with less than 2 billion parameters. in other words, while some of the identified activation directions benefit performance, they do not necessarily align with the semantic role as determined by patch-scoping methods. this is a known characteristic of patch-scoping @xcite that distinguishes it from auto-interp @xcite . while auto-interp leverages the feature's maximally activating examples from the training set of saes to prompt a language model to interpret that feature, patch-scoping captures the underlying concept represented by the feature, yet struggles to identify the "label" of the concept explicitly. examining tab. 3, we notice that larger models exhibit activation directions more clearly interpretable as corresponding to specific roles than their smaller counterparts within a given model family. this observation holds for gemma-2, qwen, and llama-3.2 3 . this indicates that larger models can capture and encode fine-grained rolespecific features within their activation spaces. in contrast, smaller models tend to develop more general, abstract representations that may blend multiple role-related cues, making it harder to isolate a clear directional signal corresponding to a specific role. this aligns with the evidence from anthropic in @xcite that as model scale increases, representations become more mono-semantic, meaning activations align more closely with specific concepts. the effect of ablating roles. results in tab. 4 indicate that ablating the optimal d * r activation directions yields heterogeneous effects. for directions associated with the role @xmath0, where c ∼ r, we observe a performance degradation, which aligns with expectations. notably, in domain-specific datasets d test c unrelated to the role r, where c ̸ ∼ r, performance generally declines but occasionally exhibits a marginal improvement. we hypothesise that this variation arises because the removal process may eliminate certain noise components without significantly disrupting the core representational structure essential for the task. as shown by @xcite , many neurons across neural networks are redundant and can be removed when optimising towards a downstream task. also, tab. 3 clearly shows that multiple directions exist in the activation space that yield an improvement; ablating a single direction can remove noise and amplify the effect of the remaining ones. smaller models have less redundancy, making them more sensitive to perturbations. while steering interventions can enhance performance, they can just as easily cause deterioration across test splits. this sensitivity likely stems from more concentrated representations, where each directional component is crucial for encoding domain-specific knowledge. ## conclusion in this work, we introduced role vectors as a novel method for guiding the behaviour of llms by directly manipulating their internal activations. by computing difference-in-means vectors between role-specific prompts and a generic baseline, our approach shows that targeted activation addition can steer models toward domain-specific expertise. our experiments, spanning multiple models and domains, reveal that such interventions can improve performance in target domains and, in some cases, more broadly across domains, while largely preserving overall capabilities. we also show that the effectiveness of role-based steering is sensitive to both model scale and the depth at which the intervention is applied; larger models and deeper layers tend to yield more robust and interpretable direc-tional signals. our findings suggest that embedding role vectors within model activations offers a promising pathway for achieving more controllable behaviour in large language models. our further work will explore this phenomenon in greater depth, considering additional analytical dimensions and potential biases, using repe techniques such as activation patching (causal mediation analysis) with sae features @xcite to better explain this phenomenon.