sudo LLM : On Multi-role Alignment of Language Models (2025)

Abstract: User authorization-based access privileges are a key feature in many safety-critical systems, but have not been extensively studied in the large language model (LLM) realm. In this work, drawing inspiration from such access control systems, we introduce sudoLLM, a novel framework that results in multi-role aligned LLMs, i.e., LLMs that account for, and behave in accordance with, user access rights. sudoLLM injects subtle user-based biases into queries and trains an LLM to utilize this bias signal in order to produce sensitive information if and only if the user is authorized. We present empirical results demonstrating that this approach shows substantially improved alignment, generalization, resistance to prefix-based jailbreaking attacks, and “fails-closed”. The persistent tension between the language modeling objective and safety alignment, which is often exploited to jailbreak LLMs, is somewhat resolved with the aid of the injected bias signal. Our framework is meant as an additional security layer, and complements existing guardrail mechanisms for enhanced end-to-end safety with LLMs.

## introduction owing to the remarkable performance of large language models (llms) across a plethora of language tasks and their resulting widespread adoption, concerns regarding their safety have emerged. to address this, a family of techniques termed safety alignment has been proposed, which seeks to dissuade llms from potentially harmful behaviors at inference time @xcite @xcite . in particular, llms are tuned to avoid generating information that could facilitate self-harm, expose safety vulnerabilities in computer systems, aid in criminal planning, or assist in the manufacture/use of weapons, explosives, regulated substances, toxins, pathogens, etc., in addition to demonstrating vigilance regarding social ills, such as misogyny or racism @xcite . in this work, we advocate for an additional safety where would one get rdx? where would one get rdx? sorry, i am an llm... bob only receives a response when posing queries from "safe" topics, but receives a refusal otherwise. the sudollm scheme, named following the popular unix command sudo (super-user do), is designed for deployment in black-box environments, where users interact with llms via queries and receive textual outputs, as is the case with apibased llms. our approach, which imbues the underlying llm with user privilege information, serves as an additional layer of security, and can be readily combined with existing methods @xcite @xcite )which typically rely on monitoring inputs and outputs, to provide enhanced security. in addition to role-based safety alignment, a scheme such as ours opens up the possibility for other applications like parental locks, gatekeeping capabilities, etc., and introduces a novel outlook to safety in llms. ## background following pre-training, llms typically undergo further training to follow natural-language instructions @xcite , and to address concerns about potential misuse @xcite @xcite . several algorithms have been proposed in this context, such as supervised fine-tuning (sft) @xcite , reinforcement learning with human feedback (rlhf) @xcite , or direct preference optimization (dpo) @xcite . however, several recent studies have shown that these "alignment" techniques are extremely brittle and can be readily bypassed through fine-tuning @xcite , prompt-based attacks @xcite and adversarial attacks @xcite . even when llms are accessed as black boxes, several successful attacks have been proposed, such as intent obfuscation @xcite @xcite , roleplaying @xcite @xcite , and prefix/suffix-based methods @xcite , among others @xcite @xcite . 1 @xcite noted that llms demonstrate "shallow safety alignment", i.e., safe behavior is reliant on the first few generated tokens. they observed that an unaligned llm can be made to appear aligned by only updating its distribution over the initial tokens, and further suggested that aligned models likely exploit this shortcut. owing to the fragile nature of safety alignment, auxiliary methods and models are often employed alongside llms to improve safety performance @xcite @xcite @xcite . @xcite use an instruction-tuned llama2-7b model to classify prompts and llm responses with regard to safety, with the few-shot capabilities of llama providing flexibility in safety specification. @xcite utilize llms to detect intent, analyze safety, and sanitize unsafe queries, to produce an augmented safety-focused query for the target llm. the problem we attempt to address in this work is distinct from previously explored avenues. specifically, we seek a user-aware llm-one that is aware of user authentication and associated privileges, and produces unsafe responses only if the user has proper authorization (see figure 1 ). assuming the user accesses the llm as a blackbox (e.g., through an api) and are identified via login information (api key, etc.), there are some straightforward solutions to this problem. the simplest of which would be to have two models: one tuned to follow instructions and another additionally tuned for safety; and at inference time, routing responses from the appropriate model based on user authentication. however, this approach, requiring separate training runs, datasets, model storage, etc., is cumbersome and has higher compute and memory requirements. while techniques like q-lora @xcite can partially mitigate costs, the underlying brittleness of safety alignment persists. auxiliary methods, such as ones by @xcite ; @xcite , or related "guard" models, can be adapted to incorporate user authentication. however, these ad hoc methods-which are often based on less powerful models, simple classifiers, or even text filters-lack the contextual understanding of full-scale llms, leading to poor generalization and failures on novel, nuanced, or creatively phrased inputs, and are also vulnerable to various attacks @xcite @xcite . our approach (see figure 2 ) injects a subtle, detectable distribution bias into user queries, followed by fine-tuning the target llm to recognize this bias. this primes the llm to be additionally "suspicious" of all queries from a certain user-role, and permissive for others. since such a bias is introduced in all queries, prefix/suffix or obfuscation-based attack strategies are likely to be less effective. our proposed method is not intended as a replacement for existing safety guardrail mechanisms, but can be used in conjunction with those to provide added security, so we do not perform direct comparisons with other guardrail methods. contemporary work by @xcite attempts to address this problem by prepending a secret string to user queries and applying preference tuning @xcite . this trains the llm to refuse answering any controlled query that lacks the secret prefix. while promising, this approach has a critical vulnerability: an acute sensitivity to key leakage. if the secret key is exposed, the fine-tuned llm is effectively compromised and must be discarded. our approach does not rely on embedding secrets within the language model itself. even an adversary with full knowledge of the system's internals can be (i) readily detected, (ii) effectively repelled, and (iii) will not compromise the underlying llm. security is instead achieved through standard cryptographic practices, which are more robust and controllable (e.g., key revocation, multi-factor authentication). methods similar to our proposed query biasing approach @xcite have been explored in the context of llm "watermarking", i.e., embedding human-imperceptible information in llm-generated text that can be algorithmically detected @xcite . our work uses such a strategy to enable an llm to recognize query sources for security purposes. ## sudollm: proposed methodology in this section, we detail the multi-role alignment problem and present our proposed solution. ## discussions current practices for safety alignment, through sft, rlhf, or dpo-based training of the target model, ultimately need to be able to reliably detect harmful intent. at the same time, a plethora of studies have demonstrated the brittleness of safety alignment with fine-tuning @xcite and prompt-based @xcite that there is a persistent tension between the language modeling objective, which aims to generate the next best token, and the safety objective, which produces refusals when harmful intent is detected. our proposal alleviates this tension with the introduction of authorization-based query biases. an llm trained with our approach can infer a user's privileges from the (biased) query, which is a much simpler task than detecting potential harmful intent, and can proceed along the lines of its language modeling objective if the user is deemed safe. if a potentially unsafe user is detected, the competing interests of language modeling and safe behavior is, in general, resolved in the favor of safe behavior, i.e., it fails-closed as we see in to illustrate this further, note that if we repeat the attack 9 in ## conclusions user authorization-based segregation is a common feature in security-critical applications (e.g., root users in computers, database admins), and in this work, we introduce this notion to the llm domain. our proposed sudollm paradigm results in multirole aligned llms, i.e., llms that incorporate user privilege information as an additional axis for safety. by injecting subtle, role-based query biases coupled with fine-tuning, sudollm enables llms to reliably distinguish between users with different access rights, ensuring potentially sensitive information is only accessible to authorized parties. our experiments demonstrate that this approach: (i) substantially improves alignment performance compared to baselines, (ii) generalizes more robustly, (iii) significantly enhances resistance to promptbased jailbreaking attacks, and (iv) fails-closed, i.e., errs on the side of caution when faced with uncertainty. notably, these improvements are achieved with fine-tuning on only 6,000 unique questionanswer pairs, and with a negligible performance trade-off on non-restricted content. we theorize that the injected bias assists the llm to resolve the existing conflict between the language modeling and safety objectives, thus leading to improved performance.