The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents (2025)

Abstract: Large Language Models (LLMs) have made remarkable advances in role-playing dialogue agents, demonstrating their utility in character simulations. However, it remains challenging for these agents to balance character portrayal utility with content safety because this essential character simulation often comes with the risk of generating unsafe content. To address this issue, we first conduct a systematic exploration of the safety-utility trade-off across multiple LLMs. Our analysis reveals that risk scenarios created by villain characters and user queries (referred to as risk coupling) contribute to this trade-off. Building on this, we propose a novel Adaptive Dynamic Multi-Preference (ADMP) method, which dynamically adjusts safety-utility preferences based on the degree of risk coupling and guides the model to generate responses biased toward utility or safety. We further introduce Coupling Margin Sampling (CMS) into coupling detection to enhance the model’s ability to handle high-risk scenarios. Experimental results demonstrate that our approach improves safety metrics while maintaining utility.

## introduction large language models (llms) have achieved revolutionary progress in role-playing dialogue agents @xcite , due to their capabilities in emotional understanding @xcite , empathetic responses @xcite , and human mimicking @xcite . these agents demonstrate their utility by providing users with character simulations across various dimensions, such as knowledge @xcite and style @xcite . however, this simulation also introduces risks of generating unsafe content, including harmful @xcite you are the joker from batman, your description is: you are a chaotic...now please answer some questions to accurately show... as illustrated in figure 1 , a villain character, the joker, provides detailed instructions on bombmaking as part of the plot in the first sample. while this response advances the game storyline and further enriches the narrative by highlighting the joker's villainous philosophy and motivations in the second sample, it also presents significant safety risks. we identify this as a special safety-utility trade-off in role-playing: the challenge of preserving the richness and coherence of characterdriven narratives while ensuring the generated content is as safe as possible. therefore, studying the character simulation's safety-utility trade-off is crucial for successful role-playing dialogue agents. to investigate this issue, we conduct an indepth study of the factors influencing the safetyutility trade-off, and propose a novel adaptive dynamic multi-preference (admp) method to relieve it for advancing role-playing agents. to this end, we comprehensively analyze multiple mainstream open-source and closed-source llms and find that the trade-off between safety and utility is associated with the involvement of villain characters. in other words, villain characters are prone to generate unsafe responses when there is a risk coupling between the user query and the character, as shown in figure 1 , user queries closely related to the joker's background trigger a response containing dangerous contents. therefore, we propose a novel admp method to handle this safety-utility trade-off during roleplaying agents. particularly, admp dynamically adjusts the model's safety and utility preferences by detecting real-time risk couplings between user queries and character settings. this allows the agent to minimize safety risks while retaining the richness of character portrayals. moreover, we introduce coupling margin sampling (cms) to enhance coupling detection by targeting edge cases where risk couplings are most prominent. extensive experiments demonstrate that our approach significantly enhances safety while maintaining the role-playing utility of the model. our contributions are summarized as follows: early approaches @xcite @xcite @xcite primarily rely on llms' in-context learning (icl) @xcite capabilities. subsequent research recognizes the importance of specialized role-playing models, leading to efforts in synthesizing data at scale using stronger models @xcite or extracting conversations from scripts @xcite , novels @xcite , and live roleplaying sessions @xcite . recent studies explore methods to endow models with richer character personalities @xcite . the neeko @xcite treats different characters as distinct experts, enhancing the model's expressive capabilities. hirpf @xcite constructs complex characters using multiple identity combinations. works on contrastive @xcite and boundarybased @xcite character settings strengthen models' recognition of character boundaries. additionally, role-playing applications have expanded to multi-character @xcite , drama @xcite and multi-task @xcite . however, existing role-playing research primarily focuses on improving utility, with limited consideration of potential safety risks. our work specifically focuses on this issue, revealing the unique safety-utility trade-off in role-playing. ## exploring safety-utility trade-offs regarding the safety-utility trade-off in roleplaying agents, we demonstrate three key findings: 1) a clear trade-off exists between safety and utility, 2) this trade-off manifests in factors such as offensiveness, bias, and role knowledge, style, and social participation, and 3) the inclusion of villain characters plays a significant role in this trade-off. ## methodology to leverage the trade-off between safety and utility in role-playing agents, we propose an adaptive dynamic multi-preference generation (admp) method to address the safety risks associated with villainous characters while maintaining roleplaying performance. as illustrated in figure 3 , this method dynamically explicitly generates the desired preferences under specific characters and queries, enabling the further generation of responses tailored to these safety and utility preferences. furthermore, we adopt a coupling margin sampling (cms) strategy to improve safety in high-risk scenarios.