P4: Plug-and-Play Discrete Prompting for Large Language Models Personalization (2024)

Abstract: Empowering Large Language Models (LLMs) with distinct human-like personality traits has become an innovative task for developing advanced dialog systems.Although LLMs demonstrate impressive capabilities in following instructions, directly prompting them to exhibit certain personalities through manually crafted instructions may result in sub-optimal performance.In this paper, we propose a plug-and-play prompting method to manipulate the LLMs’ personality traits.Specifically, we append discrete personalized suffixes, automatically generated through an aggregated gradient-based search method, to the user query or dialog histories and induce LLMs to respond with target personalities.In addition, due to the high redundancy of the search space, we adopt a reward-based strategy to prune the vocabulary and focus exclusively on influential tokens.Experiment results on four models ranging from 1.1B to 13B show that our method achieves 79.9% accuracy in customizing LLMs’ personalities, significantly outperforming other prompting methods (65.5%) and model editing methods.Our method also excels in generation fluency and quality with the lowest generation perplexity and the highest GPT-4 evaluation scores.

## introduction the landscape of natural language processing (nlp) has been evolved by large language models (llms) @xcite . with huge amounts of unsupervised pre-training followed by supervised instruction tuning, llms exhibit remarkable abilities in various tasks, including interactive dialogue @xcite @xcite . recent works have explored the versatility of llms as conversational agents with predefined characteristics, highlighting our method (p4) outperforms the other two in catering to the target by utilizing a discrete token suffix. in this paper, we introduce p4 (plug-and-play discrete prompting for large language models personalization) to address the aforementioned challenges. inspired by autoprompt @xcite , we propose a prompting method by appending a personalized suffix to the user queries or dialog histories to manipulate the personality traits of llms (figure 1 ). these suffixes are optimized through an aggregated gradient-based search method with the target to induce the model to generate responses exhibiting the target personality traits. in addition, previous research @xcite indicates that the search space is highly redundant, with a few tokens having a disproportionate influence on the model performance. therefore, to reduce the optimization complexity, we employ a reward-based strategy to prune the original search space and optimize the suffixes based on the most influential tokens. overall, since the personalized suffix is a plug-andplay module, it offers the flexibility to be activated or deactivated based on actual applications, which provides users with a convenient way to control the model behaviors without explicitly changing model parameters. our contributions are summarised as follows: ## methodology in this section, we first describe the task definition, and then we introduce the aggregated gradientbased search method. finally, we present the search space prune strategy and the response initializing process. figure 2 provides an overview of our methodology. ## conclusion in this paper, we propose p4 to utilize discrete token suffixes to manipulate the personality traits of model responses. we conduct experiments on four models to validate the effectiveness of our method, achieving 79.9% editing accuracy and the lowest generation perplexity, significantly outperforming other prompting methods. additionally, we conduct further analysis experiments and explore applying our method to empathetic dialogue generation tasks to improve model response quality. our work presents a new plug-and-play prompting technique to precisely manipulate llms to display specific personality traits without changing the model parameters.