On the Way to LLM Personalization: Learning to Remember User Conversations (2025)

Abstract: Large Language Models (LLMs) have quickly become an invaluable assistant for a variety of tasks. However, their effectiveness is constrained by their ability to tailor responses to human preferences and behaviors via personalization. Prior work in LLM personalization has largely focused on style transfer or incorporating small factoids about the user, as knowledge injection remains an open challenge. In this paper, we explore injecting knowledge of prior conversations into LLMs to enable future work on less redundant, personalized conversations. We identify two real-world constraints: (1) conversations are sequential in time and must be treated as such during training, and (2) per-user personalization is only viable in parameter-efficient settings. To this aim, we propose PLUM, a pipeline performing data augmentation for up-sampling conversations as question-answer pairs, that are then used to finetune a low-rank adaptation adapter with a weighted cross entropy loss. Even in this first exploration of the problem, we perform competitively with baselines such as RAG, attaining an accuracy of 81.5% across 100 conversations.

## introduction large language models (llms) have quickly become a go-to resource for learning about new topics or assisting with a plethora of tasks. however, to fully unlock the models' capabilities, responses require personalization, tuning the model to the user's preferences and needs @xcite @xcite . prior work on llm personalization has largely focused on adapting to the user's style and preferences via reinforcement learning from human feedback (rlhf) @xcite @xcite *work done while interning at apple. @xcite @xcite and parameter-efficient finetuning (peft) @xcite . to move beyond simple preference learning and small facts about the user, retrieval augmented generation (rag) methods have been used to integrate user profiles and conversation history into the model's generation process @xcite @xcite @xcite . however, rag-based methods require maintaining external storage and picking a suitable number of documents to retrieve, with llm performance having been shown to deteriorate with larger context windows @xcite . furthermore, @xcite show that personalized finetuning yields improved model reasoning over zeroshot prompting. this leads us to question whether parametric knowledge injection could yield a more streamlined approach to encoding knowledge of prior interactions with the user, opening new avenues for future llm personalization. current works on knowledge injection mainly focus on adding fact-based knowledge to llms @xcite @xcite and note that knowledge injection remains an open challenge @xcite . in this paper, we tackle the challenge of injecting knowledge of prior user conversations into llms via finetuning. our efforts are in support of enabling future personalization research. given the focus on user conversations, we identify and impose two important constraints: (1) finetuning must be parameter efficient and (2) conversations are sequential in nature and we do not want to store them like rag-based methods. note that we focus on inter-conversation rather than intra-conversation knowledge. we are interested in remembering the conversation holistically after it has occurred, as individual turns within a conversation can usually be injected into the context window. to the best of our knowledge, this is ## plum we propose plum, a two-stage pipeline for injecting knowledge of prior user conversations into llms. the first stage encompasses augmenting the user conversation data. the second stage focuses on training via peft with a custom loss function. we refer the reader to figure 1 for a visual overview. ## conclusion in this work, we explore injecting knowledge of prior user conversations into llms. we propose plum, a pipeline for finetuning a lora adapter on question-answer pairs about prior conversa-tions, maintaining conversation order. moreover, we propose a custom loss for improved results. we achieve competitive results to rag baselines, while not requiring to store each individual conversation. our results indicate directly injecting personalization data into llms as an interesting avenue for future research, such as reducing redundancy in subsequent conversations or extending reasoning capabilities.