Personalized Question Answering with User Profile Generation and Compression (2025)

Abstract: Large language models (LLMs) offer a novel and convenient avenue for humans to acquire knowledge. However, LLMs are prone to providing “midguy” answers regardless of users’ knowledge background, thereby failing to meet each user’s personalized needs. To tackle the problem, we propose to generate personalized answers with LLMs based on users’ past question-answering records. We dynamically generate and update a user’s domain and global profiles as the user asks questions, and use the latest profile as the context to generate the answer for a newly-asked question. To save tokens, we propose to compress the domain profile into a set of keywords and use the keywords to prompt LLMs. We theoretically analyze the effectiveness of the compression strategy. Experimental results show that our method can generate more personalized answers than comparative methods. The code and dataset are available at https://github.com/DaSESmartEdu/PQA.

## introduction the rapid advancement of large language models (llms) @xcite @xcite @xcite and the maturation of associated technologies such as rag @xcite @xcite and agent @xcite @xcite have established llms as a pivotal avenue of knowledge acquisition. however, existing qa systems based on llms lack personalization, or simply retrieve the qa history to facilitate multi-round dialogue. under such circumstances, llms often fail to generate responses that match users' background knowledge. in this work, we investigate the task of personalized question answering with llms, which attempts to respond to a user's question based on her background knowledge, thereby generating a tailored answer. existing studies have discussed different approaches to adapting llms to various personalized tasks, among which two types of approaches are generally used. the first type of approaches fine-tunes the llms @xcite @xcite using users' personal knowledge, which is resource-intensive and impractical to fine-tune for each user. the second type of approaches introduces users' historical information into prompts to obtain personalized responses from llms @xcite @xcite . however, users' knowledge in a specific domain is hard to retrieve from fragmented information, thereby resulting in incorrect prompts. to tackle the above challenges, we are inspired by daily question-answering between students and teachers. as illustrated in figure 1 , when a student asks a question, the teacher assesses the student's domain knowledge pertaining to the question and gives an answer that is particularly tailored for the student. motivated by this, we maintain domain and global profiles for each user by leveraging the user's historical qa records, which represent the user's background knowledge. when the user asks a new question, we retrieve from the profiles the relevant background knowledge as the context to make an llm generate a personalized answer. particularly, the challenge is to maintain a precise and concise profile, so that the retrieved background knowledge does not consume too many additional tokens. inspired by the recent studies on prompt compression @xcite @xcite , we propose to compress the domain profile into a set of keywords and use the compressed profile to guide personalized question-answering. we theoretically analyze the effectiveness of the compression strategy. we conduct experiments on three qa datasets to show that our proposed method outperforms comparative methods in terms of personalization. our contribution can be summarized as follows: ## the methodology figure 2 shows the overview of our proposed method, which consists of four steps, namely, domain extraction (section 2.1), domain profile generation (section 2.1), profile update and question answering (section 2.2). in addition, we optimize the process with domain profile compression (section 2.3). we also theoretically analyze why the compressed profile can replace the domain profile and maintain the effectiveness of the methodology (section 2.4). ## conclusion this study aims to generate personalized answers that match users' individual background knowledge. we extract and compress user profiles based on the content of questions asked by users in the past, and use the compressed profiles to prompt llms for personalized question answering. we theoretically analyze why the compression strategy is effective. experimental results show that our method outperforms all comparative methods. in the future, we plan to leverage other user generated data to enhance personalized question-answering, and design more semantic evaluation metrics. becomes: the error terms from approximating the sum by the integral become negligible as n ′ → ∞. therefore, we have: