Rehearse With User: Personalized Opinion Summarization via Role-Playing based on Large Language Models (2025)

Abstract: Personalized opinion summarization is crucial as it considers individual user interests while generating product summaries.Recent studies show that although large language models demonstrate powerful text summarization and evaluation capabilities without the need for training data, they face difficulties in personalized tasks involving long texts. To address this, Rehearsal , a personalized opinion summarization framework via LLM-based role-playing is proposed. Having the model act as the user, the model can better understand the user’s personalized needs.Additionally, a role-playing supervisor and practice process are introduced to improve the role-playing ability of the LLMs, leading to a better expression of user needs.Furthermore, the summary generation process is guided by suggestions from virtual users, ensuring that the generated summary includes the user’s interest, thus achieving personalized summary generation. Experiment results demonstrate that our method can effectively improve the level of personalization in large model-generated summaries.

## introduction personalized opinion summarization, which takes into account user characteristics and interests while summarizing multiple product reviews, aims to meet the individual needs of users. based on general multi-document opinion summarization, personalized opinion summarization needs to understand user preferences from relevant historical information and analyze aspects of the current product that the user may be interested in. then, based on the user's interests, it provides a targeted summary of product reviews that emphasizes content most relevant to the user. due to the difficulty in annotation, research related to personalized opinion summarization is almost nonexistent. recent studies have shown that large language models (llms), such as gpt-3.5, can achieve stateof-the-art -even human-level -performance on standard summarization benchmarks without requiring large-scale training data @xcite @xcite . moreover, some studies have demonstrated that llmbased summary evaluation metrics can more flexibly assess different aspects of summary generation, showing stronger correlations with human evaluation, compared to traditional metrics @xcite . however, these studies are mostly limited to generic summarization scenarios. in scenarios such as personalized recommendations @xcite @xcite , search @xcite , and dialogue @xcite @xcite , some studies have shown that llms have certain personalization capabilities. however, it has also been found that personalized llms also face difficulties in long-text scenarios @xcite . on the one hand, handling long texts with extensive redundant information is challenging for llms @xcite . excessively long input texts easily exceed the model's input length limit. besides, there is usually a lot of redundant information from product reviews or user history in multi-document opinion summarization, which greatly hinders llms from understanding product features and users' interests accurately. on the other hand, it is highly difficult to infer users' preferences for the current product from their complex historical data @xcite . users' shopping preferences are not explicit but hidden within a large volume of historical information. moreover, their interest in the current product cannot be directly equated to their preferences for other products, requiring intricate analysis. therefore, we propose rehearsal, a personalized opinion summarization framework based on llms role-playing. as shown in figure 1 , to alleviate the input pressure on the llms from complex user historical information and a large volume of product review texts, we adopt a multi-agent framework, which models product information and user information separately through the summary system and the user system. via the interaction between these two agent systems, the personal summary is generated. to enable the user system to better understand user interests, role-playing with supervision and practice is introduced into the user system. specifically, the framework consists of three steps: generic summary generation, user suggestions based on role-playing with supervision and practice, and retrieval-augmented personalized summary generation. first, we generate a generic opinion summary based on product reviews. second, an llm acts as the user and proposes modifications to the current summary. to ensure consistency in role-playing, a professional role-playing observer is introduced. the observer provides continuous modification suggestions based on four dimensions of user consistency, ensuring that the user model remains true to the role. before formal roleplaying, an exercise process is executed in advance, where the observer conducts role-playing practice with the user model. the results of the process are then used to strengthen the user model in the formal role-playing. third, after receiving user suggestions, retrieval augmentation is applied to eliminate irrelevant product reviews and extract important user reviews. the summary is then revised based on the retrieved text and the suggestion. our main contributions are as follows: ## methodology in this section, we provide a detailed introduction to our method for personalized opinion summarization based on user role-playing. given a set of reviews about an entity (e.g., a product) and a collection of historical comments from a specific user, the aim is to summarize the opinions that are of interest to the user, as expressed in the product reviews. next, we will introduce the overall process of the method. in the subsequent sections, we will explain the three components of the method: general summary generation, user suggestions based on practice and supervision, and how to perform retrieval-augmented personalized summary rewriting based on personalized suggestions. ## conclusion we explored a personalized opinion summarization generation method based on llms and collected a test set to evaluate the level of personalization in the generated summaries using llm-based metrics. in the design of the personalized opinion summarization method, we adopted a multi-agent framework where personalized summaries are generated through the interaction between the summarization system and the user system. to help the user system better understand the user's interests, roleplaying was introduced into the user system, along with supervision and practice. experimental results demonstrate the effectiveness of this method.