LLM -based Conversational Recommendation Agents with Collaborative Verbalized Experience (2025)

Abstract: Large language models (LLMs) have demonstrated impressive zero-shot capabilities in conversational recommender systems (CRS). However, effectively utilizing historical conversations remains a significant challenge. Current approaches either retrieve few-shot examples or extract global rules to enhance the prompt, which fail to capture the implicit and preference-oriented knowledge. To address this challenge, we propose LLM-based Conversational Recommendation Agents with Collaborative Verbalized Experience, abbreviated as CRAVE. CRAVE begins by sampling trajectories of LLM-based CRS agents on historical queries and establishing verbalized experience banks by reflecting the agents’ actions on user feedback. Additionally, we introduce a collaborative retriever network fine-tuned with item content-parameterized multinomial likelihood on query-item pairs to retrieve preference-oriented verbal experiences for new queries. Furthermore, we developed a debater-critic agent (DCA) system where each agent maintains an independent collaborative experience bank and works together to enhance the CRS recommendations. We demonstrate that the open-ended debate and critique nature of DCA benefits significantly from the collaborative experience augmentation with CRAVE. The code is available at https://github.com/yaochenzhu/CRAVE.

## introduction conversational recommender systems (crs) aim to recommend items by making dialogues with users @xcite . compared with traditional recommender systems (rs) that leverage historical interactions or item content to suggest new items @xcite @xcite , crss engage users to express their preferences in natural language, which attracts more attention in both academia @xcite and industry @xcite . traditional crs methods train sequential models such as rnns @xcite or transformers @xcite on historical conversations with groundtruth items to extract user preference, where external databases (e.g., item/word level knowledge graphs) are often introduced as the prior knowledge @xcite . afterward, efforts have been devoted to finetuning pretrained language models (plm) despite the success of llms as zero-shot crss, utilizing the knowledge in historical conversations and user feedback still remains a great challenge. first, most llms are large black-box models, which preclude model finetuning with historical conversations @xcite . one naive strategy is to retrieve conversation-feedback pairs as few-shot demonstrations in the prompt @xcite . however, the semantic gap between the conversation and user preference is substantial, making it difficult (even for the llms) to derive generalizable recommendations for new conversations in an in-context manner @xcite . to bridge the semantic gap, summarizing the implicit knowledge by reflecting on historical recommendations, as proposed in verbal reinforcement learning (vrl), @xcite @xcite appears to be a promising strategy. however, unlike typical vrl tasks such as question answering (qa) that assume llms can have multiple attempts for each question with external feedback, recommendations for each user query are usually one-time. consequently, @xcite introduced a general memory part in memocrs, which adapted vrl to llm-based crss by summarizing global rules via reflecting their reasoning and recommendations for all the training samples, which are shared across all test queries. nevertheless, these global rules often fail to account for the personalized preferences of different user queries, which are critical for crss. furthermore, all the methods introduced above focus on a single llm with chain-of-thought (cot) reasoning @xcite . nevertheless, @xcite found that llms tend to think convergently with cot (e.g., see fig. 1 , agent 1), but for crs, since user queries are usually vague (otherwise users will directly search for the item instead of exploring with the system), it is important to promote divergent thoughts on different aspects of user preference. recently, llm debate has been introduced to address such limitations @xcite . however, the tasks that llm debate focuses on (e.g., qa) usually have clear answers derived with logic/math reasoning from the question, whereas the answers for crs are more vague and personalized, resembling an open-end debate without definite answers. therefore, it is especially crucial to derive valuable experiences from historical conversations and user feedback to guide each debater to contribute new perspectives on user preferences. in addition, since there are no definite answers nor multiple attempts for crs debaters to generate the final answers, it would be challenging to comprehensively evaluate the reasoning and recommendations of different debaters. to address the challenges, we propose crave, i.e., llm-based conversational recommendation agents with collaborative verbalized experience. specifically, crave starts by sampling trajectories of llm-based crs agents on historical conversations and establishing verbalized experience banks by reflecting the agents' actions on the user feedback. afterward, a collaborative retriever network finetuned on query-item pairs with item contentparameterized multinomial likelihood is introduced to retrieve the agent-specific preference-oriented experiences for each new query. furthermore, we develop a debater-critic agent (dca) system to tackle the convergent thinking issue of cot-based crs agents, where each agent maintains an independent collaborative experience bank and works together to enhance the crs recommendations. we find that the open-end debate nature of the dca system benefits significantly from the collaborative experience augmentation with crave. ## problem formulation let i denote the set of items in the system. we use {(u t , s t , i t )} t @xmath0, where at the t-th turn, u @xmath1, system} generates an utterance s t , and i t ⊆ i denotes the set of mentioned items. when u @xmath2, we have the groundtruth items i ## methodology the overall framework of crave is illustrated in fig. 2 , which is composed of three components: (i) conversational recommendation agents, (ii) collaborative experience retrieval, and (iii) experience augmented generation. the details of crave will be introduced in the following subsections. ## diversity analysis finally, we compare the recommendation diversity of crave with cot and dca backbones in ## conclusions in this paper, we introduced crave, i.e., conversational recommendation agents with collaborative verbalized experience. we show that valuable experiences can be gained from the training set via trajectory sampling and self-reflection, which substantially augments the current state-of-the-art llmbased crs agent systems. we also find that the improvement is especially evident for the debatercritic system, which resembles an open-end debate without definite groundtruth. in addition, we show that the collaborative retriever network, which is a fine-tuned bert model that encodes preference similarity of conversations, plays a key role in crave to retrieve personalized experiences.