Beyond Persuasion: Towards Conversational Recommender System with Credible Explanations (2024)

Abstract: With the aid of large language models, current conversational recommender system (CRS) has gaining strong abilities to persuade users to accept recommended items. While these CRSs are highly persuasive, they can mislead users by incorporating incredible information in their explanations, ultimately damaging the long-term trust between users and the CRS. To address this, we propose a simple yet effective method, called PC-CRS, to enhance the credibility of CRS’s explanations during persuasion. It guides the explanation generation through our proposed credibility-aware persuasive strategies and then gradually refines explanations via post-hoc self-reflection. Experimental results demonstrate the efficacy of PC-CRS in promoting persuasive and credible explanations. Further analysis reveals the reason behind current methods producing incredible explanations and the potential of credible explanations to improve recommendation accuracy.

## introduction conversational recommender systems (crss) aims to engage in a natural language conversation with users, provide recommendations, and ultimately achieve a high level of user acceptance @xcite . to achieve this, providing proper recommendation explanations along with accurate recommendations is paramount, because users are usually not familiar with the recommendations @xcite . such explanations should be carefully crafted, incorporating persuasive elements that can influence user behavior and decision-making, thus increasing the likelihood of user acceptance of the recommendations @xcite . recently, the integration of large language models (llms) has dramatically enhanced the persuasive power of current crss. llms possess the remarkable ability to generate highly convincing content that can rival, and even surpass, human-crafted persuasion @xcite , significantly augmenting crss in delivering persuasive explanations, which improve user understanding and ultimately result in higher acceptance rates @xcite . while llm-based crs is highly persuasive, a concerning trend has been observed: these crss can mislead users by incorporating deceptive elements into their explanations. for example, as illustrated in figure 1 , a crs mistakenly recommends the film "mission: impossible" as a comedy movie to unfamiliar users, ultimately resulting in a negative user experience after viewing the film. this practice contradicts the formal definition of persuasion, which emphasizes influencing people's behaviors or attitudes without using coercion or deception @xcite . such incredible explanations can create erroneous perceptions about recommended items @xcite , ultimately damaging the long-term trust between users and the crs @xcite @xcite . while the need to enhance credibility during persuasion in crss is acknowledged @xcite , effective solutions remain elusive. to this end, we introduce a simple yet effective method for persuasive and credible crs, called pc-crs. it proactively emphasizes both persuasiveness and credibility during the generation of explanations, and then gradually refines them via post-hoc self-reflection. this is achieved by a twostage process: strategy-guided explanation generation and iterative explanation refinement. specifically, in the first stage, pc-crs utilizes the novel credibility-aware persuasive strategies to guide the generation of candidate explanations. such strategies are informed by social science research on persuasion @xcite and further tailored with credible information to ensure both persuasive and credible explanations in our scenario. in the second stage, pc-crs utilizes a self-reflective refiner to identify and correct potential misinformation in the candidate explanations. it is due to generative models have the inherent tendency to prioritize contextual coherence at the expense of faithful adherence to source information @xcite @xcite . as such, pc-crs prevents potential deception in candidates, thus enhancing credibility. its training-free nature also makes it a highly efficient and adaptable solution. we conduct extensive experiments to demonstrate the effectiveness of pc-crs. our experiments leverage the widely-used simulator-based evaluation framework 1 @xcite and employ two crs benchmarks: redial @xcite and opendialkg @xcite . experimental results show that pc-crs, on average, achieves an improvement of 8.17% on credibility score (i.e., consistency with factual information) and 5.07% on persuasiveness score (i.e., raising user's watching intention towards recommended items), compared to the best baseline. further analysis reveals the reason why llm-based crs generates incredible explanations is that they cater to user's history utterances rather than describing items faithfully. in addition, the in-depth analysis also suggests that our credible explanations promote recommendation accuracy. this is potentially due to that credible explanations avoid the 1 human evaluation in section 4.4 validates our reliability. introduction of noisy information and contribute to a reliable conversation context, making it easier to comprehend user's true preference. our main contributions are as follows: ## related work our research focuses on the explanations of crss, particularly highlighting their persuasiveness and credibility. hence, we provide an overview of crs and persuasive and credible recommender systems and then discuss our differences. crs. crs enables users to engage in free-form natural language conversations with the system to achieve their recommendation-related goals @xcite @xcite @xcite . to generate human-like responses, early studies leverage pre-trained language models (plms) as their backbones @xcite , enabling them to proactively interact and engage with users through verbal explanations @xcite @xcite . in the era of llms, crss have shifted from providing simple information to actively persuading users during explanations @xcite , ultimately increasing user acceptance @xcite . while llms enable crss to generate highly persuasive explanations, a recent study revealed a concerning trend: they may incorporate misinformation to achieve persuasiveness @xcite , jeopardizing the long-term relationship of trust between users and the crs. to address this challenge, we propose a method to enhance the credibility of crs explanations during persuasion. persuasive and credible recommender systems. early research on identifying how people persuade others with credible explanations in recommendations draws heavily on insights from social science and human-computer interaction (fogg refine strategy selector explanation generator critique conversation history item information final explanation ... an agent faces insurmountable odds , blending intense thrills and suspense that will leave you gripping the edge of your seat, fully immersed in the high-stakes world of espionage .... guide "emotion appeal" select credibility-aware persuasive strategies 1. logical appeal 2. emotion appeal 3. framing 4. evidence-based persuasion 5. social proof 6. anchoring self-reflective refiner strategy-guided explanation generation ① strategy-guided explanation generation iterative explanation refinement ## pc-crs overview. our pc-crs, as illustrated in figure 2 , involves a two-stage process, i.e., strategy-guided explanation generation and iterative explanation refinement. given the conversation history and item information, the former stage selects an appropriate strategy from credibility-aware persuasive strategies and then generates a candidate explanation accordingly. then, taking the previous candidate as input, the latter stage critiques and refines it to eliminate misinformation and yield the final explanation. pc-crs leverages llms with detailed chain-of-thought instructions @xcite to make full use of the generative capabilities of them in the above two stages. ## experiments in this section, we investigate the superiority of pc-crs on persuasiveness and credibility (section 4.2). subsequently, we provide detailed analyses on characteristics of pc-crs to gain understanding why it alleviates the incredibility and improves the recommendation accuracy (section 4.3). then, ablation studies and human evaluation indicate the necessity of two stages in pc-crs and the reliability of our evaluation, respectively (section 4.4). ## conclusion the pursuit of trustworthy ai necessitates a profound understanding of credibility. this paper delves into the crucial role of bolstering the credibility of the crs during its persuasions, recognizing that such credibility is essential for cultivating long-term trust between users and the crs. we introduce a simple yet effective method for enhancing crs with the awareness of being both persuasive and credible. our experimental findings demonstrate the efficacy of this method in promoting persuasive and credible explanations, while also shedding light on the inherent tendency of current llm-based crs to prioritize persuasion over honesty. additionally, our research highlights the delicate balance required when aiming for both persuasiveness and credibility without resorting to deception -a balance demanding sophisticated linguistic capabilities within llms. our work lays the groundwork for further exploration of this vital relationship, and we encourage future research to delve deeper into this critical area.