Aligning Recommendation and Conversation via Dual Imitation (2022)

Abstract: Human conversations of recommendation naturally involve the shift of interests which can align the recommendation actions and conversation process to make accurate recommendations with rich explanations. However, existing conversational recommendation systems (CRS) ignore the advantage of user interest shift in connecting recommendation and conversation, which leads to an ineffective loose coupling structure of CRS. To address this issue, by modeling the recommendation actions as recommendation paths in a knowledge graph (KG), we propose DICR ( D ual I mitation for C onversational R ecommendation), which designs a dual imitation to explicitly align the recommendation paths and user interest shift paths in a recommendation module and a conversation module, respectively. By exchanging alignment signals, DICR achieves bidirectional promotion between recommendation and conversation modules and generates high-quality responses with accurate recommendations and coherent explanations. Experiments demonstrate that DICR outperforms the state-of-the-art models on recommendation and conversation performance with automatic, human, and novel explainability metrics.

## introduction conversational recommendation systems (crs) @xcite aim to conduct recommendations during conversations with users @xcite . compared with traditional recommendation systems @xcite , crs's two main advantages are understanding user's dynamic interest during the conversation and making persuasive response with coherent explanations of the recommendation @xcite . in both advantages, user interest shift plays an essential role. as the dialog in fig. 1 , the successful recommendation of "iron man 3 " in the final response is achieved by tracking and reasoning the user interest shift of "the @xmath0. furthermore, the final response is persuasive because of utilizing part of the user interest shift, i.e., "@xmath1, as the explanation. due to the limited context @xcite , a recommendation module based on knowledge graph (kg) is helpful to track the user interest shift in conversation. as shown in figure 1 , formally corresponding with the paths in kg, the user interest shift in conversation not only guides the reasoning-based prediction of recommendation, but also guides the explanation generation in response. however, existing kg-enhanced crs models @xcite @xcite @xcite have not made full use of the user interest shift to tightly align the kg-based recommendation and conversation. consequently, one issue is the less accurate recommendation due to using unrelated entities in conversation to support the recommendation instead of using coherent en-tities in user interest shift. the other issue is the lack of explanation in response due to black-box representation of the user preference ignoring the explicit preference logic in user interest shift. to address these issues in crs, we propose to align the explicit behaviors of recommendation reasoning and conversation process, which are described as the recommendation paths and interest shift paths, respectively. as in figure 1 , a recommendation path is an explicit path in kg consisting of explicit relations of entity nodes and ending with a predicted recommended entity; an interest shift path is an implicit path in dialog context consisting of implicit relations of entity words. the recommendation path and interest shift path are concrete manifestations of the user interest shift in kg and dialog, respectively. the sequence of interest entities shared by the two paths facilitates the alignment of recommendation reasoning and conversation process, which can be effectively achieved by imitation learning @xcite . therefore, we propose a dual imitation framework named dicr (dual imitation for conversational recommendation). dicr designs bidirectional alignment signals from dual imitation learning to improve the crs by forcing the recommendation and conversation to behave similarly to the shared user interest shift. precisely, in a conversation-aware recommendation module, to align the recommendation reasoning to the conversational user interest shift, the recommendation side of the dual imitation, i.e., path imitation, adopts adversarial reinforcement learning to make the recommendation reasoning policy imitate the user interest shift in conversation. the reasoned recommendation paths are provided to the conversation module as alignment signals. in a recommendation-aware conversation module, to align the conversation process to the recommendation paths, the conversation side of the dual imitation, i.e., knowledge imitation and semantic imitation, which refine weight distribution and semantic encoding of recommendation paths by imitating the human response and the utterance statement semantic of golden explanations, respectively. these two imitations also provide the recommendation module with the rewards as the alignment signals indicating how well the predicated recommendation paths consist to the conversation context. our contributions are summarized as follows: ## related work conversational recommendation systems (crs) aim to obtain user interests through conversational interaction and make persuasive recommendations @xcite @xcite . to track the user interest shift in conversation, an intuitive strategy is to ask related questions @xcite which leads to question-based crs @xcite . limited by predefined templates for asking and recommending, it is difficult for question-based crs to flexibly adopt different contexts and converse in a human-like manner. towards more flexible conversation, generationbased crs @xcite @xcite capture user interests from context and generate responses containing persuasive explanations for recommendation. limited by sparse context and language complexity @xcite , it is challenging for generation-based crs to track user interest shift in conversation. as a popular solution, kg-enhanced crs @xcite @xcite involve knowledge graphs about explicit relations among potential interest items. although kg-enhanced crs has achieved significant improvements, most approaches @xcite @xcite ) adopt a black-box style to transfer implicit and sparse information between recommendation and conversation. the explicit interest reasoning path in kg and explicit interest shift path in conversation have a good chance to align the recommendation and conversation to benefit each other. prior posterior knowledge imitation decoder response 𝑌 utterance statement 𝑈 beam search: 𝑁 𝑝 recommendation paths 𝒫 context 𝐶 fusion semantic imitation path-aware response generation actor critic actor-critic path imitation discriminator path imitation recommendation-aware conversation module conversation-aware recommendation module mdp environment 𝒢 state 𝑠 𝑡 action space 𝒜 𝑡 action 𝑎 𝑡 reward 𝑅 𝑇,𝑡 golden segment action path segment (𝑠 𝑡 , 𝑎 𝑡 ) tokenize(𝒫) tokenize(𝒫) 𝑅 𝑝,𝑡 𝑅 𝑠,𝑡 𝑅 𝑘,𝑡 × 𝜉 𝑔𝑒𝑛 × (1 -𝜉 𝑔𝑒𝑛 ) 𝑃 𝒫 (𝑤) 𝑃 𝑣𝑜𝑐𝑎𝑏 (𝑤) 𝑄 𝛿 (𝑠 𝑡 , 𝑎 𝑡 ) 𝜋 𝜑 (𝑎 𝑡 , 𝑠 𝑡 , 𝒜 𝑡 ) 𝑃(𝑤) knowledge encoder context encoder semantic encoder … σ path distribution attention 𝑜 𝑆 𝑜 𝐶 𝐻 𝐶 𝑜 𝑌 𝑜 𝑆,𝒫 𝐻 𝒫,𝑖 𝑖=1 𝑁𝑝 𝑜 𝒫,𝑖 𝑖=1 𝑁𝑝 𝑜 𝒫,𝑖 𝑖=1 𝑁𝑝 𝑜 𝑈 (𝑠 𝑡 𝐾 , 𝑎 𝑡 𝐾 ) discriminator mim figure 2: the dual imitation architecture of the proposed dicr model. 3 problem formalization a kg @xmath2, r, e ′ ) | e, e ′ ∈ e, @xmath3, where e is entity set, and r is relation set. each triplet (e, r, e ′ ) indicates that the head entity e and the tail entity e ′ are connected by the relation r. in this paper, a recommendation path is a multi-hop reasoning path p on g: @xmath4. . . in this paper, given a conversation context c and a kg g, we aim to generate a response y containing the recommendation set i and explanation u . we design a novel crs model, in which kg path reasoning obtains explicit reasoning path set p from g to help generate y containing recommendation set i and coherent explanation u of i. ## conclusions we propose dicr, which adopts the dual imitation to align crs's recommendation and conversation behavior explicitly. using recommendation paths and conversational rewards as alignment signals for tight interaction between recommendation and conversation, dicr achieves accurate recommendations and coherent explanations in generated responses. the effectiveness of dicr is verified by designed novel explainability evaluations together with human and existing automatic metrics.