CRFR : Improving Conversational Recommender Systems via Flexible Fragments Reasoning on Knowledge Graphs (2021)

Abstract: Although paths of user interests shift in knowledge graphs (KGs) can benefit conversational recommender systems (CRS), explicit reasoning on KGs has not been well considered in CRS, due to the complex of high-order and incomplete paths. We propose CRFR, which effectively does explicit multi-hop reasoning on KGs with a conversational context-based reinforcement learning model. Considering the incompleteness of KGs, instead of learning single complete reasoning path, CRFR flexibly learns multiple reasoning fragments which are likely contained in the complete paths of interests shift. A fragments-aware unified model is then designed to fuse the fragments information from item-oriented and concept-oriented KGs to enhance the CRS response with entities and words from the fragments. Extensive experiments demonstrate CRFR’s SOTA performance on recommendation, conversation and conversation interpretability.

## introduction different from traditional one-shot recommendation systems @xcite , conversational recommender systems (crs) obtain users' interests through multi-turn conversation, and make recommendations with responses. typical crs consists of two parts: recommender and response generation. the recommender aims to understand users' dynamic preference from contextual utterances to find the items matching the preference best. subsequently, the response generation aims to generate appropriate sentences asking for more information or exhibiting the recommended items and related explanation. recommender and response generation are expected to be mutually beneficial. however, contextual utterances are usually insufficient to understand users' preference @xcite . external knowledge, especially knowledge graphs (kgs), helps crs to alleviate the problem in item-oriented and concept-oriented kgs, multiple high-order reasoning paths contain fragments, from which items and concepts support the recommendation and the interpretability of response. @xcite . existing kg-based crs still have some issues. the first issue is the lack of explicit reasoning, especially high-order explicit reasoning, in kg to track the deep shift of user interest in conversations. for example, in figure 1 , the user prefers "shutter island" and "the prestige" due to certain attributes, i.e., actor "dicaprio" and director "nolan". in this case, both "dicaprio" and "nolan" lead to a 2order explicit reasoning of the user's interest shift to "inception" in the kg. such multiple high-order explicit reasoning paths are strong evidence to recommend "inception". besides item kg, explicit high-order reasoning in concept kg can further describe the shift of concept words of interest, e.g., from "suspense" to "unpredictable" in figure 1 , and enhance the interpretability of response. however, because kg nodes usually have many neighbors, effective high-order explicit reasoning is challengeable. therefore, instead of explicit reasoning, one alternate in existing kg-based crs @xcite is to aggregate neighborhood of interest entities in context, which can implicitly gathering high-order relations by multiturn aggregation. another alternate strategy @xcite is to explicitly grow a high-order tree in kg starting from interest entities to cover the potential preference, without aiming to the best recommended items. both strategies can improve crs, but have not full appreciation of the advantage of tracking interest shift in kg. the second issue is that kgs are usually incomplete to track the path of interest shift. for example, in figure 1 , the user mentioned the two movies probably because he/she likes "nolan" and "di-caprio". multiple paths in kg help to locate the subspace of the user's interest and generate interpretive utterances in line with people's dialogue behavior, e.g., "...inception with christopher nolan and leonardo dicaprio...". however, kgs can not record all relations of interest entities involved in real-world diverse dialogues @xcite @xcite . therefore, it is often difficult to achieve a complete reasoning path of interest shift within limited number of hops. to address these issues, we improve conversational recommender systems with flexible fragments reasoning on kgs (crfr). crfr uses an explicit multi-hop reasoning method to model the user's interest shift in a conversation with respect to an item-oriented kg (dbpedia @xcite ) and a concept-oriented kg (conceptnet @xcite ). we formalize the interest shift as a markov decision process and propose an explicit policy-guided reasoning model based on reinforcement learning. due to the possible absence of complete reasoning path in limited hop, instead of finding single best reasoning path, the learned policy flexibly obtains a set of high-order optimal path fragments. the obtained path fragments point to the destination of the user's interest shift and are likely to be contained in complete interest shift paths. to make recommendation with multiple reasoning path fragments, we further design a fragmentsaware recommendation module to fuse fragments information to learn the final representation of user preference. subsequently, in a reasoning information enhanced dialog module, the user preference representation is integrated into the response at token level. in this way, both the entities and words in reasoning path fragments can be flexibly involved and thus enhance the informativeness and interpretability of generated response. our contributions are summarized as follows: ## related work conversational recommender systems (crs) can be divided into two types. one is agent-driven crs known as "system ask, user respond" @xcite @xcite . this strategy obtains user preference by asking for predefined attributes @xcite and makes response by the user's feedback @xcite @xcite . graph reasoning @xcite and comment information @xcite ) can make the recommendation process of agent-driven crs interpretable. although agent-driven crs is popular in specific field, predefined template limits its generalization ability. the other is user-oriented crs known as "user talk, system understand" @xcite @xcite . due to the insufficient context, user-oriented crs often need the help of external knowledge, e.g., knowledge graphs (kgs). closely related to our work, @xcite integrates kgs to connect recommender and response generation. @xcite further uses item-level and word-level kgs to align two different semantic spaces. however, these two approaches only implicitly aggregating the neighbor information with gcn-based methods, instead of conducting explicit reasoning in kgs to track the deep user preference shift. an ideal explicit kg reasoning for crs is supposed to accurately describe the shift process of user interests using a single complete path. how-ever, it is usually challengeable due to the incompleteness of the kgs, the large search space and the diversity of expressing user interests. different from the approaches looking for single path @xcite , the reasoning tree @xcite executes relatively undirected reasoning and lacks concept relations in the reasoning. ## preliminaries in general, a knowledge graph g with entity set e and relation set r is defined as @xmath0, r, e ) | e, @xmath1, @xmath2, where each triplet (e, r, e ) represents a relation r from entity e to entity e . in this paper, we denote the item-oriented knowledge graph dbpedia @xcite as g d and the concept-oriented knowledge graph con-ceptnet @xcite as g c . correspondingly, their entity sets are e d and e c respectively. a multi-hop complete reasoning path p e 0 ,et , which connects a starting entity e 0 and a target entity e t : in this work, given an n-turn conversation history h, which contains the utterances of each turn, crfr takes conversation history h with the user u, graphs g d and g c as input, utilizes context information to perform flexible policy reasoning on g d and g c to obtain high-order path fragments set p that point to the destination of the user's interest shift, and further uses p as a guide to output the response containing the recommended item and the explanation of the recommendation. ## methodology in this section, we introduce our proposed crfr model. the overall framework is shown in figure 2. crfr executes flexible multi-hop policy reasoning on kgs g d and g c to obtain high-order path fragments that point to the destination of the user's interest shift on the two kgs respectively. path fragments from g d help the fragments-aware recommendation module to obtain a more accurate user preference representation. further, the reasoning information enhanced dialog module integrates the user's preference representation at the token level to flexibly select two types of path fragment information to guide response generation. ## conclusion we propose crfr, which significantly improves the agent response in conversational recommendation by exhibiting items having more clear higherorder relations with users' contextual intention and containing more persuasive explanation. as the essential advantage of this approach, explicit reasoning of high-order fragments in two heterogeneous knowledge graphs is performed by a reinforcement learning model. high-order path fragments obtained by explicit reasoning on item-oriented kg help the model to better track the user preference shift in conversation. the same reasoning on concept-oriented kg further improves the interpretability of response with informative concept words. heterogeneous fragments are personalized encoded to finally enhance the response generation. extensive experiments demonstrated that crfr is superior to the sota baselines on recommendation, explanation and language quality. in future, we will explore to make better use of path information to further improve the interpretability of responses.