A ttn IO : K nowledge G raph E xploration with I n-and- O ut A ttention F low for K nowledge- G rounded D ialogue (2020)

Abstract: Retrieving the proper knowledge relevant to conversational context is an important challenge in dialogue systems, to engage users with more informative response. Several recent works propose to formulate this knowledge selection problem as a path traversal over an external knowledge graph (KG), but show only a limited utilization of KG structure, leaving rooms of improvement in performance. To this effect, we present AttnIO, a new dialog-conditioned path traversal model that makes a full use of rich structural information in KG based on two directions of attention flows. Through the attention flows, AttnIO is not only capable of exploring a broad range of multi-hop knowledge paths, but also learns to flexibly adjust the varying range of plausible nodes and edges to attend depending on the dialog context. Empirical evaluations present a marked performance improvement of AttnIO compared to all baselines in OpenDialKG dataset. Also, we find that our model can be trained to generate an adequate knowledge path even when the paths are not available and only the destination nodes are given as label, making it more applicable to real-world dialogue systems.

## introduction one of the milestone challenges in conversational ai is to engage users with a more informative and knowledgeable response, rather than merely outputting generic sentences. for instance, given a user's utterance saying "i'm a big fan of steven spielberg", it would be more engaging to respond "my favorite movie is his science fiction film a.i.", rather than "i like him too.". an external source of knowledge such as knowledge graph (kg) can * equal contribution. the above mentioned motivation gave rise to a conspicuous need for path retrieval model on kg, which can learn to traverse a path consisting of proper entities and relations to mention in the next response, given the dialog context. previous approaches to this knowledge selection problem rely on either an rl-based agent @xcite or a recurrent decoder @xcite , which greedily selects the most proper entity to traverse regarding its previous decision. despite their novelty, we find several rooms of improvement from the previous works, to move toward a more finegrained modeling of knowledge path retrieval for dialogue systems. first, a model could make use of the rich relational information residing at the neighborhood of each node on kg. typically, the number of entities in kg is large, while the numbers of each entity's usage in actual dialogues are small. thus leveraging the neighborhood information of each entity in knowledge graph could be crucial, to overcome the sparsity of entity usage and learn proper representation of entities and relations. also, we find that the range of knowledge paths plausible for a response to a given dialog may vary, depending on the dialog context and user intent. in response to a closed question such as "who directed the movie a.i.?", there could be only one or two knowledge paths valid as answer. on the contrary, in response to an open question such as "do you know steven spielberg?", there could be a variety of knowledge paths natural enough to carry on the conversation. therefore, a model should be able to choose the range of entities to attend, depending on the characteristics of a given dialog. lastly, one should note that it is practically hard to gather a large-scale dialog-kg path parallel corpus, fully annotated with all entities and relations comprising each path. retrieving the initial, and final entities of the kg path is relatively easier, as it only requires post-processing the {query, re-sponse} pairs in dialog. therefore, it would be more desirable if a model could be trained to traverse a proper knowledge path, only with the destination nodes provided as label. to this end, we propose attnio (attention inflow and out-flow), a novel kg path traversal model that overcomes all challenges stated above. aside to the conventional textual encoder which encodes dialog history and user utterance, attnio models the kg traversal mechanism into two subprocesses: incoming attention flow, and outgoing attention flow. inspired by attention flow @xcite , the two attention flows explore kg by propagating the attention value at each node to its reachable neighbor nodes, as shown in figure 1 . the attention propagation mechanism enables our model to start exploring kg from multiple entities (a.i., and the truman show), then find out an intermediate node drama relevant to both movies, and end the multi-hop reasoning by arriving at catch me if you can. such a complex interaction between entities cannot be modeled by a greedy decoder, limited to consider only an optimal node at each decoding step. in addition, our model provides better interpretation of its path reasoning process, by visualizing the attention distribution of nodes and edges at each step. lastly, we consider our model in a more challenging, but more realistic setting of path retrieval task, where no ground-truth path is available for supervision, but only the final desti-nation nodes are given. even in this setting, we find that attnio can be trained to infer a proper knowledge path for the input dialog. in summary, our contributions are as follows: (1) we suggest a novel path traversal model attnio, achieving state-of-the-art performance in dialogconditioned knowledge path retrieval task on the opendialkg dataset. (2) we demonstrate that at-tnio can be trained even in a challenging setting where only the destination nodes are given, and show through both qualitative and quantitative analysis that the quality of paths generated from this setting does not fall behind that of the all-path supervision setting. (3) through visualizing the attention distribution at each decoding step, we show that our model possesses better interpretability over the path reasoning process. ## related works recently, lots of research effort have been devoted to grounding dialogue systems on structuredknowledge embedded in knowledge graphs. these works can be broadly classified into two categories, depending on the range of exploration over candidate knowledge. the first line of works, namely breadth-centric approaches, tend to focus on augmenting dialog context with entity representations, by aggregating their shallow (i.e., 1-hop or 2-hop) neighborhood information from an external knowledge graph @xcite @xcite . @xcite suggest to encode an auxiliary knowledge vector by attentively reading all 1-hop relations of each initial entity that appears in user's utterance. @xcite extends the previous work's knowledge encoding scheme to 2-hop relations, encoding all initial entities and their 1-hop neighbors with two independent attention mechanisms. while these works are successful in contextualizing each entity with various relations in kg, they lack in retrieving small set of focused knowledge paths relevant to the dialog, or generalizing to multi-hop relations. we extend these approaches by suggesting a new framework that can be generalized to an arbitrary length of traversal, and dynamically updating entity features at each decoding step to facilitate multi-hop relational inference. on the other hand, the second line of works resort to depth-centric search over candidate knowledge paths. rather than augmenting entity representation with shallow but wide range of knowledge, they concentrate on traversing only a specific range of entities and relations directly usable for response generation. @xcite formulate the knowledge selection problem as partially observed markov decision process, employing a policy network to traverse kg. meanwhile, @xcite suggest a recurrent path decoder that relies on a hidden state vector to choose the next entity among reachable nodes. although these models are competent at inferring multi-hop relations, their discrete selection mechanism neglects rich relational information of nodes and edges they did not explicitly choose to traverse. to complement the weakness, attnio does not select an optimal node in advance; rather, it first propagates attention to all reachable entities, and then decode an optimal path from the output attention distribution. our work is also closely motivated from recent techniques suggested in the domain of knowledge graph completion tasks. to compensate for weak representation power of translative embedding @xcite and convolution-based embedding @xcite , several models have adopted graph neural networks (gnn), encoding structural information into entity embedding @xcite . other works perform traversal-based inference in nodeprediction tasks based on reinforcement learning @xcite , or attention propagation @xcite . we extend these previous works by adopting graph neural network and attention propagation for the dialog-conditioned path generation problem. 3 proposed method 3.1 overview we denote the external knowledge graph as g @xmath0, with nodes as entities and edges as relations between a pair of entities. we denote g v,n ⊆ g kg as a subgraph containing all nodes and edges reachable in less then or equal to n-hops, starting from vertex v. also, we define -→ n i as a set of incoming neighbor nodes of v i , i.e. nodes possessing edges toward v i , and ← -n i as a set of outgoing neighbor nodes of v i . figure 2 at each decoding step @xmath1, • • • , t , the incoming attention flow iteratively updates the kg entity features by attentively aggregating rich relational features from their incoming neighbor nodes. then, the outgoing attention flow propagates the attention value of each node to its outgoing neighbor nodes, yielding the node attention distribution a t i and edge attention distribution a t ij as step t's output. we show that each candidate entity path ## experiments and results dataset we evaluate our proposed method on opendialkg @xcite , a dialog -kg parallel corpus designed for knowledge path retrieval task. the dataset consists of 91k multi-turn conversations in form of either task-oriented (recommendation) dialog, or chit-chat on a given topic. each pair of utterances in the conversations is annotated with a kg path, where its initial entity is mentioned in the former turn, and its destination entity is mentioned in the latter turn. as the train/valid/test partitions of opendialkg are not publicly available, we create our own split by randomly partitioning into train (70%), valid (15%), and test set (15%). baselines we take 4 models suggested by @xcite as baselines. these models include di-alkg walker, a state-of-the-art model designed to traverse a dialogue conditioned knowledge path. other 3 models are seq2seq @xcite , extended encoder-decoder @xcite , tri-lstm @xcite all modified to fit the entity path retrieval task. ## conclusion in this work, we suggest attnio, a novel path traversal model that reasons over kg based on two directions of attention flows. the empirical evaluations on opendialkg dataset show the strength of at-tnio in knowledge retrieval compared to baselines. attnio can also be trained to generate proper paths even in a more affordable setting of target supervision. lastly, we show through case study that our model enjoys from transparent interpretation of path reasoning process, and is capable of intuitively modeling knowledge exploration depending on the dialog characteristics.