Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems (2022)

Abstract: Users interacting with voice assistants today need to phrase their requests in a very specific manner to elicit an appropriate response. This limits the user experience, and is partly due to the lack of reasoning capabilities of dialogue platforms and the hand-crafted rules that require extensive labor. One possible solution to improve user experience and relieve the manual efforts of designers is to build an end-to-end dialogue system that can do reasoning itself while perceiving user’s utterances. In this work, we propose a novel method to incorporate the knowledge reasoning capability into dialog systems in a more scalable and generalizable manner. Our proposed method allows a single transformer model to directly walk on a large-scale knowledge graph to generate responses. To the best of our knowledge, this is the first work to have transformer models generate responses by reasoning over differentiable knowledge graphs. We investigate the reasoning abilities of the proposed method on both task-oriented and domain-specific chit-chat dialogues. Empirical results show that this method can effectively and efficiently incorporate a knowledge graph into a dialogue system with fully-interpretable reasoning paths.

## introduction nowadays, dialogue systems are ubiquitous in customer service and voice-based assistants. one of the main uses of this technology is supporting humans in accomplishing tasks that might require accessing and navigating large knowledge bases (e.g., movies search). a dialogue system architecture is typically composed of a natural language understanding (nlu) module, a dialogue management (dm) module, and a natural language generation (nlg) module @xcite . first, the nlu component extracts a meaning representation from the user utterance based on which the dm generates the next system action by reasoning over the meaning representation and communicating with external applications if necessary. for example, the dm may retrieve information from external knowledge graphs (kg) to answer the user's query based on the dialogue history. this process requires the dm to convert the output of nlu to a query to be issued to the backend. given the difficulty of this step, which is often domain-dependent, the dm component might require the design of hand-crafted rules. however, such rules are usually not scalable to different applications. they could require considerable effort to cover all possible cases/dialogue flows, leading to expensive costs to design new applications. moreover, in several cases, users interacting with such assistants are forced to formulate specific queries in order to accomplish their objective, which might break user engagement. to alleviate the problem of having to design expensive hand-crafted rules and breaking user experience, recent works have explored the possibility of building end-to-end dialogue systems @xcite and all-in-one response generation models @xcite . among them, since graph is one of the main structure to store knowledge, recent research @xcite @xcite @xcite has proposed methods to generate natural language responses according to both the dialogue history and external knowledge graph. despite these innovative and inspiring methods, there are some shortcomings. for instance, these methods are either not fully-interpretable or limited to small-scale knowledge graphs. in this paper, we propose a novel dialogue differentiable knowledge graph model @xcite . the diffkg is a single transformer model that directly (1) generates a sequence of relations to perform multi-hop reasoning on a reified kg representation proposed by @xcite , and then (2) generates responses using the retrieved entities. to the best of our knowledge, this is the first dialogue model that can directly walk on a large-scale kg with flexibility and interpretability. diffkg allows having flexible entity values in the kg and handling novel entity values with an arbitrarily defined number of tokens. the reasoning path of diffkg consists of the predicted relations, thus allowing for transparency. we run extensive experiments to test diffkg performance on kg-grounded dialogues. we select stanford multi-domain dialogues (smd) @xcite and propose a new dataset, smd-reasoning, to simulate scenarios requiring multiple reasoning types and select the opendialkg @xcite to simulate scenarios requiring largescale kg reasoning without preprocessing. we then compare diffkg with state-of-the-art models on smd and opendialkg and an additional baseline that flattens kgs into a textual form from which transformers can learn. empirically, our experiments show that diffkg can effectively be trained on large-scale kgs and demonstrate its robustness with modified triplets in a kg. from the perspective of computation, diffkg leads to relatively low extra time and memory usage compared to transformer models not using any kg information. in summary, our contributions are: 1) we propose diffkg, a novel method that can effectively and flexibly incorporate large-scale kg; 2) we demonstrate that diffkg is a modelagnostic method and can be applied to different model architectures; 3) we show that diffkg is an interpretable method with low add-on latency at inference time. our code and processed datasets are released in https://github. com/pascalson/diffkg-dialog . ## related work recent years have seen a surge of new methods proposing end-to-end models that try to both understand natural language input text and search information. two of the widely explored tasks are question-answering (qa) and dialogue generation. qa. multiple qa methods @xcite @xcite @xcite have been proposed to tackle tasks that go beyond what is explicitly stated in the linguistic context @xcite . for example, the benchmarks @xcite @xcite are particu-larly useful for the model to extract information from external knowledge bases to answer questions. nonetheless, these studies mostly take the retrieved information from kg as the answer to a single question, while in dialogue we have to formulate an informative response to multi-turn dialogue history. ## problem statement we focus on understanding the ability of language models in performing reasoning during a conversation. we consider two tasks that are usually required in dialogue scenarios and call them semantic form and natural language (nl) form in first, given a dialogue history and a user's query, the task of semantic form is to predict the next system action, corresponding to the output of the dm module, based on the available knowledge. in this case, we assume the expected output is the essential knowledge for an nlg module. we argue that this task could help better evaluate if the response is correct or not and which type of reasoning can be more successfully handled. second, given a dialogue history and a user's query, the task of the nl form could be to directly output the response given 1 an example of the triples in g is a triple e h @xmath0, r @xmath1, and e t @xmath2. that is, "gas station is the type of chevron" to this system. by the system. this setting with annotated reasoning path can shed light on understanding if the model can learn to support chit-chat and reasoning at the same time. moreover, we aim to understand models' reasoning capability both in the form of logical reasoning and over the provided knowledge. as illustrated in we formulate the task that we focus on as follows: given the dialogue history x and currently accessible kg g, can we extend a transformer model to predict a correct response y in either semantic or nl form? as illustrated in ## proposed approach figure 1 illustrates our proposed architecture which contains four main parts: a dialogue history encoder, a differentiable kg reasoning module, a learnable logical operations module, and a response decoder (the transformer model). note that we experiment with two types of transformers: a causal language model gpt2 @xcite and an encoder-decoder model t5 @xcite . for gpt2, we reuse the same encoder that is used at the beginning of the process, i.e., f enc in figure 1 , as the final transformer that generates the response token by token. for t5, we reuse the same encoder as the encoder of the final transformer with a separate decoder that generates the response. therefore, this method contains a single transformer model. in following sections we present each module in detail. ## conclusion and future work for a dialogue system, an effective reasoning method over structured databases is important. in this work, we proposed diffkg, an end-to-end model-agnostic method that does symbolic reasoning on any scale of kgs to enhance response generation. experiments demonstrated that using dif-fkg, models are able to generate responses with interpretable kg reasoning paths at a modest extra cost. this work can be extended in various ways. while we solely consider efficient large-scale kg reasoning in dialogue generation, future work can incorporate domain fusion methods to consider the generalizability over domains or simultaneously use relation information. moreover, since dif-fkg is a simple large-scale structured knowledgeempowered transformer with flexible entity values, future work can extend it to dialogue generation that needs to do table and text mixed reasoning and that needs to do both kg reasoning and other goals such as personalized dialogues, storytelling, etc. latency is about twice at inference time. the difference in inference latency is even larger with gpt2 as the backbone model. the reason is that the computational cost of a causal language model such as gpt2 largely depends on the input sequence length, which is one of the main issues of flatinfo.