Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020)

Abstract: Task-oriented dialogue systems are either modularized with separate dialogue state tracking (DST) and management steps or end-to-end trainable. In either case, the knowledge base (KB) plays an essential role in fulfilling user requests. Modularized systems rely on DST to interact with the KB, which is expensive in terms of annotation and inference time. End-to-end systems, instead, use the KB directly as input, but they cannot scale when the KB is larger than a few hundred entries. In this paper, we propose a method to embed the KB, of any size, directly into the model parameters. The resulting model does not require any DST or template responses, nor the KB as input, and it can dynamically update its KB via fine-tuning. We evaluate our solution in five task-oriented dialogue datasets with small, medium, and large KB size. Our experiments show that end-to-end models can effectively embed knowledge bases in their parameters and achieve competitive performance in all evaluated datasets.

## introduction task-oriented dialogue systems are designed to help users achieve predefined goals, such as booking restaurants or movie recommendations via natural language interactions. these systems are deeply connected with external knowledge bases (kbs) since the system responses are guided by the output from the kb and the dialogue history. the current state-of-the-arts @xcite @xcite @xcite are end-to-end pipelined systems that rely on dialogue state tracking (dst) and speech act (s-act) annotations. aside from the annotation cost, which is knowingly high @xcite figure 1 : during training, the ke dialogues are generated by fulfilling the template with the user goal query results, and they are used to embed the kb into the model parameter θ. at testing time, the model does not use any external knowledge to generate the correct responses. on the other end of the spectrum, there are endto-end trainable models that use both the kb and the dialogue history as input, and they directly generate system responses. most of the implementations use either the gold kb as input @xcite @xcite @xcite or an intermediate api call to retrieve part of the kb (api+kb) @xcite @xcite @xcite . these systems re-quire at least the dst annotation for generating the api calls or to select the gold kb. moreover, even with the most advanced transformer architecture @xcite @xcite , end-to-end models struggle when the input becomes too large @xcite . for example, in mwoz @xcite , there are 22k entities just for one of the domains. interested readers can refer to appendix c for an overview of different task-oriented methodologies. on the other hand, @xcite discovered a simple yet effective way to query factual knowledge from bert @xcite . later on, @xcite fine-tuned a pre-trained language model, t5 @xcite , on just question-answers pairs, without letting the model access any external context or knowledge. these results suggest that the actual knowledge is stored in the model parameters. however, in task-oriented dialogue systems, kb entities do not appear in news articles or wikipedia, e.g., hotel addresses or postcodes, and thus the aforementioned methods cannot be straightforwardly applied, especially when the kb dynamically changes (e.g., weather information). in this paper, we propose a method to store the kb directly into the model parameters using a novel knowledge embedded (ke) approach. the resulting model does not use any dst or template responses, nor a kb as input at the inference time, and it can be used in dynamically changing kbs via fine-tuning. the ke approach consists of a newly defined user goal query that generates equivalents ke dialogues from the kb (i.e., table or graph) using minimal annotation effort. figure 1 shows a high level overview of our approach. to verify the effectiveness of our proposed methodology, we extensively experiment, using both automatic and human metrics, in five task-oriented datasets with small, medium, and large kbs. our experiments show that end-to-end models can effectively embed knowledge bases in their parameters and achieve competitive performance in all five datasets. ## methodology in this section, we formalize the knowledge embedded (ke) strategy and the learning algorithm. in section 2.1, we provide several preliminary definitions used thought out the paper. in section 2.2, we extend the user goal definition from @xcite to cover a broad concept that we define as user goal query. then, in section 2.3, we describe two functions, ke-delex and ke-relex, used for generating templates and ke dialogues, respectively. finally, in section 2.4, we describe the causal language model transformer @xcite used for modeling the dialogue responses. ## experiments in all experiments, if not specifically mentioned, we use the pre-trained gpt2 (small) @xcite as causal language model @xcite . when the dataset has a sufficiently small kb (i.e., less than 1024 tokens), we also fine-tune gpt2 using the kb as input. in appendix d, we report details about hyperparameters and the implementation details. in appendix e, we report the data splitting for each dataset. ## analysis and discussions templates vs. performance in all experiments, we show that given the generated ke dialogues, the model learns to embed the kb into its parameters. however, the user goal query still requires human annotations; thus, we want to analyze the effect of using increasingly less templates in ke. for instance, in figure 2 , we report the number of templates used for fine-tuning versus the bleu score and the entity f1-score in the smd dataset. in general, we observe that more templates increase significantly both the f1 and bleu score. especially, we observe that blue score linearly increase with the number of templates used in training, suggesting that a more diverse and fluent generation can be achieved using more templates. in appendix f, we report the same analysis in each datasets, where we observe a similar trend. limitation & dynamic kb throughout our experiments, we identify two major limitations: noisy ke dialogues generation and fine-tuning time for dynamic kbs. although the proposed ke results successfully embed the kb into the model parameters, the generated ke dialogues are sometimes noisy. for example, the ke-delex function converts, "i want to find an expensive restaurant..." into a template "i want to find an [price-range] restaurant...". then the ke-relex can generate "i want to find a cheap restaurant...", which has a clear grammar mistake. this type of error does not happen often, and we notice that gpt2 is robust to this kind of noisy input. in future work, we propose to improve the robustness and fluency of our model using different regularization losses. moreover, in the case of dynamic kbs a substantial fine-tuning cost is required for updating the kb. figure 2 shows the average time-per-epoch spent for fine-tuning in smd. in future work, we propose to study both a meta-learning @xcite strategy for quick fine-tuning and continual learning approach for updating the kb while retaining the previous existing knowledge. recently, several task-oriented dialogue models are introduced to tackle the resource scarcity challenges in target domains @xcite @xcite and target languages @xcite @xcite , and large pre-trained language models are shown to possess the capability to quickly adapt to taskoriented dialogue tasks by using only a few data samples @xcite @xcite . data augmentation is a widely used technique to improve both robustness and performance @xcite . task-oriented dialogue systems have been explored to improve dst @xcite @xcite , natural language understanding (nlu) @xcite , intent classification @xcite and hybrid end-to-end systems @xcite . these data augmentation methods aim to improve the final performance of the given task, e.g., zeroshot performance, template response, etc., where instead, our proposed approach aims to store the kb into the model parameters. agenda-based user simulation builds an interactive system that models the user turns @xcite ) rather than the system. user simulators are designed to cover all possible user queries while keeping a diverse and fluent user interaction. this enables models to learn a better dialogue policy via interaction @xcite @xcite , and it is especially useful in scenarios in where few or no data is available @xcite @xcite @xcite . in our work, instead, we use all the possible user goal queries to generate dialogues directly, instead of creating a reinforcement learning loop to train the model. language models as knowledge bases has been used for encoding common sense knowledge into transformers @xcite @xcite @xcite . @xcite improved story generation by training a language model with knowledge triples converted into sentences using predefined templates @xcite . differently, we extract templates from real data, and we aim to store the kb into the models parameters to be able to extract knowledge directly, instead of improving common sense generation. moreover, several studies tried to extract @xcite @xcite or use @xcite ) large pre-trained models, e.g. bert @xcite , as knowledge bases. ## conclusion in this paper, we propose to learn the kb directly into the model parameters using a novel knowledge embedded approach, that is fundamentally different from giving the kb as input or using the dst for querying the kb. we demonstrate that our approach is scalable to different kb sizes and it can be used with dynamically changing kbs via finetuning. automatic and human evaluations confirm that models with embedded kbs achieve competitive performance in all evaluated datasets. finally we show, for the first time, that end-to-end models can perform as well as pipelined modularized systems @xcite in the mwoz single domain dataset.