Q - TOD : A Query-driven Task-oriented Dialogue System (2022)

Abstract: Existing pipelined task-oriented dialogue systems usually have difficulties adapting to unseen domains, whereas end-to-end systems are plagued by large-scale knowledge bases in practice. In this paper, we introduce a novel query-driven task-oriented dialogue system, namely Q-TOD. The essential information from the dialogue context is extracted into a query, which is further employed to retrieve relevant knowledge records for response generation. Firstly, as the query is in the form of natural language and not confined to the schema of the knowledge base, the issue of domain adaption is alleviated remarkably in Q-TOD. Secondly, as the query enables the decoupling of knowledge retrieval from the generation, Q-TOD gets rid of the issue of knowledge base scalability. To evaluate the effectiveness of the proposed Q-TOD, we collect query annotations for three publicly available task-oriented dialogue datasets. Comprehensive experiments verify that Q-TOD outperforms strong baselines and establishes a new state-of-the-art performance on these datasets.

## introduction task-oriented dialogue systems are designed to help users achieve their goals, such as restaurant reservation, calendar scheduling, and movie recommendation. typically, these systems need to rely on external knowledge bases to retrieve necessary information for response generation @xcite @xcite . some end-to-end trainable approaches try to encode the knowledge base into a memory module and attend relevant knowledge records for response generation @xcite @xcite . since these end-to-end approaches need to continually refresh the memory module, a large-scale knowledge base will lead to a heavy computation burden and difficult joint optimization. recently, some works leverage the power of * equal contribution. in these circumstances, the practical deployed systems tend to employ pipelined designs and strip out the component of knowledge retrieval @xcite @xcite . these pipelined systems usually consist of natural language understanding, dialogue state tracking, dialogue policy learning, and system response generation. in order to retrieve relevant information from the external knowledge base, these approaches need to pre-define the schema of dialogue states according to the knowledge base. due to this kind of strong association, these pipelined systems have difficulties adapting to unseen domains, i.e., weak ability on domain adaptation. to tackle these issues, in this paper, we introduce a novel query-driven task-oriented dialogue (q-tod) system. the overview of q-tod is shown in figure 1 , where three sequential modules are included: 1) the query generator extracts the essential information from the dialogue context into a concise query in an unstructured format of the natural language; 2) the generated query is then utilized to retrieve relevant knowledge records with an off-the-shelf knowledge retriever; 3) the response generator produces a system response based on the retrieved knowledge records and the dialogue context. the advantages brought by the query-driven taskoriented dialogue system are two-fold. firstly, the query is in the unstructured format of natural language, which is not confined to the knowledge base and is able to mitigate the issue of domain adaptation. secondly, with the incorporation of the query, q-tod decouples the knowledge retrieval from the response generation, getting rid of the issue of the knowledge base scalability. to explore the effectiveness of the query-driven systems, we collect query annotations for three public task-oriented dialogue datasets: smd @xcite , cam-rest @xcite , and multiwoz-2.1 @xcite . experimental results demonstrate that q-tod achieves superior performance as compared to other state-of-the-art approaches. particularly, in the few-shot settings, q-tod achieves a comparable performance with the previous stateof-the-art using only 5% of the training data. our collected data, code, and models have been released at github foot_0 , hoping to facilitate further research in task-oriented dialogue systems. ## methodology the goal of this paper is to explore a novel and effective framework for task-oriented dialogue systems. as shown in figure 1 , the proposed q-tod consists of three subsequent modules: query generator, knowledge retriever, and response generator. the detailed design of these three modules will be discussed in the following. ## discussion for further analysis of q-tod, we will have discussions on the following aspects: knowledge base scalability, effect of query, performance of precise knowledge, domain adaptation, and case study. in this section, unless specified, experiments are carried out with t5-large. ## related work in task-oriented dialogue systems, there is a trend to develop end-to-end trainable approaches to incorporate the external knowledge base for response generation @xcite @xcite @xcite @xcite . some works encode the entire knowledge base into a memory module and learn to attend to the relevant knowledge entities for response generation @xcite @xcite @xcite . recently, with the advances in the pretrained language models, some works take the entire linearized knowledge base as the transformer input and generate the final system response directly @xcite . additionally, @xcite proposes to store the knowledge base in the language model parameters implicitly through dialogue augmentation. however, considering the knowledge base usually contains thousands of records in practice, these endto-end trainable approaches face the critical challenges of knowledge base scalability. meanwhile, there are many pipelined taskoriented dialogue systems, which strip out the component of knowledge retrieval @xcite @xcite @xcite @xcite . they usually decompose a task-oriented dialogue system into several pipelined modules: natural language understanding, dialogue state tracking, dialogue policy learning, and system response generation @xcite . some recent works formulate all pipelined modules as a cascaded generation task using pre-trained language model @xcite @xcite @xcite . to further boost the performance, some models attempt to introduce the pre-training strategy into task-oriented dialogue systems @xcite @xcite . however, these pipelined systems rely on the predefined schema to retrieve knowledge from an external knowledge base, leading to difficulties in adapting to unseen domains. in other research fields, there are also some works using a query to store essential information or retrieve relevant knowledge. in knowledgegrounded open-domain dialogue, to incorporate real-time external information, recent works learn to generate a search query based on the dialogue context for internet searching @xcite @xcite . in opendomain conversational question answering, to handle the reference problem and optimize retrieval performance, recent systems introduce a question rewriting task to convert a context-dependent question into a self-contained question @xcite @xcite . in context-dependent text-to-sql task, some works learn to reformulate multi-turn conversational questions into a self-contained question, and then a context-independent text-to-sql parser follows @xcite . to the best of our knowledge, q-tod is the first framework that introduces the query into task-oriented dialogue systems. ## conclusion in this paper, we propose a novel query-driven taskoriented dialogue system, namely q-tod. q-tod consists of three modules: the query generator extracts the essential and up-to-date information from the dialogue context into a concise query, the offthe-shelf knowledge retriever utilizes the generated query to retrieve relevant knowledge records, and the response generator produces the final system response using the retrieved knowledge records and the dialogue context. comprehensive experiments show that q-tod consistently outperforms all baselines on three active task-oriented dialogue datasets, achieving a new state-of-the-art performance.