Intention Reasoning Network for Multi-Domain End-to-end Task-Oriented Dialogue (2021)

Abstract: Recent years has witnessed the remarkable success in end-to-end task-oriented dialog system, especially when incorporating external knowledge information. However, the quality of most existing models’ generated response is still limited, mainly due to their lack of fine-grained reasoning on deterministic knowledge (w.r.t. conceptual tokens), which makes them difficult to capture the concept shifts and identify user’s real intention in cross-task scenarios. To address these issues, we propose a novel intention mechanism to better model deterministic entity knowledge. Based on such a mechanism, we further propose an intention reasoning network (IR-Net), which consists of joint and multi-hop reasoning, to obtain intention-aware representations of conceptual tokens that can be used to capture the concept shifts involved in task-oriented conversations, so as to effectively identify user’s intention and generate more accurate responses. Experimental results verify the effectiveness of IR-Net, showing that it achieves the state-of-the-art performance on two representative multi-domain dialog datasets.

## introduction task-oriented dialogue systems are designed to help users to achieve specific goals such as schedule arrangement or weather inquiry via natural language. compared with traditional pipeline dialogue systems @xcite that include multiple modules each requiring a huge amount of human effort to design, end-to-end approaches @xcite @xcite @xcite @xcite @xcite that can directly output system responses with plain text as input have recently gained much attention. in recent years, sequence to sequence (seq2seq) models have dominated the study of end-to-end task-oriented dialog systems, and many memory augmented seq2seq models have been proposed @xcite @xcite @xcite @xcite , which exploit both dialog history and domain-specific knowledge base (kb) to incorporate kb information and perform knowledge-based reasoning for better performance. though achieving remarkable progress, existing memory augmented seq2seq models still suffer from the following two limitations. first, prior models rely heavily on the soft attention mechanism @xcite to generate responses by adopting a weighted sum over the embeddings of memory triples (from both dialog history and external kb) as the output representation. since the representation acquired in this way is scattered by the context, it is difficult to model deterministic knowledge w.r.t. specific conceptual tokens. take the dialog in figure 1 as an example, when answering the user's query on today's (monday's) weather, the response generated by existing seq2seq models may be ambiguous or even incorrect due to the impact of contextual triples such as (tuesday, weather, sunny) and (wednesday,weather, cloudy). second, the soft attention mechanism is inherently not suitable for performing fine-grained (tokenlevel) multi-hop reasoning, which makes it hard to capture user's real intention to generate accurate responses, especially in complex cross-task scenarios where concept shifts @xcite ) may occur. for example, in figure 1 , when the system is asked "please give me the specific address for the dinner", it is expected to explore the pivot "john's_home" that connects the start token "diner" (in schedule domain) with the target token "550_alester_ave" (in navigate domain), and finally return the answer "550_alester_ave". existing attention-based models generally fail to perform such a token-level multi-hop reasoning, which hampers them from obtaining accurate responses. to address the aforementioned limitations, we propose a novel intention reasoning network (ir-net), which is a memory-augmented seq2seq model equipped with an intention reasoning module that is responsible for obtaining an intentionaware representation, with the goal of generating more accurate responses. specifically, to address the first limitation, we propose a novel intention mechanism (sec. 2.3.1), which directly incorporates the tail-token of a knowledge triple by comparing the similarity between the query vector and the triple's head-token to model deterministic knowledge. based on the intention mechanism, we further address the second limitation by proposing an intention reasoning module that consists of token-level joint reasoning and multi-hop reasoning (sec. 2.3.2), which are responsible for capturing specific target information from breadth and depth respectively to generate intention-aware representations, so as to improve the integrality and accuracy of the generated responses. we conduct experiments on two publicly available multi-domain datasets, namely smd @xcite and multi-woz 2.1 @xcite . the experimental results show that ir-net consistently outperforms the current stateof-the-art models in both automatic and human evaluation. to our best knowledge, we are the first to effectively explore fine-grained token-level in-tention reasoning in multi-domain end-to-end taskoriented dialog. ## model description our proposed model is based on a seq2seq dialog generation model (sec. 2.1), which encodes dialogue history x and knowledge base b and ultimately obtains a response sequence y . an external memory module @xmath0; b] is set up for knowledge query (sec. 2.2). moreover, to capture potential concept shift and user's intention to generate a fluid and intention-aware response, an intention reasoning module based on a novel intention mechanism is proposed (sec. 2.3). the workflow of our proposed model is depicted in figure 2 . ## related work sequence to sequence approaches, which use an encoder-decoder structure to capture the contextual dialog semantics and generate responses directly, have recently gained much attention in task-oriented dialogue systems @xcite . these models have effective language modeling ability, but cannot work well in kb retrieval, even with sophisticated attention-based mechanism. to alleviate this problem, copy augmented seq2seq models @xcite have been adopted, but still suffer from the challenge of performing reasoning over kb triples. to address this problem, memory augmented seq2seq models, such as end-to-end memory network @xcite and dqmn @xcite , have been proposed and shown promising results. later, mem2seq @xcite and glmp @xcite further augmented memory based methods by incorporating the copy mechanism @xcite , which enables ## conclusion in this paper, we propose a novel intention mechanism to directly incorporate the tail-token information of a knowledge triple to better model deterministic knowledge for multi-domain task-oriented dialog. moreover, based on the intention mechanism, we further propose an intention reasoning module that consists of token-level joint reasoning and multi-hop reasoning to obtain an intention-aware representation, aiming at improving the integrality and accuracy of the generated response. experiments on two publicly available multi-domain datasets demonstrate the effectiveness and superior performance of our model in both automatic and human evaluation.