Domain-specific Attention with Distributional Signatures for Multi-Domain End-to-end Task-Oriented Dialogue (2023)

Abstract: The end-to-end task-oriented dialogue system has achieved great success in recent years. Most of these dialogue systems need to accommodate multi-domain dialogue in real-world scenarios. However, due to the high cost of dialogue data annotation and the scarcity of labeled dialogue data, existing methods are difficult to extend to new domains. Therefore, it is important to use limited data to construct multi-domain dialogue systems. To solve this problem, we propose a novel domain attention module. It use the distributional signatures to construct a multi-domain dialogue system effectively with limited data, which has strong extensibility. We also define a adjacent n-gram pattern to explore potential patterns for dialogue entities. Experimental results show that our approach outperforms the baseline models on most metrics. In the few-shot scenario, we show our method get a great improvement compared with previous methods while keeping smaller model scale.

## introduction task-oriented dialogue systems (tod) aim to assist users in achieving specific goals, such as hotel reservations or weather inquiries, through limited dialogue turns. in contrast with chitchat systems, task-oriented dialogues generate responses based on a specific domain knowledge base (kb). traditional pipeline methods @xcite suffer from error propagation and huge cost for intermediate annotations such as dialogue states and actions. recently, end-to-end methods @xcite @xcite @xcite have achieved great success by taking the sequence-to-sequence (seq2seq) model to generate system responses directly with dialogue history and the specific domain knowledge base. these approaches have the advantages that the dialogue ## methodology as shown in figure 2(a) , given dialogue of domain d (d 2 d, d is the set of all domains) between user and system, our model takes the tokens @xmath0, x 2 , ..., x t ) from dialogue history and the corresponding multi-domain distributional signatures s @xmath1,d , s 2,d , ..., s t,d ) into semantic and signature block respectively. then we use the context vector obtained by two blocks to initialize the knowledge module. finally, the decoder generates final responses sequentially with the kb read-out vector and context hidden state. ## related work existing end-to-end approaches to modeling multidomain datasets can be divided into three categories. the first strand of work trains the model on the mixed data directly. @xcite first adopts end-to-end memory network @xcite to encode kb items and dialogue contexts. @xcite proposes a global-tolocal pointer mechanism to improve the accuracy of querying kb based on the memory network. our model retains the main framework of @xcite . @xcite uses a flow operation to strengthen the connection between the dialogue history and the knowledge base. @xcite also propose a pairwise similarity-based kb distillation to achieve the same purpose as @xcite . @xcite combines request-aware with kb-aware to better capture the latest request of users. @xcite models task-oriented dialogues as a text-to-text task and fine-tunes the t5 model @xcite on the mixed dataset. these works treat data from different domains in the same way, which ignores domain-specific knowledge. the second strand of work trains separate models for each domain. @xcite use the dialog state representation of some domain to query the knowledge base. @xcite restricts the query result from a single kb record. they both only focus on domain-specific knowledge and lack general knowledge. the third strand of work @xcite proposes a dynamic fusion network to handle multi-domain dialog, which needs multiple encoder-decoder for each domain and lacks flexibility. the distributional signature of text data contains rich semantic and structural knowledge. @xcite uses the distributional signature to generate general and class-specific attention and improve text classification performance. in our work, we leverage the dialogue data signature of different domains instead of classes. following @xcite , we employ a bi-lstm @xcite to bridge the gap caused by statistic noise. in addition, we take inspiration from @xcite , which proposes an effective pattern taxonomy model. we design adjacent n-gram patterns to discover entities better in the dialogue context. to our best knowledge, we are the first to use distributional signatures to model multi-domain task-oriented dialog. ## conclusion in this work, we propose a domain attention module with distributional signatures of dialogue corpus to capture domain-specific knowledge. we combine the features of different domains in an extensible way, and a domain loss is used to instruct our model to learn better from signatures. in addition, we define a adjacent n-gram pattern to mine the kb entities in the dialogue context. we also adopt attention with a coverage mechanism to improve the quality of generated responses. extensive experiments have demonstrated the effectiveness of our method.