Global-to-local Memory Pointer Networks for Task-Oriented Dialogue

1 INTRODUCTION . Task-oriented dialogue systems aim to achieve specific user goals such as restaurant reservation or navigation inquiry within a limited dialogue turns via natural language . Traditional pipeline solutions are composed of natural language understanding , dialogue management and natural language generation ( Young et al. , 2013 ; Wen et al. , 2017 ) , where each module is designed separately and expensively . In order to reduce human effort and scale up between domains , end-to-end dialogue systems , which input plain text and directly output system responses , have shown promising results based on recurrent neural networks ( Zhao et al. , 2017 ; Lei et al. , 2018 ) and memory networks ( Sukhbaatar et al. , 2015 ) . These approaches have the advantages that the dialogue states are latent without hand-crafted labels and eliminate the needs to model the dependencies between modules and interpret knowledge bases ( KB ) manually . However , despite the improvement by modeling KB with memory network ( Bordes & Weston , 2017 ; Madotto et al. , 2018 ) , end-to-end systems usually suffer from effectively incorporating external KB into the system response generation . The main reason is that a large , dynamic KB is equal to a noisy input and hard to encode and decode , which makes the generation unstable . Different from chit-chat scenario , this problem is especially harmful in task-oriented one , since the information in KB is usually the expected entities in the response . For example , in Table 1 the driver will expect to get the correct address to the gas station other than a random place such as a hospital . Therefore , pointer networks ( Vinyals et al. , 2015 ) or copy mechanism ( Gu et al. , 2016 ) is crucial to successfully generate system responses because directly copying essential words from the input source to the output not only reduces the generation difficulty , but it is also more like a human behavior . For example , in Table 1 , when human want to reply others the Valero ’ s address , they will need to “ copy ” the information from the table to their response as well . Therefore , in the paper , we propose the global-to-local memory pointer ( GLMP ) networks , which is composed of a global memory encoder , a local memory decoder , and a shared external knowledge . Unlike existing approaches with copy ability ( Gulcehre et al. , 2016 ; Gu et al. , 2016 ; Eric & ∗All work was done while the first author was an intern at Salesforce Research . Manning , 2017 ; Madotto et al. , 2018 ) , which the only information passed to decoder is the encoder hidden states , our model shares the external knowledge and leverages the encoder and the external knowledge to learn a global memory pointer and global contextual representation . Global memory pointer modifies the external knowledge by softly filtering words that are not necessary for copying . Afterward , instead of generating system responses directly , the local memory decoder first uses a sketch RNN to obtain sketch responses without slot values but sketch tags , which can be considered as learning a latent dialogue management to generate dialogue action template . Then the decoder generates local memory pointers to copy words from external knowledge and instantiate sketch tags . We empirically show that GLMP can achieve superior performance using the combination of global and local memory pointers . In simulated out-of-vocabulary ( OOV ) tasks in the bAbI dialogue dataset ( Bordes & Weston , 2017 ) , GLMP achieves 92.0 % per-response accuracy and surpasses existing end-to-end approaches by 7.5 % in full dialogue . In the human-human dialogue dataset ( Eric et al. , 2017 ) , GLMP is able to surpass the previous state of the art on both automatic and human evaluation , which further confirms the effectiveness of our double pointers usage . 2 GLMP MODEL . Our model 1 is composed of three parts : global memory encoder , external knowledge , and local memory decoder , as shown in Figure 1 ( a ) . The dialogue history X = ( x1 , . . . , xn ) and the KB information B = ( b1 , . . . , bl ) are the input , and the system response Y = ( y1 , . . . , ym ) is the expected output , where n , l , m are the corresponding lengths . First , the global memory encoder uses a context RNN to encode dialogue history and writes its hidden states into the external knowledge . Then the last hidden state is used to read the external knowledge and generate the global memory pointer at the same time . On the other hand , during the decoding stage , the local memory decoder first generates sketch responses by a sketch RNN . Then the global memory pointer and the sketch RNN hidden state are passed to the external knowledge as a filter and a query . The local memory pointer returns from the external knowledge can copy text from the external knowledge to replace the sketch tags and obtain the final system response . 2.1 EXTERNAL KNOWLEDGE . Our external knowledge contains the global contextual representation that is shared with the encoder and the decoder . To incorporate external knowledge into a learning framework , end-to-end memory networks ( MN ) are used to store word-level information for both structural KB ( KB memory ) and temporal-dependent dialogue history ( dialogue memory ) , as shown in Figure 1 ( b ) . In addition , the MN is well-known for its multiple hop reasoning ability ( Sukhbaatar et al. , 2015 ) , which is appealing to strengthen copy mechanism . Global contextual representation . In the KB memory module , each element bi ∈ B is represented in the triplet format as ( Subject , Relation , Object ) structure , which is a common format used to represent KB nodes ( Miller et al. , 2016 ; Eric et al. , 2017 ) . For example , the KB in the Table 1 will be denoted as { ( Tom ’ s house , distance , 3 miles ) , ... , ( Starbucks , address , 792 Bedoin St ) } . On the other hand , the dialogue context X is stored in the dialogue memory module , where the speaker and temporal encoding are included as in Bordes & Weston ( 2017 ) like a triplet format . For instance , the first utterance from the driver in the Table 1 will be denoted as { ( $ user , turn1 , I ) , ( $ user , turn1 , need ) , ( $ user , turn1 , gas ) } . For the two memory modules , a bag-of-word representation is used as the memory embeddings . During the inference time , we copy the object word once a memory position is pointed to , for example , 3 miles will be copied if the triplet ( Toms house , distance , 3 miles ) is selected . We denote Object ( . ) function as getting the object word from a triplet . 1https : //github.com/salesforce Knowledge read and write . Our external knowledge is composed of a set of trainable embedding matrices C = ( C1 , . . . , CK+1 ) , where Ck ∈ R|V |×demb , K is the maximum memory hop in the MN , |V | is the vocabulary size and demb is the embedding dimension . We denote memory in the external knowledge as M = [ B ; X ] = ( m1 , . . . , mn+l ) , where mi is one of the triplet components mentioned . To read the memory , the external knowledge needs a initial query vector q1 . Moreover , it can loop over K hops and computes the attention weights at each hop k using pki = Softmax ( ( q k ) T cki ) , ( 1 ) where cki = B ( C k ( mi ) ) ∈ Rdemb is the embedding in ith memory position using the embedding matrix Ck , qk is the query vector for hop k , and B ( . ) is the bag-of-word function . Note that pk ∈ Rn+l is a soft memory attention that decides the memory relevance with respect to the query vector . Then , the model reads out the memory ok by the weighted sum over ck+1 and update the query vector qk+1 . Formally , ok = ∑ i pki c k+1 i , q k+1 = qk + ok. ( 2 ) 2.2 GLOBAL MEMORY ENCODER . In Figure 2 ( a ) , a context RNN is used to model the sequential dependency and encode the context X . Then the hidden states are written into the external knowledge as shown in Figure 1 ( b ) . Afterward , the last encoder hidden state serves as the query to read the external knowledge and get two outputs , the global memory pointer and the memory readout . Intuitively , since it is hard for MN architectures to model the dependencies between memories ( Wu et al. , 2018 ) , which is a serious drawback especially in conversational related tasks , writing the hidden states to the external knowledge can provide sequential and contextualized information . With meaningful representation , our pointers can correctly copy out words from external knowledge , and the common OOV challenge can be mitigated . In addition , using the encoded dialogue context as a query can encourage our external knowledge to read out memory information related to the hidden dialogue states or user intention . Moreover , the global memory pointer that learns a global memory distribution is passed to the decoder along with the encoded dialogue history and KB information . Context RNN . A bi-directional gated recurrent unit ( GRU ) ( Chung et al. , 2014 ) is used to encode dialogue history into the hidden states H = ( he1 , . . . , h e 1 ) , and the last hidden state h e n is used to query the external knowledge as the encoded dialogue history . In addition , the hidden states H are written into the dialogue memory module in the external knowledge by summing up the original memory representation with the corresponding hidden states . In formula , cki = c k i + h e mi if mi ∈ X and ∀k ∈ [ 1 , K + 1 ] , ( 3 ) Global memory pointer . Global memory pointer G = ( g1 , . . . , gn+l ) is a vector containing real values between 0 and 1 . Unlike conventional attention mechanism that all the weights sum to one , each element in G is an independent probability . We first query the external knowledge using hen until the last hop , and instead of applying the Softmax function as in ( 1 ) , we perform an inner product followed by the Sigmoid function . The memory distribution we obtained is the global memory pointer G , which is passed to the decoder . To further strengthen the global pointing ability , we add an auxiliary loss to train the global memory pointer as a multi-label classification task . We show in the ablation study that adding this additional supervision does improve the performance . Lastly , the memory readout qK+1 is used as the encoded KB information . In the auxiliary task , we define the label Glabel = ( gl1 , . . . , g l n+l ) by checking whether the object words in the memory exists in the expected system response Y . Then the global memory pointer is trained using binary cross-entropy loss Lossg between G and Glabel . In formula , gi = Sigmoid ( ( qK ) T cKi ) , g l i = { 1 if Object ( mi ) ∈ Y 0 otherwise , Lossg = − ∑n+l i=1 [ g l i × log gi + ( 1− gli ) × log ( 1− gi ) ] . ( 4 )

Summary: The paper presents a new model for reading and writing memory in the context of task-oriented dialogue. The model contains three main components: an encoder, a decoder, and an external KB. The external KB is in the format of an SVO triple store. The encoder encodes the dialogue history and, in doing so, writes its hidden states to memory and generates a "global memory pointer" as its last hidden state. The decoder takes as input the global memory pointer, the encoded dialogue state history, and the external KB and then generates a response using a two-step process in which it 1) generates a template response using tags to designate slots that need filling and 2) looks up the correct filler for each slot using the template+global memory pointer as a query. The authors evaluate the model on a simulated dialogue dataset (bAbI) and on a human-human dataset (Stanford Multi-domain Dialogue or SMD) as well as in a human eval. They show substantial improvements over existing models on SMD (the more interesting of the datasets) in terms of entity F1--i.e. the number of correctly-generated entities in the response. They also show improvement on bAbI specifically on cases involving OOVs. On the human evaluation, they show improvements in terms of both "appropriateness" and "human-likeness".