Year: 2019

Abstract:
End-to-end models have achieved considerable success in task-oriented dialogue area, but suffer from the challenges of (a) poor semantic control, and (b) little interaction with auxiliary information. In this paper, we propose a novel yet simple end-to-end model for response generation via mixed templates, which can address above challenges. In our model, we retrieval candidate responses which contain abundant syntactic and sequence information by dialogue semantic information related to dialogue history. Then, we exploit candidate response attention to get templates which should be mentioned in response. Our model can integrate multi template information to guide the decoder module how to generate response better. We show that our proposed model learns useful templates information, which improves the performance of ”how to say” and ”what to say” in response generation. Experiments on the large-scale Multiwoz dataset demonstrate the effectiveness of our proposed model, which attain the state-of-the-art performance.

text: End-to-end models have achieved considerable success in task-oriented dialogue area, but suffer from the challenges of (a) poor semantic control, and (b) little interaction with auxiliary information. In this paper, we propose a novel yet simple end-to-end model for response generation via mixed templates, which can address above challenges. In our model, we retrieval candidate responses which contain abundant syntactic and sequence information by dialogue semantic information related to dialogue history. Then, we exploit candidate response attention to get templates which should be mentioned in response. Our model can integrate multi template information to guide the decoder module how to generate response better. We show that our proposed model learns useful templates information, which improves the performance of ”how to say” and ”what to say” in response generation. Experiments on the large-scale Multiwoz dataset demonstrate the effectiveness of our proposed model, which attain the state-of-the-art performance.

heading: 1 INTRODUCTION
text: Task-oriented dialogue is aim to help users to complete a task in specific field such as restaurant reservation, or booking film tickets. The traditional approach is to design pipeline architectures which have several modules: natural language understanding, dialogue manager and natural language generation (Wen et al., 2017). It’s easy to control, but the dialogue system becomes more and more complicated. With the development of deep learning, end-to-end methods have shown hopeful results and received great attention in academic. They input user queries and generate system responses, which is relatively simple. However, the disadvantage is that end-to-end approaches are difficult to control generated results.
Task-oriented responses should have correct entities and grammatical expressions, which means solving problems of ”what to say” and ”how to say”. If a user wants to ask for a restaurant of moderate price range, a good task-oriented dialogue system should return the response with right restaurants whose prices cannot be high or low and use the proper wording which in clear and unambiguous expression. Researches get right entities by looking up the knowledge base(KB). Sukhbaatar et al. (2015) introduce KB in the form of hidden states. Madotto et al. (2018) use attention and copy mechanism to produce words from KB. Most research on generating smooth response has been carried out in using templates. However, templates usually are designed by domain experts in advance (Walker et al., 2007b) , so it needs huge cost and is difficult to transfer different domains. Wen et al. (2015) use Semantically Controlled LSTM (SC-LSTM) to control semantic results of language generation. Su et al. (2018) propose a hierarchical architecture using linguistic patterns to improve response generation. These models though achieving good performance, suffer from the problems as templates fixed or poorly controllability of semantic .
In order to alleviate such issues, we propose an end-to-end response generation model via templates integration(RGTI), which is composed of a encoder encoding the retrieval candidate responses with the form of triple into template representation and a decoder that integrates templates to generate the target response. Instead of encoding the templates directly, we construct a hierarchical encoding structure to make the templates contain semantic information and sequence information. During the decoder phase, we exploit a mixed decoder via templates and dialogue history introducing copy mechanism to generate better response.
We empirically show that RGTI can achieve advanced performance using the triple encoder and mixed decoder. In the human-human multi-domain dialogue dataset Budzianowski et al. (2018), RGTI is able to surpass the previous state-of-the-art on automatic evaluation, which further confirms the effectiveness of our proposed encoder-decoder model.

heading: 2 OUR MODEL
text: We now describe the RGTI framework composed of two parts: encoder for templates in triple form as well as dialogue history and mixed decoder in generation and copy mode, as shown in Figure 1. The dialogue history X = (x1, ..., xn) is the input, and the system response Y = (y1, ..., ym) is the expected output, where n,m are the corresponding lengths. We first retrieval relevant candidate responses to the target responses. Then the encoder block uses multi-head attention to encode the candidate responses in triple form into templates representation. Next, the copy augmented decoder uses a gating mechanism for words selection from two sources: utterances and templates, while generating a target response.

heading: 2.1 RETRIEVAL REFERENCE RESPONSE
text: First we should retrieval reference responses from training sets. The benefit of this approach is we can get more semantically coherent responses. For getting the better reference responses, we use dialogue state, dialogue act and semantic information of responses as search criteria. ElasticSearch is been used, which can get faster and more acurrate results.

heading: 2.2 ENCODER BLOCK
text: We use multi-head attention to encode dialogue historyX and retrieval reference responsesR, which can automatically extract important information from sentences. If we feed the retrieved sentences directly into the model, the noise is large and not conducive to model learning, so we assume the closet part to the entity in the response is the most important. We divide the entity and its most relevant parts into triple form (head, entity, tail). Head means the part of the sentence before the entity, and tail means the after part, for example if a response is ”enjoy your stay in value-place, goodbye”, the corresponding head is ”enjoy your stay”, entity is ”value-place”, tail is ”goodbye”. Using this form, we can pay attention to word order and obtain the relationship between the sentence structure and the entity in the sentence. We treat the encoded retrieval responses as template Q, which is a refined expression of retrieval results.
2.3 MIXED DECODER
Add & Norm Add & Norm
Feed Forward
Add & Norm
Add & Norm
Feed Forward
Add & Norm
Liner
Input Embedding
Output Embedding
Masked Multi-Head Attention Multi-Head Attention
Multi-Head Attention
Softmax
MultiHead Attention Distribution
Vocabulary Distribution
P_gen
Copy Distribution
Copy_dist *(1-P_gen)
Generate_dist * P_gen
Final Distribution
Figure 3: Mixed Decoder,
We predict tokens of target response based on two mixed modes, generate-mode and copy-mode. Generate-mode is to generate words from the vocabulary directly and copy mode is to copy words from templates. Accordingly, our model uses two output layer: sequence prediction layer, template location copying layer. Then we use the gated mechanism of the above output layers to get the final generated words. The probability of generating target word yt is the sum of multiple probabilities at each time step t:
p(yt|st, yt−1) =
ppr(yt|st, yt−1, ct) · pm(pr|st, yt−1)+
pco(yt|st, yt−1,MQ) · pm(yt, ct|st, yt−1)
where pr is the predict-mode, and comeans copy words from template, pm(·|·) shows the probability for choosing different modes. The probabilities of these modes are calculated as follows:
ppr(yt|·) = 1
Z eϕpr(yt)
pco(yt|·) = 1
Z ∑ j:Qj=yt eϕco(yt)
where ϕ is score function to choose mode, Z stands for the normalization term of two modes. Z = eϕpr(v) + ∑ j:Qj=v eϕco(v)
Specifically, the score functions of two modes are given by:
ϕ(yt = vi) = v T i Wpr[st, ctem]
ϕ(yt = xj) = DNN(hj , st, histQ)

heading: 3 EXPERIMENTS

heading: 3.1 DATASET
text: To verify the results of our model, we use recently proposed MultiWOZ dataset (Budzianowski et al., 2018) to carry out experiments. The MultiWOZ is the largest existing human-human conversational corpus spanning over seven domains (attraction, hospital, police, hotel, restaurant, taxi, train), which contains 8438 multi-turn dialogues and the average length of each dialogue is 13.68. Different from current mainstream task-oriented dialogue datasets like WOZ2.0 (Wen et al., 2017),and DSTC2 which contain less than 10 slots and only a few hundred values. There are almost 30(domain,slot) pairs and over 4500 possible values in MultiWOZ dataset. Each dialogue consists of a dialogue goal and a representation of multiple pairs of users and system utterances. In each turn of the dialogue, there are still two kinds of annotations, one is belief state and the other is the dialog action which are used to mark the status of the current conversation and the potential actions of the user.

heading: 3.2 TRAINING DETAILS
text: We trained our model end-to-end using Adam optimizer (Kingma & Ba, 2014),and uses multistep learning rate with milestones in 50, 100, 150, 200, the learning rate starts from 1e−3 and the parameter γ is 0.5. All the embeddings are initialized randomly, and beam-search strategy is used during the decoding state. The hyper-parameters such as hidden size is 512 and the dropout rate is 0.2.
3.3 EVALUATION
3.3.1 ENTITY F1
The main evaluation metric is F1 score which is the harmonic mean of precision and recall at word level between the predicted answer and ground truth. By comparing the ground truth system responses with the set of entities to select useful entities, this metric can evaluate the ability to generate
relevant entities and to capture the semantics of the dialog (Eric & Manning, 2017) and (Eric et al., 2017)

heading: 3.3.2 BLEU
text: We also use BLEU score in our evaluation which is often used to compute the word overlap between the generated output and the reference response.The early BLEU metrics was used in the field of translation . In recent years, it has also referred to the BLEU metrics in the end-to-end task-oriented dialogue and in the field of chat-bots.
3.4 ABLATION STUDY
We perform ablation experiments on the test set to analyze the effectiveness of different module in our model. The results of these experiments are shown in the Table below. As one can observe from the Table, our model without copy mechanism has 2.5% BlEU drop in generate results. On the other hand, RGTI without templates means that we do not consider the relevant templates information and thus lead to a reduction in BLEU. Note that if we remove the templates, then a 0.5 % increase can be observed in the table, which suggests that instead of applying context history makes the corresponding text noise have more side effects than his positive influence.

heading: 4 RELATED WORKS
text: Task-oriented dialog systems are mainly explored by following two different approaches: piplines and end-to-end. For the dialogue systems of piplines (Williams Young, 2007; Wen et al., 2017), modules are separated into different trained models:(i)natural language understanding g (Young et al., 2013; Chen et al., 2016), which is used to understand human intention, dialogue state tracking
(Lee Stent, 2016; Zhong et al., 2018) which is for estimating user goal at every step of the dialogue, dialogue management (Su et al., 2016), and natural language generation (Sharma et al.,2016) which is aimed to realize language surface form given the semantic constraint. These approaches achieve good stability via combining domain-specific knowledge and slot-filling techniques, but additional human labels are needed. On the other hand, end-to-end approaches have shown promising results recently. BiLSTM Zhao et al. (2017) use recurrent neural networks to generate final responses and achieve good results the previous state-of-art model about memory mechanism strengthen the reasoning ability by incorporating external knowledge into the neural network, Sequicity and HDSA represent the conversation history as belief span or dialog action which is used as compressed information for downstream model generate system response considering current dialog state.
4.1 TEMPLATE
Using templates to guide response generation is a common method in task-oriented dialogue area. Stent et al. (2004) and Walker et al. (2007a) use machine learning to train the templates selection. The cost to create and maintain templates is huge, which is a challenge in adapting current dialogue system to new domains or different language. Wiseman et al. (2018) introduce HMM into text generation, which can decode the state of generation templates. It’s a useful method to generate templates, however the variability and efficiency are not satisfied.
4.2 TRANSFORMER
Recently, a new neural architecture called transformer has surpassed RNNs on sequence to sequence tasks, the paper on transformers by Vaswani et. al. [19] demonstrated that transformers produce state-of-the-art results on machine translation, which allowing for increased parallelization and significantly reduced training time. Otherwise, there has been little work around the use of transformer on end-to-end dialog system.
4.3 POINTER-GENERATOR NETWORKS
The pointer-generator was first proposed by Vinyals ,and then has been applied to several natural language processing tasks, including translation (Gulcehre et al. [4]), language modeling (Merity et al. [7]), and summarization.
The motivation of this paper is how to effectively extract and use the relevant contexts for multi-turn dialogue generation. Different from previous studies, our proposed model can focus on the relevant contexts, with both long and short distant dependency relations, by using the transformer-pointergenerator mechanism.

heading: 5 CONCLUSION
text: In this paper,we propose an end-to-end trainable model called Response generation via templates Integration.The transformer pointer generator is trained to incorporate related reference responses into the learning framework which performs remarkably well. As a result our model achieves stateof-the-art results in both BLEU and Entity F1 score in the Multiwoz Dataset. We hope that the proposed model can benefit future related research.

Review summaries:
1. Summary: The authors proposed a deep neural network-based model to generate responses fro task-oriented dialogue systems. The model mainly contains two parts, the first part is to retrieve relevant responses based on question and encode them into templates, the second part is a decoder to generate the response based on the encoded templates and input utterances.

This paper describes a method to incorporate candidate templates to aid in response generation within an end-to-end dialog system. While the motivation and task setup is interesting, the paper is clearly unfinished. Most jarringly, Table 2 which should contain the main results comparing the proposed RGTI model to existing baseline models is not filled in, and there appears to be no table showing results of the ablation study briefly described in Section 3.4.