Convolutional Neural Encoder for the 7 th Dialogue System Technology Challenge

In this paper, we demonstrate that our convolutional neural network (CNN) approach to selecting the next best system response in a spoken dialogue system is competitive on the recently released 7th Dialogue System Technology Challenge (DSTC7). Our CNN model outperforms the strong dual LSTM encoder baseline, placing us in 11th place out of 20 participants on the first subtask of the Advising dataset in track one, where the goal is to select the next best response given the previous dialogue history between a student and the advisor, who are discussing the best courses for the student to enroll in. We show that our learned CNN filters identify semantically meaningful categories of tokens, such as greetings and course names, illustrating the interpretability of convolutional models for dialogue response generation. Introduction With the rise of conversational agents such as Siri1 and Cortana,2 dialogue systems that interact with people through spoken language are becoming more and more popular. Typically, however, these commercial products are still heavily rule-based. With the abundance of data and more powerful computation available to us today, we can apply increasingly powerful models to machine learning problems. In particular, neural networks have been shown to outperform prior state-of-the-art statistical models in computer vision and speech recognition, and can learn to handle raw input without requiring any manual feature engineering. Thus, we propose applying neural methods to spoken dialogue systems, allowing the models to handle raw natural language internally, with minimal pre-processing required. The standard pipeline of steps in a spoken dialogue system is shown in Fig. 1, where the user query (either written or spoken) is fed through an automatic speech recognizer if it is spoken, and the text of the query is sent to the language understanding component. In this component, the semantic tagging step involves isolating slots and corresponding slot values (e.g., in a flight booking system, the departure city slot may have the value Boston). The database retrieval then looks up the relevant information (e.g., flights) Copyright c © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. https://www.apple. com/siri https://www.microsoft. com/en-us/cortana Figure 1: A standard pipeline for a spoken dialogue system, where the input spoken user query is passed to an automatic speech recognizer to generate its text, and the generated text or the input textual user query is sent to the language understanding component for pre-processing and semantic tagging. Then, the information is passed to the dialogue manager to update the state of the dialogue and predict the next action the system should take to generate a desired response. from a knowledge base. This information is passed to the dialogue manager, which updates the state of the dialogue (i.e., the user’s goals, which are usually represented as a set of slots and matching slot values) and determines the next action the system should take, such as asking a followup question about the user’s preferred departure time. In this paper, we focus on the final response generation step, taking the user’s query and dialogue history as the only input, completely bypassing the language understanding and dialogue management components. Therefore, the model is trained fully end-to-end, with only two inputs: the previous two utterances in a dialogue between a student and the advisor, and a candidate utterance for the next system response. Given a set of 100 possible response candidates, the goal is to select the next best response based on the dialogue history (see Table 1 for an example partial dialogue). We evaluate our convolutional neural network approach, which does not require any manual feature engineering or semantic dictionaries, on the recently released 7th Dialogue System Technology Challenge (DSTC7), focusing in particular on subtask one of the Advising dataset in track one.3 https://ibm.github. io/dstc7-noesis/public/index.html The challenge provides five subtasks: 1) selecting the next response from among 100 candidates, 2) selecting the next response from 120,000 responses, 3) selecting the next response and its paraphrases from among 100 candidates, 4) selecting the next response from among 100 candidates that may not contain the correct response, and 5) selecting the next response from among 100 candidates and incorporating the provided external knowledge. We demonstrate that our convolutional neural encoder places 11th out of 20 participants on the first subtask, with a Recall@50 score of 82.4%. Advisor: Hello Mingyang! Are you doing well? Student: Hi advisor. I’m doing alright. I would like some advice on which courses to take next semester. Student: My interested area is Software Development and Intelligent system. Advisor: you have three choices namely, EECS481 Software Engineering, EECS492 Introduction to Artificial Intelligence, and EECS 381 Object Oriented and Advanced Programming. Student: how many difficulty levels do these classes have? Advisor: EECS381 is not easy Advisor: any thoughts about that? Student: What time does the course occur? I like afternoon classes and will find something else if it’s scheduled too early. Advisor: EECS351 is after lunch. The others are before. EECS481 is from nine to ten thirty and EECS492 is from ten thirty to twelve. Table 1: Example DSTC7 dialogue snippet, between a student and the advisor. 100 candidate responses are provided, and the system must choose the correct response, given the conversation history: “481 is the early morning and is quite similar to EECS381, so you might want to skip it. ”