A lexa Conversations: An Extensible Data-driven Approach for Building Task-oriented Dialogue Systems (2021)

Abstract: Traditional goal-oriented dialogue systems rely on various components such as natural language understanding, dialogue state tracking, policy learning and response generation. Training each component requires annotations which are hard to obtain for every new domain, limiting scalability of such systems. Similarly, rule-based dialogue systems require extensive writing and maintenance of rules and do not scale either. End-to-End dialogue systems, on the other hand, do not require module-specific annotations but need a large amount of data for training. To overcome these problems, in this demo, we present Alexa Conversations, a new approach for building goal-oriented dialogue systems that is scalable, extensible as well as data efficient. The components of this system are trained in a data-driven manner, but instead of collecting annotated conversations for training, we generate them using a novel dialogue simulator based on a few seed dialogues and specifications of APIs and entities provided by the developer. Our approach provides out-of-the-box support for natural conversational phenomenon like entity sharing across turns or users changing their mind during conversation without requiring developers to provide any such dialogue flows. We exemplify our approach using a simple pizza ordering task and showcase its value in reducing the developer burden for creating a robust experience. Finally, we evaluate our system using a typical movie ticket booking task integrated with live APIs and show that the dialogue simulator is an essential component of the system that leads to over 50% improvement in turn-level action signature prediction accuracy.

## introduction goal-oriented dialogue systems enable users to complete specific goals such as making restau-rant reservations and buying train tickets. user goals may be complex and may require multiple turns to achieve. moreover, users can refer to contextual values anaphorically, can correct previously informed preferences and provide additional or fewer entities (over-cooperative or undercooperative user) than requested by the agent. this presents challenges for building robust dialogue agents that need to understand different kinds of user behavior, gather user requirements split over multiple turns and complete user goals with minimal friction. there is also limited availability of dialogue datasets and they span only a handful of application domains. designing suitable data collection for dialogue systems is itself a research area. traditional dialogue systems follow a pipelined approach that ties together machine learning components for natural language understanding (nlu), dialogue state (belief) tracking, optimal action prediction (policy learning), and natural language generation @xcite . advances in deep learning techniques have led to the development of more end-to-end neural dialogue systems that combine some or all of the components of the traditional pipeline reducing the need for component-wise annotations and allowing for intermediate representations to be learned and optimized end-to-end @xcite . on the data side, notable data collection approaches for dialogue systems include the wizard-of-oz (woz) framework @xcite , rule-based or data-driven user simulators @xcite @xcite @xcite , and the recently-proposed machines-talking-to-machines (m2m) framework @xcite where user and system simulators interact with each other to generate dialogue outlines. in this demo, we present alexa conversations, a novel system that enables developers to build ro-bust goal-oriented dialogue experiences with minimal effort. our approach is example-driven as it learns from a small number of developer-provided seed dialogues and does not require encoding dialogue flows as rigid rules. our system contains two core components: a dialogue simulator that generalizes input examples provided by the developer and a neural dialogue system that directly predicts the next optimal action given the conversation history. the dialogue simulator component extends the m2m framework @xcite in two main directions. first, instead of generating user goals randomly, we use various goal sampling techniques biased towards the goals observed in the seed dialogues in order to support variations of those dialogues robustly. second, in m2m, the system agent is geared towards database querying applications where the user browses a catalogue, selects an item and completes a transaction. in contrast, our formulation does not require any knowledge of the purpose of the apis provided by the developer. moreover, our system can generate a richer set of dialogue patterns including complex goals, proactive recommendations and users correcting earlier provided entities. the proposed neural dialogue model component follows an endto-end systems approach and bears some similarities with hybrid code networks (hcn) @xcite . however, compared to hcn, our system is more generic in the sense that it directly predicts the full api signature that contains the api name, values of the required api arguments, relevant optional api arguments and their values. the model chooses the api argument values to fill from user mentioned, agent mentioned and api returned entities present in the full dialogue context that includes the current user utterance. we showcase the significance of our approach in reducing developer burden using the example of a pizza ordering skill. compared to a rule-based system where a developer would have to code hundreds of dialogue paths to build a robust experience even for such a simple skill, alexa conversations requires only a handful of seed dialogues. to evaluate our approach, we build a movie ticket booking experience. on a test set collected via wizardof-oz (woz) framework @xcite , we quantify the impact of our novel dialogue simulation approach showing that it leads to over 50% improvement in action signature prediction accuracy. a: nlg: welcome() u: "how long is [ la la land | @xmath3: call: getduration(@xmath4, @xmath5: nlg: inform_movie_cast( @xmath6... u: "exit" a: nlg: stop() in alexa conversations, we follow a data-driven approach where the developer provides seed dialogues covering the main use cases they want to support, and annotates them in a dialogue markup language (dml). to handle the wide variation of conversations a user can have with the dialogue system, alexa conversations augments the developer provided seed dialogues through a simulator. this component takes the annotated seed dialogues as input, and simulates different dialogue flows that achieve the same user goals but also include common patterns such as when a user confirms, changes, or repeats an entity or action. optionally, it uses crowdsourcing through amazon mechanical turk (mturk) to enrich the natural language variations of user utterances provided by the developer. overall, the developer provides on the order of 10 seed dialogues and the simulator generates on the order of 10k training dialogues with flow and language variations. alexa conversations consists of three main domain-specific modeling components: 1) a named-entity recognition (ner) model that tags entities in the user utterance (e.g., "la la land" as a movietitle), 2) an action prediction (ap) model that predicts which api or nlg response should be executed next (e.g., getduration or in-form_movie_duration), and 3) an argument filling (af) model that fills required (and possibly optional) action arguments with entities (e.g., get-duration(@xmath7. we use the entire dialogue history, i.e., user utterances, system actions and responses, and api return values, as input for all modeling components. in this sense, this dialogue history is used as a generalized state representation from which models can retrieve relevant information. an overview of the runtime flow of a dialogue is illustrated in figure 1 . each user utterance initiates a turn and is followed by ner, after which one or more actions are predicted. these actions could be either an api or nlg call, or a special action indicating the end of a turn or the end of dialogue. every new action prediction updates the dialogue history and therefore influences future action predictions. for each api/nlg call the af model is called to fill in the required arguments. @xmath8, the system waits for new user input. @xmath9, the system ends the interaction. ## dialogue simulation we propose a novel component called simulator to generate diverse but consistent dialogues, which can be used to train robust goal-oriented neural dialogue systems. we presented the simulator details goal sampler api simulator user agent system agent user nlg system nlg user goal api value api + arguments dialog acts utterance utterance template id dialog acts + utterance actions + utterance in @xcite and briefly provide an overview of the overall system here. a high-level simulator architecture is illustrated in figure 2 . the simulator is structured in two distinct agents that interact turn-by-turn: the user and the system. the user samples a fixed goal at the beginning of the conversation. we propose novel goal-sampling techniques @xcite to simulate variation in dialogue flows. the agents communicate at the semantic level through dialogue acts. having the exact information associated with each turn allows us to define a simple heuristic system policy, whose output can be used as supervised training labels to bootstrap models. we note that the user policy is also heuristic-based. in each conversation, the user agent gradually reveals its goal and the system agent fulfills it by calling apis. the system agent simulates each api call by randomly sampling a return value without actually calling the api and chooses an appropriate response action. depending on the returned api value, the chosen response is associated with dialogue acts. the system agent gradually constructs an estimate of the user goal and makes proactive offers based on this estimated goal. the dialogue acts generated through self-play are also used to interface between agents and their template-based nlg model. after sampling the dialogue acts from their policy, each agent samples the surface-form from available templates corresponding to the dialogue acts. in addition to enriching the dialogue flows; we use crowd-sourcing through mturk to enrich the natural language variations of the user utterance templates. goal sampling and the self-play loop provide dialogue flow variations while crowd-sourcing enriches language variations, both of which are essential for training robust conversational models. we introduce additional variations to dialogues during simulation for more natural conversation generation. in goal-oriented conversations, users often change their mind during the course of the conversation. for example, while booking a movie ticket a user may decide to purchase three adult tickets but could eventually change their mind to book only two tickets. we used additional heuristics to introduce such variations to conversations without any additional input requirements from the developer. another important non-task-specific conversational behavior is the system's ability to suggest an appropriate next action based on the conversation history, without requiring invocation by a specific user utterance. we introduce proactive offers in the system policy of the simulator to facilitate exploration of the available api functionality in a manner consistent with human conversation. ## models for each domain, we have three separate models: ner, action prediction (ap) and argument filling (af), all of which depend on features extracted from conversation history and encoded using dialogue context encoders. ## case study 1: pizzabot to showcase how alexa conversations improves the skill-building experience of a typical developer, we discuss pizzabot, a pizza ordering skill. pizzabot is provided as one of the template alexa con-versations skills to help bootstrap a new developer. it allows a user to order a pizza of various sizes, with different kinds of crusts and their choice of any number of toppings. the skill has just one main api orderpizza with four slots types: toppingslist, crust, cheese and size. even with this simple skill, ordering a two topping pizza with a choice of say 10 toppings, 5 crusts and 5 cheese in 4 different sizes results in over hundreds of different possible dialogue paths. with a typical rule-based dialogue manager, a developer would need to code most, if not all, of these flows for building a robust experience. on the other hand, with alexa conversations, a pizzabot developer needs to provide just 7 example dialogues to support a total of 3 apis and 6 entity types. the different variations are automatically generated by the proposed simulator and are consumed by the models, resulting in a robust skill with significantly lower developer effort. alexa conversations also provides out-of-the-box support for features like context carryover, entity confirmation, user correction, over/under-cooperative user, and multi-value slots without requiring the developer to write custom rules or logic, making the skill more robust to real user interactions. see ## case study 2: ticketbot to showcase the end-customer impact of alexa conversations, we built a movie ticket-booking experience which we call ticketbot. ticketbot allows a user to browse and book movie tickets. users can browse currently playing movies by various search criteria like date, time, location, theater and movie title. they can specify one or more search criteria either within a single turn or across multiple turns. after finding their choice of movie and theater, users can select a particular showtime, provide booking details like number of tickets and finally confirm booking. the experience was built based on the information provided by the developer. this is a complex experience with 10 apis, 28 entity types, 10 nlg responses and 35 seed dialogues all provided as an input to the system. this experience was implemented using live apis that were provided by the developers and thus the users were able to actually achieve their goals and complete ticket-booking transactions. ## conclusions we presented alexa conversations, a novel datadriven and data-efficient approach for building goal-oriented conversational experiences. our proposed system significantly reduces developer bur-den while still allowing them to build robust experiences. we envision that this system will be used by a wide variety of developers who only need to provide seed dialogues and action schema to build conversational experiences 1 . we expect our system to mature in the following directions in future. we aim to reduce developer requirements for providing nlg responses by introducing a statistical nlg system. we will also develop robust mechanisms for incorporating developer feedback through supervised and semisupervised methods to improve the performance of our simulator and modeling components.