AARGH ! End-to-end Retrieval-Generation for Task-Oriented Dialog (2022)

Abstract: We introduce AARGH, an end-to-end task-oriented dialog system combining retrieval and generative approaches in a single model, aiming at improving dialog management and lexical diversity of outputs. The model features a new response selection method based on an action-aware training objective and a simplified single-encoder retrieval architecture which allow us to build an end-to-end retrieval-enhanced generation model where retrieval and generation share most of the parameters. On the MultiWOZ dataset, we show that our approach produces more diverse outputs while maintaining or improving state tracking and context-to-response generation performance, compared to state-of-the-art baselines.

## introduction most research task-oriented dialog models nowadays focus on end-to-end modeling, i.e., the whole dialog system is integrated into a single neural network @xcite . although recent end-to-end generative approaches based on pre-trained language models produce fluent and natural responses, they suffer from two major problems: (1) hallucinations and lack of grounding @xcite , which result in faulty dialog management or responses inconsistent with the dialog state or database results, and (2) blandness and low lexical diversity of outputs @xcite . on the other hand, retrieval-based dialog systems @xcite select the most appropriate response candidate from a humangenerated training set, thus producing varied outputs. however, their responses might not fit the context and can lead to disfluent conversations, especially when the set of candidates is sparse. this limits their usage to very large datasets which do not support dialog state tracking or database access @xcite . several recent works focus on combining the retrieval and generative dialog systems via response selection and subsequent refinement, i.e., retrievalaugmented generation @xcite @xcite . these models are used for open-domain conversations or to incorporate external knowledge into task-oriented systems and do not consider an explicit dialog state. our work follows the retrieve-and-refine approach, but we adapt it for database-aware taskoriented dialog. we aim at improving diversity of produced responses while preserving their appropriateness. in other words, we do not retrieve any new information from an external knowledge base, instead, we retrieve relevant training data responses to support the decoder in producing varied outputs. to the best of our knowledge, we are the first to use retrieval-augmented models in this context. unlike previous works, we merge the retrieval and generative components into a single neural network and train both tasks jointly, instead of using two separately trained models. our contributions are summarized as follows: 1 • we propose a single-encoder retrieval model utilizing dialog action annotation during training, and we show its superior retrieval capabilities in the task-oriented setting compared to twoencoder baseline models @xcite . task-oriented response generation most current works focus on building multi-domain database-grounded systems. the breeding ground for this research is the large-scale conversational dataset multiwoz @xcite @xcite . recent models often benefit from action annotation. @xcite use action-based data augmentation and a three-stage architecture, decoding the dialog state, action, and response. @xcite generate responses without state tracking, exploiting a hierarchical structure of the action annotation. on the other hand, reinforcement learning models @xcite learn latent actions from data without using annotation. recent works focus on end-to-end systems based on pre-trained language models. @xcite fine-tune gpt-2 @xcite to model task-oriented dialogs, hosseini-asl et al. (2020) enhance this approach with explicitly decoded system actions. @xcite use auxiliary training objectives and machine teaching for gpt-2 fine-tuning. @xcite introduced the encoder-decoder-based framework mintl with bart @xcite or t5 (kale and rastogi, 2020) backbones (see section 3.1). response selection can be viewed as scoring response candidates given a dialog context. a popular approach is the dual encoder architecture @xcite where the response and context encoders model a joint embedding space. the encoders can take various forms: @xcite compare encoders based on bert @xcite and custom encoders pre-trained on reddit; @xcite pretrain encoders specifically for task-oriented conversations. @xcite introduce polyencoders, which produce multiple context encodings and add an attention layer to allow rich interaction with the candidate encoding (cf. section 3.3). ## method we aim at end-to-end modeling of database-aware task-oriented systems, i.e., systems supporting both dialog state tracking and response generation tasks @xcite . we combine retrieval and generative models to reduce hallucinations and boost output diversity. we first describe our purely generative baseline (section 3.1), then explain baseline generation based on retrieved hints (section 3.2). we then introduce baseline retrieval models (section 3.3) and our action-aware retrieval (section 3.4). finally, we describe aargh, our singlemodel retrieval generation hybrid, in section 3.5. aargh is shown in figure 1 ; other setups are depicted in appendix a. ## experimental setup models our models are based on pre-trained models from huggingface @xcite : we implement gen and the generative parts in our retrieval-based models using t5-base @xcite . retrieval encoders in de, aade, pe and aae are implemented as fine-tuned bertbase @xcite . aargh is built upon t5-base, same as gen; we fork the last @xmath0. the choice of l is a trade-off between model performance and size. 3 the database embedding has size @xmath1. for simplicity, we do not use specialized backbones pretrained on dialogs such as tod-bert @xcite . pe uses @xmath2.3) and single-headed attention mechanisms. i would like to eat in the center of town at an expensive place. usr pricerange area how about african? i'd like to book a table for 7 at 16:15 on tuesday. bookpeople food booktime booktime i have booked a table at bedouin african restaurant for 7 people at 16:15 tuesday. your reference number is vazcwis6 . may i help with anything else? sys for town centre, there are african, asian oriental, and british options. which of these would you prefer? restaurant: inform-area, 3× inform-food, select-none restaurant: inform-name, inform-food, inform-bookpeople, inform-booktime, inform-bookday, inform-ref usr sys figure 2: part of a short conversation from multiwoz. it has user and system turns, and annotated slot spans. both, user and system affect the dialog state. actions are shown below system texts. ## evaluation and results we focus on end-to-end modeling, which includes dialog state tracking and response generation. all reported results are on multiwoz test set with 1000 dialogs, averaged over 8 different random seeds. we generated responses given ground truth contexts. we follow mintl and predict the dialog state cumulatively for each conversation turn, which means that state tracking errors may compound. see appendix c for an example end-to-end conversation without any ground-truth information. ## conclusion we present aargh, an end-to-end task-oriented dialog system, combining retrieval and generative approaches. it uses an embedded single-encoder retrieval component which extends a purely generative model without the need for a large number of new parameters. aargh features an action-aware response selection training objective. our experiments on the multiwoz dataset show that aargh outperforms baselines in terms of automatic metrics and human evaluation and it is competitive with state-of-the-art models such as soloist or mt-tod. we showed that our proposed action-aware retrieval training objective supports retrieval of a larger variety of unique and relevant responses in the task-oriented setting and makes efficient use of the available system action annotation. further, using the retrieval module improves dialog management in terms of the success rate. a limitation of our approach is the need for careful hyperparameter setting, coupled with the risk of overuse of retrieved responses that match the dialogue state but are not appropriate for the context. in future work, we would like to confirm our results on more datasets and explore more complex ways of usage of the retrieved responses to encourage the model to copy interesting language structures while ignoring inappropriate tokens or relics of faulty delexicalization. @xcite chris brockett, xiang gao, jianfeng gao, jingjing liu, and bill dolan. 2020b. dialogpt : large-scale generative pre-training for conversational response generation. in proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, acl 2020, pages 270-278, online.