Pretrain-Finetune Based Training of Task-Oriented Dialogue Systems in a Real-World Setting (2021)

Abstract: One main challenge in building task-oriented dialogue systems is the limited amount of supervised training data available. In this work, we present a method for training retrieval-based dialogue systems using a small amount of high-quality, annotated data and a larger, unlabeled dataset. We show that pretraining using unlabeled data can bring better model performance with a 31% boost in Recall@1 compared with no pretraining. The proposed finetuning technique based on a small amount of high-quality, annotated data resulted in 26% offline and 33% online performance improvement in Recall@1 over the pretrained model. The model is deployed in an agent-support application and evaluated on live customer service contacts, providing additional insights into the real-world implications compared with most other publications in the domain often using asynchronous transcripts (e.g. Reddit data). The high performance of 74% Recall@1 shown in the customer service example demonstrates the effectiveness of this pretrain-finetune approach in dealing with the limited supervised data challenge.

## introduction retrieval-based dialogue systems are popular in task-oriented domains. a typical retrieval-based system encodes the dialogue context and a large set of candidate responses (templates) in a joint semantic space, and then scores how appropriate each candidate is given the dialogue context; the template with the highest score is selected as the response. these systems can use a sequence-to-sequence model @xcite or a dual-encoder style architecture @xcite to encode and score the context-response pair. one major challenge for any task-oriented dialogue system is the scarcity of training data. highquality data with all the required annotations are needed to train an accurate model. such datasets are not readily available, and collecting them is a costly and labor-intensive process. a few synthetic datasets @xcite @xcite have been proposed but they do not capture the real-world variations and subtleties of the task-oriented dialogues. the limited amount of supervised training data available makes it difficult to train these models from scratch. to overcome the issue of limited training data, the idea of finetuning a pretrained model has become a popular approach in other domains like computer vision and is recently gaining popularity in the natural language processing (nlp) domain. pretrained models in nlp such as elmo @xcite , openai gpt @xcite , and bert @xcite have attracted a lot of attention and achieved state-of-the-art accuracy in multiple natural language understanding tasks. in this paper, we present a methodology for training retrieval-based dialogue systems using a small amount of supervised data and a large, low-quality, unannotated dataset. 1. we demonstrate that finetuning a model @xcite pretrained using the unannotated dataset performs better than directly finetuning on the clean, annotated data. 2. we experiment with different finetuning loss functions and show that a ranking based loss function performs better than classification loss for template-retrieval based dialogue systems. 3. we deploy the finetuned model in an agent assistance application for customer service, and present real-world results on live customer contacts. in the sections that follow, we describe the data from the customer service domain that is used for pretraining and finetuning the model in section 2. in section 3, we explain how we pretrain the model using the next-turn prediction task along with the results. next, we present the proposed finetuning strategy, and the associated experimental setup and the results. in section 5, we present the real-world results of the deployed model. section 6 describes the conclusion and direction for future work. ## data in this work, we use data from the customer service domain -customer service chats handled in english. when customers contact customer service regarding their issue (e.g., order tracking, payment questions), the routing system connects the customer to an agent based on the specific issue type. agents resolving customer issues have access to a wide variety of profile information (e.g. customer details, order status, and internal apis) to execute actions such as canceling or refunding an order. for our experiments, we select a delivery-related customer issue. in the following subsection, we explain how we collect the pretraining and finetuning data. ## pretraining in this section, we introduce the next sentence prediction based pretraining used to pretrain the model. ## finetuning simple finetuning using the small finetune training dataset can lead to overfitting. in this section, we describe our finetuning approach, which incorporates regularization to avoid overfitting and the loss function that better caters to the task of template ranking. ## conclusion and future work in this paper, we study a less explored approach of finetuning a retrieval-based dialogue system based on a small amount of high-quality, annotated data that resulted in 26% offline and 33% online performance improvement in recall@1 over a pretrained model.we deployed the model in an agent-support application, and demonstarte that the proposed model achieves 74% recall@1, suggesting these models are effective in assisting agents by recommending text responses. the results demonstrate the effectiveness of pretrain-finetune approach in dealing with the limited supervised data challenge. in this paper, we focus on a customer service delivery issue, but since this technique can scale to other task-oriented dialog systems with a wide range of applications. we believe that additional investment in contextual and profile features would help improve the model performance. as discussed in the error analysis section, 28% of the model's error is caused due to missing context information. in addition, manual study of the rejection reasons highlights the issue of subjective evaluation. more investment is needed in agent training and standardization of