Deploying a Retrieval based Response Model for Task Oriented Dialogues (2022)

Abstract: Task-oriented dialogue systems in industry settings need to have high conversational capability, be easily adaptable to changing situations and conform to business constraints. This paper describes a 3-step procedure to develop a conversational model that satisfies these criteria and can efficiently scale to rank a large set of response candidates. First, we provide a simple algorithm to semi-automatically create a high-coverage template set from historic conversations without any annotation. Second, we propose a neural architecture that encodes the dialogue context and applicable business constraints as profile features for ranking the next turn. Third, we describe a two-stage learning strategy with self-supervised training, followed by supervised fine-tuning on limited data collected through a human-in-the-loop platform. Finally, we describe offline experiments and present results of deploying our model with human-in-the-loop to converse with live customers online.

## introduction a task oriented dialogue (tod) system aims to accomplish specific tasks such as hotel reservation @xcite , flight booking, customer support @xcite and so on. an end-to-end tod system directly takes a multiturn dialogue context as input and predicts the next response with a single model @xcite . these can be developed using either retrievalbased approaches @xcite where the model ranks a response from a pre-constructed response pool; or generative approaches where a response is sequentially generated with encoder-decoder architectures @xcite . although generative models are widely studied in literature for dialogue systems @xcite et al., 2021) as they are capable to generate free text, it is nearly impossible to provide guarantees on the style, quality and privacy risks for their realworld applications. in this work, we focus on the development and deployment of a retrieval-based conversational system for an online retail store, in the customer service domain. our main contributions are: ## related work retrieval-based dialogue systems @xcite involve single-and multi-turn response matching @xcite @xcite @xcite @xcite . the selection of an appropriate response is usually based on computing and ranking the similarity between context and response. two popular model architectures for such similarity computation between inputs, is cross-encoders @xcite , which perform full self-attention over a given input and label candidate; and bi-encoders @xcite , which encode the input and candidate separately and combine them at the end for a final representation. bi-encoders have the ability to cache the encoded candidates, and reuse their representations for fast inference. cross-encoders, on the other hand, often achieve higher accuracy but are prohibitively slow at test time. a recent method, poly-encoders @xcite , combines the strengths from the two architectures, and allows for caching response representations while implementing an attention mechanism between context and response for improved performance. transformer-based architectures @xcite are widely used to encode information in tod systems. for instance, tod-bert @xcite ) incorporates user and system tokens into the masked language modeling task and uses a contrastive objective function to simulate the response selection task. in this work, we also adapt the transformer architectures and enhance poly-encoders to encode conversational history, response and profile features. ## response pool creation we semi-automatically extract a broad template pool from a large number of anonymized human dialogues. we first select the template texts from human responses in actual dialogues. this ensures that the bot language conforms to the desired style. our primary selection criteria for response candidates are frequency and novelty. we iteratively select sentences that are (1) most frequently used in human dialogues, and (2) contain information different from already selected responses (detailed algorithm in appendix a). this directly maximizes the dialogue model's coverage, as measured by the fraction of contexts for which the model has a suitable response in the pool. an alternative approach would have been clustering frequent sentences and selecting a representative for each cluster @xcite as templates. we instead opted for the deterministic procedure which is more intuitive for ingesting prior linguistic knowledge and provides interpretability. quantitative evaluation of coverage: figure 2 shows the bleu score by aligning the best matching templates to unconstrained human-human dialogues. as can be seen, with the growing size of the template pool, the bleu score approaches this upper bound, proving that the proposed approach can achieve strong conversational capacity. similar to @xcite , we see that a set of 500 -1k sentences can achieve good coverage of domain-specific conversations. template decorations: we enhance the templates through attaching metadata, like calls to external apis and constraints on profile features. for example, a template 'i have issued a refund to your credit card', will have an action that triggers an api call for issuing the refund. through profile feature constraints we enforce consistency requirements on the dialogue, for example, filtering out the above template if an order is not eligible for refund. this establishes guarantees that the bot is always consistent with business policies. ## model architecture we represent a dialogue d @xmath0, u 1 , . . . , a n } as a set of user (u i ) and agent (a i ) turns. while conversing with a user, agents look up information related to the particular order and item to determine applicable business policies and constraints. this may include item category, its delivery status, whether it was already refunded, among others; we encode these as categorical features. we create multiple input-output tuples by splitting a complete dialogue transcript at each agent turn (e.g. at turn k). the model learns to predict the next agent response (a k ) given the dialogue history so far and the features that encode item level, customer level information and applicable policies. we flatten the history into a single sequence by concatenating all agent and user turns (x h ). we introduce two marker tokens [agentstart] and [userstart] to mark the beginnings of an agent and user turn respectively. the features are also represented as a sequence, where each feature is encoded as a key_value pair (x f ). figure 1 presents the overall architecture of our proposed ranking model extended from poly-encoder @xcite . we use separate transformers @xcite as encoder blocks (m) for all inputs and encode them as: over the sequences we apply self-attentions to obtain latent representations. we represent the response using a single vector z @xmath1. for the multi-turn dialogue context and feature sequence we learn m h and m f representations, respectively, i.e. z @xmath2. we use 300 history representations (m h ) and 50 profile representations (m f ) in our experiments. to learn history-response and feature-response correlations we apply cross-attention layers. we then merge the two modalities of information from history and profile features through a 2-layer mlp to represent the complete dialogue context: a score function is used to rank the candidate responses given a (history, profile) pair through computing similarity using dot-product. we train the model in end-to-end manner using a binary cross-entropy loss. ## evaluation we conduct offline and online experiments on internal conversational datasets of an e-commerce customer service from the following two intents, 1 . we tokenize and sentence split dialogue turns using nltk toolkit @xcite . we split each dataset to train/dev/test sets with ratio 90:5:5 and use the most frequent 30k, 10k tokens as dialogue encoder vocabulary for start return and return refund intent, respectively. model training. we train and fine-tune the models on the unlabeled and labeled intent datasets, respectively. we use a learning rate of 0.00015 and train for 30 epochs with early stopping. ## conclusion we presented a neural, retrieval-based dialogue model that ranks responses from a large, datadriven template pool. pre-defined responses make it possible to enforce requirements for consistency to business policies and the proposed template mining method provides good conversational capacity. the model is accurate and efficient in terms of inference speed to handle conversations in real time. a human-in-the-loop setup lets us effectively collect a small-sized labeled dataset to improve the quality for online deployments. offline and online results demonstrate that this is a viable approach for developing tod systems for practical usecases. while rrs showed good improvements with our training protocol, it needs further work to be deployed. performance on the sr intent permitted the deployment of the model; its live success rate almost reaches the 56% upper bound that humans achieve in the controlled setting. anecdotal evidence 2 from customer feedback shows that successful dialogues by the model provide good conversational experience. 2 we include few positive user feedback in appendix b.