Improving Large-Scale Conversational Assistants using Model Interpretation based Training Sample Selection (2022)

Abstract: This paper presents an approach to identify samples from live traffic where the customer implicitly communicated satisfaction with Alexa’s responses, by leveraging interpretations of model behavior. Such customer signals are noisy and adding a large number of samples from live traffic to training set makes re-training infeasible. Our work addresses these challenges by identifying a small number of samples that grow training set by ~0.05% while producing statistically significant improvements in both offline and online tests.

## introduction large-scale, voice-based conversational assistants such as alexa, siri, google assistant and cortana process each utterance through a multi-stage pipeline that includes wakeword detection, automatic speech recognition (asr), natural language understanding (nlu), entity resolution, and textto-speech. this is a well-understood sequence @xcite and each of these steps leverage multiple machine learning models. the nlu system is often modularized into a number of domains that handle distinct classes of utterances such as music, weather, etc. @xcite . the assistant system comprises models for domain classification (dc), intent classification (ic), and named entity recognition (ner). a key challenge in building, extending and maintaining such a system is that the underlying models need annotated training data. collecting large volumes of such data through manual labeling is expensive and does not scale. our work aims at improving the efficiency of this process. in contrast to previous approaches which identify utterances with defective responses, we instead focus on identifying cases that were processed successfully by the conversational assistant, and automatically retraining models with the additional data. however, this introduces two challenges. first, the vast majority of utterances are already processed correctly by the deployed system, resulting in an overwhelmingly large set of augmentation candidates. secondly, implicit signals for satisfaction are noisy, as users might frequently ignore incorrect responses without making the effort to reformulate their query or provide a response that reflects dissatisfaction with the experience. thus, simply adding all utterances from the full candidate pool (potentially billions/week) is infeasible and might actually degrade performance due to noise. we present a novel approach to address this based on integrated gradients (ig) @xcite , a technique for understanding model behavior through feature importance scores. we propose a sample importance score that aggregates word scores and ranks the utterances in our initial candidate set, followed by training set augmentation with a small fraction of the top utterances. our experiments on live traffic data from a large scale conversational assistant indicate that retraining models with training sets, augmented by as little as 0.05% in size, produces a statistically significant (p-@xmath0.05) improvement in semantic error rate (semer) in offline test sets -0.27% overall and 0.45% on a more challenging set of less frequent "tail" utterances. in online a/b tests, we observe a 1.23% and 0.96% improvement in defect rates for all and tail utterances, respectively. in contrast, simply adding all utterances from our initial candidate set degrades semer by 1.74% and 3.13% on the full and tail data sets, respectively. finally, we demonstrate the repeatability and generalizability of our approach on public benchmark data sets. despite the lack of label noise, we see small but consistent accuracy gains of 0.21% resp. 0.65% on the snips and agnews data sets. ## related work several approaches have been proposed recently to use distant or weak supervision to address sparsity of labeled data (see e.g. the survey in @xcite ). a number of works identify utterances with processing errors through offline analysis @xcite @xcite . these approaches however still need human annotation in an active learning loop to improve production models. query rewriting based approaches @xcite @xcite aim to address this limitation and enable self-learning without the need for human annotation. they detect instances where a user reformulates a query due to an unsatisfactory response and learn to map the failed utterance to a subsequent successful one. however, such approaches do not generalize to other utterance shapes. @xcite leverage user paraphrasing behavior in dialog systems to automatically collect annotations for long-tail utterances. @xcite present an approach where implicit negative feedback from the user is used to train a re-ranker that is then applied to pick correct annotations for under-performing utterances. a range of post-hoc model interpretability methods for machine learned models has been developed in recent years (see e.g., @xcite @xcite . local black-box methods typically measure the influence of individual features of an input example (e.g., individual words in a sentence) on the output prediction. other techniques aim to score complete examples with respect to prototypicality @xcite , influence on test predictions @xcite , and difficulty @xcite . our word-occurrence based approach can be seen as a computationally scalable linear approximation to such measures. @xcite conducted a survey on how organizations use model interpretability in practice. they identified model debugging as one of the primary uses of model explainability, seeking explicit human feedback on gathering more data for improving model performance. our approach differs from previous work as follows: ## implicit user feedback based data augmentation nlu services which cater to a large number of users such as voice controlled agents typically collect implicit feedback metrics for each interaction. as a simplistic example, the absence of any negative feedback from the user to the agent's action (no interruption, no repetition, etc.) can suggest that the agent successfully served the user's request. in this paper, we propose a mechanism that relies on successful user interactions to identify additional data for building nlu models. oftentimes, an unsuccessful action from our virtual agent is followed by the user rephrasing their request. if the rephrase is successfully served by the agent, then this indicates that the nlu hypothesis for the rephrased request is likely correct. we believe that a correctly served rephrased turn is a stronger positive feedback when compared to a single-turn interaction with an implicit positive feedback (i.e no negative feedback from the user). we create a new training sample using the asr transcript of the rephrased user request and the nlu hypothesis. we call this data set weak signal labeled (wsl) data since we rely on weak supervision from the user to obtain nlu labels. we score these utterances using integrated gradient technique as described in section 4 before using them as additional data source for building nlu models. ## importance score computation we describe our approach for scoring utterances according to their importance to the performance of a classification model (e.g., a domain or intent classifier). let t be the original training data, a be a pool of augmentation data, v be a validation data set, and y be the test set. the model trained on t typically makes some mistakes on v . the objective is to use word attribution scores computed with a local black-box interpretability technique @xcite on v to score utterances in a; then, by adding some of them to t , we hope to train a model that is more robust against failures on y that are similar to those observed on v . thus, our approach can be roughly subdivided into three steps: ## experimental setup we first present initial results of our approach on the open source intent classification (ic) data sets (snips @xcite and agnews @xcite ), and then demonstrate the impact of our approach on a joint intent classification and named entity recognition (ic-ner) task on live traffic of a commercial conversational assistant. algorithm 1 diversity method for utterance scoring. with agnews (del @xcite , we chose a data set with a slightly different but related task (news topic classification), due to its sufficient size. it contains 4 classes each containing 30 000 training samples and 1 900 testing samples. the total number of training and testing data is 120 000 and 7 600, respectively. we apply stratified random sampling to subdivide the training further for a validation set of 7 600 instances. apart from the usual partitions of data into train, validation and test sets, our experiments consider a further sub-partition of the train set into base set and augmentation set. the base set is a randomly down-sampled version of the full train set to a desired target size (e.g., 50% of the data). the rest of the training data is the augmentation set from which additional samples are selected. the validation set is used for computing the feature attributions. ## conclusions and future work a key challenge in building state-of-the-art deep learning models is the cost and effort involved in obtaining large volumes of manually labeled data. our work is part of a line of investigations into leveraging unlabeled or weakly supervised data at scale. we extract large amounts of user dialogs with a conversational assistant which are deemed successful according to implicit feedback. however, it is not sufficient to add all such examples indiscriminately to the training data -doing so does not improve the model, nor is it computationally scalable. we show how to leverage model interpretability techniques to prioritize the most important instances that should be added to the training set. our approach leads to statistically significant error rate reductions of our live system. we also demonstrate transferability on public nlu data sets, snips and agnews. in the future, we will apply our approach to other challenging public data sets which suffer from significant label noise and ambiguity. we will investigate other types of data set modifications, such as removal and replacement of examples.