An Active Learning Pipeline for NLU Error Detection in Conversational Agents (2023)

Abstract: High-quality labeled data is paramount to the performance of modern machine learning models. However, annotating data is a time-consuming and costly process that requires human experts to examine large collections of raw data. For conversational agents in production settings with access to large amounts of user-agent conversations, the challenge is to decide what data should be annotated first. We consider the Natural Language Understanding (NLU) component of a conversational agent deployed in a real-world setup with limited resources. We present an active learning pipeline for offline detection of classification errors that leverages two strong classifiers. Then, we perform topic modeling on the potentially mis-classified samples to ease data analysis and to reveal error patterns. In our experiments, we show on a real-world dataset that by using our method to prioritize data annotation we reach 100% of the performance annotating only 36% of the data. Finally, we present an analysis of some of the error patterns revealed and argue that our pipeline is a valuable tool to detect critical errors and reduce the workload of annotators.

## introduction modern machine learning methods rely heavily on the availability of high-quality labeled data @xcite . as a consequence, annotating large volumes of data has become a priority across organizations. however, data annotation is a time-consuming and costly process: it requires, first, to train human experts who, then, have to manually examine large collections of raw data and assign labels. since assigning labels is often an ambiguous task, it is a standard that each sample is labeled by multiple annotators and labels are assigned based on inter-annotator agreement @xcite . the complexity of this process makes data annotation a common bottleneck when it comes to deploying data-driven systems that should operate reliably in production environments. a relevant example of these data-driven systems are conversational agents that interact directly with human users. these agents typically have at least two components, one for natural language understanding (nlu) and another for dialogue management (dm). the nlu component extracts intents and entities from the user utterance at each conversation turn, while the dm component decides on the next action based on the nlu output @xcite . once deployed, these assistants can have access to large amounts of raw data in the form of user-agent conversations. at scale, the amount of data available for annotation may soon exceed the capacity of the human annotators. the challenge then becomes how to select samples for annotation. on the nlu side, it is desirable to prioritize the annotation of utterances whose intent was misclassified during inference in order to correct existing flaws in the agent. however, automatically finding those utterances is challenging, since intent mis-classifications do not necessarily result in failed conversations and conversations can fail due to the misbehavior of other components of the digital assistant, not only due to the nlu. in this work, we consider a real-world scenario where an intent classifier needs to run with limited resources, specifically, in cpus and with low latency. this discards modern large language models (llms) as a valid option. nevertheless, llms can be used offline to detect potentially mis-classified data. we present a simple yet effective method based on voting that leverages two llms to detect problematic utterances. in particular, we compare the prediction of two llms with the intent assigned by the production classifier and if there is no unanimity between the three intents, we mark the utterance as problematic to prioritize its annotation and analysis. we embed this method in an active learning pipeline consisting of error detection, clustering and topic modeling, followed by expert annotation. this way, the human expert receives a curated set of problematic utterances clustered by topic, which facilitates the discovery of error patterns and greatly reduces the required workload. in our experiments, we simulate a real-world environment, where an intent classifier is periodically exposed to new data that can be potentially labeled and incorporated to the training data. we evaluate on a held-out test set and show that on a real-world dataset, an intent classification model trained with data labeled following the priority given by our pipeline can reach with 36% of the train data the same performance as with 100%, which represents a major reduction in annotation costs. furthermore, we show a qualitative analysis of the error patterns discovered by our method on two public datasets and argue that our pipeline is a valuable tool to early-detect intent classification errors that could be critical for the operation of a conversational agent. interactive machine learning (iml): iml is a growing field in machine learning that has demonstrated its success in building well-performing classifiers using fewer features @xcite @xcite . moreover, it improves user's trust and understanding of the system @xcite . in this context, our approach stands out as we provide a visualization of topic clusters to the annotators to facilitate their task. ## methodology our active learning pipeline consists of three stages, intent classification, error detection and topic modeling. the full pipeline is depicted in figure 1 as a block diagram. intent classification this is the production model that predicts the intent of the user utterance. due to the scalability constraints in terms of latency and computing resources, this model must have low inference time and run on cpus. without loss of generality, in this work we employ the universal sentence encoder (use) @xcite as embedder, followed by linear support vector classification (svc). during live conversations, both the user utterance and the predicted intent are stored and passed to the next stage of the pipeline for offline error detection. ## experiments and results here, we conduct a quantitative and a qualitative evaluation. in our quantitative evaluation we assess to what extent our pipeline reduces the amount of labeled data needed to reach certain performance; and in the qualitative evaluation we analyze discovered error patterns. we run our experiments on two public datasets: atis @xcite and snips @xcite ; and an internal dataset (auto) consisting of real-world data from the automotive domain. atis is a dataset of queries about flight information of 4,978 training samples and 893 test samples 1 . snips is a dataset of interactions between users and virtual assistants like siri or alexa. we use the version with 26,000 utterances and we set as label the joint fields "intent" and "scenario", which results in 64 classes. 1 we use: https://github.com/microsoft/cntk ## conclusion in this work we have presented an active learning pipeline for conversational agents which consists of intent classification, unsupervised error detection and topic modeling. in the experiments, we show that our approach helps in prioritizing data for annotation: in our real-world dataset (auto) we reach the same performance with 36% of the data when selected by our pipeline as with 100% without prioritization. therefore, this method can provide major savings for organizations with limited annotation capabilities. furthermore, we argue that our approach helps to discover intent classification errors that may be critical for the correct operation of the dialogue agent and which, if not detected on time, could jeopardize the viability of the system. in future work, we plan to extend our proposed pipeline to support also named entity recognition.