Benchmarking the Covariate Shift Robustness of Open-world Intent Classification Approaches (2022)

Abstract: Task-oriented dialog systems deployed in real-world applications are often challenged by out-of-distribution queries. These systems should not only reliably detect utterances with unsupported intents ( semantic shift ), but also generalize to covariate shift (supported intents from unseen distributions). However, none of the existing benchmarks for open-world intent classification focus on the second aspect, thus only performing a partial evaluation of intent detection techniques. In this work, we propose two new datasets ( and ) that include utterances useful for evaluating the robustness of open-world models to covariate shift. Along with the i.i.d. test set, both datasets contain a new cov-test set that, along with out-of-scope utterances, contains in-scope utterances sampled from different distributions not seen during training. This setting better mimics the challenges faced in real-world applications. Evaluating several open-world classifiers on the new datasets reveals that models that perform well on the test set struggle to generalize to the cov-test. Our datasets fill an important gap in the field, offering a more realistic evaluation scenario for intent classification in task-oriented dialog systems.

## introduction open-world classification has been extensively studied in both nlp and cv. reliably refraining from prediction on samples from out-of-scope labels is of utmost value @xcite , especially to ensure safety (e.g. autonomous driving) and high quality performance of ml models in production environments. @xcite term this as semantic shift detection. with the advent of voice/text-based task-oriented dialog assistants, it is important to distinguish between supported and unsupported intents to ensure that the classifier does not return garbage when it is barraged with queries from intents it has not been trained on. several state-of-the-art datasets have been proposed to evaluate the performance of openworld classifiers for intent detection. for example, clinc @xcite , rostd @xcite , hwu64 @xcite , etc. however, to the best our knowledge, none of the existing benchmarks for intent classification incorporate another fundamental aspect of inference in production. not only should an open-world classifier reliably handle semantic shift, it should also generalize (or be robust) to inference-time covariate shift where p train (y|x) = p test (y|x) but p train (x) ̸ = p test (x) @xcite @xcite . in industrial settings, it is common practice for ml systems to be trained on some amount of synthetic data. in general, for most real-world applications the production distribution is often unknown. the classifier, however, is still expected to output correct predictions regardless of this potential shift from what it has observed during training. in this work, we propose two new english @xcite benchmarks, clinc14-cov and hwu12-cov, that fill this gap by focusing on both semantic and covariate shift to evaluate the performance of intent classifiers. we leverage existing state-of-the-art intent classification datasets to specifically design a test set (cov-test) that, along with out-of-scope utterances, contains in-domain queries generated from a different distribution to the training set. the latter are collected by identifying equivalence clusters across different state-of-the-art intent classification datasets. elements within an equivalence cluster contain intent classes that, despite being sourced from different datasets, share the same underlying intent. in total, clinc14-cov cov-test contains 420 queries across 14 intents, while the cov-test split in hwu12-cov has 1080 queries across 12 intents. we evaluate a range of open-world intent classifiers and out-of-scope detection techniques on our datasets. our experiments show that all methods perform relatively poorly on the new cov-test sets. in the full-setting, we find a drop in performance of more than 10 absolute f1 and accuracy points from test to cov-test. we observe a smaller drop for few-shot classification suggesting that such a setting might lead to more robust intent classifiers. we also analyse the affect of covariate shift with and without semantic shift, and find that not only does the existence of both phenomena better mimic production scenarios, it also results in a more challenging setting for classifiers. our results show that the current models are less reliable when exposed to queries with covariate shift, especially in the open-world setting. we hope that the new datasets will enable future work to fill this gap in the research and development of dialog systems. 1 ## dataset we introduce two new datasets that contain utterances to evaluate the robustness of intent-classifiers to both covariate shift and semantic shift. ## benchmark evaluation we evaluate a range of open-world intent classification approaches on the new benchmarks. ## results and analysis here, we present the results of our experiments. full setting open-world classification. part of which can be attributed to their poor performance on in-domain classes (f in ) on this set. on hwu12-cov, the drop is smaller yet still significant, with performance (f 1 all , acc) going down from low 90s to mid 80s. similar to clinc14-cov, we see large differences (around 10 points) between the f 1 in scores on test vs cov-test sets. ## conclusion in this work, we propose two new benchmark datasets to evaluate open-world intentclassification techniques on their robustness to covariate shift. we leverage previously proposed intent-detection datasets to construct equivalence clusters whose elements represent intent labels that come from different datasets but refer to the same underlying intent class. the nature of these clusters results in a natural covariate shift, as utterances corresponding to each element can be assumed to be generated from a different distribution. these benchmarks test models in the presence of both semantic and covariate shift, a setting that better mimics the challenges faced in real-world production scenarios. we evaluate a range of state-ofthe-art open-world classification techniques on our datasets and find that despite their superior performance on i.i.d. test data, they fail to generalize on the covariance test samples. we believe that our datasets and analysis will lead to developing more robust systems for task-oriented dialog.