Distribution Calibration for Out-of-Domain Detection with B ayesian Approximation (2022)

Abstract: Out-of-Domain (OOD) detection is a key component in a task-oriented dialog system, which aims to identify whether a query falls outside the predefined supported intent set. Previous softmax-based detection algorithms are proved to be overconfident for OOD samples. In this paper, we analyze overconfident OOD comes from distribution uncertainty due to the mismatch between the training and test distributions, which makes the model can’t confidently make predictions thus probably causes abnormal softmax scores. We propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout. Our method is flexible and easily pluggable to existing softmax-based baselines and gains 33.33% OOD F1 improvements with increasing only 0.41% inference time compared to MSP. Further analyses show the effectiveness of Bayesian learning for OOD detection.

## introduction detecting out-of-domain (ood) or unknown intents from user queries is key for a task-oriented dialog system @xcite @xcite @xcite . it aims to know when a user query falls outside their range of predefined supported intents to avoid performing wrong operations. different from normal intent classification tasks, lack of labeled ood examples leads to poor prior knowledge about these unknown intents, making it challenging to detect ood samples. a rich line of ood intent detection algorithms has been developed recently, among which softmax-based methods demonstrated promise @xcite @xcite 2020). softmax-based methods leverage softmax outputs extracted from an in-domain (ind) intent model and operate under the assumption that the test ood samples get a lower likelihood probability than the id data. for example, maximum softmax probability (msp) @xcite detects a test query as ood if its max softmax probability is lower than a fixed threshold. however, all these models make a strong distributional assumption of the practical ood probability being uniform, which has been proven wrong because neural networks can produce overconfidently high softmax scores even for ood samples @xcite . therefore, solving the overconfidence issue is still challenging for ood detection. in this paper, we study the overconfidence issue from the perspective of bayesian learning @xcite . essentially, the reason for overconfidence is that a model cannot confidently make predictions on the input ood utterances (unknow-unknow) due to the lack of prior knowledge of ood data. in other words, even given the same input ood intent, the predicted probability distributions using different random seeds are completely different, uniform, sharp, or any distribution. fig 1 show an example. we find models with different initialization seeds can output diverse distributions for ood input, maybe cause overconfidence in several in-domain classes. but the averaged output is close to a uniform distribution. we also find models with different seeds are more robust to ind input and obtain consistent outputs (see appendix c.2). therefore, one direct way to solve the distribution uncertainty is to train multiple models independently and assemble their outputs for the final result. but this method is not applicable to practical scenarios for large training cost. in this paper, we propose a bayesian ood detection framework to calibrate distribution uncertainty. specifically, we firstly train an indomain intent classifier using ind data, then in the test stage, we perform multiple stochastic forward passes with a certain dropout rate (like 0.7) and average the output normalized logits as a final probability. without increasing any new parameters, we calibrate distribution uncertainty by tending to expectation uniform distribution via monte-carlo dropout @xcite . our method can be easily extended to existing softmax-based ood detection methods and gain significant ood improvements with only increasing little inference time compared to baselines, even outperform the state-of-the-art distance-based methods like lof @xcite and gda @xcite . our contributions are two-fold: (1) we analyze the intrinsic reason of overconfidence issue via distribution uncertainty and propose a bayesian ood detection framework to calibrate this uncertainty using monte-carlo dropout. (2) we provide theoretical and empirical analysis to demonstrate the effectiveness of our bayesian ood method. 2 method 2.1 understanding ood detection problem definition we refer to training data d as ind data. we aim to detect the input utterances x belonging to ood and correctly classify the utterances belonging to ind utilizing a well-calibrated classifier trained only on finite ind data d. the predictive uncertainty of a classification model p (υ|x, d) is commonly divided into data uncertainty (aleatoric), distribution uncertainty and model uncertainty (epistemic)(kiureghian and ditlevsen, 2009; malinin and gales, 2018): p (υ|x, d) = p (υ|µ) data p (µ|x, θ) distribution p (θ|d) model dµdθ therefore, how to alleviate the distribution uncertainty is the key to solving the overconfidence problem in ood detection. ## conclusion in this paper, we conduct an analysis of why previous softmax-based detection algorithms like msp or entropy suffer from the overconfidence issue. we find ood samples exactly yield a sparse distribution over the simplex and evenly distribute over the whole space. therefore, we propose a simple but strong bayesian approximation method to calibrate ood distribution. experiments prove the effectiveness of our method. we hope to provide new guidance for future ood detection work. zhiyuan zeng, keqing he, yuanmeng yan, hong xu, and weiran xu. 2021b. adversarial self-supervised learning for out-of-domain detection. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 5631-5639, online. association for computational linguistics. yinhe zheng, guanyi chen, and minlie huang. 2020. out-of-domain detection for natural language understanding in dialog systems. ieee/acm transactions on audio, speech, and language processing, 28:1198-1209.