U ni NL : Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning (2022)

Abstract: Detecting out-of-domain (OOD) intents from user queries is essential for avoiding wrong operations in task-oriented dialogue systems. The key challenge is how to distinguish in-domain (IND) and OOD intents. Previous methods ignore the alignment between representation learning and scoring function, limiting the OOD detection performance. In this paper, we propose a unified neighborhood learning framework (UniNL) to detect OOD intents. Specifically, we design a KNCL objective for representation learning, and introduce a KNN-based scoring function for OOD detection. We aim to align representation learning with scoring function. Experiments and analysis on two benchmark datasets show the effectiveness of our method.

## introduction out-of-domain (ood) intent detection aims to know when a user query falls outside the range of pre-defined supported intents, which helps to avoid performing wrong operations and provide potential directions of future development in a taskoriented dialogue system @xcite @xcite @xcite . compared with normal intent detection tasks, we don't know the exact number and lack labeled data for unknown intents, which makes it challenging to identify ood samples in the task-oriented dialog. previous ood detection works can be generally classified into two types: supervised @xcite @xcite and unsupervised @xcite @xcite @xcite @xcite ood detection. the former indicates that there are extensive labeled ood samples in the training data. @xcite ; @xcite , form a (n+1)-class classification problem where the (n+1)-th class represents the ood intents. further, @xcite uses labeled ood data to generate an entropy regularization term. but these methods require numerous labeled ood intents to get superior performance, which is unrealistic. we focus on the unsupervised ood detection setting where labeled ood samples are not available for training. unsupervised ood detection first learns discriminative representations only using labeled ind data and then employs scoring functions, such as maximum softmax probability (msp) @xcite , local outlier factor (lof) @xcite , gaussian discriminant analysis (gda) @xcite to estimate the confidence score of a test query. all these unsupervised ood detection methods only focus on the improvement of a single aspect of representation learning or scoring function, but none of them consider how to align representation learning with scoring functions. for example, @xcite proposes a local outlier factor for ood detection, which considers the local density of a test query to determine whether it belongs to an ood intent, but the ind pre-training objective lmcl @xcite cannot learn neighborhood discriminative representations. @xcite ; @xcite employ a gaussian discriminant analysis method for ood detection, which assumes that the ind cluster distribution is a gaussian distribution, but they use a cross-entropy or supervised contrastive learning @xcite objective for representation learning, which cannot guarantee that such an assumption is satisfied. the gap between representation learning and scoring function limits the overall performance of these methods. to solve the conflict, in this paper, we propose a unified neighborhood learning framework (uninl) for ood detection, which aims to align ind pre-training representation objectives with ood scoring functions. our intuition is to learn neighborhood knowledge @xcite to detect ood intents. for ind pre-training, we introduce a k-nearest neighbor contrastive learning objective (kncl) to learn neighborhood discriminative representations. compared to scl @xcite which draws all samples of the same class closer, kncl only pulls together similar samples in the neighbors. to align kncl, we further propose a k-nearest neighbor scoring function, which estimates the test sample confidence score by computing the average distance between a test sample and its k-nearest neighbor samples. the kncl objective learns neighborhood discriminative knowledge, which is more beneficial to promoting knn-based scoring functions. our contributions are three-fold: (1) we propose a unified neighborhood learning framework (uninl) for ood detection, which aims to match ind pre-training objectives with ood scoring functions. (2) we propose a k-nearest neighbor contrastive learning (kncl) objective for ind pre-training to learn neighborhood discriminative knowledge, and a knn-based scoring function to detect ood intents. (3) experiments and analysis demonstrate the effectiveness of our method for ood detection. ## approach overall architecture fig 1 shows the overall architecture of our proposed uninl, which includes k-nearest contrastive learning (kncl) and knnbased score function. we first train an in-domain intent classifier using our kncl objective in the training stage, which aims to learn neighborhood discriminative representation. then in the test stage, we extract the intent feature of a test query and employ our proposed knn-based score function to estimate the confidence score. we aim to align representation learning and scoring functions. knn contrastive representation learning existing ood detection methods generally adopt cross-entropy (ce) @xcite and supervised contrastive learning (scl) @xcite objectives for representation learning. both ce and scl tend to bring all samples of the same classes closer, and samples of different classes are pushed embedding layer in-domain utterances contextual encoder ce loss kncl loss ind @xmath0. they learn inter-class discriminative features in a global representation space. however, we find that when performing ood detection, we care more about the data distribution within the neighborhood of a given sample. inspired by @xcite , we hope to learn neighborhood discriminative knowledge in the ind pre-training stage to facilitate ood detection. we propose a knearest neighborhood contrastive learning (kncl) objective to learn discriminative features in a local representation space. given an ind sample x i , we firstly obtain its intent representation z i using a bilstm @xcite or bert @xcite encoder. next, we perform kncl as follows: ## conclusion in this paper, we focus on how to align representation learning with the scoring function to improve ood detection performance. we propose a unified neighborhood learning framework (uninl) to detect ood intents, in which a kncl objective is employed for ind pre-training and a knn-based score function is used for ood detection. experiments and analyses confirm the effectiveness of uninl for ood detection. we hope to provide new guidance for future ood detection work.