Unsupervised Out-of-Domain Detection via Pre-trained Transformers (2021)

Abstract: Deployed real-world machine learning applications are often subject to uncontrolled and even potentially malicious inputs. Those out-of-domain inputs can lead to unpredictable outputs and sometimes catastrophic safety issues. Prior studies on out-of-domain detection require in-domain task labels and are limited to supervised classification scenarios. Our work tackles the problem of detecting out-of-domain samples with only unsupervised in-domain data. We utilize the latent representations of pre-trained transformers and propose a simple yet effective method to transform features across all layers to construct out-of-domain detectors efficiently. Two domain-specific fine-tuning approaches are further proposed to boost detection accuracy. Our empirical evaluations of related methods on two datasets validate that our method greatly improves out-of-domain detection ability in a more general scenario.

## introduction deep neural networks, despite achieving good performance on many challenging tasks, can make overconfident predictions for completely irrelevant and out-of-domain (ood) inputs, leading to significant ai safety issues @xcite . detecting out-of-domain inputs is a fundamental task for trustworthy ai applications in realworld use cases, because those applications are often subject to ill-defined queries or even potentially malicious inputs. prior work on out-of-domain detection (e.g., @xcite @xcite @xcite ) mostly requires indomain task labels, limiting its usage to supervised classification. however, deployed applica-tions rarely receive controlled inputs and are susceptible to an ever-evolving set of user inputs that are scarcely labeled. for example, for many nonclassification tasks, such as summarization or topic modeling, there are no available classifiers or task labels, which limits the practical usage of recently proposed out-of-domain detection methods. therefore, it is natural to ask the following question: can we detect out-of-domain samples using only unsupervised data without any in-domain labels? we regard the out-of-domain detection problem as checking whether the given test samples are drawn from the same distribution that generates the in-domain samples, which requires a weaker assumption than prior work (e.g., @xcite . we suppose that there are only in-domain samples, which allows us to understand the properties of data itself regardless of tasks. therefore, methods developed for this problem are more applicable than task-specific ones and can be further adapted to tasks where no classification labels are present, such as active learning or transfer learning. to solve the problem, we utilize the latent embeddings of pre-trained transformers (e.g., @xcite @xcite to represent the input data, which allow us to apply classical ood detection methods such as one-class support vector machines @xcite or support vector data description @xcite on them. however, the best practice on how to extract features from bert is usually task-specific. for supervised classification, we can represent the text sequence using the hidden state of @xcite token from the top layer. meanwhile bert's intermediate layers also capture rich linguistic information that may outperform the top layer for specific nlp tasks. by performing probing tasks on each layer, @xcite suggest bottom layers of bert capture more surface features, middle layers focus more on syntax and semantic features are well represented by top ones. as no prior knowledge about ood samples is usually provided in practice, deciding which layer of features is the most effective for ood detection is itself non-trivial. some ood samples may just contain a few out-of-vocabulary words; while others are ood due to their syntax or semantics. based on the observations above, this paper studies how to leverage all-layer features from a pretrained transformer for ood detection in an unsupervised manner. our contributions are three-fold: ## problem setup assume that we have a collection of text inputs d n := {x i } n @xmath0, we want to construct an out-ofdomain detector that takes an unseen new input u and determines whether u u u comes from the same distribution that generates d n . we adopt a more practical setting where we have no prior knowledge of what out-of-domain inputs look like. in this case, training a domain classifier directly is not feasible. the out-of-domain detector can be described mathematically as: there are two different scenarios, considering if we have any in-domain labels for data x @xmath1. here we define in-domain labels as any specific supervised task labels, such as sentiments, intents or topics of the text. with in-domain labels suppose that we have multi-class label y @xmath2, y i )} n @xmath3. given a classifier h trained with d n , we can use maximum calibrated softmax probability with temperature scaling as the anomaly score @xcite : ## model and feature learning bert and its variants such as roberta (e.g., @xcite are pre-trained on large-scale public data (denoted as d pub ) using self-supervised tasks, such as language model and next sentence prediction. these models show ## experiments datasets we consider two distinct datasets for experiments, where one is to regard text from unseen corpora as ood, and the other one is to detect class-level ood samples within the same corpus. examples taken from the two datasets can be found in ## results and discussions in this section, we present the results for our experiments and summarize our findings. the best f ℓ (x x x) for ood is dataset-specific for the cross-corpus dataset (sst), we find that the best results come from the top layer of both (ro)bert(a). however, for the cross-intent dataset (clinic150), the middle layers perform the best when using [cls], while the bottom layers achieve the best results with avg. this indicates that ood distributions are not simply based on certain types of linguistic features and the strategy of choosing f ℓ (x x x) is dataset-specific; for some dataset, semantic features play a more important role, while sometimes we need to focus on syntactic or lexical features. this validates the assumption that it is beneficial to fully utilize all layers of the hidden representations from pre-trained transformers to detect ood instances. we find using f ℓ (x x x) of bert is generally better than roberta, especially with @xcite . we guess next sentence prediction may cause this, which pretrains on [cls] and is exclusive for bert. in later sections, (ro)bert(a)-single layer will refer to the best one in ## related work out-of-domain detection is essentially an important component for trustworthy machine learning applications. there are two lines of work proposed to perform out-of-domain detection. one is to tackle the problem in specific multi-class classification tasks, where well-trained classifiers are utilized to design anomaly scores (e.g., @xcite @xcite @xcite , those methods can only be useful when multi-class labels are available, which limits their application in more general domains. our proposed work goes beyond this limitation and can utilize large amounts of unsupervised data. another line of work is based on support estimation or density estimation, which assumes that the in-domain data is in specific support or from the high density region @xcite . in principle, our work is closely related to this line of work. besides, @xcite al. (2016); ruff et al. (2018); zong et al. (2018) also leverage the features of neural networks, though these methods require designing specific network structures for different data. our work circumvents the issues of prior work by designing a computationally efficient method that leverages the powerful representations of pre-trained transformers. finally, the fine-tuning techniques we use to improve the representation of data are closely related to unsupervised pre-training for transformers @xcite , and recently proposed contrastive learning (e.g., @xcite . lately, @xcite discover that performing pre-training (mlm) on the target domain with unlabeled data can also help to improve downstream classification performance. to the best of our knowledge, our method is the first to incorporate transformers and pre-training techniques to improve out-of-domain detection. ## conclusion we study the problem of detecting out-of-domain samples with unsupervised in-domain data, which is a more general setting for out-of-domain detection. we propose a simple yet effective method using mahalanobis distance as features, which significantly improves the detection ability and reduces computational cost in learning the detector. two domain-adaptive fine-tuning techniques are further explored to boost the detection performance. in the future, we are interested in deploying our ood method to real-world applications, such as detecting unseen new classes for incremental fewshot learning @xcite or filtering ood samples in data augmentations.