Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers (2023)

Abstract: For real-world language applications, detecting an out-of-distribution (OOD) sample is helpful to alert users or reject such unreliable samples. However, modern over-parameterized language models often produce overconfident predictions for both in-distribution (ID) and OOD samples. In particular, language models suffer from OOD samples with a similar semantic representation to ID samples since these OOD samples lie near the ID manifold.A rejection network can be trained with ID and diverse outlier samples to detect test OOD samples, but explicitly collecting auxiliary OOD datasets brings an additional burden for data collection. In this paper, we propose a simple but effective method called Pseudo Outlier Exposure (POE) that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes. The surrogate OOD sample introduced by POE shows a similar representation to ID data, which is most effective in training a rejection network. Our method does not require any external OOD data and can be easily implemented within off-the-shelf Transformers.A comprehensive comparison with state-of-the-art algorithms demonstrates POE’s competitiveness on several text classification benchmarks.

## introduction pre-trained language models (plms) have achieved remarkable success in various natural language processing (nlp) tasks such as questionanswering @xcite , sentiment analysis @xcite , and text categorization @xcite . while plms have become a de-facto standard promoting classification accuracy, recent studies have found that over-parameterized plms often produce overconfident predictions for out-of-distribution (ood) samples @xcite . for real-world language applications, these unreliable predictions can confuse users when interpreting the model's decisions. therefore, language models require the ability to detect ood samples to instill the reliability in nlp applications. the task of detecting ood samples can be formulated as a binary hypothesis test of detecting whether an input data is from in-distribution (id) or ood. to detect an outlier data, in machine learning communities, the ood detection task has been studied for many years @xcite @xcite . the prior works have proposed effective methods, including post-hoc algorithms @xcite , and training a rejection network by exposing the model to external ood datasets @xcite . however, existing post-hoc methods usually require a subset of actual ood samples to tune their hyperparameters @xcite , especially, @xcite find that hyperparameters tuned with limited ood dataset are not generalized to others. thus, these methods are not feasible in real-world applications; moreover, we often cannot know the entire distribution of ood datasets. similarly, training a rejection network not only brings an additional burden for ood data collection but also may result in sub-par ood detection performance in deciding which subset of external data to use. intuitively, ood examples that are excessively distant from training samples may not help with ood detection because easy-tolearn outlier features can be discriminated rather trivially. therefore, a desirable trait for ood sam-ples to effectively train rejection networks is that the ood sample does not belong to id but is sufficiently close to the distribution of id samples @xcite . in this paper, we primarily focus on detecting ood samples by constructing a surrogate ood dataset from training samples rather than using external ood data to train a rejection network. to this end, we propose pseudo outlier exposure (poe) which is a procedure to construct a near-ood set by erasing tokens with high attention scores in training sentences. a rejection network can then be trained on the training (id) and constructed ood datasets to detect ood samples. numerical experiments confirm that our procedure indeed generates surrogate ood data close to id examples. accordingly, a rejection network trained on this construction outperforms state-of-the-art ood detection algorithms on several benchmarks. our main contributions are: ## method given transformer-based plms with the softmax classifier, we propose a simple but effective method for detecting ood samples. we first introduce the proposed method for generating surrogate ood data and then present a rejection network that is trained with id and the surrogate ood. notation. let @xmath0, and @xmath1, ..., k} be a label. for multi-class classification tasks, bert-style transformer f can be decomposed by the attention blocks and the last dense layer. we denote each layer as f att , and f out , respectively. unless otherwise mentioned, the output of f att (•) denotes the [cls] feature vector on the last attention block. the cast is fantastic, and it is very impressive movie. the cast is [mask] and it is very impressive movie. the cast is [mask] and it is very [mask] movie. the cast is [mask] and it is [mask] [mask] movie. t 1 t 2 t 3 t 4 token replacement feature space of the plm high density low density t 4 : t 3 : t 2 : t 1 : (ood) (id) in the text boxes, the blue words denote tokens with high attention scores, and the darker words represent higher attention scores than others. ## result in this section, we present comprehensive experimental evaluations of poe. we compare poe with baselines for clinc datasets (sec. 5.1), followed by empirical results for semantic and background shift benchmarks (sec. 5.2) and detailed analysis (sec. 5.4). due to the space limitation, we evaluate all methods based on roberta in this section, and the experimental results based on bert are reported in appendix. ## conclusion in this paper, we propose a simple and intuitive ood construction to train a rejection network. motivated by the previous observation that ood samples are most effective when semantically similar to id samples, poe detects and erases tokens with high attention scores of plms. its resultant surrogate ood dataset is close to the distribution of id samples that have been observed to improve the ood detection performance of the rejection network. extensive experiments conducted on challenging id/ood pairs show poe's competitiveness.