Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks (2023)

Abstract: State-of-the-art models can perform well in controlled environments, but they often struggle when presented with out-of-distribution (OOD) examples, making OOD detection a critical component of NLP systems. In this paper, we focus on highlighting the limitations of existing approaches to OOD detection in NLP. Specifically, we evaluated eight OOD detection methods that are easily integrable into existing NLP systems and require no additional OOD data or model modifications. One of our contributions is providing a well-structured research environment that allows for full reproducibility of the results. Additionally, our analysis shows that existing OOD detection methods for NLP tasks are not yet sufficiently sensitive to capture all samples characterized by various types of distributional shifts. Particularly challenging testing scenarios arise in cases of background shift and randomly shuffled word order within in domain texts. This highlights the need for future work to develop more effective OOD detection approaches for the NLP problems, and our work provides a well-defined foundation for further research in this area.

## introduction systems based on artificial intelligence (ai) have to be safe and trustworthy @xcite . ensuring user reliance on these systems requires a cautious approach in making predictions. ai tools should avoid decisions on examples that significantly deviate from the training data. this is especially risky when the classifier shows excessive confidence in its incorrect decisions, leading to the propagation of errors in the system pipeline @xcite . however, current models are often trained under the closed-world assumption, limited to specific domains @xcite . test sets drawn from the same domain for evaluation may not reflect real-world scenarios accurately @xcite . this poses challenges when deploying such models in production environments @xcite . real-world data is often completely different from training one. the change in data distribution can be caused by several factors such as user behavior, legal regulations, market trends or seasonal changes. in an open-world scenario, the ai-based system can be even exposed to inputs that deviate from the trained task. a significant risk that may arise is the possibility of model overconfidence while predicting data of this nature. as a result, there is a business need for detecting examples outside the domain @xcite . out-of-distribution (ood) detection techniques can be well applied in a production system with human-in-the-loop technology @xcite , where it is important to quickly identify whether an input sample is characterized by a distributional shift. such an example should be handled then by a human expert in order to avoid potential misclassification by the model. the essence of such systems is to find a trade-off between the accuracy and automation @xcite (figure 1 ). this way, the model can achieve the highest possible performance on in-distribution (id) data and difficult shifted data can be given to human verification, thus increasing the credibility of the overall system. the bottleneck here is a well-designed ood detection method, which must be sensitive enough to capture all examples outside the domain. the problem of ood identification is mainly investigated for vision classification tasks @xcite , whereas in the field of nlp, studies on this topic are limited. we fill the missing gap by proposing a comprehensive analysis of existing ood approaches for text classification tasks. in this work, we focus on the post-hoc techniques which are most suitable for business applications i.e. they have to fulfil the requirement of smooth integration into existing systems, without the need for additional ood training data or any changes in model architecture. ultimately, we evaluated eight methods in two different scenarios. the first one includes grouping test data into three splits according to the similarity to the in-distribution set: near-ood, far-ood and distinct-ood @xcite . the ai system is evaluated based on the degree of domain difference between training and test samples. the second scenario considers the division of datasets according to the shift of distribution @xcite . there are many categories of distribution shift @xcite , but in this study, we consider two types -semantic and background. semantic shift occurs when new labels appear, which may be due to the lack of a sufficient number of classes representing the training data or the emergence of new classes over time. in distinction, the background shift is class independent. it appears when the characteristic features of text change (e.g. source origin, writing style), which can happen even within the same class. the reason may be language evolution, regional conditions, etc. -such factors are difficult to predict and adequately address in the training set. by preparing data separated into different kinds of shift, we gain an in-depth insight into the origin of the data, on which a particular ood detection method performs better or worse. we also provide a well-structured research environment that allows the full reproducibility of the achieved outcomes and evaluation of another nlp models. the source code is available on github 1 . to summarize, our contribution is as follows: ## related work in recent years, there has been a growing interest in developing robust methods that can detect outof-distribution examples. the work of @xcite has played a significant role in advancing this field. their maximum softmax probability (msp) method, which relies on the softmax output of a neural network, has become a reference for subsequent research and still remains as the solid baseline approach @xcite . the benefit of the msp was its independence from the specific task domain. since then, many researchers have extended this method or proposed novel techniques to address the challenge of detecting ood data. the first to popularize the interest in the ood topic were computer vision (cv) researchers @xcite . the emerged techniques in this field were summarized in a survey by @xcite . the authors proposed a unified framework that groups ood detection methods into categories based on their common underlying mechanisms. among them, the following ones can be distinguished: (1) output-based @xcite techniques which detect ood samples based on output vector obtained by classification model for given input; (2) gradient-based @xcite focus on analyzing the fluctuation of the gradient flow through the model layers to verify that the input is ood; (3) density-based @xcite methods involve modeling a density function from the training set and then determining whether a new example belongs to the same distribution; (4) distance-based @xcite measure the dissimilarity between a new input and the training data by computing standard metrics such as cosine similarity, euclidean or mahalanobis distance. another work of @xcite provides a comprehensive evaluation of 13 methods for ood detection in cv. notably, the experimental results show that simple preprocessing techniques can be highly effective, outperforming even more sophisticated methods in identifying ood examples. in addition, post-hoc methods have demonstrated considerable effectiveness in ood detection and have made significant impact in this task. the nlp community is also more and more interested in addressing the challenge of ood detection data, especially after the appearance of text processing automation systems. despite the expectation that pre-trained language models (plms) would generalize well to unseen data, many existing transformer-based architectures perform poorly in an open-world assumption setup. this was proven by the work @xcite where the authors created the glue-x benchmark to reliably test the robustness of plms against ood samples exposure, without using any of the previously mentioned techniques dedicated to ood. their achieved results confirm the necessity of further development of ood detection methods. currently, researchers are continuously proposing techniques tailored for the nlp tasks @xcite , revisiting existing ones @xcite or designing completely novel approaches that can address specific shifts in data distribution @xcite . the latter two publications particularly highlight the importance of dividing datasets into semantic and background shift sets, as they provide valuable findings and a better understanding of how the model works on different data types. evidently, there have been several nlp articles addressing ood detection, but their comparison to existing methods has been limited. a comprehensive study which evaluates various ood detection approaches on a larger scale and addressing the specific needs of businesses is still lacking. to fill this gap, we have developed a benchmark that provides a fair comparison of these techniques while testing their performance across different distributional shift scenarios. all the selected methods have been inspired by cv achievements, and we have specifically chosen those that can be easily integrated into an existing ai system with minimal complexity. ## benchmark outline this section provides an overview of the datasets and the model architecture, with a detailed description of the techniques reimplemented in our benchmark for detecting out-of-domain examples. the metrics used for evaluating the effectiveness of the detection methods are also presented. ## data preparation in our study, we have paid particular attention to provide a complete and unbiased comparison of ood detection methods. to achieve this goal, we adopted two diverse perspectives: one inspired by the field of computer vision @xcite and the other drawn from works dedicated to the nlp domain @xcite . ## experiments in this section, we describe the details of a training procedure and present the outcomes from the experiments. ## conclusions the latest advancements in ood detection techniques have surpassed the conventional msp baseline. in this work, we applied some of them to the nlp classification problems, selecting only posthoc approaches because of their easy integration to already trained plm model. most of the examined techniques achieved better results than the msp, but their performance varied when subjected to different types of data distributional shift. background shift was particularly challenging for the majority of methods to properly distinguish ood examples. the knn and vim methods were found to be the most effective, and their performance was also stable. hence, they are better alternatives to msp for out-of-distribution detection. however, it should be kept in mind that it is likely that the vim method is sensitive to cases where the language model was trained on short texts and later exposed to a long text from outside the domain. the proposed by us the unique analysis of distinct-ood scenario, allowed to draw interesting findings. the tested methods were able to identify texts in different languages very easily as a ood examples, but they had problems detecting ood on the news category random with shuffled data. this means that plm models, despite their ability to detect contextual nuances in text, still tends to behave like bag-of-words @xcite in text classification tasks. business-wise, such structurally disturbed examples should not be further processed by ai systems. therefore, ood methods employed in nlp should better address semantic disorders in input sentences. in conclusion, the overall performance of current ood detection techniques is still low and unsatisfactory, particularly when presented with the near-ood samples. further research is necessary for the development of ood detection methods, es-pecially in the field of nlp, where more and more document processing automation systems are being developed, where ensuring reliability is important for users. our work addresses the need for a comprehensive framework to evaluate the quality of ood detection and provides easy extensibility to emerging methods.