Multi-Level Knowledge Distillation for Out-of-Distribution Detection in Text (2023)

Abstract: Self-supervised representation learning has proved to be a valuable component for out-of-distribution (OoD) detection with only the texts of in-distribution (ID) examples. These approaches either train a language model from scratch or fine-tune a pre-trained language model using ID examples, and then take the perplexity output by the language model as OoD scores. In this paper, we analyze the complementary characteristic of both methods and propose a multi-level knowledge distillation approach that integrates their strengths while mitigating their limitations. Specifically, we use a fine-tuned model as the teacher to teach a randomly initialized student model on the ID examples. Besides the prediction layer distillation, we present a similarity-based intermediate layer distillation method to thoroughly explore the representation space of the teacher model. In this way, the learned student can better represent the ID data manifold while gaining a stronger ability to map OoD examples outside the ID data manifold with the regularization inherited from pre-training. Besides, the student model sees only ID examples during parameter learning, further promoting more distinguishable features for OoD detection. We conduct extensive experiments over multiple benchmark datasets, i.e., CLINC150, SST, ROSTD, 20 NewsGroups, and AG News; showing that the proposed method yields new state-of-the-art performance. We also explore its application as an AIGC detector to distinguish answers generated by ChatGPT and human experts. It is observed that our model exceeds human evaluators in the pair-expert task on the Human ChatGPT Comparison Corpus.

## introduction machine learning systems such as dialog agents are widely used in many real-world applications. 1 our code is available at https://github.com/ microsoft/kc/tree/main/papers/mlkd_ood . these systems have proved to work well when the distributions of training data and test data are the same or closely similar. however, when there is a gap between training distribution and test distribution, trained models may generate dubious, and even disastrous, predictions that could cause serious ai safety issues @xcite . therefore, it is crucial to detect out-of-distribution (ood) inputs for deployed machine learning systems. moreover, lifelong learning systems are usually required to discover ood examples during their application to create new tasks and learn them incrementally @xcite , which further highlights the importance of ood detection. in this paper, we focus on the task of ood detection with only in-distribution texts available during learning for its capability of dealing with diverse scenarios such as non-classification applications while requiring the least data collection effort. recent studies have well demonstrated the validity of self-supervised representation learning @xcite @xcite . these approaches use id examples to either fine-tune a large pre-trained language model (m finetune ) @xcite or to train a language model from scratch @xcite (m fromscratch ). given an input sequence for inference, token perplexity output by the learned/fine-tuned language model is regarded as the ood score, i.e., indication of an example being ood. however, both methods have limitations. for m finetune , since the pre-training corpus usually consists of huge-scale datasets from a diverse range of genres, it is possible that some ood examples are seen during pre-training, leading to a risk of non-distinguishing perplexities between id examples and these "leaked" ood examples as shown in figure 1a . this issue is eliminated in m fromscratch which only sees id examples during training. however, using only id examples to minimize the self-supervised language modeling loss without any other constraints may result in a less compact representation space of id data. consequently, ood examples have more chance to locate in the id data manifold, leading to the overlap between perplexity distributions of id examples and ood examples as shown in figure 1b . inspired by @xcite , which indicates that unsupervisedly trained sentence embeddings (mean pooling over all token representations) @xcite can achieve non-trivial performance in the sentence classification task, here we contemplate that the pre-training procedure of language models can facilitate their ability of capturing semantic relatedness. in other words, language models are promoted to map examples with different semantics to different manifolds via pre-training. therefore, we suggest inheriting the representation space with such characteristics gained from pretraining to mitigate the limitation of m fromscratch . in this paper, we propose to adopt multi-level knowledge distillation to integrate the strengths from both methods while mitigating their limitations. specifically, we first produce a teacher model by fine-tuning a large pre-trained language model with id training examples, so that features of the teacher model can well represent the id data manifold, while to some extent preserving the ability to map examples with different semantics to different manifolds. then, we perform knowledge distillation to learn a student model from scratch, using id training examples with supervision from the fine-tuned teacher model. to learn the teacher's representation space more thoroughly, we not only perform prediction layer distillation, but also propose a similarity-based intermediate layer distillation method to make the student model aware of the information flow inside the teacher's layers. finally, we deploy the learned student model to compute token perplexity for each inference example as its ood score and compare it with a threshold to determine whether it is ood or not. in contrast to m finetune , our student model doesn't see any ood examples during parameter learning, thus avoiding the leakage of ood examples. compared with m fromscratch , our student model is trained with the regularization inherited from pre-training via the multi-level supervision from the teacher model, thus gaining a stronger ability to map ood examples outside the id data manifold. both are conducive to more distinguishable representations for ood detection. moreover, with the development of automatic text generation technologies such as instructgpt @xcite and chatgpt foot_1 , the risk of automatically generated content to society (e.g., generating fake news or fake reviews of products) is increasing. therefore, we further adapt our model to distinguish texts generated by ai models and human experts. by conducting experiments on the human chatgpt comparison corpus (hc3), we observe that our model beats human evaluators and shows excellent capability in the pair-expert task. our major contributions can be summarized as: considering the accessibility of ood data and class labels of id data, previous work for ood detection can be divided into three categories: i) ood data available; ii) ood data unavailable, but class labels of id examples available; and iii) both types of data unavailable. methods with supervision from ood data. these methods usually train a binary classifier @xcite or a multi-class classifier @xcite to detect ood examples, where ood data is regarded as an independent class for training. ood data used as supervision is collected from other existing datasets that are disjoint with the id training data @xcite . some previous work also introduces synthesized pseudo outliers to try to find a more representative classification hyperplane for ood data @xcite . since there are various reasons for an example to be considered ood, e.g., being out-of-domain @xcite , infrequent @xcite , or adversarial @xcite , it is impractical to collect ood data for learning. methods without supervision from ood data but with supervision from id class labels. these approaches generally consider the scenario of multiclass classification such as intent detection @xcite and assume that class labels of in-distribution (id) data are available during model training. class probabilities @xcite @xcite @xcite and distance or density in latent space @xcite @xcite @xcite are the most prevalent metrics. in particular, @xcite propose a strong baseline which takes the maximum softmax probability (msp) of a multi-class classifier as a measure of ood score. based on that, lots of following studies devote to optimizing the model's calibration with temperature scaling @xcite , contrastive learning @xcite , etc. for distance and density based approaches, they first learn discriminative deep features via carefully designed loss functions, e.g., large margin cosine loss @xcite and contrastive loss @xcite . then, compute distance or density metrics such as local outlier factor (lof) @xcite @xcite and gaussian discriminant analysis @xcite @xcite to detect ood examples. methods without supervision from both ood data nor id class labels. given the indistribution data, these methods generally estimate id density and regard test examples that deviate from the estimated distribution as ood examples. previous work for this setting mainly focuses in the field of computer vision. variational autoencoders (vae) (kingma and welling, 2014) and generative adversarial networks (gan) are frequently taken as the backbone models for density estimation @xcite . in natural language processing, current studies generally perform selfsupervised language modeling on id examples and take token perplexity as ood score @xcite @xcite . @xcite further introduces an independent background model to correct confounding background statistics. moreover, @xcite learn a combination of latent representations from different layers of pre-trained transformers to represent id data manifold in a compact way. based on that, one-class classification methods such as one-class svm @xcite limitations. to our best knowledge, this is the first attempt to adapt knowledge distillation to textual ood detection. compared with @xcite , our approach does not involve any hyper-parameter sensitive one-class classification stage. compared with @xcite , our proposed method requires no prior knowledge about the id data, e.g., the number of semantic categories. moreover, our approach is orthogonal to @xcite and we can combine both to achieve better performance. ## methodology in this section, we elaborate on the proposed multilevel knowledge distillation approach for ood detection. first, we clarify how to produce a teacher model that estimates the distribution of id data. then, we describe the proposed multi-level knowledge distillation procedure to teach a randomly initialized student model via the produced teacher model. note that in this paper, both the teacher and student networks are built with transformer layers @xcite . figure 2 illustrates the overall framework. ## conclusion in this paper, we focus on the setting of ood detection without supervision from both ood data nor id class labels. we analyze the complementary characteristics of existing self-supervised representation learning-based methods and propose a multi-level knowledge distillation approach to integrate their strengths, while mitigating their limitations. we evaluate the proposed method on multiple datasets and results show that the proposed method yields new state-of-the-art performance. we analyze why our approach attains superior performance by conducting ablation studies and sentence representation visualization. we further apply our model as an aigc detector to distinguish chatgpt-generated texts from those generated by human experts and the experimental results demonstrate that our model outperforms human evaluators in the setting of paired answers.