Beyond Mahalanobis-Based Scores for Textual OOD Detection
Abstract
Deep learning methods have boosted the adoption of NLP systems in real-life applications.
However, they turn out to be vulnerable to distribution shifts over time which may cause severe dysfunctions in production systems, urging practitioners to develop tools to detect out-of-distribution (OOD) samples through the lens of the neural network.
In this paper, we introduce TRUSTED , a new OOD detector for classifiers based on Transformer architectures that meets operational requirements: it is unsupervised and fast to compute.
The efficiency of TRUSTED relies on the fruitful idea that all hidden layers carry relevant information to detect
OOD examples.
Based on this, for a given input, TRUSTED consists in (i) aggregating this information and (ii) computing a similarity score by exploiting the training distribution, leveraging the powerful concept of data depth .
Our extensive numerical experiments involve 51k model configurations, including various checkpoints, seeds, and datasets, and demonstrate that TRUSTED achieves state-of-the-art performances.
In particular, it improves previous AUROC over 3 points.
Introduction
The number of AI systems put into production has steadily increased over the last few years.
This is because advanced techniques of Machine Learning (ML) and Deep Learning (DL) have brought significant improvements over previous state-of-the-art (SOTA) methods in many areas such as finance [17, 57], transportation [59] and medicine [14, 77].
However, the increasing use of black-box models has raised concerns about their societal impact: privacy [33, 64, 74, 29], security [6, 3, 20], safety [41, 50, 18], fairness [5, 72], and explainability [13, 63] which became areas of active research in the ML community.
This paper is about a critical safety issue, namely Out-Of-Distribution (OOD) detection [11], which refers to a change of distribution of incoming data that may cause failures of in-production AI systems.
When data are tabular and of small dimension, simple statistical methods can be efficient, and one may, for instance, monitor the mean and variance of each marginal over time.
However, these traditional methods do not work anymore when data are high-dimensional and/or unstructured.
Thus, the need to design new techniques that incorporate incoming data and the neural networks themselves.
Distinguishing OOD examples from in-distribution (ID) examples is challenging for modern deep neural architectures.
DL models transform incoming data into latent representations from which reliable information extraction is cumbersome.
Because of that, designing new tools of investigation for large pretrained models, judiciously named foundation models by [9], is an essential line of research for the years to come.
In computer vision, methods are more mature thanks to the availability of appropriate deformation techniques that allow for sensitivity analysis.
In contrast, the nature of tokens in Natural Language Processing (NLP) makes it more difficult to develop such suitable methods.
In this paper, we focus on classifiers for textual data and on the ubiquitous BERT [34], DistilBERT [83], and RoBERTa [68] architectures.
Existing methods can be grouped according to their positioning with respect to the network.
Some works exploit only the incoming data and compare them with in-distribution examples through likelihood ratios [42, 19].
Another line of research consists in incorporating robust constraints during training, with [48] or without [103, 60] access to some available OOD examples.
Another line of research focuses on post-processing methods that can be used on any pretrained models.
In our view, these are the most promising tools because typical users rely on transformers without retraining.
Within post-processing methods, one can distinguish softmaxbased tools that compute a confidence score based on the predicted probabilities and threshold to decide whether a sample is OOD or not.
Notice that this does not require direct access to indistribution data.
The seminal work is due to Hendrycks [47] who uses the maximum soft-probability, and has been pushed further in [62, 52].
In [67], authors suggest looking one step deeper into the network, namely, to compute a confidence score based on the projections of the pre-softmax layer.
More recently, [75] achieved SOTA results on transformer-based encoder by computing the Mahalanobis distance [71, 32] between a test sample and the in-distribution law [58], estimated through accessible training data points.
Nevertheless, the distance-based score is computed on the last-layer embedding only, suggesting that going deeper inside the network might improve OOD detection power.
Moreover, the computation of
Mahalanobis-based scores requires inverting the covariance matrix of the training data, which can be prohibitive in high dimensions.
It is worth noticing that the Mahalanobis-based scores can be seen as a data depth [97, 104] through a simple re-scaling [66], that is a statistical function measuring the centrality of an observation with respect to a probability distribution.
Although data depths are quite natural in the context of OOD detection, they remain overlooked by the ML community.
In the present work, we rely on the recently introduced Integrated Rank-Weighted depth [76, 94] in order to remedy the drawbacks of the Mahalanobis-based scores for OOD detection.
1.1 Our contribution
We first leverage the observation introduced by previous work that all hidden layers of a neural network carry useful information to perform textual OOD detection.
For a given input x , our method consists in computing its average latent representation x
and then its OOD score through the depth score of x with respect to the averaged in-distribution law (see Fig.
1 for an illustration).
Notice that the ability to compute averaged latent representations crucially relies on the structure of transformers layers that share the same dimension.
The depth function we are using is based on the computation of the projected ranks of the test inputs using randomly sampled directions.
From a theoretical viewpoint, this novel method requires fewer assumptions on the data structure than the Mahalanobis score.
We conduct extensive numerical experiments on three transformers architectures and eight datasets and benchmark our method with previous approaches.
To ensure reliable results, we introduce a new framework for evaluating OOD detection that considers hyperparameters that were unreported before.
It consists of computing performances for various choices of checkpoints and seeds, which allows us to report a variance term that makes some previous methods fall within the same performance range.
Our conclusions are drawn by considering over 51k configurations , and show that our new detector based on data depth improves SOTA methods by 3 AUROC points while having less variance.
This result supports the intuition that OOD detection is a matter of looking at the information available across the entire network .
Our contribution can be summarized as follows:
We introduce a novel OOD detection method for textual data.
Our detector TRUSTED 1 relies on the full information contained in pretrained transformers and leverages the concept of data depth: a given input is detected as being in-distribution or OOD sample based on its depth score with respect to the training distribution.
We conduct extensive numerical experiments and prove that our method improves over SOTA methods.
Our evaluation framework is more reliable than previous studies as it includes the variance with respect to seeds and checkpoints.
We release open-source code and data to ease future research , ensure reproducibility and reduce computation overhead.
Problem Formulation
Training distribution and classifier.
Let us denote by X the textual input space.
Consider the multiclass classification setup with target space Y = { 1 , . .
, C } of size C ≥ 2 .
We assume the dataset under consideration is made of N ≥ 1 i.i.d.
samples ( x 1 , y 1 ) , . .
, ( x N , y N ) with probability law denoted by p XY and defined on X × Y .
Accordingly, we will denote by p X and p Y the marginal laws of p XY .
Finally, we denote by f N : X → Y the classifier that has been trained using ( x i , y i ) .
Open world setting.
In real-life scenarios, the trained model f N is deployed into production and will certainly be faced with input data whose law is not p XY .
To each test point ( x , y ) , we associate a variable z ∈ { 0 , 1 } such that z =
if ( x , y ) stems from p XY and z = 1 otherwise.
It is worth emphasizing that in our setting f N has never been faced with OOD examples before deployment.
This is usually referred to as the open-world setting.
From a probabilistic viewpoint, the test set distribution of the input data p test X is a mixture of in-distribution and OOD samples:
[Equation]
where α ∈ (0 , 1) .
In this work, we will not make any further assumptions on the proportion α of
OOD samples and on the OOD pdf p X | Z
( x | z = 1) , making the problem more difficult but at the same time more well suited for practical use.
Indeed, for textual data, it does not appear to be realistic to model how a corpus can evolve.
OOD detection.
The objective of OOD detection is to construct a similarity function s : X → R + that accounts for the similarity of any element in X with respect to the training in-distribution.
For a given test input x , we then classify x as in-distribution or OOD according to the magnitude of s ( x ) .
Therefore, one fixes a threshold γ and classifies IN ( i.e. ˆ z = 0 ) if s ( x ) > γ or OOD ( i.e. ˆ z = 1 ) if s ( x ) ≤ γ .
Formally, denoting g ( · , γ ) the decision function, we take:
[Equation]
Performance evaluation.
The OOD problem is a (unbalanced) classification problem, and classically, two quantities allow to measure the performance of a method.
The false alarm rate is the proportion of samples that are classified as OOD while they are IN.
For a given threshold γ , it is theoretically given by Pr (cid:0)
s ( X ) ≤ γ
| Z = 0 (cid:1) .
The true detection rate is the proportion of samples that are predicted OOD while being OOD.
For a given threshold γ , it is theoretically given by Pr (cid:0)
s ( X ) ≤ γ
| Z = 1 (cid:1) .
There exist several ways to measure the effectiveness of an OOD method.
We will focus on four metrics.
The first two are specifically designed to assess the quality of the similarity function s .
Area Under the Receiver Operating Characteristic curve ( AUROC ) [10].
It is the area under the
ROC curve γ (cid:55)→ (Pr (cid:0) s ( X ) > γ | Z = 0 (cid:1) , Pr (cid:0) s ( X ) ≤ γ | Z = 1 (cid:1) ) , which plots the true detection rates against the false alarm rates.
The AUROC corresponds to the probability that an in-distribution example X in has higher score than an OOD sample X out : AUROC =
Pr( s ( X in ) > s ( X out )) , as can be checked from elementary computations.
Area Under the Precision-Recall curve ( AUPR-IN / AUPR-OUT ) [31] .
It is the area under the precisionrecall curve γ (cid:55)→ (Pr (cid:0) Z = 1 | s ( X ) ≤ γ (cid:1) , Pr (cid:0) s ( X ) ≤ γ | Z = 1 (cid:1) ) which plots the recall (true detection rate) against the precision (actual proportion of OOD amongst the predicted OOD).
The
AUPR is more relevant to unbalanced situations.
The third metric we will use is more operational as it computes the performance at a specific threshold γ corresponding to a security requirement.
False Positive Rate at 95% True Positive Rate ( FPR ) .
In practice, one wishes to achieve reasonable level of OOD detection.
For a desired detection rate r , this incites to fix a threshold γ r such that the corresponding TPR equals r .
At this threshold, one then computes:
Pr
[Equation]
0) with γ r s.t.
TPR ( γ r ) = r .
In our work, we set r = 0 .
95 in (2).
Error of the best classifier ( Err (%)) .
This refers to the lowest classification error obtained by choosing the best threshold.
TRUSTED : Textual OOD-Detection using Integrated Rank-Weighted Depth
In this work, we focus on OOD detection when using a contextual encoder ( e.g. , BERT ).
We denote by { ϕ 1 , . . .
, ϕ L } the L functions corresponding to the layers of the encoder: for every 1 ≤ l ≤ L and a given textual input x , ϕ l ( x ) ∈ R
d is the embedding of x in the l -th layer, where d is the dimension of the corresponding embedding space.
Notice that all layers share the same dimension d .
TRUSTED in a nutshell.
Our OOD detection method is composed of three steps.
For a given input x with predicted label ˆ y : 1.
We first aggregate the latent representations of x via an aggregation function F : (cid:0) R d (cid:1) L → R d .
We choose to take the mean and compute
[Equation]
[Equation]
We will further elaborate on this choice of aggregation function in Sec. 3.2.
We compute a similarity score D ( F PM ( x F PM ( S train n, ˆ y )) between F PM ( x ) and the distribution of the mean-aggregation of the training distribution samples with same predicted target as x ( i.e. ˆ y ) that we denote by F PM ( S train n, ˆ y ) .
Formally, if x 1 , . . .
, x n are the training data, this distribution is given by (1 /n ˆ y ) (cid:80) i :ˆ y i
=ˆ y δ F PM ( x i ) , with n ˆ y =
|{ i : ˆ y i =
ˆ y }| and δ x is the Dirac measure in x .
We take as similarity score D a depth function, namely the integrated rank-weighted depth, that we introduce in
Sec. 3.3.
The last step consists in thresholding the previous similarity score D ( F PM ( x ) , F PM ( S train n, ˆ y )) : under a given threshold γ , we classify x as an OOD example.
3.2 Layer aggregation choice
Most recent work in textual OOD detection with a pretrained transformer solely relies on the last layer of the encoder [100, 75].
Although detectors using information available in multiple layers have been proposed previously, mostly for image data, they rely on post-score aggregation heuristics that are either supervised [58, 44] (and thus require having access to OOD samples) or heavily use arbitrary heuristics [85].
TRUSTED differentiates from previous OOD detection methods as it relies on a pre-score aggregation function.
Most popular layer aggregation techniques for Transformer based architecture involve either Power
Means [46, 82] or Wasserstein barycenters [26].
Motivated by both simplicity and computational efficiency, we discard the Wasserstein barycenters and decide to work with Power Mean (case p = 1 ).
3.3 OOD Score Computation via Integrated Rank-Weighted Depth
Since its introduction by John Tukey in 1975 [97] to extend the notion of median to the multivariate setting, the concept of statistical depth has become increasingly popular in multivariate data analysis.
Multivariate data depths are nonparametric statistics that measure the centrality of any element of R d , where d ≥ 2 , w.r.t.
a probability distribution (respectively a random variable) defined on any subset of R d .
Let X be a random variable.
We denote by P X the law of X .
Formally, a data depth is defined as follows:
[Equation]
The higher D ( x , P X ) , the deeper x is in P X .
Data depth finds many applications in statistics and
ML ranging from anomaly detection [87, 80, 92, 96, 91] to regression [79, 45] and text automatic evaluation [95].
Numerous definitions have been proposed, such as, among others, the halfspace depth [97], the simplicial depth [65], the projection depth [66] or the zonoid depth [56], see [90, Ch.
2] for an excellent account of data depth.
The halfspace depth is the most popular depth function probably due to its attractive theoretical properties [37, 104].
However, it is defined as the solution of an optimization problem (over the unit hypersphere) of a non-differentiable quantity and is therefore not easy to compute in practice [81, 39].
Furthermore, it has been show in [73] that the approximation of the halfspace depth suffers from the curse of dimensionality involving statistical rates of order
O( (log( n ) /n )
1 / ( d − 1) )
(see Equation (12) in [73]) where n is the sample size.
Recently, the Integrated
Rank-Weighted (IRW) depth has been introduced in [76], replacing the infimum with an expectation (see also [16, 94]) in order to remedy this drawback.
In contrast to the halfspace depth, it has been show in [94] that the approximation of the IRW depth doesn’t suffer from the curse of dimensionality (see Corollary B.3 in [94]).
The IRW depth of x ∈ R d w.r.t.
to a probability distribution P X on R d is given by:
[Equation]
where F u ( t ) =
Pr( ⟨ u , X ⟩ ≤ t ) and S d − 1 is the unit hypersphere.
In practice, the expectation can be approximated by means of Monte-Carlo.
Given a sample S n = { x 1 , . . .
, x n } , the approximation of the IRW depth is defined as:
[Equation]
where u k ∈ S
d − 1 and n proj is the number of direction sampled on the sphere.
The approximation version of the IRW depth can be computed in O ( n proj nd ) and is then linear in all of its parameters.
In addition, the IRW depth has many appealing properties such as invariance to scale/translation transformations or robustness [76, 16].
Furthermore, it has been successfully applied to anomaly detection [94] making it a natural choice for OOD detection.
Connection to Mahalanobis-based score.
Interestingly enough, the Mahalanobis distance [71] can be seen as a data depth via an appropriate rescaling as suggested in [66].
It measures the distance between an element in R d and a probability distribution having finite expectation and invertible covariance matrix differing from the Euclidean perspective by taking account of correlations.
Precisely, the Mahalanobis depth function D M ( x , P X ) is defined as: D M ( x , p X ) =
(cid:0)[-
-]
[Equation]
, where Σ − 1 is the precision matrix of the r.v. X .
Even though interesting results relying on this notion have been highlighted in [75] for OOD detection, we experimentally observe better results with the IRW depth.
Additionally, the Mahalanobis distance requires the first two moments to be finite and to compute Σ − 1 in high dimension, which can be ill-conditioned in low data regimes.
Last, inverting Σ requires O ( d 3 ) operations or storing C matrix, which can become a burden when the number of classes grows [7].
Application to TRUSTED .
The second step of TRUSTED uses an OOD score on the aggregated features.
Thanks to its appealing properties, we choose to rely on the Integrated Rank-Weighted Depth D IRW that measures the “similarity” between a test sample x and a training dataset S train n .
One independent
D IRW is computed per class on the final aggregated layer.
The decision is taken by taking the D score of the predicted class.
Formally the final score is taken as:
[Equation]
(5) where we recall that F PM ( S train n, ˆ y ) is the distribution of the mean-aggregation of the training distribution samples with same predicted target as x ( i.e. ˆ y ). [Figure] [Figure]
Experimental Settings
In this section, we first discuss the limitation of the previous works on textual OOD detection, then present the chosen benchmark, the pretrained encoders, and baseline methods.
4.1 Previous works and their limitations
Previous works in OOD detection [62, 84, 52, 67, 75] mostly rely on a single model to determine which methods are the best.
This undermines the soundness of the conclusions that may only hold for the particular instance of the chosen model ( e.g. for a specific checkpoint trained with a specific seed).
To the best of our knowledge, no work studies the impact of the several sources of randomness that are involved, such as checkpoint and seed selections [86]).
Nevertheless, these hyperparameters do impact the OOD detectors’ performances.
This is illustrated in Fig.
9 of the supplementary material, which gathers several Mahalanobis scores for various checkpoints of the same model.
In the light of Fig.
9, we choose to study both the impact of the checkpoint and the seed choice in our experiment.
Specifically, for each model, we consider 5 different checkpoints.
We save and probe models after 1k, 3k, 5k, 10k, 15k, and 20k finetuning steps.
We additionally reproduce this experiment for 3 different seeds.
As the reuse of checkpoints reduces the cost of research and allows for easy head-to-head comparison, our library also contains the probed models to draw general robust conclusions about the performance of the considered class of models [30, 38, 102].
4.2 Dataset selection
Dataset selection is instrumental for OOD detection evaluation as it is unreasonable to expect a detection method to achieve good results on any type of OOD data [1].
Since there is a lack of consensus on which benchmark to use for OOD detection in NLP, we choose to rely on the benchmark introduced by [103] which is an extension of the one proposed by [49].
Benchmark description.
The considered benchmark is composed of three different types of in distribution datasets (referred to as IN-DS ) which are used to train the classifiers: sentiment analysis ( i.e. , SST2 [88] and IMDB [70]), topic classification ( i.e. , 20Newsgroup [54]) and question answering ( i.e. , TREC-10 [61]).
For splitting we use either the standard split or the one provided by [103].
For the OOD datasets (referred to as OUT-DS ), we first consider the aforementioned datasets ( i.e. , any pair of datasets can be considered as OOD).
Then, we also rely on four other datasets: a concatenation of premises and respective hypotheses from two NLI datasets ( i.e. , RTE [12, 51] and MNLI [99]),
Multi30K [40] and the source of the English-German WMT16 [8].
We gather in Tab.
5 the statistics of the various data-sets and refer the reader to reference [103] for further details.
4.3 Baseline methods and pretrained models
Baseline methods.
We consider the three following baselines 2 :
Maximum Soft-max Probability (MSP).
This method has been proposed by [47].
Given an input x , it relies on the final score s MSP defined by s MSP ( x ) = − max
y
∈Y p Y | X ( y | x where p Y | X ( ·| x )) is the soft-probability predicted by the classifier after x has been observed.
Energy-based score (E) [67] is defined as the score s E ( x ) =
T × log (cid:104)(cid:80)
y ∈Y exp (cid:16) g y ( x ) T (cid:17)(cid:105) , where g y ( x ) represents the logit corresponding to the class label y .
Mahalanobis ( D M ) .
Following [75, 60, 103], the last layer of the encoder is considered leading to the score: s M ( x ) =
− D M ( F PM ( x ) , F PM ( S train n, ˆ y )) where ˆ y represents the label predicted by the classifier based on the observation of x .
Aggregation procedures.
Both D M and D IRW rely on feature representations of the data which are extracted from the neural networks.
Our goal is to demonstrate that our aggregation procedure F PM defined in Eq.
3 is a relevant choice to be plugged in Eq. 5.
To do so, we also perform experiments on other natural aggregation strategies we introduce in the following.
Logits layer selection .
We use the raw non-normalized predictions of the classifier.
In this case F Logits ≡ F ( ϕ 1 ( x ) , . . .
, ϕ L ( x )) =
ϕ L +1 ( x ) .
Last layer selection .
Following previous work in textual OOD detection [100], we also consider the last layer of the network.
Formally F L ≡ F ( ϕ 1 ( x ) , . .
ϕ L ( x )) =
ϕ L ( x ) .
Layer concatenation .
We follow the BERT pooler and explore the concatenation of all layers.
Formally, F cat ≡ F ( ϕ 1 ( x
ϕ L ( x )) =
[ ϕ 1 ( x ) , · · · , ϕ L ( x )] represents the concatenated vector.
The main limitation of layer concatenation is that the dimension of the considered features linearly increases with the number of layers which can be problematic for very deep networks [98].
Pretrained encoders.
To provide an exhaustive comparison, we choose to work with different types of pretrained encoders.
We test the various methods on DISTILBERT ( DIS. )
[83], BERT [35] and ROBERTA ( ROB. )
We trained all models with a dropout rate [89] of 0 . 2 , a batch size of 32 , we use ADAMW [55].
Additionally, the weight decay is set to 0 .
01 , the warmup ratio is set to 0 .and
the learning rate to 10 − 5 .
[Table]
Static Experimental Results
In this section, we demonstrate the effectiveness of our proposed detector using various pretrained models.
Due to space limitations, additional tables are reported in the Supplementary Material
5.1 Methods comparison
In Tab. 2, we report the aggregated score obtained by each method combined with a different aggregation function.
We observe that TRUSTED obtains the best overall scores followed by D IRW using F cat .
Similarly to previous works [75], we notice in general that score leveraging information available from the training set ( i.e. , D M and D IRW ) achieve stronger results than those relying on output of softmax scores solely ( i.e. , E and MSP).
Interestingly, we observe that D M achieves the best results when relying on the last layer solely ( i.e. , using F L +1 ).
Considering additional layers through concatenation or mean hurts the performances of
D M .
This is not the case when relying on D IRW .
Indeed layer aggregation improves the performance [Table] of the detector demonstrating the relevance of using D IRW over D M as an OOD score.
Relying on Mahalanobis as OOD score suppose that the representation follows a multivariate Gaussian distribution which might be too strong assumption in the case of layer aggregation.
On the contrary
D IRW do not rely on any distributional assumption
[Table]
5.2 On the pretrained encoder choice
[Table]
When training and deploying a classifier [Table] a key question is choosing a pretrained encoder.
It can be beneficial in critical applications to trade off the main task accuracy to ensure better OOD detection.
In Tab. 3, we report the individual performance of OOD methods on three types of classifiers.
Although TRUSTED achieves state-of-the-art on all configurations, it is worth noticing a difference in performance concerning the type of pretrained model.
It is also important to remark that for ROB both
MSP and E achieve on-par performances with D M while not requiring any extra training information.
Overall, for a given method ( i.e. D M or D IRW ), the ranking of detectors performances according to the type of feature extractor remain still.
This validates the use of the mean-aggregation procedure of
TRUSTED .
Overall, based on the difference of OOD detection performance of Tab. 3, we recommend to use TRUSTED on BERT or DIS.
if accurate OOD detection is required.
5.3 Impact of the training dataset
OOD detection performance depends on the nature of what is considered in-distribution (the training distribution in our case).
Thus, it is interesting to study the performance per-
IN-DS as reported in
Tab. 4.
Even though TRUSTED achieves strong results in terms of AUROC , we observe a high FPR on
SST2.
From Fig.
4, we observe that IMDB and SST2 are harder to detect, especially for D M .
Finally, since high AUROC does not necessarily imply a low FPR , it is crucial to take both into account when designing an OOD detector.
[Figure]
A UR O C IN-DS imdbtrecsst220ngMethod D IRW D M
[Figure]
Dynamic Experimental Results
Most OOD detection methods are tested on specific checkpoints, where the selection criterion is often unclear.
The consequences of this selection on OOD detection are rarely studied.
This section aims to respond to this by measuring the OOD detection performance of methods on various checkpoint finetuning of the pretrained encoder.
We will use 5 different checkpoints taken after 1k, 3k, 5k, 10k, 15k, and 20k iterations.
Training curves of the models are given in Fig.
Notice that after 3k iterations models have converged and no over-fitting is observed even after 20k iterations ( i.e., we do not observe an increase in validation loss).
Overall analysis.
We report the results of the dynamic analysis on the various pretrained models in Fig.
For all the methods and models (except D M on ROB ), we observe that training longer the classifier hurts detection.
Interestingly, this drop in performance has a higher impact on FPR compared to AUROC .
Thus, it is better to use an early stopping criterion to ensure proper OOD detection performance.
In addition, we observed that TRUSTED (corresponding to D IRW ) achieves better detection results and that D M outperforms TRUSTED for checkpoints larger than 10k on ROB .
[Figure]
Analysis Per In-Dataset.
Fig.
6 reports the results of the dynamical analysis per IN-DS .
We observe that D IRW is consistently better than D M with sensible improvement on IMDB and SST2, which are the hardest benchmarks.
We observe a similar trend to the previous experiment: training longer the classifiers hurt their OOD detection performances .
Similar observations hold for FPR ,
AUPR-IN [Figure] and AUPR-OUT that are postponed to the
Supplementary Material (see Fig. 10)
[Figure]
Conclusions and Future Directions
In this work we introduced TRUSTED a novel OOD detector that relies on information available in all the hidden layers of a network.
TRUSTED leverages a novel similarity score built on top of the Integrate Rank-Weighted depth.
We conduct extensive numerical experiments proving that it consistently outperforms previous approaches, including detection based on the Mahalanobis distance.
Our comprehensive evaluation framework demonstrates that, in general, OOD performances vary depending on several hyperparameters of the models, the datasets, and the detector’s feature extraction step.
Thus, we would like to promote the use of such exhaustive evaluation frameworks for future search to assess AI systems’ safety tools properly.
Another interesting question is the detection inference-time / accuracy trade-off, which is instrumental for the practitioner
Acknowledgments
This work was also granted access to the HPC resources of IDRIS under the allocation 2021AP010611665 as well as under the project 2021-101838 made by GENCI.
This work has been supported by the project PSPC AIDA: 2019-PSPC-09 funded by BPI-France
A Experimental Details
A.1 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.2 Benchmark Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.3 Training Parameters . .
A.4 Training Curves . .
B Additional Static Experimental Results
B.1 Analysis Per In-Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B.2 Trade-off between metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B.3 Analysis Per Out-Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C Additional Dynamical Experimental Results
C.
On the importance of dynamical probing . . . . . . . . . . . . . . . . . . . . . . .
C.2 Analysis Per IN-Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.3 Impact of the pretrained encoder . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.4 All Combinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.5 Futur works . .
A Experimental Details
In this section we gather additional experimental details.
For completeness we provide the used algorithms for both TRUSTED and D IRW (see Sec.
A.1), we also gather additional benchmarks details (see Sec.
A.2), hyperparameter used during training (see Sec.
A.3) as well as the training curves (see
Sec.
A.4).
A.1 Algorithms
In this part, we present algorithms to compute D IRW (see Algorithm 1) and TRUSTED (see Algorithm 2). Initialization
[Table]
Initialization: x , n proj , S n , ˆ y .
1: Compute F PM ( x ) 2: for y = 1 , . . .
C do
Compute F PM ( S train n,y ) 3: end for 4: Compute D IRW ( F PM ( x ) , F PM ( S train n, ˆ y )) using Algorithm 1 with F PM ( x ) , n proj and
F PM ( S train n, ˆ y )
Output : s TRUSTED ( x ) =
D IRW ( F PM ( x ) , F PM ( S train n, ˆ y ))
A.2 Benchmark Details
We report in Tab.
5 statistics related to the datasets of our benchmark.
The only difference between our work and the one from [103] is that we considered the pair IMDB and SST2 a valid OOD pair of
IN-DS/OOD-DS as this can be seen as a background shift (see [78]).
Remark
We initially started to work with the appealing benchmark introduced by [60] which introduces an alternative standard that addresses the limitation of the benchmark proposed by [100] (e.g. there is no category overlap between training and OOD test examples in the non-semantic shift dataset).
Additionally, [60] put effort into designing a dataset that can categorize shifts as belonging to semantic or background shift [2, 78].
However, we failed to reproduce the baseline results from [60] even after contacting the authors.
A.3 Training Parameters
In this section, the detail the main hyper-parameters that were used for finetuning the pretrained encoders.
It is worth noting that we use the same set of hyperparameters for all the different encoders [26, 24 ? ].
The dropout rate [89] is set to 0 .
2 [], we train with a batch size of 32 , we use ADAMW [55, 69, 4].
Additionally, we set the weight decay to 0 .
01 , the warmup ratio to 0 .
06 , and the learning rate to 10 − 5 .
All the models were trained during 20k iterations with different seeds.
A.4 Training Curves
In order to understand the change of performance while finetuning the model, it is crucial to understand when and if the different models have converged.
Thus we report in Fig.
7 dev losses and dev and test accuracy.
From Fig.
7, we can observe that a pretrained encoder finetuned on 20ng takes more time to converge compared to the same pretrained encoder finetuned on either SST2, IMDB, or trec.
Additionally, after 2k updates, both dev and test accuracy are stable on SST2, IMDB, and trec, while for 20ng, it requires 6k steps.
Last, it is worth noting (see Tab. 1) that ROB.
achieves the best accuracy overall.
BERT is the second best and achieves stronger test accuracy than DIS.
on the different datasets.
[Figure]
D e v A cc u r a c y
[Figure]
D e v A cc u r a c y
[Figure]
D e v A cc u r a c y
[Figure]
t
[Figure]
cc u r a c y
[Figure]
(i)
[Figure]
B Additional Static Experimental Results
In this section, we report additional static experiment results.
Formally, we aim to gain understanding:
• on the role of the IN-DS (see Sec.
B.1)
and on the impact of the choice of the pretrained encoder [27].
• on the different trade-off that exists between the different metrics (see Sec.
B.2) • on the impact of performance of the detection methods depending on the OOD distribution (see Sec. B.3).
B.1 Analysis Per In-Dataset
We present in Tab.
the average performance per IN-DS .
On 3 out of 4 datasets TRUSTED achieves the best results and outperforms other methods.
Interestingly, the table shows the key importance of the training corpus.
As an example, we observe that detection methods applied to classifiers trained on sst2 are less efficient (by several AUROC points) compared to other IN-DS .
A finer analysis of this phenomenon can be conducted using Tab. 7.
From Tab. 7, we observe that this phenomenon is consistent accross all the pretrained classifiers ( i.e., BERT , ROB. and DIS. ).
We observe that different TRUSTED does not work uniformly better on all pretrained models.
For example, the best results on 20ng are obtained for BERT while on sst2 it is obtained for DIS. .
Takeaways: Both IN-DS and pretrained model choices are essential to ensure good detection performance.
[Table]
B.2 Trade-off between metrics
We report Fig.
8 the different trade-off that exist between the considered metrics.
Interestingly, a high AUROC does not imply a low FPR .
Similar conclusions can be drawn regarding AUPR-IN and
AUPR-OUT .
Takeaways.
Fig.
8 illustrates that all metrics matters and should be considered when comparing detection methods.
[Figure]
B.3 Analysis Per Out-Dataset
We report on Tab. 8 the detailled results of the detection methods for different OUT-DS .
It is worth noting that TRUSTED achieves best results on 22 configurations.
Takeaways.
Tab. 8 illustrates the impact of the type of OOD sample on the detection performances.
C Additional Dynamical Experimental Results
In this section, we gather additional dynamical analysis [25, 23].
We further illustrate the importance of dynamical probing in Sec.
C.1.
We then conduct a dynamical study of the impact of the OOD dataset in Sec.
C.2.
In Sec.
C.3, we study the role of the pretrained encoder and in Sec.
C.4.
Last, we gather per encoder, per IN-DS and per OUT-DS analysis.
[Table]
C.1 On the importance of dynamical probing
In Fig.
[Table]
we report histograms for TRUSTED and Mahanalobis distance.
Interestingly, the shape of histograms is changing across checkpoints demonstrating the need for dynamical probing [22, 93] [Table]
[Table]
C.2 Analysis Per IN-Dataset
In Fig.
10, we conduct a dynamical analysis per IN-DS .
We observe a variation of performance ( e.g., up to 10 AUROC points for sst2 for D M ) while probing accross time.
Takeaways: Although our method achieves strong results a key dimension when deploying OOD detection methods is to carefully select the checkpoints when learning the classifier.
C.3 Impact of the pretrained encoder
In Fig.
11, we report the results of the dynamical analysis and study the influence of the encoder choice on the detection performance
Takeaways Although, TRUSTED achieves strong results on a large number of configurations.
We observe different behaviours while considering different success criterion or different pretrained
[Table]
[Figure]
models.
For examples, on BERT and DIS. , TRUSTED is uniformly better across checkpoints when considering Fig.
For ROB. , TRUSTED is not better on all metrics on last checkpoint ( e.g., 20k).
C.4 All Combinations
For completeness of the paper, we report all the results of the dynamical analysis for all considered combinations in this section (see Fig.
12 Fig.
13 Fig.
14 Fig. 15).
We believe this will allow the curious reader to gain more intuition and draw nuanced conclusions from our experiments.
C.5 Futur works
Futur works include testing our method on different settings such as sequence generation, multimodal learning or automatic evaluation [21, 36, 53, 101, 43, 28, 15].
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]