Beyond Mahalanobis Distance for Textual OOD Detection
Authors: Pierre Colombo
Eduardo Dadalto Câmara Gomes
Guillaume Staerman
Nathan Noiry
Pablo Piantanida
Venue: NeurIPS 2022 Accept (2022)
Keywords: NLP
Out-of-Distribution detection
text classifiers

Abstract: As the number of AI systems keeps growing, it is fundamental to implement and develop efficient control mechanisms to ensure the safe and proper functioning of machine learning (ML) systems. Reliable out-of-distribution (OOD) detection aims to detect test samples that are statistically far from the training distribution, as they might cause failures of in-production systems. In this paper, we propose a new detector called TRUSTED. Different from previous works, TRUSTED key components (i) include a novel OOD score relying on the concept of statistical data depth, (ii) rely on the idea’s full potential that all hidden layers of the network carry information regarding OOD. Our extensive experiments, comparing over 51k model configurations including different checkpoints, seed and various datasets, demonstrate that TRUSTED achieve state-of-the-art performances by producing an improvement of over 3 AUROC points.

Beyond Mahalanobis-Based Scores for Textual OOD Detection
Pierre Colombo
Mathématiques et Informatique pour la Complexité et les Systèmes CentraleSupelec, Université Paris Saclay
pierre. colomio@ceetrralesseeeee.....
Eduardo D. C. Gomes
Guillaume Staerman
Laboratoire des signaux et systemes CentraleSupelec, CNRS, Université Paris-Saclay eduardo. cadaaloooctttraleeeepe......
Inria, CEA, Universite Paris-Saclay guillaume staerman@inria. fr
Nathan Noiry
althiqua.io moirynathanngmail.com
Pablo Piantanida
International Laboratory on Learning Systems
-cGill-ETe MILA CNRS CentraleSupélec, Université Paris-Saclay
pablo. http.oyyyaaaaaaddeeeaaaaaalppppp
Abstract
Deep learning methods have boosted the adoption of NLP systems in real-life applications. However, they turn out to be vulnerable to distribution shifts over time which may cause severe dysfunctions in production systems, urging practi- tioners to develop tools to detect out-of-distributiin (OOD) samples through the lens of the neural network. In this paper, introduce TRUSTED, new OOD detector for classifiers based on Transformer architectures that meets operational requirements: it unsupervised and fast to compute. The efficiency of TRUSTED relies on the fruitful idea that all hidden layers carry relevant information to detect OOD examples. Based on this, for given input, TRUSTED consists in aggre- gating this information and (ii) computing similarity score by exploiting the training distribution, leveraging the powerful concept of data depth. Our extensive numerical experiments involve 51k model configurations, including various check- points, seeds, and datasets, and demonstrate that TRUSTED achieves state-of-the-art performances. In particular, it improves previous AUROC over 3 points.
Introduction
The number of AI systems put into production has steadily increased the last few years. This because advanced techniques of Machine Learning (ML) and Deep Learning (DL) have brought significant improvements over previous state-of- tiie-oo-thhearr (SOTA) methods in many areas such as finance [17, 57], transportation [59], and medicine [14, 77]. However, the increasing use of black-box models has raised concerns about their societal impact: privacy [33, 64, 74, 29], security [6, 3, 20),
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

safety [41, 50, 18), fairness [5, 72], and explainability [13, 63] which became areas of active research in the ML community.
This paper is about critical safety issue, namely Out-Of-Distribution (OOD) detection [11], which refers to change of distribution of incoming data that may cause failures of in-production Al systems, When data are tabular and of small dimension, simple statistical methods can be efficient, and one may, for instance, monitor the mean and variance of each marginal over time. However, these traditional methods do not work anymore when data are high-dimensional and/or unstructured. Thus, the need to design new techniques that incorporate incoming data and the neural networks themselves.
Distinguishing OOD examples from in-distribution (ID) examples is challenging for modern deep neural architectures. DL models transform incoming data into latent representations from which reliable information extraction is cumbersome. Because of that, designing new tools of investigation for large pretrained models, judiciously named foundation models by [9], is an essential line of research for the years to come. In computer vision, methods are more mature thanks to the availability of appropriate deformation techniques that allow for sensitivity analysis. In contrast, the nature of tokens in Natural Language Processing (NLP) makes it more difficult to develop such suitable methods.
In this paper, we focus on classifiers for textual data and the ubiquitous BERT [34], DistilBER: [83], and RoBERTa [68] architectures. Existing methods can be grouped according to their positioning with respect the network. Some works exploit only the incoming data and compare them with in-distribution examples through likelihood ratios [42, 19]. Another line of research consists in incorporating robust constraints during training, with [48] or without [103, 60] access to some available OOD examples. Another line of research focuses on post-processing methods that can be used on any pretrained models. In our view, these are the most promising tools because typical users rely on transformers without retraining. Within post-processing methods, one can distinguish softmax- based tools that compute a confidence score based on the predicted probabilities and threshold to decide whether a sample is OOD or not. Notice that this does not require direct access to in- distribution data. The seminal work is due Hendrycks [47] who uses the maximum poot-probability. and has been pushed further in [62, 52]. In [67], authors suggest looking one step deeper into the network, namely, to compute a confidence score based on the projections of the pre-softmax layer. More recently. [75] achieved SOTA results on transformer- based encoder by computing the Mahalanobis distance [71, 32] between test sample and the in-distribution law [58], estimated through accessible training data points.
Nevertheless, the distance-based score is computed on the last-layer embedding only, suggesting that going deeper inside the network might improve OOD detection power. Moreover, the computation of Mahalanobis-basee based scores requires inverting the covariance matrix of the training data, which can be prohibitive in high dimensions. It is worth noticing that the Mahalanobis- bbsee scores can be seen as a data depth [97, 104] through a simple re-scaling [66], that is a statistical function measuring the centrality of an observation with respect to probability distribution. Although data depths are quite natural in the context of OOD detection, they remain overlooked by the MI. community. In the present work, we rely on the recently introduced Integrated Rank- Weighted depth [76, 94] in order to remedy the drawbacks of the Mahalanobis- scores for OOD detection.
1.1 Our contribution
We first leverage the observation introduced by previous work that all hidden layers of neural network carry useful information to perform textual OOD detection. For a given input x, our method consists in computing its average latent representation X and then its OOD score through the depth score of X with respect to the averaged in-distribution law (see Fig. for an illustration). Notice that the ability to compute averaged latent representations crucially relies on the structure of transformers layers that share the same dimension. The depth function we are using is based on the computation of the projected ranks of the test inputs using randomly sampled directions. From theoretical viewpoint, this novel method requires fewer assumptions on the data structure than the Mahalanobis score.
We conduct extensive numerical experiments on three transformers architectures and eight datasets and benchmark our method with previous approaches. To ensure reliable results, we introduce new
2

---

framework for evaluating OOD detection that considers hyperparameters that were unreported before. It consists of computing performances for various choices of checkpoints and seeds, which allows us to report a variance term that makes some previous methods fall within the same performance range. Our conclusions are drawn by considering over 5/k configurations, and show that our new detector based on data depth improves SOTA methods by 3 AUROC points while having less variance. This result supports the intuition that OOD detection a matter of looking at the information available across the entire network. Our contribution can be summarized as follows:
1. We introduce a novel OOD detection method for textual data. Our detector TRUSTED relies on the full information contained in pretrained transformers and leverages the concept of data depth: a given input is detected as being in-distribution or OOD sample based on its depth score with respect to the training distribution.
2. We conduct extensive numerical experiments and prove that our method improves over SOTA methods. Our evaluation framework more reliable than previous studies as it includes the variance with respect to seeds and checkpoints.
3. We release open-source code and data to ease future research, ensure reproducibility and reduce computation overhead.
Problem Formulation
Training distribution and classifier. Let denote by X' the textual input space. Consider the multiclass classification setup with target space y = (1...., C} of size C' 2 2. We assume the dataset under consideration is made of N 2 1 ii.d. samples (1). ..... (XN: UN) with probability law denoted by pxy and defined on X' x y. Accordingly, we will denote by px and py the marginal laws of pxy. Finally, we denote by fN X -> y the classifier that has been trained using (x,, Vs)-
Open world setting. real-life scenarios, the trained model fN is deployed into production and will certainly be faced with input data whose law is not pxy. To each test point (x, y). we associate a variable z E {0, }} such that 2 = 0 if (x, y) stems from pxy and = 1 otherwise. It is worth emphasizing that in our setting SN has never been faced with OOD examples before deployment. This is usually referred to as the open-world setting. From a probabilistic viewpoint, the test set distribution of the input data pest is mixture of in -idistribbtiin and OOD samples:
PX (x) a px1z (x|2 1) (1 - a) Px 2x (x|= = 0),
where a (0. 1). In this work, we will not make any further assumptions on the proportion a of OOD samples and on the OOD pdf px z (x|2 1), making the problem more difficult but at the same time more well suited for practical use. Indeed, for textual data, does not appear to be realistic to model how corpus can evolve.
OOD detection. The objective of OOD detection is construct a similarity function s x + R4 that accounts for the similarity of any element in X' with respect to the training in in-distribution. For given test input x, we then classify x as in-distribution or OOD according the magnitude of sxx). Therefore, one fixes a threshold y and classifies IN (i.e. & = 0) if s(x) or OOD (i.e. 2 = 1) if s(x) S y. Formally, denoting ((+7) 7) the decision funetion, we take:
{
if x
< 7: x
g(x,))
0 if
7
Performance evaluation. The OOD problem is a (unbalanced) classification problem, and classically. two quantities allow to measure the performance of a method. The false alarm rate is the proportion of samples that are classified as OOD while they are IN. For given threshold 7, it is theoretically given by Pr (x(X) < 7 Z = 0). The true detection rate is the proportion of samples that are predicted OOD while being OOD. For a given threshold 7, it is theoretically given by Pr (s(X)= < 7|==1 1).
There exist several ways to measure the effectiveness of an OOD method. We will focus on four metrics. The first two are specifically designed to assess the quality of the similarity function s.
TRUSTED stands for ReTect USing inTegrated rank- WEggttd Depth.
3

---

Area Under the Receiver Operating Characteristic curve (AUROC) [10]. It is the area under the ROC curve y + (Pr (s(X) > 7 Z 0). Pr (s(X) S > Z 1)), which plots the true detection rates against the false alarm rates. The AUROC corresponds to the probability that an in-distribution example x has higher score than an OOD sample Xout: AUROC (r() > s(Xout)) )): as can be checked from elementary computations.
Area Under the Precision
curve (AUPR- IN/AUPR-OUT) is the area under the precision- recall curve 7 + (Pr (Z = s(X) I 7), Pr (x(X) S < = 1)) which plots the recall (true detection rate) against the precision (actual proportion of OOD amongst the predicted OOD). The AUPR is more relevant to unbalanced situations.
The third metric will use is more operational computes the performance specific threshold corresponding to a security requirement.
False Positive Rate at 95% True Positive Rate (FPR). In practice, one wishes to achieve reasonable level of OOD detection. For a desired detection rate r, this incites to fix a threshold 7, such that the corresponding TPR equals At this threshold, one then computes:
(r(s(X) 2 Yr = 0) with "r s.t. (PR(%) =r.
(2)
In our work, we set 0.95 in (2).
Error of the best classifier (Err (%)). This refers to the lowest classification error obtained by choosing the best threshold.
3 TRUSTED: Textual OOD-Detection using Integrated Rank- Weighted Depth
In this work, we focus on OOD detection when using a contextual encoder (e.g., BERT). We denote by fø1...., Φ1] the L functions corresponding to the layers of the encoder: for every 1 S L and given textual input 0r(x) E IR' is the embedding of in the /-th layer, where d is the dimension of the corresponding embedding space. Notice that all layers share the same dimension
3.1 TRUSTED in nutshell.
Our OOD detection method is composed of three steps. For a given input with predicted label ÿ:
We first aggregate the latent representations of x via an aggregation function F (RR') 4 Rd We choose to take the mean and compute
1.
1
(x)) FPM == (().
Σ oi (x) = x. 1=1
L
We will further elaborate on this choice of aggregation function in Sec. 3.2.
We compute a similarity score (x) FPM (PP((St)) (Strgin)) between FPM(x) and the distribution of the mean- ageenagggregt of the training distribution samples with same predicted target as (i.e. ÿ) that we denote by FrM (Strai))))) Formally, if x X,, are the training data, this distribution is given by (1/rg) Ei.g; ò (esa((..). with ng Ki ú = أ and 6, is the Dirac measure in æ. We take as similarity score D depth function, namely the integrated rank weighteighted depth, that we introduce in Sec. 3.3.
3. The last step consists in thresholding the previous similarity score ((FPM((x) FPM (Straia) under given threshold y, classify x an OOD example.
3.2 Layer aggregation choice
Most recent work in textual OOD detection with a pretrained transformer solely relies on the last layer of the encoder [100, 75]. Although detectors using information available in multiple layers have been proposed previously, mostly for image data, they rely on post-score aggregation heuristics that are either supervised [58, 44] (and thus require having access to OOD samples) or heavily use arbitrary heuristies [85]. TRUSTED differentiates from previous OOD detection methods it relies on pre-score aggregation function.

---

Most popular layer aggregation techniques for Transformer based architecture involve either Power Means [46, 82] or Wasserstein barycenters [26]. Motivated by both simplicity and computational efficiency, we discard the Wasserstein barycenters and decide to work with Power Mean (case 1).
3.3 OOD Score Computation via Integrated Rank-Weighted Depth
Since its introduction by John Tukey in 1975 [97] to extend the notion of median to the multivariate setting, the concept of statistical depth has become increasingly popular in multivariate data analysis. Multivariate data depths are nonparametric statistics that measure the centrality of any element of R. where d 2 2. w.r.t. a probability distribution (respectively a random variable) defined on any subset of Rd Let X be random variable. We denote by Px the law of X. Formally, a data depth is defined as follows:
D: Rd x P(R4) (x, =
[0.1]. Pxx,
(4)
The higher D(x, Px), the deeper x is in Px. Data depth finds many applications in statistics and ML ranging from anomaly detection [87, 80, 92, 96, 91] to regression [79, 45] and text automatic evaluation [95]. Numerous definitions have been proposed, such as, among others, the halfspace depth [97]. the simplicial depth [65], the projection depth [66] the zonoid depth [56], see [90, Ch. 2] for an excellent account of data depth. The halfspace depth is the most popular depth function probably due to its attractive theoretical properties [37. 104]. However, it is defined as the solution of an optimization problem (over the unit hypersphere) of a non-difeerentiable quantity and is therefore not easy compute in practice 1 391. Furthermore, it has been show in [73] that the approximation of the halfspace depth suffers from the curse of dimensionality involving statistical rates of order O(log(n)) (m))))))//)))) (n))/////1)/))) (d-1)) (see Equation (12) in [73]) where n is the sample size. Recently, the Integrated Rank- weighee (IRW) depth has been introduced in [76], replacing the infimum with expectation (see also [16, 94I) in order
to remedy this drawback. In contrast to the halfspace depth, it has been show in [94] that the approximation of the IRW depth doesn't 't suffer from the curse of dimensionality (see Corollary B.3 in [94]). The IRW depth of x E IRd w.r.t. to a probability distribution Px on Rd is
given by:
f min (Fr ((,,)), Fu ((u,))) du, 3d-1
Duw (x, Px) Px)=
where (t) (u, X) S and Sdd1 is the unit hypersphere. In practice, the expectation be approximated by means of Monte-Carlo. Given a sample S, = {xI...., xx), , the approximation of the IRW depth is defined as:
Pproj
1
DIRW (x,w(x,00 (x, Sn)
min
II - x) <O) n2 M X -x 0}
prroj
n
where Un E Sdd1 and Tirooj is the number of direction sampled on the sphere. The approximation version of the IRW depth can be computed in (nrrjitdd) and is then linear in all of its parameters. In addition, the IRW depth has many appealing properties such as invariance to scale/translation transformations robustness [76, 16]. Furthermore, it has been successfully applied to anomaly detection [94] making it a natural choice for OOD detection.
Connection to Mahalanobis- based score. Interestingly enough, the Mahalanobis distance [71] can be seen as a data depth via an appropriate rescaling as suggested in [66]. It measures the distance between an element in d and probability distribution having finite expectation and invertible covariance matrix differing from the Euclidean perspective by taking account of cor- relations. Precisely, the Mahalanobis depth function (x Px) is defined as: DM (M,x,px)
(1 - (x-[(x) ( - E(X))) where --1 is the precision matrix of the r.v. X. Even though interesting results relying on this notion have been highlighted in [75] for OOD detection, we experimentally observe better results with the IRW depth. Additionally, the Mahalanobis distance requires the first moments to be finite and to compute Σ- in high dimension, which can be ill-conditioned in low data regimes. Last, inverting Σ requires (dd) operations or storing C matrix, which can become burden when the number of classes grows [7].
Application to TRUSTED. The second step of TRUSTED uses an OOD score on the aggregated features. Thanks to its appealing properties, we choose to rely on the Integrated Weik-Weghtted Depth DIRW
5

---

pre-tenined
1.50
C Teqquency
1.28
1.00
Out
0.75
0.50
0.28
0.00
0.36 0.38
0.40
0422
0.44 0.46
OOD secns DIS.P)
(a) Draw
Exaction
Frequenoy:
1.5
OuE
1.0
0.5
0.50 0.75 1.00 1.25 1.50 1.75 2,00 0
Fig. I: TRUSTED detector. It relies on two steps: mean layer aggregation followed by the computation of DIRW-
(b) DM
Fig. 3: Histogram scores.
that measures the "similarity" between test sample x and a training dataset Strain, One independent DIaw is computed class on the final aggregated layer. The decision is taken by taking the D score the predicted class. Formally the final score taken as:
Strusted (x) DIRW (FPM(x), FPM (Strg
(5)
where recall that (S is the distribution of the mean-aggregation of the training distribu- tion samples with same predicted target (i.e. 9).
4
Experimental Settings
In this section, we first discuss the limitation the previous works on textual OOD detection, then present the chosen benchmark, the pretrained encoders, and baseline methods.
4.1 Previous works and their limitations
Previous works in OOD detection [62, 84, 52, 67, 75] mostly rely on single model to determine which methods are the best. This undermines the soundness of the conclusions that may only hold for the particular instance of the chosen model (e.g. for a specific checkpoint trained with a specific seed). To the best of our knowledge, are involved, such as checkpoint and seed selections [86]). Nevertheless, these hyperparameters do impact the OOD detectors' performances. This illustrated in Fig. 9 of the supplementary material, which gathers several Mahalanobis scores for various checkpoints the same model.
work studies the impact of the several sources of randomness that
In the light of Fig. 9, choose study both the impact of the checkpoint and the seed choice in our experiment. Specifically, for each model, we consider different checkpoints. We save and probe models after 1k, 3k, 5k, 10k, 15k, and 20k finetuning steps. We additionally reproduce this experiment for 3 different seeds. As the reuse of checkpoints reduces the cost of research and allows for easy read-to-head comparison, our library also contains the probed models draw general robust conclusions about the performance the considered class models [30, 38, 102].
4.2 Dataset selection
Dataset selection is instrumental for OOD detection evaluation it is unreasonable to expect detection method to achieve good results on any type of OOD data [1]. Since there a lack of consensus which benchmark to use for OOD detection in NLP, we choose rely the benchmark introduced by [103] which an extension of the one proposed by [49].
Benchmark description. The considered benchmark is composed of three different types of in distribution datasets (referred to as IN-DS) which are used to train the classifiers: sentiment analysis (i.e., SST2 [88] and IMDB [70]), topic classification (i.e., 20Newsgroup [54]) and question answering (i.e., TREC-10 [61]] D). For splitting we use either the standard split or the one provided by [103]. For the OOD datascts (referred to OUT-DS), first consider the aforementioned datasets (i.c., any pair

---

of datasets can be considered as OOD). Then, we also rely on four other datasets: concatenation of premises and respective hypotheses from two NLI datasets (i.e., RTE [12, 51] and MNLI (99]), Multi30K [40] and the source of the English- of the various data-sets and refer the reader to reference [103] for further details.
WMT16 [8]. We gather in Tab. 5 the statistics
4.3 Baseline methods and pretrained models
Baseline methods. We consider the three following baselines
1. Maximum Soft-max Probability (MSP). This method has been proposed by [47]. Given an input x,
it relies the final score SMSP defined by sMSP(x) = maxyey PYIx (ylx), where py (-12:)
is the soft-probability predicted by the classifier after x has been observed.
2. Energy-based score (E) [67] is defined as the score (x) T x log Σ vEy exp
9v T
where (x) represents the logit corresponding to the class label y.
3. Mahalanobis (Da). Following [75, 60, 103], the last layer of the encoder is considered leading to the score: $M(x) = - -x FFM (x), FFM where represents the label predicted by the
classifier based on the observation x.3
Aggregation procedures. Both DM and Draw rely feature representations of the data which extracted from the neural networks. Our goal is to demonstrate that our aggregation procedure Fra defined in Eq. is relevant choice be plugged in Eq. 5. To do so, we also perform experiments on other natural aggregation strategies we introduce in the following.
Logits layer selection. We use the raw nom-nozmalized predictions of the classifier. In this case F Foggiss = F (..
Φ (x).
2. Last layer selection. Following previous work in textual OOD detection [100], we also consider the last layer of the network. Formally Fz = (Ø1 (x). (x)) = (xxx).
Layer concatenation. We follow the BERT pooler and explore the concatenation of all layers. Formally, Fewr F (x). OL = [ (x).
[x) represents the concatenated vector.
The main limitation of layer concatenation that the dimension of the considered features linearly increases with the number of layers which can be problematic for very deep networks [98].
Pretrained encoders. To provide an exhaustive comparison, we choose to work with different types of pretrained encoders. We test the various methods on DISTILBERT (DIS.) [83], BERT [35] and ROBERTA (ROB.) ) [68]. We trained all models with dropout rate [89] of 0.2, a batch size of 32, we use ADAMW [55]. Additionally, the weight decay is set to 0.01, the warmup ratio is set to 0.06
IN-DS
BERT Acc
DIS. Acc
ROB. Acc
20ng
92.9
92.0
92.7
imdb
91.7
90.6
93.6
sst2
92.7
91.7
95.2
trec 96.8
97.0
97.0
Tab. I: Average test accuracy achieved by dif- ferent classifier when training is initialized with different seeds.
and the learning rate to 10-5
5 Static Experimental Results
In this section, we demonstrate the effectiveness our proposed detector using various pretrained models. Due to space limitations, additional tables are reported in the Supplementary Material.
5.1 Methods comparison
In Tab. 2, we report the aggregated score obtained by each method combined with a different aggregation function. We observe that TRUSTED obtains the best overall scores followed by Drnw using Feat- Similarly to previous works [75], we notice general that score leveraging information
Contrarily to [75], we do use likelihood ratio would require using extra language models, which not available in our setting.
An alternative is to compute the minimum of all Mahalanobis distance computed on all the classes. However. observe slightly better performance when using the predicted label
7

---

Tab. 2: Average OOD detection performance (in %). The averages are taken over 1440 configurations and include four different IN-DS (20ng. imdb, sst2, trec), eight ODD-DS, three different seeds, five different checkpoints and three different pretrained encoders. Due to space constraints, different aggregations and related discussions are relegated to Appendix B.
Score
Aggregation
AUROC
AUPR. IN
AUPR- OUT
FPR
Brr
E.
F2i1
89.9 7
84.9 1990
79.9 1273
44,9 2014
23.9
22200
MSP
Soft.
89.7
84.3 #18.9
80.4 #25.6
45.5 =20.5
25.4
#21.4
Dat
Pz
93.8 19.8
89.2 200-1
91.5 116.4
19.8 2707
12.7
1:700
FL+1
71.7 =ta.7
54.7 =320
73.3 228.4
62.6
=28.1
37.0
#22.9
FL,LI1
81.7
60.7
83.8
73.4
33.0
83.6 #00.6
61.9 =30.3
79.4 2:0.9
81.5 =10.1
30.4 #18.8
Feot
1:1.5
84.0 =:2.1
88.0 119.7
=06.2
118.8
FPM
81.2 =16.3
67.7 #26.7
82.1 #:2.22
40.2 =26.0
23.1 #20.5
Diaov
Fp.
92.6 ::600
88.5 117.7
86.3 1:9.7
37.8
=27.3
23.6
#204
Fi+1
82.4 =14.0
77.2 #24.0
72.1 #220.8
68.5 =20.5
38.0
#25.3
FL.L41
95.5 0:0.0
91.2 #15.0
94,1 8:20.0
23,5 =31.6
13.7
1:5.3
FLaL+1
95.9 =10.0
91.0 =20.0
94.0 #11.0
15.5 =20.5
13.0
#16.0
Feot
96.1
91.8
94.1
19.1 =21.6
14.1
TRUSTED
FPM
97.0 :440
93.2 =11.5
95.1 4:000
15.4
-19
11.7 #137
available from the training set (i.e., DM and DIRW) achieve stronger results than those relying on output of softmax scores solely (i.e., E and MSP).
Interestingly, we observe that DM achieves the best results when relying on the last layer solely (i.e., using F1++). Considering additional layers through concatenation or mean hurts the performances of DM- This is not the case when relying on Drnw. Indeed layer aggregation improves the performance of the detector demonstrating the relevance of using DIRw over Das as an OOD score. Relying on Mahalanobis as OOD score suppose that the representation follows multivariate Gaussian distribution which might be too strong assumption in the case of layer aggregation. On the contrary Draw do not rely on any distributional assumption.
5.2 On the pretrained encoder choice
Tab. 3: Average (over 480 model configuration) performance per pretrained encoder type.
Morld
Scare
Appreazim
AUBCC
AU73-077
27F
M52
10tb.
1775
241
11:-3
B013
45.4
25:3
91.1
11.3
5140
11. mm
71. -350
07.5
17.4
-101
01.0
1220
81.9
Dan
02.4
35.0
23.8
7606
71.
TRUSTED
$7.4
93.6
96.
12.6
18.4
235.
M52
300
15.2
22.7
7122
31.0
77.1 -210
21.0
12.3
667
12.3
61.
03.0
95.6
Drw
91.2
955
94.5
43.0
715
61.8
76.
TRUNTED
07.3
93.3
95.
14.1
M52
31.4 175
230 116-1
$30 -100
351 1354
50.9
12.1
56.1
754-364
617
15.9
16.9
Drw
83.8
90.3
574
32.0
385
Fial
87.
81.0 2229
764
38
TRUSTED
92.7
959
When training and deploying a classifier, a key question is choosing a pretrained encoder. It can be beneficial in critical applications to trade off the main task accuracy to ensure better OOD detection. In Tab. 3, we report the individual performance of OOD methods on three types of classifiers. Although TRUSTED achieves state-of-the-an art on all configurations, it is worth noticing a difference in performance concerning the type of pretrained model. is also important to remark that for ROB both MSP and E achieve on-par performances with D& while not requiring any extra training information. Overall, for a given method (i.e. D' or DIRW), the ranking of detectors performances according to the type of feature extractor remain still. This validates the use of the mean-aggregation procedure of TRUSTED. Overall, based on the difference of OOD detection performance of Tab. 3, we recommend to use TRUSTED on BERT DIS. if accurate OOD detection is required.
8

---

5.3 Impact of the training dataset
OOD detection performance depends on the nature of what is considered in-distribution (the training distribution our case). Thus, it interesting study the performance per-IN as reported in Tab. 4. Even though TRUSTED achieves strong results in terms of AUROC, we observe a high FPR on SST2. From Fig. 4, we observe that IMDB and SST2 are harder to detect, especially for DM. Finally, since high AUROC does not necessarily imply low FPR, it is crucial to take both into account when designing an OOD detector.
AUROC
AUP3- 15
AUPR: OUT
FPR
Err
IN-DS
Score
Alln
TRESTFF
VK.4
16.3
98.0 21.7
K.8
2:3-3
6.4
97.6
97.4
10.1
imdh
THUSTED
98.6
99.8
88.6
95.3
27.3
19.3
30.7 2229
212.3
332.3
223.2
30,4
122.
15
Dic
TRUSTED
-2.3
91.8
12.2 613
11.0 2107
FFR
90.0 01.3
99.8
86.1
67.4
Fig. 4: AUROC&FPR tra
Tab. 4: Average OOD detection performance per IN-DS.
6 Dynamic Experimental Results
Most OOD detection methods are tested on specific checkpoints, where the selection criterion is often unclear. The consequences of this selection OOD detection are rarely studied. This section aims to respond to this by measuring the OOD detection performance of methods on various checkpoint finetuning of the pretrained encoder. We will use 5 different checkpoints taken after 1k, 3k, 5k, 10k, 15k, and 20k iterations. Training curves of the models are given in Fig. 7. Notice that after 3k iterations models have converged and over-fitting is observed even after 20k iterations (i.e. we do not observe an increase in validation loss).
Overall analysis. We report the results of the dynamic analysis on the various pretrained models in Fig. 5. For all the methods and models (except DM on ROB), we observe that training longer the classifier hurts detection. Interestingly, this drop in performance has a higher impact on FPR compared to AUROC. Thus, it is better to use an early stopping criterion to ensure proper OOD detection performance. In addition, observed that TRUSTED (corresponding to DIRW) achieves better detection results and that D& outperforms TRUSTED for checkpoints larger than 10k on ROB.
056
1.90
0m6
1.44
2005
3625
3000
064
042
Stapa
6.5
2.4
12
0.1
1.0
Bayo
Staps
Fig. 5: Detection performance of different pretrained encoder during finetuning. First column correspond to BERT, second to ROB and last to DIS.

---

Analysis Per In-Dataset. Fig. 6 reports the re- sults of the dynamical analysis per IN-DS. We ob- serve that DIRW is consistently better than DM with sensible improvement on IMDB and SST2, which are the hardest benchmarks. We observe similar trend the previous experiment: train- ing longer the classifiers hurt their OOD detection performances. Similar observations hold for FPR, AUPR-IN, and AUPR-OUT that are postponed to the Supplementary Material (see Fig. 10).
AUROG
md
146
DAv
0.25 0.50
0.75
1.00
1.25
1.50
1.75
2.00
Steps
Fig. 6: AUROC per IN-DS during fine-tuning.
7 Conclusions and Future Directions
In this work, we introduced TRUSTED a novel OOD detector that relies on information available in all the hidden layers of network. TRUSTED leverages a novel similarity score built on top of the Integrate Rank Weighted depth. We conduct extensive numerical experiments proving that it consistently outperforms previous approaches, including detection based on the Mahalanobis distance. Our comprehensive evaluation framework demonstrates that, in general, OOD performances vary depending on several hyperparameters of the models, the datasets, and the detector's 's feature extraction step. Thus, would like to promote the use of such exhaustive evaluation frameworks for future search to assess AI systems' safety tools properly. Another interesting question is the detection inference-time accuracy trade-off, which is instrumental for the practitioner.
Acknowledgments
This work was also granted access to the HPC resources of IDRIS under the allocation 2021- APO1061 1665 as well under the project 2021-101838 made by GENCI. This work has been supported by the project PSPC AIDA: 2019-PSPC-09 funded by BPI-France.
References
[1] Faruk Ahmed and Aaron Courville. Detecting semantic anomalies. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3154-3162, 2020.
[2] Udit Arora, William Huang, and He He. Types of out-of--ditrrbuutin texts and how detect them. arXiv preprint arXiv:2109.066 0.0070 2021.
[3] Ho Bae, Jaehee Jang, Dahuin Jung. Hyemi Jang. Heonseok Ha, Hyungyu Lee, and Sungroh Yoon. Security and privacy issues in deep learning. arXiv preprint arXiv:1807.11655. 2018.
[4] Anas Barakat and Pascal Bianchi. Convergence and dynamical behavior of the adam algorithm for non-convex stochastic optimization. arXiv preprint woXiv:/110.02263. 2018.
[5] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness in machine learning. Nips tutorial, 1:2, 2017.
[6] Marco Barreno, Blaine Nelson, Anthony D Joseph, and J Doug Tygar. The security of machine learning. Machine Learning, (222)122 2010.
[7] K. Bhatia, K. Dahiya, H. Jain, P. Kar, A. Mittal, Y. Prabhu, and M. Varma. The extreme classification repository: Multi-label datasets and code, 2016. URL http::/manikee org/ dwww//ods/////ddoooooooodddoa /C/ /tt html.
[8] Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri. Findings of 2016 conference on machine translation. In Proceedings of the First Conference on Machine Translation, pages 131-198, Berlin, Germany, August 2016. Association for Computational Linguistics. URL http: / /wwwwwwww...oo////ooo/// aclweb. org/ Aattholoyy///W11//// 2301.
10

---

[9] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx. Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, al. On the opportunities and risks of foundation models. arXiv preprini arXiv: 108.07258, 2021.
[10] Andrew P Bradley. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern recognition, 30(7): (110::1155111550 1997.
[11] Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod Varshney, and Dawn Song. Anomalous example detection in deep learning: A survey. IEEE Access, 8: 00133333-133344 2020.
[12] J. Burger and L. Ferro. Generating an entailment corpus from news headlines. In Proceedings of the ACI. Workshop on Empirical Modeling of Semantic Equivalence and Entailment, pages 49 54. Association for Computational Linguistics, 2005.
[13] Nadia Burkart and Marco F Huber. A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research, 70:245-317, 2021.
[14] Ewen Callaway al. Deepmind's s ai predicts structures for a vast trove of proteins. Nature, 595(7869):635-666 635, 2021.
[15] Emile Chapuis, Pierre Colombo, Matteo Manica, Matthieu Labeau, and Chloe Clavel. Hierar- chical pre-training for sequence labelling spoken dialog. arXiv preprint arXiv:2009. 2020.
[16] Bo Chen, Kai Ming Ting, Takashi Washio, and Gholamreza Haffari. Half-spacc mass: maximally robust and efficient data depth method. Machine Learning, 100(2):677-699, 2015.
[17] Jiahao Chen, Victor Storchan, and Eren Kurshan. Beyond fairness metries: Roadblocks and challenges for ethical ai in practice. arXiv preprint arXiv:2108.005 2021.
[18] Cyril Chhun, Pierre Colombo, Chloé Clavel, and Fabian M Suchanek. Of human criteria and automatic metrics: benchmark of the evaluation of story generation. arXiv preprint arXiv:2208. /646, 2022.
[19] Hyunsun Choi, Eric Jang, and Alexander A Alemi. Waic, but why? generative ensembles for robust anomaly detection. arXiv preprint arXiv: 1810.01392, 2018
[20] Pierre Colombo. Learning to represent and generate text using information measures. PhD thesis, Institut polytechnique de Paris, 2021.
[21] Pierre Colombo, Wojciech Witon, Ashutosh Modi, James Kennedy, and Mubbasir Kapadia. Affect-driven dialog generation. arXiv preprint arXiv: http:/9444.9999 2019.
[22] Pierre Colombo, Emile Chapuis, Matthieu Labeau, and Chloé Clavel. Code-switched inspired losses for spoken dialog representations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8320-8337, 2021.
[23] Pierre Colombo, Emile Chapuis, Matthieu Labeau, and Chloe Clavel. Improving multimodal fusion via mutual dependency maximisation. arXiv preprint arXiv:2 (tfi::2009009222 00922, 2021.
[24] Pierre Colombo, Chloe Clave, and Pablo Piantanida. Infolm: A new metric to evaluate summarization& & data2text generation. arXiv preprint arXiv:2112.01588) 2021.
[25] Pierre Colombo, Chloe Clavel, and Pablo Piantanida. novel estimator of mutual information for learning to disentangle textual representations. arXiv preprint arXiv:2
2021.
[26] Pierre Colombo, Guillaume Staerman, Chloé Clavel, and Pablo Piantanida. Automatic text evaluation through the lens of Wasserstein barycenters. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10450-10466, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguis- tics. doi: 10. 00.18653/v1/2022 http:////22...mmmmmmm........ mainp- emnlp-main 817.
URL stttps://aclantholll0/ org or2/2021.
[27] Pierre Colombo, Chouchang Yang, Giovanna Varni, and Chloé Clavel. Beam search with bidirectional strategies for neural response generation, arXiv preprint 270.022110003389, 2021.
11

---

[28] Pierre Colombo, Nathan Noiry. Ekhine Irurozki, and Stéphan Clémençon. What are the best systems? new perspectives on nlp benchmarking. arXiv preprint arXiv:2202.03799, 2022.
[29] Pierre Colombo, Guillaume Staerman, Nathan Noiry, and Pablo Piantanida. Learning disentangled textual representations via statistical measures of similarity, arXiv preprint arXiv:2205.0353 2022.
[30] Alexander D' Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al. Under- specification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03399 2020.
[31] Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pages 233-240, 2006.
[32] Roy De Maesschalck, Delphine Jouan-Ringe Sitan-Rimbuud, and Désiré L. Massart. The mahalanobis distance. Chemometrics and intelligent laboratory systems, 50(1):1-18, 2000.
[33] Yves-Alexandre de Montjoye, Ali Farzanehfar, Julien Hendrickx, and Luc Rocher. Solving artificial intelligence's 's privacy problem. Field Actions Science Reports, 17(1):80-83, 2017.
[34] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810004805. 2018.
[35] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguisties: Human Minneapolis,
Language Technologies. Volume 1 (Long and Short Papers), pages Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/N19-1 1423. URL https:// acl http//aaannnoooogyyyooooo org/N19- 1423.
[36] Tanvi Dinkar, Pierre Colombo, Matthieu Labeau, and Chloé Clavel. The importance of fillers for text representations of speech transcripts. arXiv preprint arXiv:2009. 11340, 2020.
[37] David L. Donoho and Miriam Gasko. Breakdown properties of location estimates based on half space depth and projected outlyingness. The Annals of Statistics, 10:1803-1827, 1992.
[38] Rotem Dror, Segev Shlomov, and Roi Reichart. Deep dominance-how to properly compare deep neural models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2773-2785, 2019.
[39] Rainer Dyckerhoff and Pavlo Mozharovskyi. Exact computation of the halfspace depth. Computational Statistics & Data Analysis, 98:19-30, 2016.
[40] Desmond Elliott, Stella Frank, Khalil Sima' an, and Lucia Specia. Multi30K: Multilingual English-German image descriptions. Proceedings of the 5th Workshop on Vision and Language, pages 70-74, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10. 18653/111/11633222 URL httpps://aclaatthlllooee/
org/W16-32111
[41] José M Faria. Machine learning safety: An overview. Proceedings of the 26th Safety-Critical Critical Systems Symposium, York, UK, pages 6-8, 2018.
[42] Varun Gangal, Abhinav Arora, Arash Einolghozati, and Sonal Gupta. Likelihood ratios and generative classifiers for unsupervised out-of-domain detection in task oriented dialog. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7764-7771, 2020.
[43] Alexandre Garcia, Pierre Colombo, Slim Essid, Florence d' A'chéé Buc, and Chloé Clavel. From the token to the review: A hicrarchical multimodal approach to opinion mining, arXiv preprint arXiv: 1216, 2019.
[44] Eduardo Dadalto Camara Gomes, Florence Alberge, Pierre Duhamel, and Pablo Piantanida. Igeood: An information geometry approach out-of-distribution detection. arXiv preprint XXXv:2203.07799 2022.
12

---

[45] Mare Hallin, Davy Paindaveine, and Miroslav Siman. Multivariate quantiles and multiple- output regression quantiles: From 11 optimization to halfspace depth. The Annals of Statistics, 669,, 04 2010.
[46] Godfrey Harold Hardy, John Edensor Littlewood, George Pólya, György Pólya, al. Inequali- ties. Cambridge university press, 1952.
[47] Dan Hendryeks and Kevin Gimpel. A baseline for detecting misclassified and out-of- distribution examples in neural networks. arXiv preprini arXiv: 16,0.0016,, 2016.
[48] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv::8122 04606, 2018.
[49] Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. Pretrained transformers improve out-of-distribution robustness. arXiv preprint arXiv:2004.000 000/000000000000 100, 2020.
[50] Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety. arXiv preprint arXiv:2109, 13916, 2021.
[51] A. Hickl, J. Williams, J. Bensley, K. Roberts, B. Rink, and Y. Shi. Recognizing textual entailment with lec's groundhog system. In Proceedings of the Second PASCAL. Challenges Workshop, 2006.
[52] Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and 7solt Kira. Generalized odin: Detecting out-of- distribution image without learning from out-of-distribution data. 2020 JEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 10948-10957, 2020.
[53] Hamid Jalalzai, Pierre Colombo, Chloé Clavel, Éric Gaussier, Giovanna Varni, Emmanuel Vignon, and Anne Sabourin. Heavy-tailed representations, text polarity classification & data augmentation. Advances in Neural Information Processing Systems, 33:4295 4307, 2020.
[54] Thorsten Joachims. A probabilistic analysis of the rocchio algorithm with tfidf for text categorization. Technical report, Carnegie-mellon univ pittsburgh pa dept of computer science, 1996.
[55] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint ar1iv: 69800, 2014.
[56] Gleb A. Koshevoy and Karl Mosler. Zonoid trimming for multivariate distributions. The Annals of Statistics, 25(5):1998-201 , 10 1997.
[57] Eren Kurshan, Hongda Shen, and Jiahao Chen. Towards self-regulating ai: Challenges and opportunities of ai model governance in financial services. In Proceedings of the First ACM International Conference on Al Finance, pages 1-8, 2020.
[58] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting dusttr-disttitution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018
[59] Jesse Levinson, Jake Askeland, Jan Becker, Jennifer Dolson, David Held, Soeren Kammel, Zico Kolter, Dirk Langer, Oliver Pink, Vaughan Pratt, et al. Towards fully autonomous driving: Systems and algorithms. 2011 IEEE intelligent vehicles symposium (IV), pages 163-168. IEEE, 2011.
[60] Xiaoya Li, Jiwei Li, Xiaofei Sun, Chun Fan, Tianwei Zhang, Fei Wu, Yuxian Meng, and Jun Zhang. k folden: k-fold ensemble for out-of-distribution detection. arXiv preprint arXiv:2 10t.110..111111 12731, 2021.
[61] Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th Interna- tional Conference on Computational Linguistics, 2002. URL aawlaacooaootoyyyc.o org/ C02-1155
13

---

[62] Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distributee image detection neural networks. In International Conference on Learning Representations, 2018. URL.
net/forum? htte:foorummdd-HHGGk...... id=H12G
[63] Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. Explainable ai: A review of machine learning interpretabilliyy methods. Entropy, 23(1):18, 2020.
[64] Bo Liu, Ming Ding, Sina Shaham, Wenny Rahayu, Farhad Farokhi, and Zihuai Lin. When machine learning meets privacy: A survey and outlook. ACM Computing Surveys (CSUR), 54 (2):1-36, 2021.
[65] Regina Y. Liu. On a notion of data depth based on random simplices. The Annals of Statistics, 18(1):405 414, 1990.
[66] Regina Liu. Data depth and multivariate rank tests, pages 279-294. 1992.
[67] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of of-dissributtinn detection. Advances in Neural Information Processing Systems, 2020.
[68] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint
2019.
[69] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:/711.05100 2017.
[70] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christo- pher Potts. Learning word vectors for sentiment analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguisties: Human Language Technologies, pages 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. URL https: // acl www.hnlnnoooyoooooooooooooooooot org/P11-
[71] Prasanta Chandra Mahalanobis al. On the generalised distance in statistics. In Proceedings of the National Institute of Sciences of India, volume 2, pages 49-55, 1936.
[72] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6): 1-35, 2021.
[73] Stanislav Nagy, Rainer Dyckerhoff, and Pavio Mozharovskyi. Uniform convergence rates for the approximated halfspace and projection depth. Electronic Journal of Statistics, 14(2): 3939-3975 2020.
[74] Georg Pichler, Pierre Jean Colombo, Malik Boudiaf, Günther Koliander, and Pablo Pi- antanida. A differential entropy estimator for training neural networks. In International Conference on Machine Learning, pages 17691-17715. PMLR, 2022.
[75] Alexander Podolskiy, Dmitry Lipin, Andrey Bout, Ekaterina Artemova, and Irina Pio- ntkovskaya. Revisiting mahalanobis distance for transformer-baut out-of- detection. arXiv preprint arXiv:2101.03778. 2021.
[76] Kelly Ramsay, Stéphane Durocher, and Alexandre Leblane, Integrated rank-weighted depth. Journal of Multivariate Analysis, 173:51-69, 2019.
[77] Bharath Ramsundar, Steven Kearnes, Patrick Riley, Dale Webster, David Konerding, and Vijay Pande. Massively multitask networks for drug discovery. arXiv preprint 2015.
[78] Jie Ren, Peter J Liu, Emily Fertig, Jasper Snock, Ryan Poplin, Mark A DePristo, Joshua V Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. arXiv preprint arviv: /906. 02845, 2019.
[79] Peter Rousseeuw and Mia Hubert. Regression depth. Journal of the American Statistical Association, 94(446):388-4p)) 402, 1999.
14

---

[80] Peter J. Rousseeuw and Mia Hubert. Anomaly detection by robust statistics. Wiley Interdisci- plinary Reviews: Data Mining and Knowledge Discovery, 8(2),e 2018.
[81] Peter J. Rousseeuw and Anja Struyf. Computing location depth and regression depth in higher dimensions. Statistics and Computing, 8(3):193-22) 1998.
[82] Andreas Rücklé, Steffen Eger, Maxime Peyrard, and Iryna Gurevych. Concatenated power mean word embeddings as universal tross-lingual sentence representations. arXiv preprint arXiv:/803.01400, 2018
[83] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 2019.
[84] Chandramouli Shama Sastry and Sageev Oore. Detecting -uttr---ditriuutionn examples with Gram matrices. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th Interna- tional Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 8491-8501. PMLR, 13-18 Jul 2020. URL https: / http///epoo.e....c//e/// mlr.
press/v119 v11///111///// //st///y/sssaaaaaa/ html.
[85] Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribution examples with gram matrices. International Conference Machine Learning, pages 8491-8501, PMLR, 2020.
[86] Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D'Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, et al. The multiberts: Bert reproductions for robustness analysis. arXiv preprint arXiv:2106.16166 2021.
[87] Robert Serfling, Depth functions in nonparametric multivariate inference. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, 72, 2006.
[88] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang. Christopher D. Manning, Andrew Ng. and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL https::/aalaatthlll.g/ org/D13- 1170.
[89] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-199 1958, 2014.
[90] Guillaume Staerman. Functional anomaly detection and robust estimation. PhD thesis, Institut polytechnique de Paris, 2022.
[91] Guillaume Staerman, Pavlo Mozharovskyi. Stéphan Clémençon, and Florence d' Alché Buc. Functional isolation forest. In Proceedings of The Eleventh Asian Conference on Machine Learning, pages 332 332-347, 2019.
[92] Guillaume Staerman, Pavlo Mozharovskyi, and Stéphan Clémençon. The area the convex hull of sampled curves: robust functional statistical depth measure. In Proceedings of the 23nd International Conference on Artificial Intelligence and Statistics (AISTATS 2020), volume 108, pages 570-579, 2020.
[93] Guillaume Staerman, Pavlo Mozharovskyi, Stéphan Clémençon, and Florence d' Alché Buc. A pseudo-metric between probability distributions based on depth-trimmed regions. arXiv preprint arXiv:2103. 1, 2021.
[94] Guillaume Staerman, Pavlo Mozharovskyi, and Stéphan Clémençon. Affine-invariant inte- grated reikh depth: Definition, properties and finite sample analysis. arXiv preprint a0Xiv:2106. 1068, 2021.
[95] Guillaume Staerman, Pavlo Mozharovskyi, Pierre Colombo, Stéphan Clémençon, and Florence d' d'Acch Buc. pseudo-metric between probability distributions based on depth-trimmed regions. arXiv preprint arXiv:2103.12711 2021.
15

---

[96] Guillaume Staerman, Eric Adjakossa, Pavlo Mozharovskyi, Vera Hofer, Jayant Sen Gupta, and Stephan Clémençon. Functional anomaly detection: a benchmark study. arXiv preprint arXiv:2201.05 10220010511333 15, 2022.
[97] John W. Tukey, Mathematics and the picturing of data. In R.D. James, editor, Proceedings of the International Congress of Mathematicians, volume 2, pages 523-531. Canadian Mathematical Congress, 1975.
[98] Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, and Furu Wei. Deepnet: Scaling transformers to 1,000 layers. arXiv preprint arXiv:2203.00533 2022.
[99] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), pages 1112-1122, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10. 10.18653/v1/N18 1101. URL https: https://acclae htaps://aaclanthooogyyy. www.echhooooooooooooooooooo www//o/-////00000
[100] Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R. Ledsam, Patricia Mac Williams Pushmeet Kohli, Alan Karthikesalingam, Simon A. A. Kohl, taylan. cemgil, S. M. Ali Eslami, and Olaf Ronneberger. Contrastive training for improved out-of-distributtin detection. ArXiv, abs/2007 .05566,,, 2020.
[101] Wojciech Witon, Pierre Colombo, Ashutosh Modi, and Mubbasir Kapadia. Disney at iest 2018: Predicting emotions using an ensemble. In Proceedings of the 9th Workshop on Computational Approaches Subjectivity, Sentiment and Social Media Analysis, pages 248-253, 2018.
[102] Ruigi Zhong, Dhruba Ghosh, Dan Klein, and Jacob Steinhardt. Are larger pretrained language models uniformly better? comparing performance at the instance level. arXiv preprint arXiv:2105.080 06020, 2021.
[103] Wenxuan Zhou, Fangyu Liu, and Muhao Chen. Contrastive out-of-distribution detection for pretrained transformers. arXiv preprint S.Ki::2044008122 2021.
[104] Yijum Zuo and Robert Serfling. General notions of statistical depth function. The Annals of Statistics, 28(2):461-44) 4822, 2000.
16


Reviews:
[{"review_id": "zrYNrqNvnCf", "reviewer": "Reviewer_FF5o", "summary": "This paper proposes an OOD detector for Transformer models. Their motivation is that all hidden layers of transformer-based models contribute to OOD detection. They verify their method with extensive experiments and show the performance jump on AUROC.", "questions": "- Their method is based on the Mahalanobis score, which depends on the gaussian assumption. However, this is questionable; such an assumption still fits a real-world setting. Instead of the Mahalanobis score, I am wondering if the authors consider other feature-space scores for their method. Also, the computation burden or Mahalanobis also needs to be discussed.\n- They use Integrated Rank-Weighted depth in their method, which has been used in some previous related works. This downgrade the novelty of the contribution in this paper.\n- Woking on textual OOD problem is reasonable; however, since their methods rely on the transformer model, the natural question is, could it be applied to other domains with Transformer-based models, such as the vision domain?\n- Their selected datasets have different domain characteristics. For example, IMDB is a movie review, while QA is another domain. I wonder if they consider some harder OOD settings during their experiments?\n- It is interesting their observation that training longer the classifier hurts detection. Do the authors consider training those models from scratch without pretraining, for example, train 6-layer transformer vs. 12-layer transformer from scratch and show the power of their metric without pretrained weights.", "limitations": "The detector proposed in this paper is simple and effective. However, this method is still incremental and still has some room to be improved. The detector proposed in this paper is simple and effective. However, this method is still incremental and still has some room to be improved. For example, if this paper uses intermedia hidden layers, then if the pretrained model affects the use of hidden states needs to be discussed. Also, when it comes to the vision domain, could this method still work?", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Their motivation that going deeper inside the network can improve OOD detection is somehow limited. Also, intermediate representation for detection tasks is not new, at least in the vision domain. This paper works on NLP domain but is still somehow incremental.\n- Their observation that using all hidden layers is straightforward and reasonable, but simply taking an average of those latent representations is simply.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "uIcbAnI9-E", "reviewer": "Reviewer_Ezx2", "summary": "This paper develops a new tool for out-of-distribution (OOD) detection in the NLP domain, which aims to complement the need of deformation techniques as that in computer vision. The key to detecting OOD relies on the computation of the OOD score that is realized by an integrated rank-weighted (IRW) depth.  Extensive experiments are conducted to show the superiority of the proposed method.", "questions": "1. As there are many different tools to solve the OOD problem, what is the exact motivation of borrowing the IRW depth to solve this problem? \n2. The reviewer expects to see the connection of the proposed IRW depth and the deformation technique in CV.  Can the authors provide some theoretical results?\n", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper aims to solve an interesting topic.\n2. Extensive experiments are conducted and show competitive performance for the proposed method.\n3. This paper has delivered a thorough analysis of the related work.\n\nWeakness:\n1. The paper suffers from limited technical novelty.  For example, \n\n1) The proposed IRW depth is borrowed for existing work. \n\n2) The paper has claimed that one of the contributions is to aggregate all hidden layers of a neural network.  However, this kind of trick has been proposed in the literature.  The authors should cite the related work and highlight their main differences.\n\n3) The reviewer doubts whether there is any theoretical guarantee on the proposed method.  In particular, the reason for choosing \"the halfspace depth\" in line 174 is unknown.  \n\n2. Some main technical implementations are not clear.  For example, \n\n1) In line 149, the notation of \\delta is not defined.  It makes it hard for the reviewer to know how to compute the similarity score and justify its correctness.  Moreover, \\delta has appeared twice in the paper.  One is in lines 132-133, and the other is in line 149.  This makes the reviewer a little confused about understanding the meaning.  \n\n2) In line 147, the authors define F_{PM}(S) as the distribution of the mean-aggregation of the training distribution samples with the same predicted target as x.  However, it applies different aggregation strategies in Sec. 4.3, lines 246-255.\n\n3) In line 159, “a pre-score aggregation function”: the definition is not shown.\n\n4) In lines 182-183, u_k and n_{proj} are not defined. \n  \n\n3. Critical issues in the experiments:\na) Though the authors claim that the motivation for introducing IRW depth is to reduce the computational complexity, there are no such experiments to verify this issue.  The paper should address it through experimental support.\nb)\tIn Table 2, it contains some contradictory conclusions and lacks sufficient experiment comparison.\n\ni.\tThe paper claims the information from aggregating all layers in the neural network is more useful. However, for the D_M score in Table 2, the result of the last layer F_L is much better than of all layers F_{cat} and F_{PM}, which shows a contradiction result.  The authors have tried to explain the results in lines 269-270, i.e., the multivariate Gaussian assumption may not hold for the dataset.  It makes the reviewer confuse about the experimental comparison and makes unconvincing the experimental results because the aggregation of F_L is also under the Gaussian assumption and it beats F_{PM}. \n\nii.\tIn D_{IRW}, it only compares F_{cat} and F_{PM} with F_L and F_{L+1} and shows that  F_{cat} and F_{PM} beat F_L and F_{L+1}.  The comparison is not sufficient.  For example, the authors can consider other combinations, such as concatenating F_L and F_{L+1} or averaging F_L and F_{L+1}.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GCU8yl9nYK5", "reviewer": "Reviewer_o5Ud", "summary": "This work proposes TRUSTED: an out-of-distribution detection approach that uses integrated rank weighted (IRW) depth to assess whether an example falls within a set of in-distribution samples in the embedding space of a transformer model. The average embeddings across all transformer layers are used for optimal performance. TRUSTED is compared against 1) integrated rank weighted depth using final embedding layer, 2) Mahalanobis distance instead of IRW and 3) maximum softmax probability on a text OOD benchmark and shows superior performance.", "questions": "See Weaknesses.", "limitations": "Limitations are sufficiently addressed", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Approach and experiment settings are clearly explained.\n2. Code is provided for reproducibility.\n3. The detector can be applied to new transformer models without any training, as opposed to data-driven methods.\n\nWeaknesses:\n1. Experiments didn't show comparison against (any) existing works. \n- [36]\n- [82]\n- Arora, Udit, William Huang, and He He. \"Types of Out-of-Distribution Texts and How to Detect Them.\" EMNLP 2021.\n\n2. Approach is fairly simplistic. Novelty is limited. More suitable as a \"strong baseline\" type of approach.\n- There's even very limited contributions to benchmarking of text OOD detection given that approach is simple. At the current state, a sound & large OOD benchmark with diverse text, different types of OOD, with text from multiple domains may be the easiest way to boost the contributions of this work.\n\n3. Questions:\n- Table 2 uncertainties are large even after averaging over 1440 configurations. Why is that?\n\n=========================\n\nWith comparisons to existing works, the authors have adequately addressed my strong concerns on the experiments. I have updated the score.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "EqF3V14QmAB", "reviewer": "Reviewer_gSwg", "summary": "The authors propose a novel out-of-distribution (OOD) detector for text classification based on similarity scores from the training distribution. The goal is to identify OOD samples with an unsupervised and fast method by leveraging information from the hidden layers of the representation model.  The main contributions are: i) use of hidden representations to compute a distance score for OOd detection, ii) comprehensive comparison of models and hyper-parameters, and iii) open source for replicability. The study shows that the proposed method has competitive results compared to related work.", "questions": "Questions to the Authors\n\nPlease address the following questions during the rebuttal:\n\n- Could you elaborate on the hyper-parameter selection and define the classifier architecture. \n- During training, are the pre-trained models freeze?\n- As an extra contribution, the authors can compute calibration and sharpness of the predicted probability outputs. Please speculate on the relation between well calibrated output probabilities and your method to identify OOD samples. \n", "limitations": "The authors have addressed limitations of the proposed approach.\n", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n\n- Clear description of background knowledge and related work needed to understand the proposed approach.  \n\n- Clear description of the proposed approach.\n\n- The authors perform a comprehensive comparison with related work.\n\n\n\nWeaknesses\n\n\n- It is not clear how the parameter initialization  and selection of hyper-parameters could affect the method performance.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept