Mahalanobis++: Improving OOD Detection via Feature Normalization
Authors: Maximilian Müller
Matthias Hein
Venue: ICML 2025 poster (2025)
Keywords: OOD detection
Mahalanobis distance
out-of-distribution detection
Primary area: deep_learning->robustness
TL;DR: We show that the Mahalanobis distance estimation is degraded by strong variations in the feature norm and provide a simple fix (projection to the unit sphere) that consistently improves the method and leads to new SOTA results.

Abstract: Detecting out-of-distribution (OOD) examples is an important task for deploying reliable machine learning models in safety-critial applications. While post-hoc methods based on the Mahalanobis distance applied to pre-logit features are among the most effective for ImageNet-scale OOD detection, their performance varies significantly across models. We connect this inconsistency to strong variations in feature norms, indicating severe violations of the Gaussian assumption underlying the Mahalanobis distance estimation. We show that simple $\ell_2$-normalization of the features mitigates this problem effectively, aligning better with the premise of normally distributed data with shared covariance matrix. Extensive experiments on 44 models across diverse architectures and pretraining schemes show that $\ell_2$-normalization improves the conventional Mahalanobis distance-based approaches significantly and consistently, and outperforms other recently proposed OOD detection methods.

Mahalanobis++: Improving OOD Detection via Feature Normalization
Maximilian Müller
Matthias Hein
Abstract
FPR Improvement of Mahalanobis++ over Mahalanobis
Detecting out-of-distribttion (OOD) examples is an important task for deploying reliable ma- chine learning models in safety-critial applica- tions. While post-hoe methods based on the Ma- halanobis distance applied pre-logit features are among the most effective for ImageNet-scale OOD detection, their performance varies signif- icantly across models. We connect this incon- sistency to strong variations in feature norms, indicating severe violations of the Gaussian as- sumption underlying the Mahalanobis distance es- timation. We show that simple fz-oormalization of the features mitigates this problem effectively, aligning better with the premise of normally dis- tributed data with shared covariance matrix. Ex- tensive experiments on 44 models across diverse architectures and pretraining schemes show that l --normalization improves the conventional Ma- halanobis distance-based approaches significantly and consistently, and outperforms other recently proposed OOD detection methods. Code is avail- able at cwwp///ellee-mmmmmaaaaoooooo mp/maha-norm.
15
FPR Difference
10
5
Models sorted by FPR difference
Figure I. Normalizing features improves OOD detection with the Mahalanobis distance consistently. Shown is the difference in false-positive rate true positive rate of 95% between unnormal- ized and normalized features for 44 ImageNet models, averaged over five OOD datasets of the OpenOOD benchmark.
genuine ID samples to pass through normally. OOD de- tection methods are commonly divided into methods that require modifications to the training process and so-called post-hoc detection methods that can be applied to any pre- trained network. For many downstream tasks (not only OOD detection), the best results are achieved by models that have been pretrained on large datasets, some of which might not be publicly available. Since adjusting the training scheme for these networks is usually not feasible, simple post-hoc OOD detection is most often used in practice.
1. Introduction
Deep neural networks have demonstrated remarkable perfor- mance across a variety of real-world tasks. However, when faced with inputs that fall outside their training distribu- tion, they can behave unpredictably and even result in high- confidence predictions (Hendrycks& & Gimpel, 2017; Hein et al., 2019). These so-called out-of-distribution (OOD) in- puts are often misclassified with high confidence as belong- ing to the in-distribution (ID) classes, creating significant risks for real-worid deployments. OOD detectors aim to identify and reject such anomalous inputs potentially prompting human intervention, transitioning safe state, or declining to provide prediction while still allowing
Common post-hoc OOD detection methods are based on a scoring function that typically inputs either the logit/softmax outputs of a model (Hendrycks & Gimpel, 2017; Hendrycks et al., 2022; Liu al., 2020), or the pre-logit features (Lee et al., 2018b; Ren et al., 2021; Sun et al., 2022), both (Sun et al., 2021; Wang et al., 2022). VisionTransforment have shown particular success in this area (Koner et al., 2021). For large-scale settings where, e.g., ImageNet is the ID dataset, they perform particularly well (Galil et al., 2023), especially when paired with feature-based methods (Bitterwolf al., 2023). Among those, the Mahalanobis distance (Lee et al., 2018b; Ren et al., 2021) stands out as a particularly effective and simple scoring function. However, despite leading for some models to trate-of-the-ar OOD performance, it fails for others and shows high performance variation across different models and pretraining schemes, and brittleness when confronted with supposedly easy noise distributions as OOD data (Bitterwolf al., 2023).
Uniierssity of Tübingen and Tübingen Al Center. Corre- spondence Maximilian Müller <maximilian, maaaiiiiiaan.uulllr@@e tuebingen.de>>
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Unnormalized
Normalized
Class 10
Class
Class 300
Class 300
Class 633
Class 633
global bal fit
global
ID
FPR: 18.42%
DDD
DOD TPR -
TPR-999 95%%
FPR = 33.58%
95%
SMaha
SMaha +
Figure 2. Mahalanobis++:: We illustrate how to improve Mahalanobis-hasee OOD detection. Left: For unnormalized features, assuming a shared covariance matrix for all classes leads to suboptimal OOD detection (bottom) with the Mahalanobis score. Center: Normalizing the features, i.c. projecting them onto the unit sphere mitigates this problem effectively. Right: After normalization, the fit of the shared covariance matrix is tighter for all classes, leading to improved OOD detection as in- and out-distribution are better separated. Shown are the Mahalanobis++ scores for pretrained ConyNex(V2-L on NINCO, which achieves new state-of-the-art FPR of 18.4% (see Tab. 6).
In this work, we observe that for models where the Maha- lanobis distance does not work well as OOD detector, the assumptions underlying the method are often not well satis- fied. In particular, the feature norms vary much more than expected when assuming a Gaussian model with a shared covariance matrix. To mitigate this problem, we provide a simple solution, called Mahalanobis++, which we visualize in Figure 2: By projecting the features onto the unit sphere before estimating the Mahalanobis distance, we significantly reduce the class deeendet feature variability and obtain better fit of the covariance matrix, which ultimately leads to consistent improvements in OOD detection, as demon- strated in Fig. 1 or Tab. 4.
2. Related Work
Mahalanobis distance. Most closely related to our work are the well-established OOD detection methods based on the Mahalanobis distance. Lee et (2018h) proposed to esti- mate class-conditional Gaussian distribution with a shared covariance matrix "with respect to (low- and upper-level) features", and to use the minimal Mahalanobis distance to the respective mean vectors as OOD score. Since then, the community has transitioned to using only the pre-logit fea- tures. Ren et al. (2021) proposed to additionally estimate class-agnostic mean and covariance matrix and use the difference between the two resulting scores as OOD score, called relative Mahalanobis distance. These methods have demonstrated broad applicability, spanning domains such as medical imaging (Anthony & Kamnitsas, 2023) and self- supervised OOD detection (Sehwag et al., 2021). Gaussian mixture models (GMMs) represent a more comprehensive framework for modelling feature distributions. They have been applied to small-scale setups but require tweaks to the training process (e.g. spectral normalization) (Mukhoti et al., 2023). Adapting them to ImageNet-scale setups as post-hoc OOD detectors has so far not been successful.
In summary, our contributions are the following:
We observe that the assumptions underlying the Ma- halanobis distance as OOD detection method, in par- ticular that the features are normally distributed with a shared covariance matrix, are often not well satisfied
We relate this to variations in the feature norm, which can vary strongly across and within classes, and corre- lates with the Mahalanobis distance
Feature norm, The role of the feature norm for OOD detec- tion has been investigated in several works (Yu et al., 2020). Park et al. (2023b) underline that the norm of pre-logit fea- tures are equivalent to confidence scores and that the feature norms of OOD samples typically smaller than those of ID samples. Their observations are mostly based on results obtained with strong over-training and simple networks. We will show that this observation does not hold generally. Gia & Ahn (2023) investigate the role of the & norm in con- trastive learning and OOD detection. Regmi et al. (2024)
We provide an easy solution, which we call Maha- lanobis++: Normalizing the features by their norn before computing the Mahalanobis distance
We evaluate Mahalamobis++ across large range of models with different pretraining schemes and archi- tectures on ImageNet and Cifar datasets and find that it consistently outperforms the conventional Maha- lanobis distance and other baseline methods, and im- proves the detection of far-OOI noise distributions
2

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
and Haas et al. (2024) try to leverage the feature norm to dis- criminate between ID and OOD samples. In particular, they concurrently suggested training with ég-normalized features and then using the norm of the unnormalized features as OOD score at inference time, similar to Yu et al. (2020) and Wei et al. (2022).
where OOD detection on NINCO with Mahalanobis score performs significantly worse (FPR of 58.2%) than for other similar models like the ViT-B VT--I1-IIn2I1 lellinIk-augeee k-augreg with 84.5% accuracy (Steiner et al., 2022) but low FPR of 31.3% using the Mahalanobis score.
3.1. Mahalanobis Distance
Spherical embeddings. Spherical embeddings have been investigated and leveraged across several fields (Liu al., 2018; Zhou et al., 2022; Sablayrolles et al., 2018; Yaras et al., 2022), also within the OOD detection literature (Zheng et al., 2022). Ming et al. (2023) proposed CIDER, a contrastive training scheme that creates well-separated hyperspherical embeddings via a dispersion loss and applies KNN as de- tection method at inference time. Sehwag et al. (2021) also train with a contrastive loss, and apply the Mahalanobis dis- tance OOD detection method on the normalized features at inference time. Haas et al. (2023) observe that normaliz- ing features during train and inference time improves perfor- mance on the DDU benchmark (Mukhoti et al., 2023). They hypothesize that their training scheme induces early neural collapse, which might benefit out-of-distribution detection capabilities of networks. Importantly, all those methods are train-time methods, i.e. require modifications to the training process, including feature normalization either explicitly in the case of Haas et al. (2023), or implicitly through the contrastive loss in Ming et al. (2023) and Mukhoti et al. (2023). They then apply normalization at inference time, because they also normalized at train time. In contrast, we highlight the benefits of feature normalization when apply- ing the Mahalanobis distance post-hoe OOD detection method in this work- which is non-obvious for generic pretraining schemes.
The Mahalanobis distance is a simple, hyperparameter--fi post-hoc OOD detector that has been suggested by Lee et al. (2018b). Given the training set (I,, yo) with input o, and class labels yi one estimates: i) the class-wise means A. and ii) a shared covariance matrix Σ:
Me =
Σ ((x))
No
1 C
5 =
Σ
Σ 1:BL=C
(M(m)) - (a)(())))) - Ra)T
(2)
N
C
where 6(x;;) are the pre-logit features of xq. No the number of train samples in class e, N the total number of train sam- ples, and C the total number of classes. The Mahalanobis distance of test sample x; to class mean jee is then
disaha (Iz, Pc) = (δ(z)) - - ( - Ae) (3)
and the final OOD MMah( of xu is the negative smallest distance to one of the class means:
SMaha
min dMaha Pe)
(4)
If Maha ( < T then the sample is rejected as OOD, where for evaluation purposes T is typically determined by fixing TPR of 95% on the in-distribution. The core assumption of Lee et al. (2018a) that "the pre-trained features of the softmax neural classifier might also follow the class- conditional Gaussian distribution". Indeed, one implicitly uses a probabilistic model where each class is modelled as a Gaussian ((ee, Σ) with a shared covariance matrix E, which can be seen as a weighted average of the covariance matrices of the features of each class: E = Ee= N. S N with E. N: Eiy (d(x))) (ob())) - fa." with the weight No N being an estimate of P(Y = =c).
Cosine-based detection scores. Many previous works have suggested using the angle, or more specifically, the cosine, for OOD detection, but those mostly require modifications to training or architecture (Techapanurak et al., 2020; Tack et al., 2020), or are used for unsupervised setups (Radford et al., 2021; Ming et al., 2022). Park et al. (2023a) and Sun et al. (2022) use nearest neighbour search in the normal- ized feature-space, which amounts to a nearest neighbour search in the cosine space. We show that Mahalanobis++ outperforms cosine-based OOD detection methods.
3. Variations in feature norm degrade the performance of Mahalanobis-based OOD detectors
The Mahanalobis score is a strong baseline for OOD detec- tion as noted in Bitterwolf et al. (2023) where they report for a particular Vision Transformer (ViT) trained with augreg (a carefully selected combination of augmentation and regular- ization techniques) by Steiner et al. (2022) state-of the-art art results on their NINCO benchmark comparing several mod- els and OOD detection methods. On the other hand other ViTs like DeiT or Swin that are equally strong in terms of classification performance showed degraded OOD detection results. Moreover, Bitterwolf et al. (2023) report that the
In this Section, we investigate the assumptions underlying the Mahalanobis distance as OOD detection method. We report results for NINCO (Bitterwolf et al., 2023) as OOD dataset. For all experiments, we use a pretrained ImageNet SwinV2-B-In22 k model (Liu et al., 2022) with 87.1% Ima- geNet accuracy. This strong model is a prototypical example
3

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Mahalanobis-basee OOD detector performs worse on their "unit tests" of simple far-OOD test sets than other methods.
particular class are Gaussian distributed. In particular, the feature norm would be concentrated, the following lemma shows. Lemma 3.1. Let (xX) (u, E). Then
In the remainder of this section, we will try to identify the reasons for the varying performance of Mahalanobis- based OOD detection. Our main hypothesis is that is due to violations of its core assumptions:
Var (10(())12
P (1110(X31B (tr(2)) + ||a11)1 2()
e²
Assumption I: the classwwise features (ó(a;)) )))m; fol- low a multivariate normal distribution NHue, 2).
where Var(I = ==1 and (N.) are the eigenvalues of E.
-
Assumption II: the covariance matrix Σ the same for all classes.
This implies that [[x(X) should be concentrated around v (r(()) + -1ml[/.- In the right part of Fig. 3, we show the distribution of the norms of the training features across classes for the SwinV2-B model, i.e. the feature norms of those samples that were used for estimating class means and covariance. In the left part of Fig. we show the distribution of feature norms when sampling from M(se, Σ) for every class c. As expected from the derived Lemma, the sampled norms vary little around their mean value. It is evident by the differences of the left and right part of Fig. 3. that the fit with class-cona coasscoondiiinall means and shared covariance matrix does not represent the structure of the data well as the observed feature norms of V2- show heavy tails (right) which would not be present if the data was Gaussian (left). In Figure 8 we show that similar heavy-tailed feature norm distributions but with different skewness can be found even for the same ViT-architecture where the Mahalanobis score does not work well. This shows that Assumption I of the Mahalanobis score is not fulfilled across models, and models can deviate heavily from it. In contrast, for the ViT-augreg (Steiner et al., 2022), which has been shown to have very good OOD detection performance with the Mahalanobis score (Bitterwolf et al., 2023), the feature norms behave roughly as expected under the Gaussian assumption (right plot in Figure 8).
Below, we will show that these assumptions do not hold for some models, as indicated in Fig. 2. One strong indicator of this violation is the norm of the features, which turns out to be strong confounder, ultimately degrading the OOD detection performance with Mahalanobis-based detectors.
For completeness, we mention the Relative Mahalanobis score here, proposed by Ren et al. (2021), also suggested as a fix to the Mahalanobis score. They argue that for the detection of near- near-OOD, one should use likelihood ratio of two generative models compared to the likelihood used in the Mahalanobis method. Thus they fit global Gaussian distribution with mean Àgiohal and covariance matrix Egiohal, and use the difference between the dlass-conditioaal and the global Mahalanobis score as OOD score.
Expected
Observed
Aug. Norm
50
Min/Max
Std Dev
40
30
20
0
500
1000
0
500
1000
Sorted Classes
Sorted Classes
To further evaluate the adherence to Assumption I, we cen- ter training features of the Swin V2-B by their class means: geenter (a:) = -(cf)) These centered features, used for covariance estimation, should ideally follow a zero-mean multivariate normal distribution.
The feature norms vary strongly across and within classes. Left: We simulate how the feature norms per class would be distributed if they were sampled from Gaussians with the means and covariance matrix used for the Mahalanobis distance estima- tion. Right: The actual feature norm distribution observed in practice. Both the average norms across classes and the norms within each class vary much stronger than expected.
Figure 3.
To quantify deviations from normality, we use Quantile-uuantile (QQ) plots, stan- dard approach in statistics (see, e.g. Wilk & Gnanadesikan (1 (1968)) which compares sample quantiles against those of a theoretical distribution (here, the standard normal). A straight diagonal line indicates agreement with the theo- retical distribution; deviations highlight mismatches. To enable direct comparison between models (and later be- tween normalized and unnormalized features), we standard- ize (gcente(xx) by its empirical standard deviation. While standardization technically alters the distribution (as the em- pirical variance is cample-dependent) we expect this to be
3.2. Is the Gaussian fit in feature space justified?
As the features o(r;) E Rdd for input x are high-dimensional, e.g. d = 1024 for a V2-B we expect some concen- tration of measure phenomena if the features ((x) of a
4

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
Swin
DeiT
Table Variance aligmment. measure how much the class- variances deviate from the global variance via the deviation score (see Eq. 5). Lower values indicate better alignment. Normalization aligns the features of SwinV2 and DeiT3, but not ViT-augreg.
Small EV Dir. Ord. vals
-5
unnormalized
normalized
SwinV2-B- B---2-2-B22kk
0.26
0.12
0.0
DeiT3-B16-In22
0.24
0.15
10
10
Random Dir. Ord. vals
ViT-B16-In22k-auurre
0.05
0.05
negligible due to the large dataset size (> 10th samples). We report QQ-plots for three directions for a Swin and DeiT3 (blue lines in Figure 4), and observe strong deviations from the ideal diagonal line, indicating that the centered fea- tures have much stronger tails than expected if the features followed Gaussian distribution, further refuting Assump- tion I. We observe similar heavy tails in QQ-plots of other models where the Mahalanobis score not working well for OOD detection (see Fig. 9 App. D). Only the VIT with augreg training has QQ plot close to the expected one.
-5
-10
-10
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
Large EV Dir. Ordered values
10
5
Maha
Maha++
-10
-10
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
Theo. quantiles
Theo. quantiles
To assess the validity of Assumption II, we measure how strongly the individual class variances deviate from the global variance. To this end, we compute the expected relative deviation over all directions:
Figure 4. QQ-plot: & normalization helps transform fea- tures to be more aligned with normal distribution. For Swin and DeiT3 model (where the feature norms vary strongly across and within classes) normalization shifts the distribution towards a Gaussian (black line).
Eul(u" A 2tr(A²) + +t((A)²
(5)
across classes observed above. In Fig. 7, observe the same correlation for other models (again, the ViT-B- augreg being an exception). In Fig. 6 in the Appendix, we substantiate this observation by artificially scaling the fea- ture norm of OOD samples, leading to improved detection when the feature norm is increased and worse detection when the feature norm decreased.
2)
where has uniform distribution the unit sphere and A = Σ- 2-2 -E, ΣンΣー & (see App. C for derivation). We average over all classes i and report the results for a SwinV2. B, a DeiT3-B and a ViT-augreg in Table We observe that the V2 and DeiT3 show significantly larger deviations than the ViT-augreg, indicating that the class-specific vari- ances differ more. More models in Tab. 7 in the Appendix.
The heavy correlation between feature norm and OOD score implies that images yielding small feature norm are not detected as OOD (see Fig. 6 for discussion). This also explains why the simple OOD unit tests in Bitterwolf et al. (2023), using synthetic images of little variation, c.g. black or uni-colour images, often fail. These synthetic images contain little variation in color, which often results in small activations in the network, and thus small pre-logit features, see Figure 10 for an analysis.
3.3. Correlation feature norm and SMaha-SCrre
The strong variations within and across classes we observed in Figure 3 indicate that the feature norm might impact the Mahalanobis estimation. To investigate this, plot the fea- ture norm against the Mahalanobis score 8 Mahe assigned by the V2i-V22B model for ID and OOD test samples (i.e. samples that were not used for estimating means and covari- ance) Figure 5. We observe a clear correlation: Samples with large feature norms consistently receive a large OOD score, and vice versa for samples with small feature norms irrespective of whether they belong to the in out distri- bution. Ideally, detector should be able distinguish ID from OOD samples irrespective of the norm of the OOD samples. Since a large fraction of the OOD samples have a comparably small feature norm, the resulting OOD de- tection performance is, however, poor. The reason for this strong correlation the strongly different average feature
4. Mahalanobist+: Normalize your features
A challenge with Mahalanobis distance-based OOD detec- tion is its sensitivity feature norms, which can strongly correlate with Mahalanobis scores. We further find the feature distribution to strongly contradict the theoretical Gaussian assumption (with shared covariance), empirical feature norms vary much more in practice than expected. To address this mismatch, we propose a simple but effective fix: Discarding the feature norm and leveraging only directional information in the features by l E3-normalization.
5

---

Mahalanobist+: Improving OOD Detection via Feature Normalization
5PR=58.2%
FPR=31.3%
directions, normalization (green line in Figure 4) shifts the feature quantiles closer to the diagonal line, confirming that Mahalanobis++ better satisfies the Gaussian assumption of Mahalanobis- detection. We validate this for more models in Figure 9 in the Appendix.
Feature norm
Variance alignment. In Table I, we observe lower vari- ance deviation scores for normalized features of the Swin V2 model compared to unnormalized features, indicating that normalization aligns the class variances in Mahalanobis+ +. We illustrate this effect in Figure 2. which visualizes cen- tered training features for three selected classes along a random direction. Without normalization, class feature vari- ances differ substantially, and the shared covariance matrix fails to jointly capture their distributions. After normaliza- tion, class variances become more consistent, making the shared variance assumption more appropriate. To further validate this, we examine which in-distribution test samples are flagged as OOD at 95% true-positive rate: unnormal- ized Mahalanobis rejects samples from 634 classes, while Mahalanobis+ rejects samples from 728 classes. In an ideal setting with a perfect covariance fit, one would expect samples be drawn uniformly from all 1,000 classes. The increase from 634 728 classes suggests that normalization reduces bias in the covariance estimation, better aligning with the shared variance assumption. We substantiate our observations in Figure 11 and Table 7 in the Appendix for more models. We find that class variances are more similar to the global variances after normalization for all models except the ViT-augreg.
20
-15000
-10000
-5000
4000 -3000 -2000 -1000
MMata
SMakr+++ +
ID
000
TPR-95%
Figure 5. The feature norm correlates with the Mahalanobis score for Swin Left: The smaller the feature norm, the smaller the Mahalanobis OOD score Maaaa irrespective of whether sample is ID or not. OOD samples with small feature norms are systematically classified as ID. Right: After normaliza- tion, OOD samples with small feature norms can be detected, and OOD detection is significantly improved.
Method. Instead of the original features (xx), we use lz- normalized ones for computing the Mahalanobis score:
d(z;)) = <<c,)//ló(z))))
(6)
The class-means and covariance matrix of the Mahalanobis score are estimated using the normalized features, and also test features are normalized when computing their score. We denote this simple modification as Mahalanobis++.
We note that l nooormaalzttiin has been used with non- parametric post-hoc OOD detection methods like KNN (Sun et al., 2022; Park et al., 2023a) or cosine similarity (Techapanurak et al., 2020). With the Mahalanobis score, however, lg n--omallzaaiinn has- to the best of our knowl- edge only been investigated for train-time methods like SSD+ (Sehwag et al., 2021) or CIDER (Ming et al., 2023). Those methods normalize their features for OOD detection because they also normalize during training. This is orthog- onal to our work: The standard Mahalanobis method for OOD detection is a post-hoc method, where adjusting the pretraining scheme is not feasible. We show below that Mahalanobis++ + outperforms KNN and cosine similarity in all considered cases and, in particular. improves OOD detection consistently across tasks, architectures, training methods and OOD datasets as post-hoc method.
Decoupling of feature norm and OOD score. In Figure 5 the right, we plot the feature norm of ID and OOD sam- ples against their OOD scores obtained via Mahalanobis+ +. In contrast to the conventional Mahalanobis score, the corre- lation between OOD score and feature norm (before normal- ization) is much weaker. In particular, OOD samples with small feature norm are now also detected as OOD, which was not the case for unnormalized Mahalanobis.
5. Experiments
ImageNet. Our main goal is to investigate the effective- ness of Mahalanobis+ across a large pool of architectures, model sizes and training schemes for ImageNet-scale OOD detection, as this where the conventional Mahalanobis distance showed the most varied results in previous studies (Bitterwolf et al., 2023; Mueller & Hein, 2024). To this end, we use 44 publicly available model checkpoints from timm (Wightman, 2019) and huggingface co. Following the OpenOOD setup (Yang et al., 2022), we report results on Ninco (Bitterwolf et al., 2023), iNaturalist (Van Horn et al., 2018), SSB-hard (Vaze et al., 2022), OpenImages-O (Krasin et al., 2017) and Texture (Cimpoi et al., 2014). We report the false positive rate at true positive rate of 95% (FPR)
Improved normality. To evaluate how Mahalanobis++ improves the adherence to the assumption of Gaussian model with shared covariance matrix, we compare the resulting feature distributions via QQ-plots to the unnormal- ized features. Like for the unnormalized features, we center Joenter (x.) = )(x:) - Ao(t) divide by the empirical stan- dard deviation, and plot the resulting quantiles against the quantiles of standard normal. We observe that across all
6

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Table ImageNet. FPR (lower is better) on five OpenOOD datasets. Green indicates that normalization improves over unnormalized features, bold indicates the best and underlined the second best method. Mahalanobis++ consistently improves over Maha and baselines.
model
MonwNextV2-B-I211
SwinV2-B-In2I1
3- 16--i2k
dataset
NIN SSB
TxT OpO iNat
Avg
NIN SSB
TxT OpO iNat
Avg
NIN SSB TxT OpO iNat Avg
MSP (Hendrycks& & Gimpel, 2017)
41.4 60.1 47.4 24.6
8.7
36.5
48.2 63.8 51.7 32.5
21.1
43.4
61.0 73.2 66.0 46,5 32,9
55.9
MaxLogit (Hendrycks et al., 2022)
31.9 51.1 40.7
16.5
4.9
29.0
38.6
52.6 47.7 24.6
13.0
35.3
55.2 67.2 61.9 41.4 34.3
52.0
Energy (Liu et al., 2020)
30.1 47.8 39.5
14.6
4.2
27.2
38.3
47.8 50.9
26.3
13.9
35.5
55.9 65.2 63.2 43.3
45.6
54.7
GEN (Liu et al., 2023)
29.7 52.3 35.9
13.8
3.5
27.1
37.0
57.0 38.7
17.6
8.9
31.8
45.2 61.5 50.1 24.8
13.7
39.0
Reect (Sun al., 2021)
29.5 48.0 38.6
13.9
3.7
26.7
35.1
48.8
44.8
18.5
7.2
30.9
50.9 63.8 55.2 32.1
27.3
45.9
fDBD (I.iu& & Qin, 2024)
37.0 60.4 37.9
15.4
3.8
30.9
50.5
74.5
41.9
19.1
6.1
38.4
53.5 70.9 50.7
24.8
11.8
42.4
VIM (Wang et al., 2022)
26.9 47.7 28.1
79
1.1
22.4
50.4
75.7 35,4
15.4
17
35.7
55.3
75.7 48.8
21.1
4.8
41.1
KNN (Sun al., 2022)
40.9 59.0 32.9
16.5
6.5
31.2
57.2
82.3
35.4
18.5
6.8
40.0
52.6
73.7 43.7
21.1
9.7
40.2
Neco (Ammar et al., 2024)
27.7 45.9 35.0
12.5
2.8
24.8
32.6
48.7
39.8
17.6
588
28.9
51.5
64.8 57.4
34.1
24.2
46.4
NNguide (Park al., 2023u)
31.7 53.5 31.6
12.7
3.3
26.6
42.7
72.5 33.0
12.3
3.5
32.8
46.4
68.4
44.3
19.3
9.3
37.5
Rel.-Mahalanobis (Ren al., 2021)
28.1 54.6 33.2
11.7
2.0
25.9
48.2
74.0
39.7
19.3
3.5
36.9
47.4 69.8
46.2
20.1
6.0
37.9
Rel-MMaallnnobis++
24.7 51.6 32.1
11.3
2.2
24.4
34.4
62.5
36.8
15.7
3.9
30.6
38.3 61.6
42.5
17.1
4.0
32.7
Mahalanobis (Lee et ul., 2018b)
30.3
53.8
30.4
9.4
1.4
[25.0
58.2
81.4
41.5
23.2
3.5
41.6
52.5
72.8
47.0
21.4
5.5
39.8
Mahalanobis++
22.4 46.9
26.5
7.8
1.3
21.0
31.3
62.0
28.7
9.7
1.6
26.7
38.8 62.8 42.0
15.6
3.1
32.5
as the OOD detection metric and refer the appendix for other metrics, such as AUC, details on the model check- points, baseline methods, and extended results. In addition to Mahalanobis++, we also report relative Mahalanobis+ Le the relative Mahalanobis distance with lg normalization.
Table 3. CIFAR100. Green indicates that normalization im- proves the baseline, bold and underlined indicate the best and second best method, We report FPR averaged over OpenOOD datasets. Maha++ is the best method. The best FPR is achieved by
Maha++ for ViT-S16-21k highlighted in blue.
Model
MSP
Ash
ML
KNN
VIM
MD
MD++
We report results on the five OOD datasets in Table using three pretrained base-size models: ConyNextV2 (Woo et al., 2023), Swin (Liu et al., 2022) and DeiT3 (Touvron et al., 2022). For all models, Mahalanobis++ + outperforms the con- ventional Mahalanobis distance consistently across datasets, and is the best-performing method on average, and in most cases also per dataset. Also the relative Mahalanobis++ outperforms its counterpart across models and datasets, but slightly worse on average. In Table 4, we show the results averaged over the five datasets for 44 models with different training schemes, model sizes and network types. With the exception of three models (two of which are trained with augreg), Mahalanobis++ outperforms its counterparts in all cases. relative Mahalanobis++ + outperforms its coun- terpart in 39/44 cases. In 30/44 cases, the best performing method is Mahalanobis++ (in 6/44 cases it is relative Ma- halanobis+ +) and the differences the baseline methods are often large. Averaged over models, Mahalanobis++ is the best method, followed by relative Mahalanobis++ and outperforming the previously best method ViM by 7 FPR points. We note that Mahalanobis++ + is particularly effective for the best-performing models, as it is the best method for 4 of the top-5 models.
SwinV2-S-Ik
47.28
92.66
40.96
36.27
34.02
40.10
26.01
Deit3-S-21k
48.92
94.47
42.37
36.81
39.99
41.99
31.72
ConvN-T-21k
60.60
92.11
57.44
51.16
51.18
52.48
42.69
VIT-B32-21k
48.02
93.98
31.28
26.49
27.14
26.28
18.94
VIT-S16-2IK
52.17
80.45
37.63
31.91
24.90
25.51
18.58
RNI8
80.59
78.98
79.87
76.61
79.61
79.48
72.92
RN34
76.93
78.27
75.33
74.44
77.17
76.63
74.51
RNx129-32
82,31
72.59
82.30
73.17
76.40
77.67
67.71
Average
62.10
85.44
55.90
50.86
51.30
52.52
44.13
EVA and DeiT networks). For several other networks (e-g.- ConvNexts, Mixer, ResNets, EfficientNets, Swins,.... differ- ences are, however, larger and often in the range of 8- 15% FPR. We note that most of the OOD datasets in OpenOOD show contamination with ID samples, as reported in Bitter- wolf et al. (2023). Therefore, we report results on Ninco, which has been cleaned from ID data, separately in Table 6, and find even clearer improvements of Mahalanobist+.
Two of the three models for which Mahalanobis++ does not bring an improvement are ViTs trained with augreg by Steiner et al. (2022). Those are the models that showed state- of-the-art performance in Bitterwolf et al. (2023). We extend our observations from the previous Sections regarding these models in Appendix D, where we show that their feature norms are already well- therefore, l2 normalization does not improve the normality assumptions.
We further note that NNguide (Park et al., 2023a) and KNN (Sun et al., 2022), both of which operate in nor- malized feature space, are consistently outperformed by Mahalanobis++. The most competitive baseline method that is not based on the Mahalanobis distance is ViM (Wang et al., 2022), which for certain models shows similar or slightly better performance than Mahalanobis++ (c.g. for
Bitterwolf et al. (2023) reported that Mahalanobis- based detectors sometimes fail to detect supposedly easy-to-detect noise distributions (called "unit tests") "). In Section 3, we connected this to the small feature norm those samples ob-
7

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Table FPR on OpenOOD datasets, Green indicates that a normalized method is better than its unnormalized counterpart, bold indicates the best and underlined the second best method, Maha++ improves over Maha on average by 7.6% in FPR over all models. Similarly, +Maha++ is, on average, 2.9% better in FPR than rMaha. In total, Maha++ improves the SOTA compared to the strongest competitor rMaha among all OOD methods by 6.9%, which is significant improvement. The lowest FPR is achieved by Maha++ for the
EVA02-L144M.8m-Inn2222 k highlighted in blue.
Model
Acc
MSP
E+R
ML
VIM
AshS
KNN
NNG
NEC
GMN
GEN
fDBD
Maha
Maha++
rMaha
rMaha++
Con-NoXc-B-I22I
86.3
41.7
40.1
36.0
37.3
29.5
88.5
37.2
31.8
31.4
54.2
32.6
57.9
33.6
24.3
31.7
29.5
ConyNeXI-B
84.4
61.4
90.9
86.9
70.2
52.8
99.5
58.7
51.2
66.5
73.9
60.1
60.3
54.2
44.6
50.0
45.4
Com-NeXIV2.T.I1222
44.7
37.3
38.6
27.0
96.7
41.6
36.4
33.2
47.2
42.3
32.5
28.6
34.6
33.4
ComNeeXVV-BB-n22Ik
87.6
36.5
27.2
26.7
29.0
22.4
95.3
31.2
26.6
24.8
38.9
27.
30.9
25.0
21.0
25.9
24.4
CowwNcXIV--LIn211
88.2
35.0
27.0
26.5
28.5
28.7
95.6
30.8
25.9
24.1
32,9
26.4
31.6
27.8
18.8
25.8
23.0
Cow.NeXIV2-T
83.5
60.5
66.1
58.6
58.9
49.9
99.2
72.1
62.8
54.1
73.9
53.6
61.8
55.4
44.4
48.9
44.6
ConvNeXIV2-B
85.5
58.8
70.8
64.1
59.5
46.8
99.6
53.4
47.9
55.9
71.2
48.6
53.7
46.3
37.5
43.0
39.1
ConvNeXtV2-L
58.6
68.0
58,3
48.4
99.1
48.9
44.7
55,9
63.8
41.7
36.2
39.0
DeiT3 I16k
84.8
60.5
53.3
SO.4
54.4
47.6
99.2
49.8
47.5
51.6
$2.0
47.8
54.4
50.2
42.4
48.9
43.6
DeiT3-BR66In21k
86.7
55.9
54.7
45.9
52.0
41.1
99,2
40.2
37.5
46,4
46.2
39.0
42.4
39.8
32.5
37.9
32.7
DeiT3-LI6-In2IIk
87.7
55.0 O
45.5
38.3
46,9
36.4
98.1
35.0
32.1
38.5
36.8
34,6
38.2
34.9
30.1
33.4
29.9
DeiT3-S16
83.4
56.9
54.0
58.1
52.2
43.3
85.6
69.8
48.2
52.2
64.1
46.6
54.2
52.4
46.5
49.1
44.9
Dei13-B16
85.1
59.7
82.33
88.4
64.4
44.7
99.2
66.1
71.3
63.5
63.5
46.0
54.8
51.5
46.7
48.3
45.0
DeiT3-L.16
85.8
60.3
80.5
89.3
64.0
46.1
78.4
54.0
72.5
64.3
56.9
45.1
52.4
45.3
39.7
42.7
38.6
EVAUZ-B14-In21k
88.7
32.4
26.8
28.8
22.0
87.9
29.6
25.08
25.0
36,0
25.5
21.0
26.2
EVA02 M38mm In2kk
90.1
27.0
22.6
22.4
24.3
18.0
91.0
25.8
22.8
21.88
39.9
20.3
23.9
19.7
17.7
21.1
20.4
EVA02-TI4
80.6
8
66.2
66.8
63.1
49.3
98.4
60.8
57.1
55,4
54.1
57.9
66.4
51.0
48.1
52.6
50.7
EVA02-S14
85.7
52.2
53.4
53.1
49,5
34.8
99.1
44.1
40.3
42.9
43.0
41.7
48.9
36.6
35.4
38.1
36.8
EIINetV2-S
83.9
59.3
7L.0
58..7
61.1
52.2
99.4
45.6
45.2
59.3
77.9
49.7
54.0
47.3
40.2
43.6
40.4
EIINeeVV2LL
85.7
57.1
74.2
$7.4
58.8
48.9
99.2
48.9
47.1
56.0
58.1
44.7
49.2
41.3
34.6
38.0
34.8
FffNeeV2-M
85.2
57.0
69.3
56.7
57.3
54.7
99.5
51.3
48.6
54.9
66.8
45.2
52.6
46.0
37.1
41.1
36.8
Mixer (iker-B16-In211
76.6
71.5
75.0
71.8
95.8
77.8
83,9
75.5
71.8
63.3
52.5
60.0
$2.9
SwinV2-B-In21k
87.1
41.4
35.5
30.9
35.3
35.7
77.0
40.0
32.8
28.9
57.7
31.8
38.4
41.6
26.7
36,9
30.6
SwinV2-L.In2lk
87.5
40.4
35.9
31.2
34.5
39.0
85,1
38.9
32.9
29.0
48.8
31.5
37.5
41.8
24.7
36.2
28.7
SwinV2-S
84.2
61.2
68.1
62.1
60.9
51.1
99.9
58.6
52.9
56.0
61.2
$2.8
60.7
52.4
38,9
48.7
39.3
SwinV2-B
84.6
62.4
66.2
58.2
60.5
49.9
99.1
55.0
51.1
55.4
56.1
49.9
56.9
47.9
40.1
45.2
39.7
ResNet101
81.9
67.7
82.8
99.6
70.7
50.5
80.2
53.6
51.4
70.6
82.3
62.5
71.3
45.9
433.5
55.6
66.8
ResNet152
82,3
66.4
82.1
99.5
70.0
49.7
80.0
52.0
46.8
69.1
77.2
60.3
69.3
44.4
38.3
51.8
64.7
ResNe150
72.0
99.4
75.8
53.1
80.3
67.8
64.1
76.6
89.5
74.8
$2.0
62.5
70.4
ResNet50-supoon
78.7
54.0
47.3
42.1
48.4
72.0
40.6
47.0
41.9
47.8
78.8
$3.5
48.0
95.5
44.5
90.2
63.7
75.5
70.7
55.3
48.4
58.3
51.1
94.9
76.2
71.0
52.8
58.2
64.7
58.5
55.5
48.0
59.2
57.7
HI--S1--n2IK-aaurrg
81.4
57.0
38,9
42.5
41.7
33.4
76.7
55.6
48.9
38.1
44.3
46.5
44.0
36.7
31.7
43.9
40.6
VIT-B16 In21k augreg?
85.1
55.3
45.9
41.1
47.5
53.9
98.6
47.5
42.2
43.7
60.1
42.9
51.4
54.2
38.2
47.0
39.1
VIT-B16-In21k. 12236-1221--mmrree
84.5
46.5
33.7
36.0
34.6
26.9
94.9
54.3
45.6
32.4
38.6
36.5
36.4
25.7
28.3
30.8
31.5
VIT-B16-In2Ikee 16-oo2l
81.8
44.6
30.7
30.9
33.1
29.0
62.6
6
35.4
30.5
48.8
38.4
35.7
30.9
27.5
35.4
33.9
VIT-B16-In2Ik.ne miil
48.0
38.8
37.8
96,9
45.0
38.5
33.9
57.1
47.1
30.4
43.6
36.7
VIT-LI6-In2 Ik-angreg
85.8
40.2
29.4
25.0
30.0
23.6
94.5
50.6
41.2
28.0
41.6
30.7
30.4
21.0
23.9
25.2
25.8
ViT-L16-In2Ik-orig
81.5
40.8
29.3
29.2
31.1
30.4
49.3
34.0
31.6
29.4
47.9
35.2
33.0
30.9
26.8
33.6
32.6
ViT-S16-augreg
78.8
64.8
59.0
60.6
60D
68.1
96.9
71.5
68.9
60.2
61.8
61.8
64.8
49.3
49.2
48.4
48.2
VIT augreg
79.2
64.3
59.6
56.2
60.1
63.4
90.2
65.5
64.1
59.9
60.3
61.5
63.5
49.6
48.0
47.6
46.7
VIT-BI6-CCLPP..22-IIIZX
86.2
42.2
37.7
35.5
37.2
35.5
99.5
35.6
31.6
34.0
41.4
33.4
38.0
43.2
28.1
38.0
32.4
WTF-JJA-CLIP--L2--I122
88.2
31.5
25.2
24.6
26.5
21.5
97.6
29.9
22.3
26,3
36.3
24.3
27.3
28.2
22.4
27.1
25.4
VIT:HI
88.6
32.0
26.5
26.1
27.7
22.3
97.8
31.2
23.4
27.5
53.0
28.9
27.1
22.0
26.8
25.1
VIT=so400M-Sggiip
89.4
45.5
47.1
39.4
41.8
30,6
93.5
28.7
26.1
39.6
64.3
28.3
29.9
28.8
24.5
27.3
25.5
Average
84.4
52.7
53.0
51.0
49.0
41.9
90.7
48.9
44.8
46.0
56.3
43.6
47.8
42.5
34.9
41.8
38.9
tain. In Table 5 we report the number of "failed' unit tests (a unit test counts as failed when detector shows FPR values above 10%) and observe that normalization, in particular Mahalanobis++ remedies this effectively. For results on all models, we refer to Table 17 Appendix.
ImageNet (Le & Yang, 2015), Mnist (LeCun et al., 1998).
SVHN (Netzer et al., 2011) 1), Texture (Cimpoi et al., 2014), Places (Zhou et al., 2017) and Cifar as OOD datasets for a range of architectures and training schemes.
We report results averaged across the OOD datasets in Ta- ble for the most competitive methods and standard base- lines (full results in Appendix E). We observe that Maha- lanobis++ consistently outperforms the conventional Maha- lanobis distance, but the differences are smaller compared to the ImageNet setup. We hypothesize that this is because the problems of the Mahalanobis distance are less drastic at smaller scale, and therefore the conventional Mahalanobis distance is already fairly effective for OOD detection. ViM and KNN the most competitive baseline methods, but Mahalanobis+ + remains the most consistent and effective method across models.
Table 5. Normalization improves robustness against noise dis- tributions. We report the number of failed unit tests (noise dis- tributions with FPR values 2 10%) from Bitterwolf et al. (2023). Noralizatiion remedies the brittleness of Mahalanobis-baat based de- tectors. Full Table in Appendix E.
model
ConvNeX1V2
SwinV2
ViT-CLIP
Maha
5/17
10/17
14/17
Maha++
0/17
0/17
0/17
CIFAR We investigate Mahalanobis++ on CIFAR100 (Krizhevsky, 2009), following the OpenOOD setup with tiny
8

---

Mahalanobist+::
Improving OOD Detection via Feature Normalization
Table FPR on NINCO, Green indicates that normalized method is better than its unnormalized counterpart, bold indicates the best method, and underlined indicates second best method. Maha++ improves over Maha on average by 10.9% in FPR over all models. Similarly, rMaha++ is 6.0% better in FPR than rMaha. In total, Maha++ improves the SOTA compared to the strongest competitor rMaha among all OOD models by 6.3% which is significant. The lowest FPR is achieved by Maha++ for the ConwNeXIV2-L--In) blue.
highlighted
Model
Val Ace
MSP
E
E+R
ML
VIM
AshS
KNN
NNG
NEC
GMN
GEN
IDBD
Maha
Mahn++
rMaha
MMaha++
ConvNeXi-B
86.3
43.0
40.5
92.7
38.0
48.3
48.7
32.5
Com-NeXt-B
84.4
64.1
89.4
86.2
71.4
64.7
99.6
70.1
62.2
68.2
68.3
65.9
68.6
64.6
50.5
59.5
49.7
CowNNcXVV2-TIn221k
85.1
51.6
42.4
42.4
44.1
34.5
97.5
54.1
45.0
38.7
52.5
44.2
51.4
40.9
32.8
40.0
36.8
Com-NeXIV2.B-In21k
87.6
41.4
30.1
29.5
31.9
26.9
95.8
40.9
31.7
27.7
40.6
29.7
37.0
30.3
22.4
28.1
24.7
CowNeXXV2-L-I1221
88.2
38.7
30.7
29.8 8
31.5
37.2
96.0
38.7
29.9
27.0
43.4
29.0
36.6
34.6
18.4
27.7
21.4
ConvNcXIV2-T
83.5
66.1
73.3
65.7
99.2
82.3
72.3
64.1
71.6
66.1
52.3
49.7
ConwNeXIVV-B
85.5
62.9
73.9
69.8
63.5
61.1
99.4
67.3
60.3
60.7
69.2
$7.1
65.1
58.9
44.7
52.4
44.1
ComvNeXiV2.L
86.1
72.3
63.6
62.4
99.5
56.9
71.2
55.6
59.7
53.8
42.1
DeiT3-S16-In21k
84.8
68.7
61.8
59.8
62.6
60.6
99.7
62.9
59.3
60.1
58.5
58.7
65.8
62.4
50.8
59.8
50.9
DefT3-B16-n221k
86.7
61.0
55.9
50.9
55.2
55.3
99.5
52.6
46.4
51.5
52.2
45,2
53.5
52.5
38.8
47.4
38.3
DeiTS-L.66-I211k
87.7
59.7
4622
41.8
48.9
45.7
98.4
43.8
37.8
42.3
43.7
38.2
46.6
42.0
33.9
38.1
32.8
DeiT3-S16
83.4
64.3
63.0
63.8
60.6
54.3
84,1
75.6
57.8
60.6
66.4
57.4
65.0
61.1
53.5
56.3
50.5
Der33-B1
85.1
87.8
72.6
59.7
99.1
71.9
66.7
57.3
67.1
63.7
57.2
53,2
De1
85.8
67.8
62.3
86.6
70.5
$7.9
81.1
67.2
77.9
70.8
62.9
58.4
64.4
$7.0
50.4
$2.0
46.6
EVA02-R14-In2IK
88.7
35.8
28.2
27.4
30.9
28.7
92.7
30.0
27.3
39.0
25.8
32.3
31.7
25.9
EVA00-L14-M55mm-n22kk
90.1
29.0
24.3
24.1
25.7
21.44
94.8
30.3
26.1
22.9
39.5
20,3
26.0
22.2
18.6
22.1
20.1
EVA02-T14
80.6
72.7
74.5
75.2
72.1
67.7
98.8
74.5
71.0
68.4
65.6
70.5
73.5
65.9
64.0
65.9
64.4
EVA02-S14
85.7
61.2
61.4
61.5
57.8
51.6
98.9
60.0
54.0
53.2
51.9
53.1
60.0
49.3
48.0
49.1
47.8
EffNetV2-S
83.9
67.7
77.5
73.3
69.5
74.0
99.7
60.9
59.6
69.4
79.9
62.9
67.9
67.5
59.9
59.2
52.1
EIINNeV2-LL
85.7
63.7
77.2
69.4
988.9
56.3
62.4
47.8
44.3
EIfNeiV2-M
85.2
63..4
75.1
69.1
63.8
72.3
99.6
63.1
60.6
63.3
67.5
56.3
64.5
61.7
50.0
$2.2
45.3
Mixer-B16-In2Ik
76.6
77.4
83.4
83.5
79.5
78.0
94.8
85.8
83.7
79.8
66.7
75.9
80.3
73.4
70.3
63.1
SwinV2-B-In2lk
87.1
48.2
38.3
35,1
38.6
50.4
86.0
57.2
42.7
32.6
56.3
37.0
50.5
58.2
31.3
48.2
34,4
SwinV2-L-ln21k
87.5
45,9
38.7
35.4
38.6
55.3
89.9
55.1
41.7
32.3
63.1
36.5
50.5
57.8
28.3
47.6
32.2
SWin
84.2
67.6
71.7
70.1
66.7
66.8
99.8
73.1
66.8
62.7
61.0
63.8
73.4
68.0
49.8
63.7
48.5
Vwin
84.6
69.5
72.6
69.3
67.4
66.6
97.8
69.4
65.2
64.2
60.7
62.0
70.5
63.3
$2.2
59.1
50.2
ResNet101
81,9
73.4
85.2
100.0
76.1
75.8
74.9
66.4
77.2
83.5
72.5
66.8
53.5
R5sNer152
82.3
71.2
83.2
100.0
74.4
74.6
B8.1
72.0
61.6
75.0
79.5
69.9
82.4
64.9
46.5
52.7
52.2
ResNe150
80.9
76.0
94.7
99.9
78.6
79.6
89.9
83.7
75.0
80.0
89.
75.0
85.7
69.9
61.0
58.2
56.9
ResNet50-supcon
78.7
60.6
$7.0
56.1
56.8
84.6
59.1
65.8
58.4
56.9
80.3
60.0
63.9
98.3
59.6
90.7
61.8
VII-T16-In2lk-augee
75.5
79.0
72.9
69.6
74.0
66.4
90.1
81.7
81.9
71.1
69.4
78.4
72.3
61.6
63.2
67.6
68.0
V1T 1166-n21k-ann augrey
81.4
67.0
53.3
55.4
54.7
47.4
85.0
70.9
64.0
51.9
58.3
61.4
$7.6
44.8
44.6
51.1
50.5
VIT-B16- 1516-1n21l-mugre2
85.1
62.1
52.1
49.4
54.6
71.0
9%..7
64.0
$7.0
$1.3
65.2
53.4
64.4
69.8
45.9
SH.5
44.5
ViT In21k augreg
84.5
45.2
45.4
50.1
48.2
49.8
31.3
35.2
37.1
WTTB166In22I--n0
81.8
52.2
39.2
39.0
41.0
35.6
71.4
52.7
47.6
38.3
56.9
48.2
44.5
35.6
31.6
38.3
36.5
WT-BI66In2ll--m
84.3
57.2
46.4
46.5
49.3
46.1
98.0
59.6
51.7
43.6
59.4
50.0
58.1
56.2
35.4
48.6
40,2
Vi--L16-in2Ik-uugoog
85,8
47.0
39.0
27.3
37.7
31.7
95.2
68.6
58.9
35.4
52.7
37.6
40.3
24.2
28.9
26.5
28.1
I121166-221k-mo
81.5
46.2
37.3
37.1
37.7
42.2
58.5
45.8
40.7
36.4
55.8
42.1
40.1
39.4
32.4
37.6
36.1
ViT-S16-mngreg
78.8
72.8
72.8
73.6
72.5
80.7
97.0
82.1
80.0
72.7
71.0
72.8
75.4
63.2
63.1
59.2
58.9
VIT-B16-and augrey
79.2
72.2
71.7
69.6
71.1
73.5
90.9
77.6
75.9
70.9
65.5
72.0
73.8
62.9
61.3
58.4
57.4
86.2
49.7
44.7
44.0
49.6
99.9
42.3
40.9
50.5
41.4
48.4
57.2
38.9
VTT-LI-CCLI--.2---II2XX
88.2
35.5
28.8
28.1
29.9
24.5
97.9
39.5
25.4
29.7
41.5
26.8
31.4
35.3
25.4
30.3
27.2
VI-HII4-CLIP-LD---II2
88.6
36.4
31.1
30.8
31.6
24.9
97.0
41.7
27.4
31.5
53.4
27.4
33.6
33.5
23.7
29.5
26.1
VoT- SigLip
89.4
50.3
47.4
42.1
44.2
40.2
95.5
36.3
30.0
42.6
65.1
30,9
36.1
36.4
27.4
31.3
26.1
Average
84.4
58.9
58.6
57.6
55.2
54.9
92.9
61.5
53.1
52.9
61.0
32.0
38.1
53.8
42.9
49.2
43.2
6. Conclusion
more aligned with a normal distribution, less heavy-tailed, and the class variances are more similar, leading to improved OOD detection results across a wide range of models. Maha- lanobis+ + outperforms the conventional Mahalanobis dis- tance in 41/44 cases, rendering clearly the most effective method across models. It outperforms the previously best baseline ViM by 7 FPR points average on the OpenOOD datasets, and is the best method for 4 of the 5 top models.
We showed that the frequently occurring failure cases of the Mahalanobis distance as an OOD detection method are related to violations of the method's basic assumptions. We showed that the feature norms vary much stronger than ex- pected under a Gaussian model, that the feature distributions strongly heavy-tailed and that feature norms correlate with the Mahalanobis score irrespective of whether a sam- plc is ID or OOD. These insights explain why certain models despite impressive ID classification performance showed strongly degraded OOD detection results with the Maha- lanobis score in previous studies (Bitterwolf et al., 2023). We introduced Mahalanobis++. simple remedy consisting of l2 normalization that effectively mitigates those prob- lems. In particular, the resulting feature distributions are
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
9

---

Mahalanobist+::
Improving OOD Detection via Feature Normalization
Acknowledgements
Haas, J., Yolland, W., and Rabus, B. T. Linking neu- ral collapse and 12 normalization with improved out-of- distribution detection in deep neural networks. Transac- tions on Machine Learning Research, 2023, ISSN 2835- 8856. URL https:: wwwpreeeewww....m net wwt.:/rrtrroommiii/ id- EjK550r2d6.
We thank Yannic Neuhaus for insightful discussions about distances on unit spheres. Further, we acknowledge sup- port from the DFG (EXC number 2064/1, Project number 390727645) and the Carl Zeiss Foundation in the project "Certification and Foundations of Safe Machine Learning Systems in Hcalthcare". Finally, we acknowledge support from the German Federal Ministry of Education and Re- search (BMBF) through the Tübingen AI Center (FKZ: 011S18039A, and the European Laboratory for Learning and Intelligent Systems (ELLIS).
Haas, J., Yolland, W., and Rabus, B. T. Exploring sim- ple, high quality out-of-distribution detection with 12 normalization. Transactions on Machine Learning Re- search, 2024. ISSN 2835-8856. URL https://
openreview wwtp:w..eeeeeeeeeee www.eoooumidddaaaaaaaaa
References
Hein, M., Andriushchenko, M., and Bitterwolf, J. Why
ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the prob- lem. In CVPR, 2019.
Ammar, M. B., Belkhir, N., Popeseu, S., Manzan- era, A., and Franchi, G. NECO: NEural collapse based out-of-distribution detection. In The Twelfth In- ternational Conference on Learning Representations, http:///ppereeeeeeeeeeeeeeeeeeee
Hendryeks, D. and Gimpel, K. A baseline for detecting mis- classified and qut-of-distribution examples in neural net- works. In ICLR, 2017. URL https: /openrevie.. net form/o 2idimmidd----- Hol-HHKK4XXX...
2024. URL id= id=9ROuKblmi7
Anthony, H. and Kamnitsas, K. On the use of mahalanobis distance for out-of out-of-distribution detection with neural net- works for medical imaging. In Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, pp. 136-146. Springer Nature Switzerland, 2023, doi: 10.10079978-33-31-4446------ URL /ttpp::/// /doi. org/10. 100.1.00022200
Hendrycks, D., Basart, S., Mazeika, M., Zou, A., Kwon, J., Mostajabi, M., Steinhardt, J., and Song, D. Scaling out-of-distribution detection for real-world settings. In ICML, 2022.
3000--31---- 001-777--333-1143---22-1-- 14.
Koner, R., Sinhamahapatra, P., Roscher, K., Günnemann, S., and Tresp. V. Oodformer: Nut-of-distributin detection transformer, 07 2021.
Bitterwolf, J., Mueller, M., and Hein, M. or out? fix- ing imagenet out-o--dist detection evaluation. In
ICMI, 2023. URL
ngs.ml* mlr.
Krasin, I. Duerig, T., Alldrin, N., Ferrari, V., Abu-El-Haija, S., Kuznetsova, A., Rom, H., Uijlings, J., Popov, S., Ka- mali, S., Malloci, M., Buset, J., Veit, A., Belongie, S., Gomes, V., Gupta, A., Sun, C., Chechik, G., Cai, D., Feng, Z., Narayanan, D., and Murphy, K. Openimages: A public dataset for large-scale multi-label and multi- class image classification. Dataset available from stor- wttgg//aapp..oooo/ennnnnee///see
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In CVPR, 2014.
Djurisic, A., Bozanic, N., Ashok, A., and Liu, R. Extremely simple activation shaping for out-of-distribution detection. In The Eleventh International Conference Learning Representations, 2023. /attts:://ppereeee net/
html, 2017.
Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, 2009.
Galil, I., Dabbah, M., and El-Yaniv, R. A frame- work for benchmarking maas-out-of-distriuutt detec- tion and its application to imagenet. In The Eleventh International Conference on Learning Representations, 2023. URL httpt:tt/////. http///aeweeiieeeeeeeeeeeeeeeeee id-Iubbb9W6Jtk
Le, Y. and Yang, X. S. Tiny imagenet visual recog- nition challenge. 2015. URL https: /api. semant www.ccoo..........rrrroo..o. orrg//orpuusD::16647777
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient- based learning applied to document recognition. Pro- ceedings of the IEEE, 86(1 86(11):278-2232, 1998. doi: 10.1 10.1/9997.76691.
Gia, T. L. and Ahn, J. Understanding normaliza- tion in contrastive representation learning and out- of-distribution detection. ArXiv, abs/2312.15288, 2023. URL org/CorppsSID:2655055
Lee, K., Lee, H., Lee, K., and Shin, J. Training confidence- calibrated classifiers for detecting disstiriuiititionnn sam- ples. ICLR, 2018a.
10

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Lee, K., Lee, K., Lee, H., and Shin, J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, 2018b.
Park, J., Long Chai, J. C., Yoon, J., and Jin Teoh, A. B. Understanding the Feature Norm for Out-of-Distribution Detection In 2023 IEEE/CVF International Con- ference on Computer Vision (ICCV), 2023b. URL
Liu, L. and Qin, Y. Fast decision boundary based out-of- distribution detector. ICML,, 2024.
h www.:eeeeeecccomteeercc...o. wtt.eeet.rrrtttt...o.o... 10. (AL109//CCV11000000 002.0....01550
Liu, W., Wen, Y., Yu, Z., Li, M., Raj. B., and Song, L.. Sphereface: Deep hypersphere embedding for face recog- nition, 2018. URL https: arxiv. org/ abs/ 1704.08063.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In ICML, 2021.
Liu, W. Wang. X., Owens, J., and Li, Y. Energy-based out- of-distribution detection. Advances in Neural Information Processing Systems, 2020.
Regmi, S., Panthi, B., Dotel, S., Gyawali, P. K., Stoyanov, D., and Bhattarai, B. T2fnorm: Train-time feature nor- malization for ood detection in image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 153-162, June 2024.
Liu, X., Lochman, Y., and Christopher, Gen: Pushing the limits of softmax-based out-of-distribution detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong. L., Wei, F., and Guo, B. Swin transformer v2: Scaling up capacity and resolution. In CVPR, 2022. URL https: arxiv. org/ abs/ 2111 09883.
Ren, J., Fort, S., Liu, J., Roy, A. G., Padhy, S., and Lak- shminarayanan, B. A simple fix to mahalanobis dis- tance for improving near-ood detection, 2021. URL https: httpss///xxxvv.rr//.o// wwt.o////////..././///////////oe /atobb//216600900000
Sablayrolles, A.. Douze, M., Schmid, C., and Jégou, H. Spreading vectors for similarity search. arXiv: Machine Learning, 2018. URL attps://api.. semanti cww.nntcsccollr...mmmm wtg/CorppssDo.....ooooo. : 62841600810
Ming. Y., Cai, Z., Gu, Sun, Y., Li, W., and Li, Y. Delving into out-of-distribution detection with vision-language representations. In NeurlPS, 2022. URL https:// http:/eee//oourtre/e formmmormm dd=kkcs330.
Sehwag. V., Chiang, M., and Mittal, P. Ssd: A unified framework for self-supervised outlier detection. In In- lernational Conference on Learning Representations, / http///ppnneeeeeee/e//e/ee net.//runmtff forum?
Ming. Y., Sun, Y. Dia, O., and Li, Y. How to exploit hyper- spherical embeddings for out-of-distribution detection? In The Eleventh International Conference Learning Representations, 2023. URL https: // /ppprrevie...... net/ Pott/oorumiccm aEF aEOW5pAd.
2021. URL https: id=v5g JXpmRBJ.
Steiner, A. P., Kolesnikov, A., Zhai, X., Wightman, R., Uszkoreit, J., and Beyer, L.. How to train your vit? data, augmentation, and regularization in vision trans- formers. TMLR, 2022. URL https: httpp://///peeeiiee. KcP.
Mueller, M. and Hein, M. How to train your vit for ood detection, 2024. URL 2405. 17
org/abs/
net Sun, Y., Guo, C., and Li, Y. React: Out-of-distribution detection with rectified activations. NeurlPS, 2021.
Mukhoti, J., Kirsch, A., van Amersfoort, J., Torr, P. H., and Gal, Y. Deep deterministic uncertainty: A new simple baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. -4394,, June 2023.
Sun, Y., Ming, Y., Zhu, X., and Li, Y. Out-of-distribution detection with deep nearest neighbors. ICMI,, 2022.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. Reading digits in natural images with unsupervised feature learning, NIPS Workshop Deep Learning and Unsupervised Feature Learning, 2011.
Tack, J., Mo, S., Jeong, J., and Shin, J. Csi: Novelty de- tection via contrastive learning on distributionally shifted instances. In NeurlPS, 2020.
Park, J., Jung, Y. G., and Teoh, A. B. J. Nearest neighbor guidance for -ittof-distribution detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1686-1695, 2023a.
Techapanurak, E., Suganuma, M., and Okatani, T. Hyperparameter-fiee free diistr-iiitiittii detection using cosine similarity. In Proceedings of the Asian Conference on Computer Vision, 2020.

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Touvron, H., Cord, M., and Jegou, H. Deit ii:: Revenge of the vit. ECCV, 2022.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Tor- ralba, A. Places: A 10 million image database for scene recognition. IEEE transactions pattern analysis and machine intelligence, 40(6):1452-15 2017.
Van Horn, G., Mac Aodha, O., Song. Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S. The inaturalist species classification and detection dataset. In CVPR, 2018.
Zhou, C.,, Po, L. M., and Ou, W. Angular deep supervised vector quantization for image retrieval. IEEE Transac- tions Neural Networks and Learning Systems, 33(4): 1638-1649, 2022. doi: 10. 1011009TNNLS 2020.3043103.
Vaze, S., Han, K., Vedaldi, A., and Zisserman, A. Open-set recognition: a good closed-set classifier is all you need? In International Conference on Learning Representations 2022.
Wang, H., Li, Z., Feng. L., and Zhang, W. Vim: Out-of- distribution with virtual-logit matching. In CVPR, 2022.
Wei, H., Xie, R., Cheng. H., Feng, L., An, B., and Li, Y. Mitigating neural network overconfidence with logit normalization. 2022.
Wightman, Pytorch image models. https://githi com/ wwwiightan/pyttooo pptoorcttthhhoo 2019.
models,
Wilk, M. and Gnanadesikan, R. Probability plot- ting methods for the analysis for the analysis of data. Biometrika, 55(1):1-17, 03 1968. ISSN 0006-3444, doi: 10. 10.10933biomeu/5...11 URL https:tttti....o org/10. 1093/bicmet/ee 1.1.
Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, S., and Xie, S. Convnext v2: Co-designing and seal- ing convnets with masked autoencoders. arXiv preprint arXiv:2301. 00808, 2023.
Yang, J., Wang, P., Zou, D., Zhou, Z.,, Ding, K., Peng. W., Wang, H., Chen, G., Li, B., Sun, Y., et al. Openood: Benchmarking generalized ouitt---ditiibutionn detection. arXiv preprint arXiv:2210.072. 2022.
Yaras, C., Wang. P., Zhu, Z., Balzano, L., and Qu, Q. Neural collapse with normalized features: A geometric analysis over the riemannian manifold. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https: / /oopereeiiww net forum? dd2wh6115555 6N.
Yu, C., Zhu, X., Lei, Z., and Li, Z. Out-of-distribution detection for reliable face recognition. IEEE Signal Pro- cessing Letters, 27:710-714 2020. doi: 10.1109/LSP. 2020.2988140.
Zheng, J., Li, J., Liu, C.. Wang, J., Li, J., and Liu, H. Anomaly detection for high-dimensional space using deep hypersphere fused with probability ap- proach. Complex & Intelligent Systems, 8(5):4205- 4220, Oct 2022. ISSN 2198-6053.
doi: 10.1007/ -40747-022-00695-9, URL https: http///ooooggg////// 1007/840747-022-00989
12

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
A. Overview
The Appendix is structured as follows: In Section we provide the proof for Lemma 3.1. In Section D we provide extended analysis on feature norm and normalization. In particular,
we show the feature norm distribution for more models in Figure 8
we provide QQ-plots for more models in Figure 9
We report the feature norm distribution for ID and OOD data in Figure 10, showing that OOD features can be larger than ID features for off-the-shelf pretrained models
we highlight that the class variances become more similar to the global variance after normalization in Figure 11
we plot the correlation between feature norm and OOD-score in Figure 7
In Section we report extended results. In particular,
we show additional ImageNet numbers (AUC for NINCO in Table 11, OpenOOD near and far in Table and Table 9, OpenOOD averaged AUC Table 10)
we compare cosine-based methods on ImageNet explicitly in Table 12
we show robustness noise distributions (unit tests) in Table 17
we show additional CIFAR numbers (Cifarl0 AUC in Table 13 and FPR in Table 14, Cifar100 AUC in Table 15 and FPR in Table 16
compare Mahalanobis++ SSD+ in Table 18 to highlight the benefits of post-hoe OOD detection methods
In Section F we report details on the model checkpoints used throughout the experiments (ImageNet models in Table 19 and Cifar models in Table 20). Section G we provide details on the OOD detection methods evaluated in the main paper.
B. Proof of Lemma 3.1
Proof. Let Σ = UAUT be the cigendecomposition of the covariance matrix with U being an orthogonal matrix containing the eigenvectors of E and A the diagonal matrix containing the eigenvalues of a. Let X a random variable with distribution ((,, Σ) (in the main paper, we denoted the features as & (X),, here we write them as X for notational simplicity). Then holds = TT X has distribution, NUT μ,, A) and since UT is an orthogonal matrix: ||x] 2 | We have
B|||I 21122121
E E
Var (Zr +
X
U m1 tr(E) + 11
We note that
Var(I
- = - + 1|a11))2
and it remains to compute E|| 2
We note that
d
Σ
E[Z +
E =
Σ (3x + 6, + +++++ + -0u+ ++²+
-- (Ns + n2
where we have used the following calculations:
0 = E|<4 - - E[2 -3 1.
= - 1=) E|2 4u.E|2;1 64 E 3
13

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
and thus
B[Z = 3/4 + 14 B[Z = 3X, + 6/ +14
This yields
d Var(1 = Σ (3 + G3ę + +14
d
Σ (Ax + 19)
Applying Chebychev's 's inequality yields the result.
C. Derivation of expected squared relative variance deviation
Here want to derive the statement about the expected squared relative variance (denoting the covariance matrix C instead of E):
2trace(A2) + (race A))
Eu (
d(d + 2)
where u has uniform distribution the unit sphere and A= - - We note that A is symmetric and thus has an eigendecomposstiin A UAUT. We have
1
d
E.[(u" Au)?] = u) E.
Σ
X Bu + Σ Au
it
It remains compute these moments on the unit sphere. For this purpose note that ||u||2 = Dí uf = 1 and thus
= |lul12
ufuu
i=1
We note that u for 1,.... d and uZ for i # j are all equally distributed and thus for i + j
= dE d(d 1)
Moreover, we note that rotations do not change the distribution for a unifom distribution on the sphere and thus (u,, uz) and
have the same distribution and
V2
2
27
[ ug]
E
14:
Uj
U/ +
Uj
Eluf 2タ E[u uz]
V
2
This yields B[u:] = (1 Plugging this into (9) yields
3
Eluf| d(d + +)
El Elujujj
d(d ( 2)
Thus
d
Eul(u² (((TTAA))))) Au) = d(d + 2)
3
x +
Σ
Aidy
=
d(d + 2)
>
x
Σ Arl
i
:=1
Using that trace(A) -d > finishes the derivation.
14

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
D. Extended Analysis
Here, we report extended results on the experiments of Section of the main paper. In particular, we show that the observations made hold beyond the SwinV2 model. If not stated differently, all experiments are with ImageNet as ID dataset and NINCO as OOD dataset.
Feature Norm Correlation. In the main paper, we showed that for a S2 model, the feature norm of sample correlates strongly with the OOD score received via the Mahalanobis distance. Here, we show this phenomenon for more models. In Figure 7, we plot the feature norm against the OOD score assigned by Mahalanobis and Mahalanobis++ for four models. For Swin ConvNext and ViT-clip, the feature norms correlate strongly with the OOD score. Normalizing the features (bottom) mitigates this dependency, as OOD samples with small feature norms are detected as OOD, and thus improves OOD detection strongly. notable exception is the augreg-Viz ViT, for which there is no correlation between feature norms and OOD score. In Figure we further investigate the dependency of the Mahalanobis score on the feature norm by artificially scaling the feature norm of the OOD features with a prefactor a. That is, for a S2 model, we use a o; for each OOD feature and leave the II) validation features unchanged. We report the FPR values against or in Figure 6, and again observe a clear correlation between the scaling factor and the FPR: Perhaps unexpectedly, upscaling the features reduces the aalse-positive rate, up to scaling factor of 2, where zero false positives are achieved. When scaling down the feature norm, the FPR increases, and for a Rs 0.5, i.c. at half the original feature norm, all OOD samples are identified as ID. Notably, this does not change for smaller a values, not even for a = 0, where all OOD samples collapse to the zero vector, In other words, everything in the vicinity of the origin is identified as in-distribution, which contradicts the intuition of tight Gaussian clusters centered around class means. In the main paper, we hypothesized that this might explain why the Mahalanobis distance sometimes fails to detect the unit tests since those might receive small feature norm. In Figure 10, we plot the feature norms of different datasets for range of models with (top) and without (bottom) pretraining. We find that the feature norms of natural OOD images like those from NINCO tend to be even larger than the ImageNet feature norms. This violates basic assumptions in feature-norm-baen based OOD detection methods like the regative-aware-none (Park et al., 2023b), indicating that special training schemes might be necessary for those methods. However, noise distributions like the unit tests from Bitterwolf et al. (2023) can lead to fairly small feature norms for most models. Since we showed that small feature norms lead to small Mahalanobis distances for many models, this highlights why these supposedly easy-to-detect images were not detected with the Mahalanobis distance in previous studies.
Feature norm distribution. In Figure 8, we plot the feature norms for four ViTs of exactly the same architecture (ViT-B 16). In order to make the plots comparable, we normalize by the average feature norm per model. We observe that, like for the V2 in the main paper, the norms vary strongly across and within classes except for the augreg-Vit VIT. This model is one of the models that performed well with Mahalanobis "out-of-the-boxx i.e., not requiring normalization.
QQ plots. In Figure 9 we show QQ plots for four models along three directions in feature space. We observe that the normalized features (green) more closely resemble a normal distribution compared to the unnormalized features (blue), which is best visible via the long tail. The only exception is the augreg-Vii ViT, for which normalized and unnormalized features are similarly close to a Gaussian distribution.
Variance alignment. We report extended results on the expected relative deviation scores (see Eq. 5) for more models in Table 7. We observe that for all models- except the ViT-augreg- normalization lowers the deviations, indicating better alignment of the global variance with the individual class variances. In Figure 11, we illustrate this further: Instead of the score reported in Table 7, which computes an expectation over all directions, we pick three specific directions: 1) random direction, 2) an eigendirection with large eigenvalue, and 3) an eigendirection with a small eigenvalue. Ideally, along each direction, the 1000 class variances would coincide with the globally estimated, shared variance. For each direction, we divide the 1000 class variances by the global variance and plot the resulting distribution. Distributions peaked around I indicate that the global variance can capture the class variances well. We observe that the distributions of the variances after feature normalization peak more towards one for all models, except the ViT-augreg.
Augreg VITs. The ViTs that showed the best performance with Mahalanobis distance in previous studies were base-size ViTs pretrained on ImageNet21k l and fine-tuned on ImageNetlk by Steiner et al. (2022). The training scheme is called augreg, a carefully tuned combination of augmentation and regularization methods. In this paper, we made several observations regarding those models (applies for both base-size and large-size models with pretraining on ImageNet21k). In particular, they
show strong OOD detection performance with Mahalanobis distance without normalization, and normalization does
15

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
Table Deviations from global variance. report the mean squared relative variance deviation as defined in Equation (5) multiple models. In all cases, except for the ViT-augreg, normalization significantly improves the fit of the global covariance matrix to the covariance structure of the individual classes. noted the text for the ViT-augreg the features already follow very well the assumptions of the Mahalanobis score and normalization leads to no improvements.
model
unnormalized
normalized
ViT-B16-In2 -i---ge--n2ll--uugrree
0.05
0.05
SwinV2-B-In2Ik
0.26
0.12
ViT-BI6-CLII-LL2--nnI
0.17
0.08
ViT-B16-In2 I
0.14
0.07
ViT-B16-In2Ik-mill
0.12
0.09
DeiT3-B16-In2Il
0.24
0.15
ConwNeeXI-B-In2kk
0.17
0.11
EVA02-B14-n22Ik
0.21
0,14
ConvNeXttV2---nn2
0.23
0,18
ConwNeXtV2-B
0.22
0.14
ConwNeXt-B-In2Ik
0.17
0.11
ConvNeXt-B
0.22
0.12
not improve Mahalanobis-basel OOD detection
show little variations in feature norm compared to all other investigated models
show no correlation between feature norm and Mahalanobis score (in contrast to all other investigated models)
show much weaker heavy tails than the other models
show low values for the variance deviation metric
loose their advantage for unnormalized Mahalanobis-basee detection when the fine-tuning scheme is changed (the augreg2 model is fine-tuned from 2 2lk-augreg-checkpoin but the fine-tuning scheme differs in learning rate and augmentations)
In short, the augreg models omit all the points that identified problematic for Mahalanobis- OOD detection. This indicates that the augreg training scheme induces feature space that lends itself naturally towards normal distribution, aligning well with the assumptions of the Mahalanobis distance as OOD detection method. Understanding the exact reason why the augreg scheme induces those features is beyond the scope of this paper. The connection of training hyperparameters and OOD detection performance was, however, investigated by Mueller Hein (2024). should be stressed that for post-hoc OOD detection, we ideally want a method that works well with all models, not only those obtained via a certain training scheme. We provide such a method with Mahalanobis++.
100
True sample norm
FPR (%)
40
0
0.00
0.25
1.00
1.25
1.50
Scaling Factor of OOD samples
Figure 6. Impact of the feature norm of OOD samples their Mahalanobis score. When scaling down the of the features while leaving the feature direction unchanged, OOD samples receive a smaller Mahalanobis score and are incorrectly classified as ID samples. When the feature norm is artificially increased, the opposite happens.
16

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
SwinV2
ConvNext FPR 47R.4
ViT-clip
ViT-augreg
FPR=58.2%9
FPR 57.2%
FPR 31.3%
45
50
100
Feature norm
Feature norm
Feature norm
Feature norm
40
35
90
30
25
80
25
20
20
-15000
-10000
-5000
-6000
-4000
-2000
-4000
-2000
-1500
-1000
-500
8Maha
Maala
Moto
Safak
FPR=31.3%
FPR=28.8%
FPR=35.8%
FPR=35.79
50
35
Feature norm
Feature norm
40
Feature norm
40
30
35
90
Feature
30
30
80
25
20
70
20
-4000
-2000
-000
-1000
-3000
-2000
-1000
MMala
3Maka
SMale
Salako
ID
ODD (NINCO)
TPR=95%
Figure 7. M.hhlannbbis+ resolves feature-norm dependency of Mahalanobis score. With unnormalized features, OOD samples with small pre-logit feature norm were systematically identified as ID. after normalization, OOD samples with small feature norm are rightfully detected as OOD, resulting in significantly improved OOD detection with Mahalanobis+ The only exception is augreg ViT, which does not show a correlation between feature norm Mahalanobis score, even without normalization.
ViT-augreg2
ViT-mill
VIT-clip
ViT-augreg
Avg. Norm
144, Milliiihllll ang
Std Dev Min/Max
0
250
500
750
1000
250
500
750
1000
0
250
500
750
1000
250
500
750
1000
vary strongly across and within classes. The plot the "observed" part of Figure in the main paper, but normalized by the global average feature norm to make the scales of different models comparable. We thus show how strongly the feature norms vary relative their scale. We report results for ViT-B16 models with different pretraining schemes. Only the augreg VIT shows little variation in feature norm and is the only model that does not benefit from normalization. Interestingly, the augreg2 model finetuned ImageNet- Ik from the same 21k-checkpoint the augreg model and even achieves higher classification accuracy. but shows very different feature norm distribution which reflects in the OOD detection performance with Mahalanobis and Mahalanobis models except for augreg model benefit from normalization.
Figure 8. For most models, the feature
17

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
Swin
DeiT
ViT-augreg2
ViT-augreg
10
10
10
10
Small EV Dir. Ord. vals
5
5
-5
-5
-5
-5
2.5
10
10
Random Dir.
Ord. vals
5
5
0
0
0
-10
-10
-10
-10
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
Ordered values
Large EV Dir.
5
5
5
5
0
-5
-5
-5
-5
-10
-10
-10
-10
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
Theo. quantiles
Theo. quantiles
Theo. quantiles
Theo. quantiles
Figure 9. QQ-plot: & normalizzziii helps transform the features to be more aligned with normal distribution. Normalized features green, unnormalized features in blue. a SwinV2, DeiT3 ViT-augreg2, the feature norms vary strongly across classes (see e.g. Fig. 3 and Fig. 8) and normalization shifts the distribution towards a Gaussian. For ViT-B-augreg the feature norms are similar across classes (see Fig 8) and the feature norms are already fairly normal, so l2 nomommalization has almost no effect.
18

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
ConvNext-B
ViT-B-orig
ViT-B-mill
ViT-B-clip
ViT-B-augreg
Density
10
15
20
25
10
20
30
20
40
50
100
Feature norm
Feature norm
Feature norm
Feature norm
Feature norm
ID
OOD (unit tests)
OOD (NINCO)
DeiT3
ResNet
SwinV2
ConvNext
ViT-S
Density
10 20
30
10
20
30
10
20
30
10
20
30
30
40
Feature norm
Feature norm
Feature norm
Feature norm
Feature norm
ID
OOD (unit tests)
OOD (NINCO)
Figure 10. Feature norm distribution. In contrast to previous work (c.g., (Park al., 2023b)), find that the feature norm of natural OOD samples (NINCO in green) is often larger than that of ID samples (orange). Far-OOD data, like noise distributions, tend to have lower feature norms. This holds for models with (top) and without (bottom) pretraining.
19

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Random direction
Large EV direction
Small EV direction
unnoraaliaed nomalized
SwinV2-B 8--221k
1.0
global
0.5
1.0
0.5
0.2
0.0
0.0
vartclass () varlgiobal)
variclass () /var(global)
var(class l /arlgllobll)
Random direction
Large direction
Small EV direction
VIT-816 VIT-816-CLIP- CLIP. (e11:/CLP..22---11111 L.bbiinn2k
1.5
narmallzed .... global variance
1.0
0.30
0.5
0.5
0.0
0.00
0.0
var(class i) var(global)
variclass i) varlglobal)
varíciass i)/ var(global)
Random direction
Large EV direction
Small EV direction
VIT-816-2228 00--224-1n 1016--2--nn244 114--n22k-aain augreg?
1.5
unnomalized nermalized global variance
1.5
1.0
0.5
0.5
0.5
2
var[class i) var(global)
variclass /arrtglobal)
var(class variglobal)
Random direction
Large direction
Small direction
unnormaizaed normalzed global variance
2.0
NT-816-224-n221--a
1.0
1.0
0.5
0.5
0.0
0.0
0.0
var(class i) var(global)
var(class i) varíglobal)
varíclass i) var(global)
Figure 11. Mahalanobist+ aligns class-variancss. We report the distribution of the variances of the train features for each class along three directions: 1) random direction, 2) a large eigendirection, small eigendirection. For each class, we compute the variance divided by the global variance, and plot the resulting distributions. Larger deviations from one indicate larger deviations of the class variance from the global variance.
directions the distribution of variances more peaked around after normalization, indicating that after normalization the shared variance assumption is more appropriate- except for the ViT-augreg.
20

---

Mahalanobis++: Improving OOD Detection via Feature Normalization
E. Extended results
Table 8. FPR on OpenOOD near datasets, Green indicates that normalized method is better than its unnormalized counterpart, bold indicates the best method, and underlined indicates second method. Maha++ improves over Maha on average by 9.6% FPR over all models. Similarly, rMaha++ + is 5.5% better in FPR than rMaha. The lowest FPR is achieved by ViM for the EVAO2-L11-M38--In22222 highlighted blue, closely followed by Maha++ the same model.
Model
AshS
NNG
rMaha
86.3
53.9
46.1
44.2
46.7
53.8
91.3
63.1
52.7
42.2
53.8
46.7
58.3
59.6
41.7
51.8
44.7
Con-NeXI-B
84.4
70.0
89.0
86.8
75.1
72.1
99.0
77.5
72.1
72.1
72.9
73.3
76.1
72.0
59.1
67.3
58.1
Com:NeXIVV.TTIn211
85.1
59.7
51.2
51.5
53.3
45.4
95.7
62.7
55.6
48.3
56.2
54.4
61.7
52.7
44.9
52.1
48.8
ConvNeXIV2-B-Ia21k
87.6
50.8
39.0
38.8
41.5
37.3
95.5
49.9
42.6
36.8
46.1
41.0
48.7
42.0
34.7
41.3
38.2
Com-NeXiv2-LLIn21k
49.5
35.6
Con-NeXiV2-T
83.5
72.7
79.1
75.8
72.9
72.6
98.6
$6.7
81.0
69.7
74.6
72.4
78.4
71.3
60.5
66.6
58.2
Com-NeXiV2-B
85.5
69.6
79.2
76.3
70.5
69.0
99.0
70.3
68.2
65.8
72.6
55.0
61.5
54.1
Con-NeXIV2-L
86.1
69.8
77.4
73.7
69.9
70,6
98.4
70.3
66.8
68.8
77.8
64.1
68.2
62.4
53.6
56.4
51.9
DeiT3-S16-nnl In2k
84.8
73.1
65.8
64.8
67.4
68.7
98.8
70.8
67.1
65.4
64.5
65.5
72.6
70.7
60.4
68.4
60.6
DeiT3-B16-In21k
86.7
67.1
60.6
57.4
61.2
65.5
98.9
57.4
58.1
59.1
53.4
62.2
62.7
50.8
58.6
49.9
Dei13-L16-In21k
87.7
66.7
54.3
51.2
57.1
57.3
97.2
$3.9
49.4
52.2
52.9
48.1
56.9
54.5
47.3
51.6
46.3
71.3
86.7
67.7
70.5
DeiT3-B16
85.1
71.8
89.5
90.1
76.8
67.3
98.7
79.6
83.6
76.
71.9
65.4
74.7
71.8
64, 8
67.4
61.7
DeiT3-L16
85.8
72.5
83,3
86.7
74.4
65.9
85.0
75.3
80.0
74..7
67.9
65.8
72.3
66.4
60.7
61.9
57.2
EVAI2-B14-In211
88.7
45.1
37.0
36.5
40.3
36.5
93.8
46.9
40.8
36.6
45.2
36.6
43.3
41.9
34.8
42.0
38.0
EVA02 I38k-In2lk
90.1
38.3
31.9
31.7
34.6
28.1
95.2
40.2
35.7
31.5
45.2
30.8
36.3
31.8
28.7
33.1
32.3
EVA02-TI4
80.6
77.6
76.9
77.3
76.5
74.6
97.8
77.3
73.5
67.5
76.4
79.2
73.0
71.0
72.7
71.2
EVA02-SD S14
85.7
67.7
67.0
67.1
64,8
58.1
98.2
67.7
62.9
60.8
56.4
61.5
68.1
58.3
$7.0
58.7
57.3
78.4
72.9
79.7
98.7
72.7
74,9
67.7
EANNetV2-L
85.7
69.0
81.4
75.6
70.1
74.7
98.6
70.0
68.6
69.4
70.5
63.5
69.9
65.
56.4
59.5
53.8
EnNetV2-M
85.2
68.9
78.6
75.2
69.3
77.7
99.0
71.2
69.3
68.8
70.2
64.0
72.1
69.5
58.1
61.5
54.7
Mixer B16-In21k
76.6
82.4
87.7
87.8
84.6
83.3
94,9
89.4
87.9
84.8
69.4
82.4
85.6
78.4
71.8
74,9
68.8
SwinV2-B-In211
87.1
56.0
43.1
41.9
45.6
63.1
84.5
69.8
57.6
40.7
60.9
47.0
62.5
69.8
46.7
61.1
48.4
SwinV2-L-In2Ik
87.5
54.0
45.4
44.4
46.4
66.6
87.2
57.3
41.6
47.4
62.2
60.6
46.1
SwinV2-S
84.2
73.4
77.5
77.0
73.0
75.4
99.7
79.8
75.7
70.2
65.5
72.0
79.7
75.3
59.2
71.1
58.0
77.8
75.3
73.1
73.6
98.4
73.4
69.6
76.4
ReaNet101
81.9
78.4
87.0
99.7
80.4
82,1
91.55
82.6
75.0
81.1
87.4
78.8
87.6
73.1
58.7
62.8
59.9
ReaNet152
82.3
76.9
85,9
99.7
79.2
81.4
90.7
$1.0
72.2
79.8
84.4
76.8
86.3
71.3
S5.5
60.6
58.6
RexNe150
80.9
80.7
95.0
82.5
84.1
91.5
88.3
81.9
83.4
91.0
80.7
88.5
75.5
65.4
64.6
62.4
ResNet50-supcon
78.7
70.5
67.5
67.7
67.7
87.4
71.5
77.3
69
67.8
85.5
71.2
74,1
98-2
71.3
91.3
68.5
VTT-T16-In221-aam
75.5
83.7
78.7
75.2
79.8
73.5
92.1
86.5
77.4
83.5
78.6
7L.6
74.3
75.0
V1-S16-fam Inkkk-22kaaa angreg
81.4
73.8
61.0
63.6
63.1
56.6
88.7
77.2
72.1
60.6
62.4
69.3
66.6
56.7
$6.0
61.3
60.8
69.4
59.1
57.4
62.4
77.8
97,4
59.7
63.5
73.2
77.4
68.1
WT--B1--n22Ik-augee
84.5
64.4
54.3
58.4
54.8
47.9
95.3
74.9
68.0
52.8
55.9
$7.7
$8.7
44,6
49.0
48.3
49.9
VIT-B16-In21 1221--122k--rii
81.8
60.6
44.8
44.9
48.0
41.4
62.5
60.1
$4.6
44.9
61.2
$7.1
$2.6
45.3
40.6
49.9
47.8
VIT-B16-InD0 In21k-mill
84.3
65.6
$2.6
53.3
56.8
55.3
96.2
61.5
$2.1
62.4
60.1
67.3
66.3
47.5
60.1
52.1
VIT-LI6-In2Ik- I-lkgrreee
85.8
56.5
49.6
38.3
48.9
42.6
96.3
73.2
65.9
46.5
$4.2
48.7
50.8
36.4
41.9
39.2
40.8
ViT-L16-In2IA-orig
81.5
53.1
40.2
44.4
55.9
46.0
39.7
47.5
Vi1-S16-augreg
78.8
78.3
79.3
80.2
78.6
85.5
96.6
86.8
85.0
78.9
75.6
78.9
81.2
69.8
69.9
66.5
66.4
VIT-B16-nugrog
79.2
77.7
78.1
75.5
77.4
79.0
93.2
83.1
81.4
77.3
69.2
78.0
79.8
69.4
65.4
64.7
VET-B16-CLIP-LDD L-2-CCLLI------1122
86.2
56.2
46,9
46.0
49.1
55.4
99.3
57.9
50.6
45.9
52.5
49.1
56.6
65.0
44.7
59.2
49.2
CLILII
88.2
44.1
34.8
34.3
37.2
32.0
96.9
48.6
33.1
37.0
46.2
36.3
41.6
45.3
35.9
41.9
39.1
WTT-114.C..P...bb..lll
88.6
44.5
36,8
36.7
38.7
33.6
97.0
50.5
35.5
38.6
55.5
36.9
44.1
44.4
35.8
41.6
38.9
VTT-so400--SigLipp
89.4
58.6
56.1
52.5
53.4
49.9
95.3
48.6
43.7
51.3
63.9
42.8
48.9
47.0
38.6
43.7
39.2
Average
93.0
63.9
53.3

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
45
FPR Mahalanobis++ (%)
88
Validation Accuracy (%)
86
82
30
80
20
20
30
40
50
60
FPR Mahalanobis (%)
Figure 12. We plot the FPR with Mahalanobis++ against the FPR with the conventional Mahalanobis score averaged over the five OpenOOD datasets. With three minor exceptions, Mahalanobis+ improves OOD detection performance for all models, In particular, significantly improves all models with high accuracy.
22

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Table FPR on OpenOOD-fal datasets, Green indicates that normalized method better than its unnormalized counterpart, bold indicates the best method, and underlined indicates second best method. Maha++ improves Maha average 6. FPR all models. Similarly, rMaha++ is 1.2% better in FPR than rMaha. In total, Maha++ improves the SOTA compared to the previously strongest
method, ViM, by 5.1%, which significant. The lowest is achieved by Maha++ for the EVA02-L14 4-34m----------- In21k highlighted in
bluc.
Model
Val Ace
MSP
E
E+R
ML
VIM
AshS
KNN
NNG
NEC
GMN
GEN
1DBD
Maha
Maha++
rMaha
rMaha++
86.3
33.6
36.1
30.5
31.1
13.3
86.6
20.0
17.9
24.1
34,5
23.2
24.3
16.3
12.8
18.2
19.4
Com-NeXi-B
39.9
99.8
37.2
49.8
38.5
37.0
Now-NeXiVV-TTIn211
85.1
34.7
28.0
27.5
28.8
14.7
97.3
27.5
23.6
23.1
41.2
24.6
29.4
19.1
17.7
23,0
23.1
Com-NeXiV2-B-In21k
87.6
26.9
19.4
18.7
20.7
12.4
95.2
15.9
19.1
13.7
15.7
15.2
ConvNeXiV2-L-In221
88.2
26.0
19.3
18.5
20.4
15.3
95.6
18.4
15.6
16.3
18.2
17.5
19.6
15.3
10.5
15.5
14.6
Con~NeXIV2-T
83.5
52.4
57.4
47.1
49.6
34.8
99.6
62.4
50.6
43.6
73.4
41.1
50.7
43.5
33.7
37.2
35.5
Com-NeXIV2.B
85.5
51.7
65.2
56.0
52.2
32.0
99.9
39.1
32.9
47.7
71.3
37.1
41.1
32.5
25.8
30.6
29.0
Con-NeXIV2-L.
86.1
51.2
61.7
51.0
50.6
33.6
99.6
34.6
30.0
47.2
$4.5
34.4
35.3
27.9
24.6
27.4
28.6
DeiT3-S16
44.9
33.6
99.5
36.0
32.2
DeiT3-R16-In2lk
86.7
48.5
50,7
38.2
45.8
24.9
99.5
24.8
24.3
38.6
37.7
29.5
29.1
24.6
20.2
24.1
21.2
DeiT3-L16-In21k
87.7
47.2
39.7
29.6
40.0
22.4
98.7
22.5
20.7
29.3
26.
25.6
25.8
21.9
18.6
21.3
19.0
Dei13-S16
83.4
47.5
42.5
49.3
41.3
29.6
84,9
62.3
35.1
41.3
59.9
33.1
41.8
40.9
35.8
38.3
34.8
DeiT3-B16
85.1
51.6
77.4
87.2
56.1
29.6
99.6
$7.2
63.1
55.1
$7.8
33.1
41.6
37.9
34.7
35.6
33.8
DeiT3-L.16
85.8
52.2
78.6
91.0
57.1
32.8
73.9
39.8
67.6
57.5
49.5
31,4
39.1
31.3
25.7
29.9
26.3
EVA02-B14-Irg
88.7
23.9
20.0
19.3
21.0
12.4
83.9
18.0
15.8
17.3
29.8
16.2
18.8
14.6
11.8
15.6
14.2
EVA02-L.4 M:8m--1.mmmmm
11.2
15.6
EVA02-T14
80.6
56.2
59.0
59.7
54.2
32.5
98.8
47.6
43.7
43.4
45.2
45.5
57.9
36.2
32.9
39.3
37.1
EVA02-S14
85.7
41.9
44.4
43.8
39.2
19.2
99.6
28.3
25.2
31.0
34.2
28.5
36.1
22.1
21.0
24.4
23.1
ENNevV2-S
83,9
50.8
66,1
46.6
53.3
33.8
99.8
29.2
29.4
50.3
76.6
36.7
40,1
28.8
23,9
27.6
27.2
EBNe:VV2-L
85.7
49.1
69.4
45.3
51.3
31.7
99.6
34.8
32.9
47.1
49.8
32.2
35.4
25.0
20.0
23.7
22.2
FANerVV2M
85.2
49.1
63.2
44.4
49.3
39.4
99.8
34.7
45.7
64.6
32.7
39.6
30.3
23.1
27.5
24.8
Mixer B16 In21k
76.6
64.2
79.8
80.7
68.55
64.1
96.4
70.1
81.2
69.3
55.8
57.8
62.6
$3.3
39.7
50.0
42.1
SwinV2-B.ln211
17.5
72.0
21.1
55.5
21.7
SwinV2-L-In2Ik
87,5
31.3
29.7
22.4
26.6
20.7
83,7
19.6
16.6
20.7
32.9
20.8
21.1
23.5
11.9
20.0
17.1
SwinV2-S
84.2
53.1
61.8
52.2
$2.9
34.9
99.9
44.5
37.6
46.5
58.3
39.9
48,1
37.1
25.3
33.8
26.8
SwinV2-B
84.6
54.6
58.5
46.9
52.1
34.1
99.6
41.0
36.2
45.2
49.3
36.8
43,9
33.5
27.1
31.6
27.6
ResNet101
81.9
60.5
79.9
99.5
64.3
29.4
72.7
34.2
35.6
63.6
78.9
51.6
60.4
27.7
33.4
50.8
71.4
ResNet152
82.3
59.3
79.5
63.8
28.5
72.8
29.9
62.0
49.3
58.0
26.5
45.9
68.7
ResNer50
80.9
66.2
96.5
99.2
71.3
32.4
72.8
$4.2
$2.2
72.0
88.6
55.2
65.7
32.2
43.0
61.2
75.7
ResNet50-supcon
78.7
33,9
25.0
61.7
20.0
23.5
34.4
30.7
89.5
WT--T66-ln2Ik-aggrr
75,5
62.1
39.6
30.6
44.0
36.1
96.8
69.1
60.7
36.4
48.3
52.1
45,2
46.0
32.2
49.1
46.1
VII-S16-In2Ik-auge augreg
81.4
45.8
24.2
28.5
27.4
18.0
68.8
41.2
33.4
23.1
32.2
31.4
29.0
23.4
15.5
30.9
27.1
B16 In21k augreg2
85.1
46.0
37.1
30.2
37.6
38.0
99.5
30.3
25.4
33.1
54.5
29.2
36.8
38.7
25.8
33,0
27.8
VTT-B16-In2lk-agge
84.5
34,5
19.9
21.0
21.2
13.0
94.6
40.6
30.7
18.8
27.0
22.3
21.6
13.0
14.5
19.1
19.3
VIT-B16-II2lkkoorig
81.8
33.9
21.3
23.2
20.7
62.7
22.6
20.9
40.5
25.9
24.4
21.2
18.8
25.8
24.6
FTEBI6-In21l-mmnl
84.3
36.3
23.3
22.2
26.8
26.2
97.4
28.9
23.2
21.7
$3.5
23.7
29.5
34..4
19.0
32,6
26.3
WT-LL6--In2Ik-aggree
85.8
29.3
16.0
16.1
17.4
11.0
93.4
24.7
15.7
33.1
18.7
16.9
10.7
11.8
15.8
15.8
WT--L16-In2Ik-orig
81.5
32.7
22.1
22.0
23.6
21.0
44.9
22.8
22.0
21.8
40.3
25.8
24.3
20.6
18.3
24.3
23.6
516--e6-
78.8
55,8
45.4
47.5
47.6
56.6
97.1
61.3
58.2
47.7
$2.5
50.3
$3.8
35.6
35.5
36.1
36.0
VTT-B16-aagre anggee
79.2
55.3
47.2
43.3
48.6
53.0
88.2
53.8
52.5
48.2
54.3
50.5
52.7
36.4
34.6
35.8
34.7
VTT-B16-CLLI--.2---n1PP
86.2
32.9
31.6
28.5
29.3
22.3
99.7
20.8
19.0
26.0
34.0
22.9
25.7
28.7
17.1
23,9
21.1
WIFLIA-CLII-LLD--1112
18.8
14.5
15.1
17.7
13.4
17.2
VITHI44CULP..226m1222
88.6
23.7
19.6
19.0
20.3
14.8
98.3
18.3
15.3
20.1
51.3
16.3
18.7
15.6
12.8
16.9
15.9
VIT-so400M-SigLipp
89.4
36.8
41.1
30.7
34.0
17.7
92.4
15.4
14.3
31.9
64.6
18.7
17.3
16.8
15.1
16.3
16.4
Average
84.4
44.0
45.7
42.6
40.4
28.1
89.1
35.1
32.1
36.7
50.3
32.3
35.4
29.1
23.0
30.5
29.3
23

---

Mahalanobist+: Improving OOD Detection via Feature Normalization
Table 10. AUC on OpenOOD, Green indicates that normalized method is better than its unnormalized counterpart, bold indicates the best method, and underlined indicates second best method. Maha++ improves over Maha on average by 2.1% % AUC over models. Similarly, rMaha++ is 0.6% better in AUC than rMaha. In total, Maha++ improves the SOTA compared to the previously strongest methods rMaha by 1.4%, which significant. The highest AUC achieved by Maha++ for the EVA02-LI4-M38m- blue.
highlighted
Model
Val
MSP
E+R
ML
VIM
AshS
KNN
NNG
NEC
GMN
GEN
fDBD
Maho
Maha++
rMaha
(Maha++
Com/NeXt-B-Ina -n2kkk
86.3
88.8
86.8
88.9
88.4
93.3
52.1
90.1
92.6
91.6
88.1
91.5
90.1
91.9
93.7
92.0
92.5
Con-NeXt-B
84.4
80.9
71.2
74.7
85.4
19.8
84.4
76.4
84.7
87.1
88.7
87.9
Co
85.1
87.4
88.1
88.3
88.3
93.3
40.3
88.6
91.0
91.3
89.1
90.0
86,4
91.8
92.7
91.2
91.5
ComvNeXIV2-B-In211
87.6
89.9
90.4
91.0
90.4
94.8
45.2
92.9
93.6
93.4
91.1
92.3
91.4
94.0
94,9
93.7
94.0
Comv<cXtV2-L-nn1I
88.2
90.6
91.1
45.4
92.1
91.7
95.3
93.8
Com-NeXVV2-T
83.5
K3.5
78.1
82.2
81.7
86.7
21.6
81.2
85,3
83.5
82.2
86.6
84.9
87.0
88.9
88.2
89.1
Com-NeXIV2.B
62.8
78.7
79.0
86.8
Com/NeXiV2-1.
86.1
83.0
73.0
79.9
78.9
84.5
17.3
87.2
88.8
78.9
85.8
87.3
88.0
89.8
90.6
90.3
90.6
DeiT3-S16-In21k
84.8
80.3
78.7
81.2
79.5
87.1
19.5
86.0
87.4
81.1
87.9
85.2
85.5
87.9
89.5
88.1
De1T3-B16-In2IK
86.7
82.4
74.8
82.2
78.4
90.6
18.4
89.7
90.4
82.7
89.8
87.4
88.7
91.0
92.3
91.1
92.1
DeTT3-L16-I22Ik
87.7
84.3
81.5
86.6
82.7
92.0
20.5
91.1
91.4
88.3
90.9
89.3
89.4
92.0
93.0
91.9
92.8
De133-S16
83.4
84.2
84.7
85,4
86.5
88.6
Dei33-B16
85.1
83.5
69.5
633.3
79.0
87.6
22.4
83.1
79.1
79.8
85.11
87.6
%6.1
88.1
88.9
88.9
89.6
Dei33-L.6
82.9
80.5
86.7
89.7
EVAM2-B14-1o21
88.7
91.0
90.4
91.3
90.9
95.0
54.0
92.2
93.6
93.4
92.5
93.3
92.6
94.0
94.9
93.6
94,
EVA02-L14-M33m-nnn22
90.1
92.4
91.7
92.2
92.3
95.9
48.0
93.6
94.6
94.4
92.0
94.3
94.1
95.5
95.9
95.1
95.3
EVAU2-T14
80.6
81.3
79.1
78.9
80.8
87.2
43.1
81.7
85.0
85.0
86.4
85.0
80.0
86.6
87.2
86.3
86.7
EVA02-514
85.7
84.4
79.7
80.1
82.1
91.6
22.7
87.2
89.5
86.4
90.5
87.8
85.4
90.8
91.2
90.4
90.7
BIINarVV-8
83.2
20.8
90.0
E/fNeiVV2.L
85.7
83.9
73.5
83.2
80.6
85.6
17.8
86.9
87.8
82.2
86.3
87.8
86.66
89.6
90.8
90.7
91.3
RdNNetV2-M
83.7
83.3
80.6
86.3
86.2
90.5
90.2
Mixer-B16-In2lk
76.6
90.4
78.9
78.7
79.8
82.5
48.6
79.3
79.3
79.9
84.0
82.2
81.3
83,1
86.4
85,1
86.4
SwinV2-B-In2Ik
87.1
58.0
87.0
90.4
88.3
92.1
47.5
88.8
91.8
91.9
87.0
91.4
90.1
90.6
92.9
90.8
91.9
SwinVV-LLIIn21
87.5
88.7
86.8
90.8
88.1
91.9
38.4
89.8
92.2
91.8
88.9
91.6
90.8
91.1
93.7
91.5
92.7
SwinV2.5
84.2
82.3
74.8
81.4
78.9
87.0
14.8
84.3
86.8
81..
85.7
86.4
85.7
87.9
90.0
8R.3
90.0
SwinV2-B
#2.4
79.4
21.2
86.1
ResNet101
81.9
79.9
68.2
23.1
75.8
83.7
56.4
82.8
R5.5
76.7
74.9
$3.0
79.2
BSLD
89.5
87.0
85.2
R5sNer152
82.3
80.4
67.6
21.0
75.8
56.0
82.7
77.6
77.7
80.6
90.0
87.7
ResNet50
80.9
77.5
52.6
27.6
73.3
82.0
60.8
79.7
81.5
73.3
65.1
81.9
77.4
86.5
87.5
84.6
83.1
Respee50-suppon
78.7
85.2
87.7
88.7
87.66
82.3
188.6
86.4
83.6
87.8
78.9
86.7
87.1
$2.9
89.2
76.5
85.6
V16 In21k augreg
75.5
79.9
85.5
86.9
85,11
86.55
45.7
75.9
86.4
84.1
83.9
85.5
84.3
86.5
83.8
84.0
VIT-S16-In2 Inggrgggggggggg
81.4
84.4
90.4
89.9
90.0
91.6
66.5
85.3
SR.2
90.8
8R.8
89.
89.2
90.4
91.5
89.1
89.4
85.1
84.4
87.3
85.4
85.9
24.6
86.8
89.1
84.5
90.3
88.6
1TT-B6--In22 muggrgg
84.5
$7.5
91.9
91.3
91.7
93.6
54.8
85.6
89.1
92.4
90.1
91.6
91.4
93.4
92.5
92.0
91.9
VIT-B16-In2 k--org
81.8
88.0
93.2
93.1
92.7
93.5
77.1
90.4
91.9
93.3
92.0
92.7
93.5
91.6
91.8
11-B16-In21kmmil
84.3
87.7
90.6
91.0
90.1
91.8
32.1
188.4
91.1
91.7
86.6
91.0
89.1
89.6
92.6
90.2
91.2
VTT-L16 (116-1n21k-aaaa lang--gg2lk
85.8
89.6
93.0
94.1
92.9
940.6
58.1
87.8
9006
93.5
89.9
93.0
93.1
94..9
94.0
93.9
93.7
16T-L.6--I21Ikorr
81.5
89.5
93.3
93.0
93.4
86.2
92.0
93.5
87.3
92.5
92.8
93.8
92.2
92.4
VIT-S VIT-S16-augreg
78.8
82.3
84.9
84.6
84.8
80.4
46.7
77.9
81.8
84.6
83.9
85.
83.5
86.9
86.8
87.4
87.4
ViT-B16-augreg
79.2
82.7
86.4
85.7
$2.9
84.8
87.6
12k
86.2
87.9
85.8
87.8
87.1
92.3
19.5
90.4
92.5
89.6
91.2
90.8
90.6
90.5
93.3
91.2
92.2
WT-LIA-CLIP-L2--I1222
88.2
90.5
89.9
90.6
90.4
94.6
38.2
92.2
94.1
90.9
92.3
92.6
92.7
93.6
94.8
93.7
94.0
88.6
90.6
90.1
90.2
94.2
27.6
92.0
93.6
90.8
88.9
92.7
92.1
93.7
94.7
93.7
94.0
s0000 Sigl-ip
89.4
87.6
79.7
85.8
83.8
92.6
25.7
91.8
93.1
85.7
88.8
91.6
92.0
93.2
93.8
93.4
93.6
Average
85.0
84.4
40.4
87.5
24

---

Mahalanobist+: Improving OOD Detection via Feature Normalization
Table 11. AUC on NINCO datasets, Green indicates that normalized method is better than its unnormalized counterpart, bold indicates the best method, and underlined indicates second best method. Maha++ improves Maha average 2.6% in AUC all models. Similarly, + is 1.0% better in AUC than rMaha. In total, Maha++ improves the SOTA compared to the previously strongest methods rMaha by 1.0%, which significant. The highest AUC achieved by Maha++ for the EVA02-L14 ------1-----8---- M-8mmIn21k highlighted
bluc.
Model
Val Ace
MSP
E
E+R
ML
VIM
AshS
KNN
NNG
NEC
GMN
GEN
1DBD
Maha
Maha++
rMaha
rMaha++
MowNccX-BBIn22k
86.3
88.2
85.7
87.8
87.6
92.3
45.4
88.0
91.3
90.7
87.7
90.9
89.3
91.2
94.3
92.3
93.5
Com-NeXi-B
25.1
85.2
87.2
88.8
NmwNNeNiV2-TTIn21
85.1
86.7
87.2
87.4
87.5
92.7
39.7
86.8
90.2
90.6
88.8
89.4
85.9
91.7
92.9
91.4
91.9
Com-NeXiV2-B-In21k
87.6
89.4
89.1
89.9
89.3
94.8
43.8
91.1
93.5
92.7
92.0
91.8
94.5
95.0
ConvNeXiV2-L-Inn21
88.2
90.2
89.3
90.7
89.7
93.9
39.8
91.6
93.9
93.4
92.6
92.3
92.8
94.1
96.2
94.7
95.5
Con~NeXIV2-T
83.5
82.9
76.2
80.3
80.5
83.5
26.0
78.0
83.1
82.0
81.0
85.6
82.5
85
88.
87.5
89.0
Con-NeXIV2.B
85.5
82.6
73.2
78.2
79.2
83.6
20.0
83.7
86.6
79.8
82.9
86.1
85.2
87.9
89.0
90.3
Con-NeXIV2-L.
86.1
82.2
71.7
78.6
78.0
80.8
19.1
84.8
87.1
77.5
84.2
86.4
86.1
88.7
90.2
89.9
90.6
DeiT3-S16
74.8
23.2
77.6
89.0
DeiT3-R16-In2lk
86.7
81.55
74.5
80.5
77.5
89.3
21,3
87.6
89.2
81.3
89.6
86.6
87.5
90.2
91.8
90.6
91.9
DeiT3-L16-In21k
87.7
83.9
81.9
86.3
82.7
90.9
25.3
90.8
87.4
90.6
89.1
88.7
91.6
92.8
91.9
92.9
Dei13-S16
83.4
82.9
81.4
81.2
83.1
86.3
64.6
$1.1
85.0
83.0
84.4
86.4
85.1
87.5
S8.6
88.4
89,3
DeiT3-B16
85.1
82.2
66.6
62.4
76.7
85.1
28.4
80.5
77.1
77.3
83.9
86.4
84.0
87.2
S8.3
B8.1
89.2
DeiT3-L.16
85.8
81.2
75.5
68.8
78.6
84.7
62.3
81.1
78.8
78.7
84.8
86.0
84.9
88.2
89.
89.2
90.1
EVA02-B14-IRg
88.7
90.6
89.7
91.0
90.3
94.7
50.1
91.1
93.2
92.7
92.6
93.2
93.0
94.0
95.1
93.9
94.6
EVA02-L.4 M:8m-
91.1
91.8
40.9
94.6
95.9
EVA02-T14
80.6
79.2
75.3
75.0
77.7
83.6
41.1
78.3
81.9
81.5
83.7
82.3
78.0
84.2
84.8
84,3
84.7
EVA02-S14
85.7
81.8
76.1
76.5
78.8
88.6
27.0
84.3
87.1
82.8
89.6
85.0
82.8
89.6
89,3
89.6
ENNevV2-S
83,9
81.1
71.1
77.6
76.3
81.7
20.7
83.5
84,9
78.3
80.6
84.6
83.7
85.5
87.1
88,1
89.4
EBNeeVV2-L
85.7
82.6
73.3
80.3
79.6
80.2
20.9
84.0
85.4
79.7
85.3
86.4
84.0
87.4
89.0
89.5
90.5
FANerVV2M
85.2
82.3
71.8
79.5
78.6
81.0
19.3
85.4
79.4
83.9
86.6
84,1
87.2
89.
89.5
90.6
Mixer B16 In21k
76.6
79.3
78.1
78.1
78.8
S0.8
49.8
76.2
78.6
78.9
82.3
80.9
79.1
80.5
84.6
B3.8
85.2
SwinV2-B.ln211
87.1
86.0
89.2
90.8
44.1
90.5
91.2
90.4
SwinV2-L-In2Ik
87,5
88.2
85,9
89.9
87.4
90.8
36,9
87.8
91.5
91.4
87.7
91.2
89.8
90.0
94,1
91.2
93.5
SwinV2-S
84.2
80,9
74,8
79.7
78.1
83.9
17.6
80.7
83.8
80.3
84.5
84.7
82.9
86.0
88.
86.6
88.9
SwinV2-B
84.6
80.8
74,9
S0.4
78.0
81.4
28.9
$2.3
84.2
79.7
84.5
85.2
83.1
86.8
S8.5
87.3
88.7
ResNet101
81.9
78.7
67.3
21.1
74.6
76.9
47.5
77.6
82.6
74.7
74.
80.9
73.9
85.3
89.1
87.8
8R.1
ResNet152
82.3
79.1
66.22
20.0
74.6
77.3
48.7
83.1
75.6
81.4
75.7
85.9
89.5
87.9
88.6
ResNer50
80.9
76.8
$5.0
23.7
73.0
73.5
$2.6
75.1
78.8
72.3
66.2
79.7
71.7
82.7
86.5
86.1
86.6
ResNet50-supcon
78.7
87.2
79.2
86.0
86.8
87.3
86.9
85.0
48.2
78.7
87.0
WI--T66-ln2Ik-aggrr
75,5
78.1
81.0
82.1
81.3
82.2
54.4
73.3
76.2
82.3
81.0
80.8
83.0
84,0
83.7
83.0
82.8
VII-S16-In2Ik-uug augreg
81.4
83.4
88.6
88.4
88.5
89.7
61.1
82.0
85.6
89.2
86.7
87.77
88.4
90.7
90.8
89.5
89.4
B16 In21k augreg2
85.1
83.2
82.7
85.2
83.9
83.6
28.7
87.6
86.1
8.5.5
87.88
82.8
86.1
90.7
88.6
90.8
VIT-B16-In2lk-aggee
84.5
86.3
91.1
90.3
91.0
92.4
56.2
82.8
87.5
91.5
89.3
91.1
90.3
94.1
93.2
93.2
93.0
VIT-B16-In2lk-oorig
81.8
91.8
92.6
71.0
92.1
90.3
91.2
93.8
92.7
92.9
84.3
86.5
88.1
8
88.0
91.2
31.3
87.2
89.7
90.0
$6.5
89.8
88.2
89.9
93.1
91.1
92.2
WTFLL6-In2lk--agrree
85.8
89.5
92.7
94.9
92.7
93.9
48.1
88.8
93.2
88.
93.2
92.4
95.3
95.1
1I-LLi--n21kkooig
81.5
89.3
91.7
91.8
91.6
91.9
81.7
89.7
91.3
92.1
86,1
91.1
91.6
92.4
93.5
92.9
93.1
516--e6-
78.8
80.8
81.8
81.5
82.2
74.8
44.4
73.8
78.2
81.9
81.4
83.0
81.0
85.3
85.
86.6
86.6
VTT-B16-aagee angreg
79.2
81.0
83.3
83.8
83.4
81.0
52.4
81.1
81.3
82.0
83.9
82.5
85.7
85.7
86.9
86.9
VTT-BI6-CLIP--22---n
86.2
86.6
82.9
85.5
84.9
91.1
21.5
88.4
91.1
87.9
90.6
89.6
90.1
89.4
92.9
90.6
92.3
WIHLLA-CLLI-LLD---1122
88.2
87.9
94.4
33.4
94.2
94.8
VTTHI44-ULP..22611222
88.6
89.7
87.2
88.2
88.5
93.9
27.6
91.0
92.4
89.2
89.8
91.8
92.7
94.2
95.3
94.3
94.8
VIT-so400M-SigLipp
89.4
87.3
79.5
85.1
83.2
92.1
26.1
91.8
93.3
84.7
88.3
91.7
92.4
93.3
94.6
94.2
94.8
Average
84.4
84.1
80.2
79.3
83.1
86.6
39.9
84.1
86.7
84.6
85.4
87.3
85.9
88.1
90.7
89.7
90.7
25

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Table 12. FPR NINCO for bosine-based methods, Green indicates that the normalized method is better than its unnormalized counterpart, bold indicates the best method, and underlined indicates the second best method. Mahalanobis++ consistently outperforms other cosine-basce methods. In only 2 out of 44 models, another method (once NNguide and once Cosine) is better than Maha++.
Model
Accuracy
Maha++
Cosine
KNN (Sun al., 2023)
NNguide (Park et al.,, 2023a)
SSC (Techapanuaak 2020)
Con-NeXI-B
84.434
50.5
60.6
62.2
69.3
ComNNeXi-BB B-Ikkk In2kk
86,270
28.8
42.2
$1.6
41.2
$1.3
Con/NeXrV2-B
85.474
44.7
57.1
60.3
66.6
CoorNeXIV2-B-In211
87.642
22.4
31.1
40.9
31.7
40.4
CoV2NeXiV2.1.
86.120
43.0
53.8
62.4
56.9
59.7
Com-NeXiV2-L-In2lk
88.196
18,4
27.0
38.7
29.9
39.2
ConvNeXIV2-T
83.462
52.3
69.4
82.3
73.9
72.8
Com-NeXiVV-TTIn211.
85.104
45.0
55.1
DeiT3-B16
85.074
57.2
67.1
74.5
80.1
65.7
DelT3-B16-In2Ik
86.744
38.8
46.9
46.4
53.2
DeiT3-L16
85.812
50.4
62.2
67.2
77.9
68.7
DeTT3-LI6-In2Ik
87.722
33.9
38.9
43.8
37.8
46.3
DeiT3-S16
83,434
53.5
67.6
57.8
63.6
Dei13-S16-In21k
84.826
50.8
58.4
62.9
59.3
65.2
WA402-B14Inn21.
23.8
29.1
30.0
34,4
1n402--.14-M33m-nn221kk
90.054
18.6
22.7
30.3
26,1
28.8
EVA02-S14
85,720
48.0
53.6
60.0
54.0
63.8
EVA02-T14
80,630
64.0
69.8
74.5
71.0
75,5
Mixer Mixer-B16-Ia22 I16-
76,598
65.4
78.2
85.8
83.7
79.5
ResNet101
81,890
50.4
61.5
66.4
87.2
ResNet152
82.286
46.5
62.1
72.0
61.6
85.8
ResNe150
64.0
75.0
88.2
ReiNet50-supoon
78.686
59.6
58.9
65.8
58.4
74,1
SwinV2-B-In2ik
87.096
31.3
45.9
57.2
42.7
53.7
SwinV2-B
84,604
52,2
63.4
69.4
65.2
71.6
SwinV2.L. In2kk
87.468
28.3
42.8
55,1
41.7
53.6
SwinV2-S
84,220
65.2
66.8
75,2
E
85.664
47.8
$7.0
62,5
60.1
62.4
FANNrVV2-M
85,204
50.0
58.0
60.6
64.4
EANNetV2-S
83,896
59.9
58.8
60.9
59.6
68.7
VIT-B16-In2IK-uunnn22
85.096
45,9
56.4
64.9
$7.0
64.8
VTT augree
79.152
61.3
73.1
77.6
75,9
75.7
VIT.B16-I221. 1TEB16.I2llk-mgree
84.528
35.7
53.4
67.7
$9.0
$4.0
WII-B16-In21k-ee
81.790
31.6
47.6
45,2
666 In21k miil
84,268
35.4
49.7
59.6
51.7
62.1
VIT-BI6--LIP--22---0
86.172
35.8
43.5
49.4
42.3
48.8
VI-HII--CLIP-L2D--Dl
88,588
23.7
31,5
41.7
27.4
36.5
VIFLL1-CLIP-L2---il
58.178
25.4
30.9
39.5
25,4
33.3
L16 In21k augreg
85,840
28.9
50.0
68.6
58.9
48.2
VT-LL16-In21 In2
81.508
32.4
39.2
45.8
40.7
42.0
augreg
63.1
75.8
78.1
81.388
44.6
60.3
70.9
64.0
61.5
VIT-so400M-Sige SiggL
89,406
27.4
29.0
30,0
35.6
VII-T16-1n2 Hk-agreg
75.466
63.2
77.2
81.7
81.9
74.2
Table 13. AUROC for CIFARIO, Green indicates that the normalized method is better than its unnormalized counterpart, bold indicates
the best method, and underlined indicates the second best method. Maha++ is clearly the best method. Only for the WRN28- 10 Maha better (but not significantly). Maha++ improves in all cases over the previously beset methods ViM. We highlight the best AUC achieved
by Maha++ for VIT-B16-2IK- in blue.
Model
Ash
Dice
Ebo
KIM
KNN
ML
MSP
O-Max
React
Sbe
NNguide
T-Scal
VIM
Neco
cMD
rMD++
MD
MD++
SwinV2-S-Ik
69.96
92.85
95.61
98.04
99.25
93.83
96.60
97.02
96.83
96.88
67.51
96.61
99.53
98.86
98.83
98.77 79)
99.50
99.57
VIT-R16-1IK-IK
82.75
99.33
99.42
99. 64
99.41
96.88
97.66
99.45
98.99
87.30
99.06
99.67
99.56
99.03
99.04
99.60
99.71
RNI8
87.15
89.60
91.09
79.62
91.58
90.97
89.93
89.04
90.78
87.62
63.57
90.32
91.12
90.67
89.92
90.05
86.87
91.69
RN34
78.29
84.84
87.26
82.75
92.15
87.20
88.11
87.38
87.50
81.40
55.07
85.07
92.50
86.39
90.34
90.49
91.53
93.61
RNx129-32
78.33
71.90
88.45
83.19
90.46
88
80
87.98
85.65
85.27
87.90
29.57
87.97
91.36
89.62
89.84
88
90.70
91.56
Average
79.29
87.70
92.37
88.11
94.62
92.32
92.30
91.35
91.97
90.56
60.60
92.41
94,84
93.02
93.59
93.41
93.64
95.23
SC
96.76
94.46
94.30
59.00
96.80
RC
96.15
94.72
94.24
64.21
96.77
26

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Table 14. FPR for CIFARIO, Green indicates that the normalized method is better than unnormalized counterpart, bold indicates the best method, and underlined indicates the second best method. Maha++ is the best method average. We highlight the best FPR achieved by Maha++ for VIT-B 16-2 Ik in blue.
Model
Ash
Dice
Fbo
KIM
KNN
MI.
MSP
(-Max
She
NNguide
T-Scal
VIM
Neco
rMD
rMD++
MD
MD3++
SwinV2-S-Ik
93.43
19.51
8.82
6.02
4,03
8.07
6.74
5.97
7.21
12.08
63.18
6.73
2.17
3.66
3.42
3.18
2.35
2.16
VIT-R16-2IK-IK
60.41
2,42
1.93
6.23
1.75
3.27
1.99
4.48
41,88
2.91
1.29
1.57
2.59
2.66
1.66
1.24
RNI8
47.52
41.18
39.22
56.48
45.55
40.28
56,42
76.47
40.22
45.89
77.65
52.58
51.99
41.10
52.51
54.09
69.46
46.15
RN34
46.12
44.20
38.05
52.89
46.24
39.14
51.99
78.47
42.36
45.81
76.79
48.98
48.15
41.93
52.67
53.67
54.45
38.36
RNx:29-32
97.43
63.57
47.36
56.54
55.91
50.41
53.15
89.85
56.26
38.13
99.97
53.36
36.13
51.32
58.07
61.31
41.17
34.64
68.98
34.18
27.08
35,63
30.70
27.97
34.31
50.78
29.61
29.28
71,89
32.91
27.95
27.92
33.85
34.98
33,82
24.51
RN
19:48
33.23
35.
81.77
18.59
RN
22.47
30.52
32.89
78.68
17.58
Table 15. AUROC for CIFAR100, Green indicates the normalized method is better than unnormalized counterpart, bold indicates the best method, and underlined indicates the second best method. Maha++ + is clearly the best method. Only for the RNxt29-33 32 She slightly better. Maha++ improves in all cases over the previously best methods ViM, Maha and KNN. We highlight the best AUC achieved by Maha++ for the ViT-B32-2Ik in blue.
Model
Ash
Dice
Bbo
KIM
KNN
M1.
MSP
O-Max
React
She
NNguide
T-Scal
VIM
Neco
rMD
rMD++
MD
MD++
SwinV2-S-Ik
48.67
63.60
84.72
82.52
90.06
85.20
85.68
85.82
87.53
89.66
71.36
85.93
91.34
90.38
89.77
89.30
90.29
92.99
Deil3-S-11k
49.99
44.78
85.69
81.59
88.06
86.18
86.21
84.52
88.88
87.47
55,19
86.43
90.41
89.86
87.44
87.85
88.30
90.54
ComvN-T-21k
63.80
53.50
77.76
80.48
86.60
78.51
79.09
82.60
80.17
82.94
62.98
79.22
87.67
81.31
85.00
84.89
87.95
89,55
MT--32-211
39.23
88.31
90.28
89.13
94.87
89.99
85.36
88.00
88.59
94,10
87.17
86.73
94.62
90.70
92.37
92.82
95.59
96.84
MT-S16-21k
65.78
84.35
89.85
84.23
93.97
89.44
83.87
88.09
88.45
92.32
80.08
85.38
95.91
90.80
93.09
93.32
95.63
96.81
RN18
74,20
74.77
80.31
74.11
81.22
80.31
79.70
68.22
80.27
79.18
81,06
80.02
78.50
80.66
H1.27
80,91
78.46
81.71
RN34
65.82
78.86
79,88
75.63
81.51
79.76
79.13
73.14
80.48
77.15
74.13
79.56
82.13
80.61
81.22
80.94
82.03
82.16
32
79.46
82.01
78.58
70.79
80.89
78.47
78.37
66.11
78.36
82.59
73.21
78.22
75.33
79.68
76.87
77.06
76.18
82.48
Average
63.37
71.90
83.38
79.81
87.15
83.48
82.18
79.56
84.09
85.68
73.13
82.69
86.99
83.50
85.88
83.39
86.80
89.14
RC
76.80
SC
$2.41
77.90
79.67
59.01
82.44
Table 16. FPR for CIFARI00, Green indicates that the normalized method is better than its unnormalized counterpart, bold indicates the best method, and underlined indicates the second best method. Maha++ is improving in all cases over Maha and is on average the best method. We highlight the best FPR achieved by Maha++ for the ViT-S16-21k blue.
Model
Ash
Dice
Ebo
KIM
KNN
ML
MSP
O-Max
React
She
NNguide
T-Scal
VIM
Neco
rMD
iMD++
MD
MD++
S.
92.66
15.98
40.95
49.65
36.27
41.96
47.28
67.04
39.54
39.64
80.29
45.58
34.02
$3.59
41.40
47.14
40.10
26.01
Deil---21k
94.47
96.34
41.61
47.86
36.81
42.37
48.92
66.00
40.46
40.93
96.35
47.15
39.99
37.12
41.02
41.36
41.99
31.72
C-21N-T-21k
92.11
89.10
57.67
65.50
51.16
57.44
60.60
66.86
58.23
53,76
91.25
60.04
51.18
53.92
62.79
61.66
52.48
42.69
VTT-B32-11k
93.98
46.59
30.51
43.24
26.49
31.28
48.02
53.68
32.53
33.25
64.86
40.74
27.14
28.61
33,80
31.03
26.28
18.94
VIT:S16-1Ik
80.45
56.38
36.06
50.09
31.91
37.63
52.17
57.38
36.48
38.89
77.85
46.68
24.90
33.24
34.10
32.83
25.51
18.58
RN18
78.98
80.53
80.19
78.85
76.61
79.87
80.59
97.36
80.18
80.46
68.16
80.25
79.61
79.89
76.14
77.49
79.48
72.92
RN34
78.27
78.31
75.19
78.08
74.44
75.33
76.93
94.07
74.51
78.76
75.07
76.20
77.17
74.25
75.82
76.22
76.63
74.51
RN
72.59
67.03
82.22
87.56
73.17
82.30
82.31
96.32
81.87
69.42
81.89
82.60
76.40
80.54
86.58
84.39
77.67
67.71
Average
83.44
73.78
55.35
62.60
30.86
53.90
62,10
74.84
33.47
54.39
79.47
59.90
$1.30
52.65
36.46
56.51
$2.52
44.13
RN34-SC
74.37
93.76
63.51
SC
66.69
83.53
78.15
82.38
67.95
27

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
Table 17. Normalization improves robustness against noise distributions. We report the number of failed unit tests (noise distributions with FPR values 2 10%) from (Bitterwolf et al., 2023). Normalization improves the brittleness of Mahalanobis-bor detectors.
model
Maha
Maha++
ConvNeXt-B
16
15
Con-NeXit-BIn21k
0
ConvNeXtV2-B
14
6
ConvNeXtV2-B-In21k
ConvNeettV2-L
13
4
ConvNexttV2-L-In21
2
0
ConvNeXIV22-T
17
9
ConvNeXtV2-T-In211
6
0
DeiT3-B16
14
15
DeiT3-B16-In21k
6
3
DeiT3-L16
8
DeiT3-L.16-n211k
DeiT3-S16
15
10
DeiT3-S16-In21k
17
11
EVA02-B14-In21k
3
0
EVA02-LL14MM8mm-n22k
0
0
EVA02-S14
8
0
EVA02-T14
0
17
10
ResNet101
0
ResNet152
0
O
ResNet50
O
ResNet50-supcon
17
SwinV2-B-In2Ik
10
0
SwinV2-B
12
6
SwinV2-S
15
EffNetV2-L
13
7
ErNetV22M
13
4
EfNetV22S
11
3
VITB16--24-I2221-aarrggg
16
7
VYT-B16-224-aurreg
4
VIT-B16-244-I2II-oorig
0
VIT-B16-224-In2Ik-mmnt
17
VIT-BI6-CLIP--D--nn12k
14
VIT-H14-C11P-.2--nI11k
4
VITLL1-CCLIP-L2b-nn11
7
VYTHL16-224-In21k-oorgg
5
VIT-S16-224-augree
2
1
VIT-so400M-SigLip
8
0
28

---

Mahalanobis++: Improving OOD Detection via Feature Normalization
Table 18. Comparison to SSD+. SSD+ consists of a) training with contrastive loss (implicitly normalizing the features), b) estimating cluster means the normalized feature space k-means, c) centering the train features with closest class mean and estimating shared covariance matrix, and d) using the Mahalanobis distance at inference time for OOD detection. SSD+ is therefore not readily applicable as post-hoc OOD detection method. highlight benefits of post-hoc methods, we report the performance of SSD+ with ResNet-50, which was trained for 700 epochs with supervised contrastive loss, and compare it to ConvNext model and an EVA model with varied pretraining schemes. The latter models outperform SSD+ clearly, underlining importance of post-hoc methods for OOD detection.
Model
FPR (%) )
SSD+ 100 clusters
66.0
SSD+ 500 clusters
65.7
SSD+ w. 1000 clusters
67.8
CnvNxtV2-L + Maha++
18.4
EVA02-L14 + Maha++
18.6
29

---

Mahalanobist+: Improving OOD Detection via Feature Normalization
F. Models
30

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
Table 19. Imagenet model checkpoints.
Modelname
Checkpoint
source
ViT-B16-In2 laaagggg
vit vit.base.patchl 22666222.auuu
22..222..uugggegg.aa -in..nn222 lk_ it
timm / huggingface
ViT-L16-In2Ik l-agrre augreg
vit vat_Jarge.patchl wwputch16622000 wtt.ll...222222... 22w.auureg..in222. lt_t2liffftiill_
timm / huggingface
ViT-T16-In2
-it-r16-In2k-auurreg k-augreg
vit_tiny ww..inyyppatchh... 22e.66_22224___ l_ugre_.i2llkfffiii.
timm / huggingface
WTT-S16-In2IK-aupee
vit small patch16 1at.ll 224.augreg in2 2 ft inlk
timm / /uggingface
ViT-B16-augreg
vit. www.bbseseppchhhcc 224.augreg inlk
timm huggingface
ViT-S16-augreg
vit pammalllpatcch12222auuug vit 10s.:000mm.pccch....ggpp......b0 ftnk vit hitngg_pacchh4.cclp333...aa2222 itt.iinnn22ff..ii22iiiiii. vit large patch14 clip 336.laion2b ft in in
timm huggingface
ViT=so400M-SigLipp
timm huggingface
VITHI14-CLUP-L22--II1k
timm huggingface
VTTL144CI.JP-122--112
timm / huggingface
ViT-Bi6-In2Ik-orig
vit vit www.lareeeaacc22..353 vit httpcpppichh222.....ognnnnn.... vit base patch16 224.augreg2 in21k ft in
timm / huggingface
ViT-L16-In21-oriig
timm huggingface
ViT-B16-In2Ik-miill
timm huggingface
ViT-BI6-In2II-aaurree2
timm huggingface
VIT-B16-CLIP--2D---122
vit htt.:pppicchioo..cl..224.eiaii in in
timm huggingface
EVA02-L144MM3mmnn22k
wwd0__aareeppacch4..4..... 2tt.mmmmmmmm22222222221 in22k_inlk
timm huggingface
EVA02-B14-In21k
eva02 base patch14 448.mim in22k ft in22k inlk
timm / huggingface
EVA02-S14
wa002ssmall_paae 1..pch14433..mmm 33m.mim in22kk ft. inikk
timm / huggingface
EVA02-T14
eva02 tiny www.patch114. 11011144333mmmmm 336.mim in22k ft inlk
timm huggingface
DeiT3-B16
deit3 Satts_aas..pacchl_22
timm / huggingface
DeiT3-B116In21k
deit3 base patchi1 224 in21ft1k
timm huggingface
DeiT3-L16-In21k
deit3 1art_llare-pachhh6.333
timm huggingface
DeiT3-B16-In2Ik
deit3 base patch16 384.fb in22k ft inlk
timm / huggingface
DeiT3-L16
deit3 Biirs3_aagee_paahh163..........
timm / huggingface
DeiT3-B16
deit3 Seit3_base_ppe
timm / huggingface
DeiT3-S16-In21k
deit3 small patplllcchhcpc 384.fb in22k ft in1k
timm huggingface
DeiT3-S16
deit3 deif3_small.pacch16338... srinv2_small..innoww 16 156.ms_inkk
timm / huggingface
V2-S
timm huggingface
Swin V2-B
swinw2_base_window 16 11_22566msiimm
timm huggingface
Swin SwinV2-L-In21k
swinv2 large window 12to24 192to384.ms in22k ft in
timm huggingface
SwinV2-B-In2Ik
base_window wtt.eeiddoww/2222210101 120224112113884.m mw..mmsnn.222. [n22k_ft_im in
timm / huggingface
ResNet50
resnet50.tv2 in
timm / huggingface
ResNet101
resner101.tv2 01.tt2nvinnn 52.n2
timm / hwwggiggggaacccc
ResNet152
timm / huggingface
ResNee50Osupcon
rn50supcon
github.com. www//oooo////eiieooooooooooooooc /nnguide/
ConvNeXt-B
bonseext.base..b.....
timm / huggingface
ConNNeXt-B-In21k
Bonvnext_base.th faaase.fh_n22kfffinlll
timm / huggingface
Con-NeXtVV2-L-In2211
timm / huggingface
Con NeXttV2-BBIn221k
convnextv2 base.fcmae ft in22k in 384
timm / huggingface
Con/NeXIIV2-T-n22k
timm / huggingface
ConvNeXtV2-TT
convnexuv2.iiny.fmmee...i..
timm / huggingface
ComvNeXtV2-B
convnextv2_base.femmee st
timm huggingface
ConvNeXtV2-L
convnexıv2 large.femae ft inlk
timm huggingface
Mixer-B16-In2Ik
mixer www.bb6...........o. wtwpgggin2nn.ffffi in
timm huggingface
EffNetV2-M
F_.fficientnnet2..mnnll
timm / huggingface
EffNetV2-S
tf ef_efficientteet2_s.ii
timm / huggingface
EANeiV2-L
If effficientnett2..... Linlk
timm / huggingface
31

---

Mahalanobis++:: Improving OOD Detection via Feature Normalization
Table 20. Cifar model checkpoints.
SwinV2-S-Ik 1k
ft from timm model
Deit3-S-21k
ft from timm model
ConvN-T-21k
ft from timm model
ViT-B32-21k
ViT-S16-21k
https///gahbbboooooooooogoreeeee
RN18
https//hwuggiinnfacccccooeeddaad
RN34
https//hhuggiingficccccoccccloog
RN34-SC
htts://hungiinggaacccooolllaala
RN50-SC
httpss//uuggingffccc.coocccallll
RNx129-32
self trained
G. Methods
We describe OOD detection methods evaluated in work. Let neural network nø(x) g(((z)) decompose into feature extractor Φ and linear layer (((:) W di + b. For input ((x) denotes the feature embedding, and g(((x)) produces logits o, which can be transformed to probability vector via the softmax function.
MSP (Hendrycks & Gimpel, 2017): Classifer confidence, i.c. nax-sofftaxx-rrbbbbil
max(p.)
Max-Logit (ML or MLS) (Hendrycks al., 2022): Max-Logit returns the largest entry of the logit-vector o, i.e.
s = max(o.)
Energy (E) (Liu et al., 2020): Energy or log-sum-exp of logits:
log Σ exp (0c)
KL-Matching (KLM) (Hendrycks et al., 2022): KL-Matching computes the average probability vector de for each of the C' classes. For a test input, the KL-distances of all d, vectors to probability vector p are computed, and the OOD-score the negative of the smallest of those distances:
= - min [L]p
In the original paper by (Hendrycks et al., 2022), the average for d, is computed over an additional validation set. Since none of the other methods leverages extra data and we are interested in fair comparison, we deploy KL-Matching like in (Wang et al., 2022; Yang et al., 2022), where the average is computed over the train set.
KNN (KNN) (Sun et al., 2022): Computes the k-nearest neighbour in the normalized feature-space: The feature vector of a test input is normalized to z ø/1lø112 and the pairwise distances rz(z) =|z zilla to the normalized features z (.., aN) of all samples of the training set computed. The distances rz(z) are then sorted according their magnitude and the Kth smallest distance, denoted , (z) is used as negative OOD-score:
- (z)
Like suggested in (Sun et al., 2022), we use K 1000.
ReAct (E+R) (Sun et al., 2021): The authors propose to perform a truncation of the feature vector, z = min(ó, r). where the min operation is to be understood element-wise and the truncation threshold. The truncated features can then be
32

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
converted to so-called rectified logits via g(0) w क b. While the rectified logits can now be used with variety of existing detection methods, we follow (Sun et al., 2021) and use the rectified Energy OOD-score:
log
exp (oc)
As suggested in (Wang et al., 2022), we the threshold r such that 1% of the activations from the train would be truncated.
Virtual Logit Matching (ViM) (Wang et al., 2022): The idea behind VIM is that meaningful features are thought to lie in low-dimensional manifold, called the principal space P. whereas features from OOD-samples should also lie in p-, the space orthogonal to P. P is the D-dimensional subspace spanned by the eigenvectors with the largest D eigenvalues of the matrix FT F, where F is the matrix of all train features offsetted by u = -(WT)+ b (+ denotes the Peooree inverse). A sample with feature vector o is then also offset to h = é u and can be decomposed into h = hn + hn and p- is referred to the Residual of Φ. ViM leverages the Residual and converts it to virtual logit Og a [h 1|2, where
N C 1=
max, of 1|2
is designed to match the scale of the virtual logit to the scale of the real train logits. The virtual logit is then appended to the original logits of the test sample, i.e. to o, and new probability vector is computed via the softmax function. The probability corresponding to the virtual logit then the final OOD-score:
exp (op)
8 =
c=1 exp (o.) + exp (oo)
Like suggested in (Wang et al., 2022), we use D = 1000 if the dimensionality of the feature space is d 2 2048, D = 512 2048 2 d 2 768, and D = d/2 rounded to integers otherwise.
Cosine (Cos) (Techapanurak et al., 2020; Galil et al., 2023): This method computes the maximum cosine-similarity between the features of test-sample and embedding vectors ú, (sometimes also called concept-vector):
TT U
max
(10)
Ash (Ash) (Djurisic et al., 2023): Ash applies activation shaping at inference time by pruning acitvations below a certain threshold, and then binarizing (Ash-b) or scaling (Ash-s) the remaining activations, which are then processed as usually in the network. As suggested by the authors, we apply ash to the pre-logit feature activations.
Settmax-scaled-Cosine (SSC) (Tack et al., 2020): Normalize the rows of the weight matrix w, and the features, and compute the cosine between the two:
cos 0g W:
w |
Then scale by a scalar and apply the softmax, to finally use the max-softmax as OOD score:
max (softmax(! f)))
In Tack et al. (2020) the scalr s is learned, for our post-hoc setup we set s = 1.
NeCo (Nec) (Ammar et al., 2024): Compute the covariance matrix of the feature space, and project to the d eigenvectors with largest eigenvalues with the corresponding projection matrix P. The difference in norm of the projected features and the original features is then scaled with the max-logit and serves as OOD score.
[P(())l |
((x) ((x)
maxo;
max
(xx) d(x)
Like suggested by the authors, we standardize data and select d such that 90% of the train variance are explained.
33

---

Mahalanobist+:: Improving OOD Detection via Feature Normalization
Gaussian Mixture Model (GMM GMN) (Mukhoti et al., 2023): Estimate Gaussian mixture model on the train features o(æ)), and use the prgp-rbbabilities OOD score. We use GMN for Gaussian mixture model with normalized features, and GMM for Gaussian mixture model with regular features.
NNguide (NNG) (Park et al., 2023a): "Guide" the energy score by nearest-neighhor score:
Energy * 8KNN
where SKNN is a KNN score in the normalized feature space, estimated subset of the train features. Like suggested by the authors, we use 1% of the train features and K = 10 neighbors for ImageNet experiments. We also tried K 1000, as increasing K showed promising results in an ablation by the authors (Figure the paper), but found that it performs worse on average than K = 10.
Relative Mahalanobis distance (rMaha) (Ren et al., 2021): A modification of the Mahalanobis distance method, thought improve near-OOD) detection, is to additionally fit global Gaussian distribution to the train set without taking class- information into account:
Σ di.
E global
1 N
E
(iglobal Plglobal) I
Aglaball
N
The OOD-score is then defined as the difference hetween the original Mahalanobis distance and the Mahalanobis distance w.r.t. the global Gaussian distribution:
(1o- 4e) (o- -co)" - (Ø - fAglobal) Eglobar *Aglobal)
= min
34


Reviews:
[{"review_id": "ASpfPW3iJZ", "reviewer": "Reviewer_jfEM", "summary": "This paper presents an holistic empirical analysis illustrating the current violation of the gaussian distribution of the representations of most vision backbones. From this constatation, the paper introduces a variation of the Mahalanobis distance for OOD detection called Mahalanobis++. Extensive experiments on multiple recent OOD benchmarks and various bacbones are proposed to assess the good behavior of the proposed approach.\n\n## Update after rebuttal\n\nI am satisfied with the rebuttal and will thus keep my positive rating", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable comments and address the remarks below:\n- __\"the features should be concentrated around $\\sqrt{\\mathrm{tr}(\\Sigma)-\\|{\\mu}\\|^2_2}$\" (in Lemma 3.1)__\n\n    We thank the reviewer for checking our proof, but we strongly believe that the term $\\sqrt{\\mathrm{tr}(\\Sigma)+\\|{\\mu}\\|^2_2}$ is correct. The term stated by the reviewer can even become complex if the norm of the mean is large enough. As the variance goes to zero, the norm of the random variable should be concentrated around $\\|\\mu\\|_2$, which is exactly what our term states. We are happy to answer any questions on a particular step of the proof to resolve any potential confusion.\n- __\"the higher the dimension, the looser is the upper-bound\" (in Lemma 3.1)__\n\n    This is expected, as the squared $\\ell_2$-norm grows with dimension $d$. However, the deviation per dimension decreases: $$\\Pr\\left(\\frac{1}{d}\\left|\\|\\Phi(X)\\|^2_2 - \\left(\\mathrm{tr}(\\Sigma) + \\|\\mu\\|^2_2\\right)\\right| \\geq \\epsilon\\right) \\leq \\frac{\\mathrm{Var}\\left(\\|\\Phi(X)\\|_2^2\\right)}{d^2 \\epsilon^2}$$ The right side decreases with $d$ as the variance grows linearly in $d$. Lemma 3.1 shows that under the Gaussian assumption, we should see some concentration of the feature norms. To illustrate this, we simulate it in Figure 3 (left) by sampling from class-specific Gaussian distributions with the estimated means and shared covariance matrix, noting that the actual feature norms (Fig. 3 right) deviate strongly from the Gaussian model (e.g. via heavy tails). This suggests severe violations of the Gaussian assumption, which we substantiate by QQ plots and the variance alignment analysis.\n- __\"the sentence _'Adapting them to IN-scale setups ... has so far not been successful'_ ... is a bit of an overstatement ... as Mahalanobis ... has been successful on IN-1k\"__\n\n    We agree, this statement only refers to Gaussian mixture models (GMMs), and not to the Mahalanobis distance. We will clarify this in the paper and explain the difference between GMMs and the Mahalanobis distance.\n- __\"the sentence _'l2-normalization...is ... not used with the Mahalanobis score'_ ... is a bit of an overstatement ... as SSD ... has been successful on IN-1k\"__\n\n    We agree that Mahalanobis has been applied to normalized features in other works like SSD[1] and CIDER[2], and we should have chosen our statement more carefully. However, these are _train-time_ methods where normalization is implicitly part of their contrastive loss. Those methods thus normalize their features for OOD detection _because_ they also normalize during training. This is orthogonal to our work: The standard Mahalanobis method for OOD detection is a _post-hoc_ method, where adjusting the pretraining scheme is not feasible. We show that in this setting, the Gaussian assumption underlying this method is often severely violated, and that normalizing the features better aligns with this assumption, consistently improving OOD detection across architectures and pretraining techniques. We will clarify this distinction and expand the discussion of [1-4] in the paper (see below for SSD).\n- __\"a broader discussion and comparison with SSD\"__ and __\"which challenges are unaddressed by normalized approaches\"__\n\n  SSD involves three steps:  \n    1) Training with a supervised (SSD+) or unsupervised (SSD) contrastive loss (implicitly normalizing features),  \n    2) Cluster estimation via k-means in the normalized feature space,  \n    3) Mahalanobis-based OOD detection using cluster labels instead of class labels.\n   \n    This setting differs fundamentally from ours, as SSD, like [1,3,4], is a _train-time_ method. Methods like [1-4] cannot be directly applied to the pretrained checkpoints we evaluate. To demonstrate the advantages of post-hoc approaches, we evaluate SSD+ on NINCO using the ResNet50 from [5] (trained for 700 epochs). SSD+ is clearly outperformed by our top models, with FPR >3× higher. Those are obtained from various pretraining schemes, and retraining models with SSD or [1,3,4] on this scale is typically not feasible. \n| model | FPR|\n|-------------|-------|\n|SSD+ w. 100 clusters|66.0% |\n|SSD+ w. 500 clusters|65.7% |\n|SSD+ w. 1000 clusters|67.8% |\n|CnvNxtV2-L + Maha++ | 18.4%|\n|EVA02-L14 + Maha++| 18.6%|\n- __comparison with DINO__\n\n    We report the FPR of a DinoV2-S model (ft on IN1k) on NINCO. Maha++ outperforms Maha clearly.\n\n| Maha|Maha++|\n|------|------|\n| 77.3%| 53.4%|\n\n[1]Ming et al. How to exploit hyperspherical embeddings for out-of-distribution detection? ICLR2023\n\n[2]Sehwag et al. Ssd: A and unified framework for self-supervised outlier detection, ICLR 2021\n\n[3]Regmi et al. T2fnorm: Train-time feature normalization for ood detection in image classificatio, CVPR 2024 workshop\n\n[4]Haas et al T. Linking neural collapse and l2 normalization with improved out-of-distribution detection in deep neural network, TMLR 2023\n\n[5]Sun et al. Out-of-distribution detection with deep nearest neighbors. ICML 2022", "claims_and_evidence": "The principal claim concerns the violation of the class-wise unimodal Gaussian hypothesis of the representations. This is a reasonable claim as the Mahalanobis method does rely on strong relaxations for computational reasons. Moreover, this claim is supported by strong empirical evidence in this paper, see Fig. 3, 4, 5, and Table 1.", "methods_and_evaluation_criteria": "Evaluation criteria and benchmarks are standard for OOD detections.  \n\nReporting only FPR95 is not a standard practice as this metric is not robust to small changes in the decision function and is particularly sensitive to class imbalance. FPR@95 highlights performance at a specific critical threshold but is typically complemented by AUC, ensuring a more holistic evaluation.  I see that AUC scores in the supplementary are still in favor of the proposed approach.", "theoretical_claims": "Lemma 3.1. does not support the indicated conclusion. First, features should be concentrated around $\\sqrt{\\text{tr}(\\Sigma) - ||\\mu||^2_2}$. Moreover, the higher the dimension, the looser is the upper-bound.", "experimental_designs_or_analyses": "The evaluation protocol is well designed. However, as many backbones pretrained with a contrastive loss also normalize the representations, comparison of Mahalanobis with SSD [1] or on other DINO-like backbones would give important insight on the method and the importance of normalization for OOD detection.\n\n\n[1] Sehwag, Vikash, Mung Chiang, and Prateek Mittal. “SSD: A Unified Framework for Self-Supervised Outlier Detection,” ICLR 2021", "relation_to_broader_scientific_literature": "This paper shares the violation of the Gaussian distribution with multiple other distance-based papers. Other approaches proposed a pre-training strategy to mitigate this limitation. \n\nThe proposed extension to Mahalanobis is particularly incremental. However, it is well illustrated both by empirical statistical evaluation of the feature dispersion and extensive evaluations.", "essential_references_not_discussed": "* Good performances of Mahalanobis distance for OOD detection on normalized features have already been explored in SSD [1]. In the related work section, authors state that \"Adapting them to ImageNet-scale setups as post-hoc OOD detectors has so far not been successful\". Same in the end of the method section: \"While $\\ell_2$-normalization has been used with non-parametric methods like KNN (Sun et al., 2022; Park et al., 2023a) or cosine similarity (Techapanurak et al., 2020), it is - to the best of our knowledge - not used with the Mahalanobis score\". This is a bit of an overstatement as Mahalnobis is a strong and cheap baseline even on large scale datasets and SSD has been successfully experimented on ImageNet-1k.Thus, a broader discussion and comparison with SSD is missing in the current paper. \n\n[1] Sehwag, Vikash, Mung Chiang, and Prateek Mittal. “SSD: A Unified Framework for Self-Supervised Outlier Detection,” ICLR 2021", "other_strengths_and_weaknesses": "The paper is very well written and supported with extensive evaluations.", "other_comments_or_suggestions": "NA", "questions_for_authors": "Despite bringing appreciated insights into the distance-based OOD literature, the related work section misses clear positioning *e.g.* which challenges are unaddressed by normalized approaches such as [2,3] or CIDER [4]? What makes the proposed method better suited for OOD detection?   \n\n\n[2] Regmi, S., Panthi, B., Dotel, S., Gyawali, P. K., Stoyanov, D., and Bhattarai, B. T2fnorm: Train-time feature normalization for ood detection in image classification, CVPR Workshop 2024\n[3] Haas, J., Yolland, W., and Rabus, B. T. Exploring simple, high quality out-of-distribution detection with l2 normalization, TMLR, 2024.\n[4] Ming, Y., Sun, Y., Dia, O., and Li, Y. How to exploit hyperspherical embeddings for out-of-distribution detection? Neurips 2023", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FOKNyrkjIt", "reviewer": "Reviewer_vddp", "summary": "The paper revisits the Mahalanobis distance for out-of-distribution detection. It first examines how the assumptions underlying the Mahalanobis distance for OOD detection are violated by a variety of models. It then proposes a maximally simple but effective remedy by applying l2-normalization to the pre-logit features. The evaluation shows that this outperforms previous works by a significant margin.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for carefully reading and evaluating our paper, and we are glad that the reviewer finds that our claims are __“supported by clear and convincing evidence”__, that our method is __“well motivated”__, that that they appreciate the __“wide variety of model types, architectures and sizes”__ in our __“thorough evaluation\"__. We agree with the reviewer that the results about augreg ViTs stand out, and think that investigating the underlying reasons for the behaviour of those models (i.e. the why the augreg training scheme results in the favourable structure of the feature space) is an interesting direction for future research. We thank the reviewer for pointing out the incorrect ImageNet reference, which we will fix. For the rebuttal, we have included a more thorough discussion and comparison to SSD (see response to reviewer jfEM), an evaluation of a DinoV2 model (also in response to reviewer jfEM) and more variance deviation measures (see response to reviewer qhSY). If there is anything else the reviewer would like to see addressed, we would be happy to discuss this.", "claims_and_evidence": "The claims made by the paper are supported by clear and convincing evidence. The paper demonstrates that, empirically, feature distributions of some models do not fit the assumptions made by prior Mahalanobis distance-based OOD detection. Figure 5 further shows that, for SwinV2-B models, the feature norm is strongly correlated with the Mahalanobis distance, while beeing a bad OOD predictor, which in turn leads to suboptimal OOD detection performance. In contrast, applying l2-normalization as proposed reduces correlation between feature norms and Mahalanobis distance, which allows drawing a better decision boundary. The findings are furthermore validated by the quantitative evaluation of the proposed method on a wide variety of pre-trained models.", "methods_and_evaluation_criteria": "The method is well motivated by pointing out how the assumptions in prior Mahalanobis based OOD detection methods can be violated by some models. The evaluation metrics (false-positive rate at true positive rate of 95% in particular) and benchmark datasets make sense and are in line with prior work on OOD detection. I appreciate that the evaluation is performed on a wide variety of model types, architectures and sizes.", "theoretical_claims": "The main theoretical claim can be found in equation 5 and is elaborated upon in the appendix, which I did only check superficially.", "experimental_designs_or_analyses": "The main experimental design is focused on evaluating the OOD false positive rate at a fixed true positive rate of 95% across different models and datasets, which is in line with prior work. The experimental analysis demonstrates that models suffer from violations of Mahalanobis based OOD detection with varying degree, but that most models benefit somewhat from l2-normalization as proposed.", "relation_to_broader_scientific_literature": "While the purely methodological innovation of this paper is minimal, its value lies in identifying and empirically demonstrating violations of key assumptions of Mahalanobis based OOD detection in practice, proposing a maximally simple remedy, and providing thorough evaluation of this remedy on a wide variety of models.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "1. ImageNet reference renders as \"(University, 2015)\"", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bfaKZCzf8v", "reviewer": "Reviewer_ruxa", "summary": "This submission focuses on the OOD detection task and it proposes a simple yet effective method to improving the Mahalanobis distance approach.\n\n## update after rebuttal\n\nThe authors rebuttal has largely addressed my concerns and I thus maintain my positive rate.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback, and for appreciating our work. We address the remarks below:\n1.  __\"organize related work section\"__ and __\"elaborate difference to existing similar methods\"__\n\n    We will extend the discussion about related work, and emphasize the differences to previous work that used feature normalization [3,4] or the Mahalanobis distance, or both [1,2]. Most importantly, other works have investigated _train-time_ methods that involve normalization. Either implicitly through contrastive losses (CIDER [1], SSD[2]), or explicitly to improve OOD detection [3,4] It is then natural to also apply normalization at inference time. For instance, CIDER applies KNN, and SSD performs k-means and then Mahalanobis. Those methods thus normalize their features for OOD detection _because_ they also normalize during training. This is orthogonal to our work: The standard Mahalanobis method for OOD detection is a _post-hoc_ method, where adjusting the pretraining scheme is not feasible. We show that in this setting, the Gaussian assumption underlying this method is often severely violated, and that normalizing the features better aligns with this assumption, consistently improving OOD detection across architectures and pretraining techniques. We will clarify this distinction and expand the discussion of other approaches in the paper (see the answer to reviwer jfEM for a more thorough discussion and quant. comparisons to SSD). If there is a specific reference the reviewer would like us to discuss, please let us know.\n\n2. __Lemma 3.1, not sufficient condition__\n\n    We will clarify that a concentrated feature norm is not a sufficient, but a necessary condition for a Gaussian distribution. Lemma 3.1 only shows that - under the assumption of a Gaussian distribution in feature space - we expect some concentration of the feature norm. To illustrate this, we sample from class-specific Gaussian distributions with the estimated means and shared covariance matrix (Figure 3-left), noting that in practice (Figure 3-right) the feature norms deviate strongly from the Gaussian model (e.g. via heavy tails). This suggests severe violations of the Gaussian assumption, which we substantiate by QQ plots and the variance alignment analysis. Our remedy - normalization - aligns the features better with the premise of normally distributed data with shared covariance matrix.\n\n3. __\"elaborate on neligibility\" (in QQ plot analysis)__\n\n    In QQ-plots, we compare empirical quantiles against a theoretical standard normal distribution. Since normalized and unnormalized features have different variances, their QQ-plots would have different slopes, making direct comparison difficult. To align comparisons, we divide both samples by their empirical standard deviation—this ensures both are evaluated against the same reference slope (black line in Figure 4). Dividing by the empirical variance technically transforms a normal distribution to a Student's *t*-distribution with *n*-1 degrees of freedom.  As the reviewer pointed out correctly, this matters for small $n$. However, the *t*-distribution converges to a Gaussian as $n\\to\\infty$, and for $n>30$, the difference is typically negligible [5]. We use all ImageNet train features ($n>10^{6}$) in our QQ plots, making the *t*-distribution practically Gaussian, allowing for the analysis we performed in the paper. We would like to stress that all of this is only a technicality in the analysis of the features via QQ plots, and irrelevant for Maha++ as an OOD detection method.\n\n5. __concentration of feature norm is \"key motivation\"__\n\n    Our key motivation is not to concentrate the feature norm. Instead, feature norm concentration is a necessary condition IF the features were indeed normally distributed. As we find, the feature norms are, however, not concentrated, but for instance show extremely heavy tails. We take this as an indication that the Gaussian assumption is violated, and further validate it via QQ plots and our variance analysis. Regarding the reviewers question about __concentrating even further__: We are not sure we understand what the reviewer means by this. One could, in principle, normalize by a different norm (e.g. $\\ell_1$ or $\\ell_\\infty$), but this would change the direction of the features. We therefore opted for $\\ell_2$ normalization. Does this answer the question?\n\nWe are happy to clarify any of the points further!\n\n[1] Ming et al. How to exploit hyperspherical embeddings for out-of-distribution detection? ICLR2023\n\n[2] Sehwag et al. Ssd: A unified framework for self-supervised outlier detection, ICLR 2021\n\n[3] Regmi et al. T2fnorm: Train-time feature normalization for ood detection in image classificatio, CVPR 2024 workshop\n\n[4] Haas et al. Linking neural collapse and l2 normalization with improved out-of-distribution detection in deep neural network, TMLR 2023\n\n[5] https://www.jmp.com/en/statistics-knowledge-portal/t-test/t-distribution", "claims_and_evidence": "While in a mixture of theoretical and emphrical way, the reviewer believees that the submission is supported by clear evidences.", "methods_and_evaluation_criteria": "Yes, the reviewer believes that the proposed method makes sense.", "theoretical_claims": "The reviewer hasn't carefully checked the proof of Lemma 3.1. Yet, Lemma 3.1 is at least not contradicting with the reviewer's intuition.", "experimental_designs_or_analyses": "Yes, the reviewer has gone through the experimental settings in the main paper and finds that they largely make sense.", "relation_to_broader_scientific_literature": "The reviewer believes that Gaussian distribution assumption can be very common in different scientific literatures. From this perspective, it is very interesting for a method to appear if it does can help mitigating the violation of this assumption in a relatively simple way.", "essential_references_not_discussed": "N.A.", "other_strengths_and_weaknesses": "I really appreciate the authors effort in perfoming extensive experimental analysis and I believe that they can to a large scale strength this submission. Below, I still have several queries over this submission that I hope can be addressed to further improve the quality.\n\n1. I hope the related works section can be better organlized and the differences between the proposed method and existing methods to be better elaborated. For example, when the method names a subsection called Mahalanobis distance, it actually wants to review Mahalanobis-distance-based existing methods from my understanding. Thus, it is important for this to be clarified. Meanwhile, it is appreciated if the difference between the proposed method and existing similar methods to be better elaborated.\n\n2. When the authors present Lemma 3.1, if I am not wrong, it is only the property of Gaussian distributed features but not sufficient condition. If this is the case, I appreciate the authors to make this more clear there to avoid reader's misunderstanding.\n\n3. The authors claim that \"we expect this to be negligible due to the large dataset size\". I first appreciate more explanation or elaborate on this negligibility. Meanwhile, the authors seem to require the size to be very large (>10^6). What if in some cases that this is not the case? If the negligibility still holds?\n\n4. Finally, if I am not wrong, the key motivation seems to be concentrating the feature norm. I am thus a bit curious here, what if we not only normalize like Eq. 6 but concentrate the feature even further? What will happen? Meanwhile, while I admit its naturaness, is there any specific reason for the authors to choose to perform concentration via normalization?\n\nI still have these queries yet I remain positive on this submission. I thus vote for weak accept now.", "other_comments_or_suggestions": "N.A.", "questions_for_authors": "(see above in other weaknesses.)", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xKTlKSJvKV", "reviewer": "Reviewer_qhSY", "summary": "The paper proposes a simple fix to the Post-Hoc OOD detection technique based on the Mahalanobis distance computed on the feature space of the neural network of interest. This simple fix consists of normalizing the features by their $l_2$ norm before computing the distance. The authors emphasize how the samples violate the assumptions underlying Mahalanobis distance in the feature space:\n- Assumption 1: the class wise features follow a multivariate normal distribution\n- Assumption 2: class conditional covariance matrices are the same\nThey do so by analyzing the magnitude of the feature norm, emphasizing how the fix can alleviate this problem. Experiments on a comprehensive benchmark of various models empirically demonstrate the effectiveness of this method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful comments and appreciate the positive feedback. Below we address the reviewers remarks:\n- __“The fix intends to alleviate the difference between the feature norms of samples from different classes”__\n\n    We would like to clarify that different feature norms for samples from different classes are not a problem per se. For instance, if the mean vectors of different classes were of different magnitude (which is typically not the case), the Gaussian assumption could still be satisfied. However, our analysis shows that the observed feature norm distribution and the one that we would expect under the Gaussian model are significantly different, for instance showing very heavy tails. We take this as indication that the Gaussian assumption is violated, and substantiate this with further analysis (QQ plots, etc).\n- __“nothing ensures that after the fix, which is just a normalization, the features follow a normal gaussian, and that their covariance matrices are equal.”__ and __“QQ plots are here to show that the obtained features are closer to normal gaussian, but they might still be non gaussian.”__\n\n    We agree that we cannot guarantee that the features follow a normal distribution. In fact, we believe that there is no reason to believe that the features follow any particular distribution. However, we provide strong empirical evidence that _modelling_ the feature distribution with a normal distribution with shared covariance is _more appropriate after normalization_. In particular, 1) the QQ plots are less skewed, 2) the shared covariance assumption is better satisfied, and 3) the feature norm does not act as a confounder for OOD detection anymore.\n- __\"What is called a 'fix' might not be an actual 'fix'\"__\n\n    In addition to the above, we are happy to rephrase from \"fix\" to e.g. \"remedy\"\n- __“Could you plot the distribution of normalized features in a plot similar to Figure 3?”__\n\n    The feature norms of the normalized features would show as a straight line at 1 with no deviation. Please let us know in case this does not clarify the question.\n- We thank the reviewer for the remarks about the trace notation and for noting that $\\Phi(X)$ has not been properly introduced. We will adjust the notation and clarify that $\\Phi(X)$ is a random variable representing the feature distribution for input $X$.\n- __”two very different covariance matrices could have a deviation of zero with this metric. “ (Eq. 5)__\n\n    We respectfully disagree with the reviewer. In particular, we can write $$\\mathbb{E}_u \\left[ \\left( \\frac{u^T \\hat{\\Sigma}_i u}{u^T \\hat{\\Sigma} u} - 1 \\right)^2 \\right]=\\mathbb{E}_u \\left[ \\left( \\frac{u^T (\\hat{\\Sigma}_i-\\hat{\\Sigma}) u}{u^T \\hat{\\Sigma} u}\\right)^2 \\right]$$ Since $\\Sigma$ is pd, $u^T {\\Sigma} u>0$, and the only way for the expectation to be zero is that $u^T (\\hat{\\Sigma}_i-\\hat{\\Sigma}) u=0$ for all $u$, wich is only the case when $\\Sigma=\\Sigma_i$.\n- __\"expected squared variance deviation ... is not a standard metric\"__\n\n    We agree that this metric is not commonly evaluated, but we argue that it is the right one to look at. In particular, the Mahalanobis distance performs a whitening by the variances: Deviations in a certain direction are measured _relative_ to the sample variance in this direction. Small absolute deviations can thus result in large distances when they are along a direction of small variance. We therefore need a measure that can capture _relative_ deviations instead of absolute deviations, since absolute deviations would be dominated by directions of large variance. Our proposed measure computes the _relative deviation_ of the variance of $\\Sigma_i$ from $\\Sigma$ in _every direction u_ and averages this deviation over all directions. This is a natural way to assess whether $\\Sigma_i$ and $\\Sigma$ are similar in all possible directions in the feature space. A similar measure that is commonly used to compare covariance matrices is the Riemannian metric (see e.g. [1,2]) $d(\\Sigma_1, \\Sigma_2) := \\sqrt{\\mathrm{tr}\\left(\\ln^2\\left({\\Sigma_1^{-0.5}}\\Sigma_2{\\Sigma_1^{-0.5}}\\right)\\right)}$. It is also possible to compute an appropriate measure with divergences like the KL divergence: $KL_{\\text{normal}}(I_n,\\Sigma_1^{-0.5}\\Sigma_2\\Sigma_1^{-0.5})$. We evaluate both, confirming that normalization aligns the covariance structure in a meaningful way (lower is better):\n\n||Riemann|Riemann|KL| KL|\n|:-------------|------------------------:|:----------------------|----------------------------:|:--------------------------|\n||unnormalized |normalized|unnormalized|normalized|\n| mean|98.2 | **88.2**| 1090.6|**982.0**|\n| median|93.6 | **84.7** | 1011.0  |**908.3** |\n\nWe are happy to discuss any of the points further!\n\n\n[1] Förstner & Moonen. (2000). A Metric for Covariance Matrices. 10.1007/978-3-662-05296-9_31.\n\n[2] Pennec, Fillard, & Ayache. A Riemannian Framework for Tensor Computing. Int J Comput Vision 66, 41–66 (2006).", "claims_and_evidence": "The problem with the feature norm is clearly illustrated with experiments based on Lemma 3.1, expected squared variance deviation, and QQ plots.\n\nHowever, I have some concerns with the link between the fix and the assumptions. The fix intends to alleviate the difference between the feature norms of samples from different classes, but I do not see how it makes the features satisfy assumptions 1 and 2. Specifically, nothing ensures that after the fix, which is just a normalization, the features follow a normal gaussian, and that their covariance matrices are equal. \n- QQ plots are here to show that the obtained features are closer to normal gaussian, but they might still be non gaussian.\n- I do not see the relation between assumption 2 and the expected squared variance deviation, which I think is not a standard metric. Two very different covariance matrices could have a deviation of zero with this metric. Why not conducting statistical tests, or using standard probability distribution divergence measures that can be estimated?", "methods_and_evaluation_criteria": "- The experiments to emphasize the problem with feature norm are thorough and theoretically grounded\n- The evaluation benchmark is extensive.", "theoretical_claims": "I checked the proof of Lemma 3.1 which is Ok.\nI skimmed through Appendix C (proof of expected squared variance deviation) but did not check thoroughly.", "experimental_designs_or_analyses": "yes", "relation_to_broader_scientific_literature": "The contributions are closely related to (Lee et al. 2018b) and (Ren et al, 2021) which are appropriately discussed.", "essential_references_not_discussed": "All essential references are discussed to the best of my knowledge", "other_strengths_and_weaknesses": "### Strenght\n\n- The problem with Assumptions 1 and 2 is clearly emphasized \n- The method is simple\n- It consistently improves the performance of Mahalanobis method\n\n### Weaknesses\n\n- What is called a \"fix\" might not be an actual \"fix\" but just a tool to make Mahalanobis method better\n- Concerns with expected squared variance deviation", "other_comments_or_suggestions": "- In the proof of Lemma 3.1, $X$ is not introduced (the lemma is about $\\Phi(X)$)\n- Eq. 5 \"trace\" please use the same notation as lemma 3.1 (\"tr\")", "questions_for_authors": "- Could you plot the distribution of normalized features in a plot similar to Figure 3?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: After review, the paper received four positive evaluations. Following the authors' rebuttal, one reviewer increased the rating, while the other three maintained their original scores. All reviewers acknowledged the paper's contributions and expressed satisfaction with its technical merits.

The AC concurs with the reviewers' assessments and recommends acceptance.
Decision: Accept (poster)