Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Authors: Kaiyu Guo
Zijian Wang
Tan Pan
Brian C. Lovell
Mahsa Baktashmotlagh
Venue: ICML 2025 poster (2025)
Keywords: Out-of-distribution detection
Mahalanobis distance
Covariance adjustment
Primary area: social_aspects->safety

Abstract: Out-of-Distribution (OOD) detection is essential for the trustworthiness of AI systems. Methods using prior information (i.e., subspace-based methods) have shown effective performance by extracting information geometry to detect OOD data with a more appropriate distance metric. However, these methods fail to address the geometry distorted by ill-distributed samples, due to the limitation of statically extracting information geometry from the training distribution. In this paper, we argue that the influence of ill-distributed samples can be corrected by dynamically adjusting the prior geometry in response to new data. Based on this insight, we propose a novel approach that dynamically updates the prior covariance matrix using real-time input features, refining its information. Specifically, we reduce the covariance along the direction of real-time input features and constrain adjustments to the residual space, thus preserving essential data characteristics and avoiding effects on unintended directions in the principal space. We evaluate our method on two pre-trained models for the CIFAR dataset and five pre-trained models for ImageNet-1k, including the self-supervised DINO model. Extensive experiments demonstrate that our approach significantly enhances OOD detection across various models. The code is released at https://github.com/workerbcd/ooddcc.

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Kaiyu Guo
Zijian Wang'
Tan Pan
Brian C. Lovell Mahsa Baktashmotlagh
Abstract
Sumpple Carnt
Sumple Cazund
CIFAR-100
CIFAR-100
lilto---Ditribttion (OOD) detection is essential for the trustworthiness of Al systems. Methods us- ing prior information (i.e., subspace-base meth- ods) have shown effective performance by extract- ing information geometry to detect OOD data with more appropriate distance metrie. How- ever, these methods fail to address the geometry distorted by ill-distributed samples, due to the limitation of statically extracting information ge- ometry from the training distribution. In this pa- per, we argue that the influence of ill-distributed samples can be corrected by dynamically adjust- ing the prior geometry in response to new data. Based on this insight, we propose novel ap- proach that dynamically updates the prior covari- ance matrix using real-time input features, refin- ing its information. Specifically, we reduce the covariance along the direction of real-time input features and constrain adjustments to the resid- ual space, thus preserving essential data char- acteristics and avoiding effects on unintended directions in the principal space. We evaluate our method on two pre-trained models for the CIFAR dataset and five pre-trained models for ImageNet-Ik, Ik, including the self-supervised DINO model. Extensive experiments demonstrate that our approach significantly enhances OOD detec- tion across various models. The code is released at https://jgthbb.cooo//oottter////
4000
SVHN
4000
SVHN
Overiap
Overlap
2000
2000
Overlap Area: Area:1989
2000
Overlsg Area: 2329
1000
1000
Score
Score
(a) with dynamic adjustment (b) without dynamic adjustment
Figure I. Visualization of Mahalanobis distance score distributions with (w) and without (w/o) Dynamic Adjustment. We compare scores from the CIFAR-I00 dataset (ID) and the SVHN dataset (OOD) using a CIFAR- 100 pre-trained DenseNet. Without dy- namic adjustment, the scores have high variance and a large over- lapping region. With dynamic adjustment, the variance is lower and the overlap is reduced.
ever, deep models are often well-fitted to the data they trained on, known as the in-distribution (ID) data, and may perform poorly on unseen or novel data, referred to as out- of-distribution (OOD) data. Recognizing OOD samples is crucial for ensuring the safety and reliability of Al sys- tems, especially in critical applications like autonomous driving (Filos et al., 2020), medical diagnostics (Fu et al., 2023), and surveillance (Idrees et al., 20. Therefore, OOD detection is proposed to separate ID and OOD data to define the performance boundaries of deep learning models.
The goal of OOD detection is to define a score function s(.) that assigns higher scores to ID data and lower scores to OOD data. Using a threshold X, we can classify inputs as:
1. Introduction
D(z, 8, = { I I IEID ID
sf > if sf < A
I IE OOD
Deep learning has achieved remarkable success in pattern recognition tasks (He al., 2016; Han et al., 2022). How-
A variety of score functions have been proposed for OOD detection (Yang et al., 2024). this paper, we focus on de- signing score functions for test-time OOD detection, where models do not require retraining with ID data.
'School Electrical Engincoring and Computer Science, Uni- versity of Queensland, Brisbane, Australia 'Shanghai Academy of AI for Science (SAIS). Shanghai, China ' Artificial Intelli- gence Innovation and Incubation Institute, Fudan University, Shanghai, China. Correspondence to: Mahsa Baktashmotlagh <m. www.abaaoooolgggeeeeee www.hghuuuuuuuuu........
With the widespread use of pre-trained models and con- trastive learning, dlassifier-free methods for test-time OOD detection have been proposed, among which measuring the distance between features is an effective approach. Previous works (Sehwag et al., 2021; Lee al., 2018) have shown
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
the effectiveness of the Mahalanobis distance in OOD de- tection. Mahalanobis distance computes distances using covariance matrix derived from the ID data, capturing its information geometry. However, the distortion of informa- tion geometry caused by the outlier features in the training data is neglected in these methods. Specifically, if the ID data has high variance deviating from semantic directions due to outliers, the Mahalanobis distance may become less sensitive to OOD samples that align with these directions. This can lead to misclassifying OOD data as ID.
features dynamically adjust the prior geometry. Our proposed OOD detection score function reduces re- dundant information in the covariance matrix in real time, making it better tailored to OOD detection. By restricting adjustments to the residual space, we en- hance sensitivity to OOD samples while preserving the essential structure of the ID data.
Our regularization technique method can be applied to distance measures that rely on covariance structures. This broadens the applicability of our approach to var- ious problems where adjusting the sensitivity of dis- tance measures can improve performance.
Recently, methods that project features onto the subspace of ID data (Wang et al., 2022; Ammar et al., 2024; Chen et al., 2023) have been proposed, which can be viewed as matrix- induced distance scores. These approaches can be seen as attempts to mitigate this issue by replacing the covariance matrix in the Mahalanobis distance with tailored matrices. By eliminating components associated with extreme vari- ances, these methods aim to reduce the influence of outliers. However, this may also remove important information in the ID distribution and does not provide a targeted adjustment for specific OOD directions. This raises the question: How can we reduce the effect of outlier features on the informa- tion geometry without losing important characteristics of the ID data?
We evaluate our method on CIFAR benchmarks with 2 pre-trained models and on Imagenet with 5 pre-trained models including the self-supervvied pre-trained model DINO, showing it outperforms state-of-the-art (SOTA) in most cases.
2. Related Work
2.1. Out-of-distribution Detection
With the observation that the deep learning models are over- confident in classifying samples over different semantic distributions, out-of outto--distribution detection emerges to reject the samples with different semantic information from the training data ( (Yang et al., 2024). Unlike outlier detection methods (Sehwag et al., 2021), OOD detection aims to de- termine whether real-time data belongs to the in-distribution or -uttof-distribution. Thus, the group information can not be obtained from the unseen distribution.
To address this challenge, we propose a novel approach that dynamically adjusts the prior matrix using real-time input features. Our key insight is that real-time input features capture local information about the unknown distribution, which can be used to refine the prior information geometry. Specifically, we propose to adjust the covariance matrix by decreasing the covariance in the direction of the real-time features. This effectively contracts the covariance structure along targeted directions, making the distance measure more sensitive to deviations in those directions and improving OOD detection performance.
Many methods develop new training strategies VDevs & Taylor, 2018; Wang et al., 2021; Vyas et al., 2018; Chen et al., 2021) or utilize the data augmentation (Hein et al., 2019; Hendrycks et al., 2022) to manipulate the confidence. However, this kind of method requires training phase, which may take a long time to achieve score function with different deep learning models. Moreover, the computa- tional resource is also a limitation of these methods. So, in this paper, we focus on the test-time OOD detection task which consumes less time to get the score function.
However, not all information in the real-time features is suit- able for adjustment. To avoid disrupting the main structure of the ID data, we restrict changes to the residual space. By adjusting the covariance in the residual space, we preserve the main patterns of the ID data while enhancing OOD de- tection by increasing sensitivity to samples that deviate in these less significant directions. This focus also reduces noise and redundancy, improving the distinction between ID and OOD data. Our OOD detection score function is computed on the transformed features, which allows the ID distribution to form a denser manifold. Figure la shows the effectiveness of our method. With dynamic adjustment, the scores have lower variance and a smaller overlapping region between ID and OOD data, indicating improved separability. In summary, Our main contributions are as follows:
2.2. Test-time OOD detection
As aforementioned in the introduction, test-time OOD detec- tion aims to design the OOD score D(x, s, A) with fixed pre-trained model. According (Yang al., 2024; Ammar et al., 2024), the test-time OOD detection method can be mainly divided into five categories: logit-based, gradient- based, feature-based, distance-based, density-based, and subspace-base based. For the logit-based methods (Hendrycks & Gimpel, 2017; Liu et al., 2020; Basart et al., 2022), these works tend to detect the OOD data based on the high confi-
We introduce a new perspective for designing distance- based OOD score functions by utilizing real-time input
2

---

Improving Out-of-Distribation Detection via Dynamic Covariance Calibration
dence lying in the logit or probability from the ID data. For gradient-based methods (Huang et al., 2021; Behpour et al., 2023; ElAraby et al., 2023), these method aims to detect OOD data with the differences of gradients. The gradient- based method, GradNorm, has been proven to be related to the logits and feature norms (Huang et al., 2021). For the feature-based methods, feature manipulations like clip- ping (Sun et al., 2021) and scaling (Xu et al., 2024) are pro- posed to remove some redundant information. Similarly, the weight pruning methods (Sun & Li, 2022; Ahn et al., 2023) also tend to select important neurons with this motivation. In addition, weight pruning methods are the same as feature pruning, where different masks are given to the features corresponding to different logits. For density- ods, these methods (Morteza & Li, 2022; Peng et al., 2024) design the OOD score function by modeling the density of certain distribution. Distance- based methods (Basart et al., 2022; Lee et al., 2018; Liu & Qin, 2023) utilize the property that the OOD features reside in the outlier of the feature clusters. However, these methods require high-quality fea- tures and may result in large variances with low-quality features. For subspace methods (Wang et al., 2022; Ammar et al., 2024; Chen et al., 2023), these methods project the real-time features to subspace of ID data, and the scores are designed based on norms or distances. So, these meth- ods can be regarded as sittance-bssed score functions where the distances are matrix-induced In this paper, these score functions are highly related to the matrix-induced distance scores. Thus, the design of the subspace methods can be viewed as defining new distances with various prior matri- ces. Unlike previous studies that only estimate the prior matrix on the training samples, we dynamically estimate the matrix on the geometry of training distribution and local geometry of unknown distribution from real-time data.
matrix-induced distance:
- S fa);
-
(I)
where the matrix M defines the geometry and f is real- time input feature.
For the Mahalanobis distance OOD detection score (Lee et al., 2018; Sehwag et al., 2021), the matrix M is the pre- cision matrix Σ-1 for the distribution of each class. These methods consider each class of ID data as distribution, and the score is defined as the minimum distance from the data point to the mean point over the geometry of each class.
meth-
For residual space method (Wang ct al., 2022), M is defined as R R. where R is the basis of the residual space of the ID data. Since the score function is / v(f 11) R R(f - p), this method effectively measures the distance between the data point and the mean point over the manifold of distribu- tion N N(,, (R R)
Similarly, for the principal space method (Ammar et al., 2024), R is the basis in the principal space of the data. As the features are standardized in the first stage, the score considers the distance between the data and the zero point over the manifold (O, (PT P)-)), where P is the basis of the principle space of ID data. The dominator in the score function can be viewed as normalization for each feature.
As discussed above, previous works tend to consider the static geometry from ID data. Therefore, outlier features from the training samples can consistently affect the distance for every real-time input data. To mitigate this issue using the local information from real-time features, the distance in Equation can be transformed as follows:
dM(s.fa) -V - -a)TT((MEEEE - fa)
3. Preliminary and Insight
where f is the real-input feature, Ja is the anchor feature, and M Rd - Rddd maps the d dimension features to the d d-matrix space. Figure 2 shows the difference between the Euclidean distance and the matrix-induced distance as defined in equations 1 and 2. In the following section, we introduce our design for the matrix M in Equation 2.
Given X as data space and y as the label space, the ID and OOD distributions can be defined as joint distributions Px,,, Y... over Xaa x Via, and Yua.d over Xood x Vood. respectively. Given a pre-trained model h o g, where h is an encoder and is a classifier, we obtain the ID feature distribution Fea = ((X,a) and the OOD feature distribution Food = )(ooodd) Considering an anchor feature fa E Fed, an intuitive insight is that d(fia, fa) S d(food, fa), where .( is pre-defined distance metric, with fia E Fu and food E Foodd Previous work (Liu & Qin, 2023) has revealed the shortcomings of using Euclidean distance with global mean μ as an anchor point in OOD detection. However, the Mahalanobis distance has been demonstrated to be effective as a detection score (Lee et al., 2018; Sehwag et al., 2021), highlighting the necessity of considering information geom- etry when designing distance-based scores. Therefore, we simplify the distance designed for OOD detection scores as
4. Methodology
In Section 4.1, we propose a method to mitigate the impact of outlier features in the covariance matrix of the Maha- lanobis distance, with the core objective of finding a pro- jection matrix M that maximally separates ID and OOD samples. We hypothesize that these outlier features may align with the features from novel distributions. We also present the condition to ensure M to induce a valid distance. In Section 4.2, we suggest constraining adjustments to the residual space. We argue that, within the principal space of
3

---

Improving Out-of-Distribunn Detection via Dynamic Covariance Calibration
Identity
Static
Dynamic M
M(oooa
M(aaa
ff(f V0 fa) fa)
(f.fa)== V0 ( f.y f.
V MOM (a)
Euclidean distance
Mahalanobis distance
Dynamic distance
())
(3)
Training data features
Real time ID feature fia
Real time 0OD feature food
Cluster center fa (Anchor)
Figure 2. Visualization of the difference among distances calculated with prior matrix, static prior matrix, and dynamic prior matrix. The distance with no prior matrix Euclidean distance where is an identity matrix. the case the static prior matrix, the geometry not be adjusted, even if it is distorted by outlier features in the training samples. With the dynamic prior matrix, the geometry can be refined using the local information from the real-time input features.
the training distribution, there exists information that cannot be shrunk, such as semantic information or model distribu- tion. Based on these findings, in Section 4.3, we propose an OOD detection score and illustrate how its design meets the properties of valid distance function induced by M.
Theorem 4.2. Given a feature I E R a non-zero feature a E R and a symmetric positive definite matrix E Raxd, we define p = E- -= -= I-1aa a, and 8
f I a. Setting d(f) = (E-
if
p 1, then d(f) 2 0: if 1 and (s-1)² 1)2 <(p-((1))))) (p-1)(1-1).
then d(f) 2 0.
4.1. Dynamic Matrix Estimation
For OOD) features, we can not estimate the magnitude of p. Therefore, to maintain d(f) positive, we ensure that (s 1)² S (p () ( 1). Interestingly, under the condition where p > and q > 1, this inequality simplifies to (s 1) pq. condition satisfied by the Cauchy-Schward inequality when 2 0 In Section 4.3, we will discuss why this condition can be satisfied.
Here, we aim to design a mapping M to better separate ID and OOD samples. As discussed in (Pinele et al., 2019), the Mahalanobis distance measures distances within the space of Gaussian distributions, influenced by their covari- ance structures. Therefore, we can consider the covariance matrix Σ as a key matrix encoding the primary geometric characteristics in-distribution (ID) data under Gaussian assumptions. When feature f from a novel distribution feeds in, we aim reduce its influence within the covari- ance Σ. To achieve this, modify the covariance matrix to E- T.- The following proposition explains this property mathematically.
4.2. Projection in Residual Space
As mentioned earlier, foundational design of M in- volves subtracting ff However, this operation may affect the principal information of the training distribution, which could align with real-time ID features. Furthermore, studies in compatible learning (Pan et al., 2023) have illustrated that features from different pre-trained models can come form different distributions. We also visualize this phc- nomenon in Figure 3 with ImageNet-Ik 1k (Deng al., 2009) pre-trained VIT (Dosovitskiy et al., 2021) and DeiT (Tou- vron et al., 2021) models. The separated clusters demon- strate that model distributions are principal factors affecting feature distributions. Consequently, shared model distribu- tions may influence the direction of adjustment. To pre- serve semantic information while minimizing the impact
Given vector u, T uT --= S S T De. The difference can be determined by the square of similarity (v f)²
Proposition 4.1.
To enhance the Mahalanobis distance for OOD detection, given a feature f from a new distribution, we define the mapping M(f) = (Σ - )- However, with f fixed, M(f) may not be a positive definite matrix. Fortunately, find that with given anchor point a & 0, the quadratic form (f a) M(f(J a) can be guaranteed have positive value under certain conditions and introduce theorem to provide this condition.
Wheen s < 0.5, the inequality is obviously satisfied.

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
of shared model information on adjustment direction with real-time features, we confine the adjustment the residual space (Wang et al., 2022) of the training distribution.
each class mean Pes we construct zero-mean distribution and adjust the within-class covariance matrix accordingly.
Specifically, from the mean of each class pi, we can derive deviation feature in the training sample as Tin = fin Me. where e is the class label to which fin belongs. Then, we have R = {Tin) forming a zero-mean distribution, and we define M(f)
(ER B aa Ba aaaaBB B)" where ER is the covaraince matrix of R. namely the within-class covariance of {fin) J. Given the real-time feature f, the score function can be defined as:
s s(f) = min v
f-,, 0, Ne
(3)
Where Ne is the number of classes. The function s(2) aims to find the minimum distance between the real-time feature and different class means, calculated over the dynamically adjusted geometry based on the within-class covariance matrix Σg.
ViT DeiT
Rethinking the question in Section 4.1, if we consider the Σ in Theorem 4.2 as the within-class covariance Σn and a as one class mean jes we can easily obtain p > 1 and >> 1, since Σn reflects the compactness of the class clusters and none of the features tends cluster around zero point. As f and He mostly lie in the span of the positive eigenspace of MR. thus 2 O. Therefore, we can build a distance with M. Similarly, the same property can also be preserved when f in the residual space as fr and a as - - where f = fr + fr. We visualize the distributions of p and q in the Section 5.7 and s in the appendix.
Figure 3. Visualization of the model distribution between ImageNet-Ik Ik pre-trained ViT and DeiT. We labeled the features from VIT with orange colour and DeiT with blue colour. The features from different models distributed separately, which suggests the model distribution is also the principal information affecting the feature distribution.
In this context, the feature from the unknown distribution in the penultimate layer can be represented as Σ; a₂b; B. where B- } is the basis matrix of the residual space, and } is the coefficient vector. Consequently, M(f) is defined as M(f) = (Σ- B aa B)- Accord- ing to (Wang et al., 2022), B is the eigenvectors correspond- ing to the smallest eigenvalues of Σ.
5. Experiments
5.1. Experiment Setup
VIM research (Wang et al., 2022) suggests that OOD fea- tures typically exhibit more energy in residual space. More- over, Neco (Ammar et al., 2024) builds on the Neural Col- lapse phenomenon (Papyan et al., 2020) by suggesting that features in the penultimate layer of ID and OOD data can be- come increasingly orthogonal. This support the assumption that the OOD features predominantly occupy the null space of ID data, which also aligns with observations in VIM. Further enhancing this, we assume that the stronger col- lapse phenomenon may allow the residual space capture information from OOD features. To strengthen the collapse phenomenon, all features undergo La normalization in the initial stage (Haas et al., 2023).
To validate the effectiveness of our method, we conduct experiments on two standard dfitt--dist (OOD) detection benchmarks. In the first benchmark, we use the official test split of CIFAR-IO/CIFAR
as the in- distribution (ID) datasets, with six datasets serving as OOD data: SVHN (Netzer et al., 2011), Textures (Cimpoi et al., 2014), Places365 (Zhou et al., 2017), LSUN (Yu al., 2015), 2015).
(Yu et al., 2015) and iSUN (Xu et al.,
For the second benchmark, we follow (Huang & Li, 2021) to adopt ImageNet- Ik (Deng et al., 2009) as the ID dataset. There are six OOD datasets selected, including Texture (Cimpoi et al., 2014), SUN (Xiao et al., 2010), Places (Zhou et al., 2017), iNaturalist (Van Hom et al., 2018), Imagenet-O (Hendrycks et al., 2021) and OpenImage- O (Wang et al., 2022). Particularly, we use the subset of SUN, Places and Texture curated by Mos (Huang & Li, 2021) as the OOD dataset. The OpenImage-O O is a subset of OpenImage (Krasin et al., 2017) selected by VIM (Wang
more out-of-distribuu
4.3. Dynamic Covariance for Enhanced OOD Score
Building upon the theoretical findings from the previous sections, we aim to define a score function that achieves our ultimate goal of effectively separating ID and OOD samples. To this end, by utilizing deviation features derived from
5

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Table Comparision with different post-hoe OOD detection methods on CIFAR benchmarks. We present the AUROC and FPR9S results on DenseNet and WedeResNet and the average results over 2 ID datasets. The results of CIFAR- CAAARIO 100 are averaged over 6 OOD datasets. The detailed results can be viewed in the appendix.
NenseNet
WedeResNNtt
Method
CIFAR-IO 10
CIFAR-I0O
Avg.
CIFAR-I0
CIFAR-IO 100
Avg.
AUROCT
FPR95,
AUROCT
FPR951
AUROCT
FPR95L
AUROCT
FPR95L
AUROCT
FPR95
AUROCT
FPR951
(Hendrycks Gimpel, 2012)
92.5
48.72
14.37
80.13
83.44
64,43
91.07
50.64
76.93
77.48
84
64.06
Energy (Liu et al., 2020)
94.65
26.2
81.17
68.44
$7.91
47.32
91.86
33.74
79.83
71.95
85.85
52.85
maxl.ggi (Basart et al. 2022)
94.64
26.36
68.53
87.85
47.45
91.84
33.61
79.92
72.37
85.88
$2.99
ODIN (Liang sl., 2018)
94.65
26.35
81.06
68.53
$7.86
47.44
91.85
33.62
79.93
72.38
85.89
53
Mahalanobis (Lee et al., 2018)
85,9
47.64
77.56
$8.08
81.73
52,86
90.88
47.58
79.35
59.63
85.12
$3.61
GEM (Morteza & 2022)
88.01
31,73
84.19
56,93
86.1
44,33
93.22
37.28
82.71
57.15
87.97
47.22
KNN (Sun et al., 2022)
96.79
16.16
87.50
42.3
92.18
29.23
93.68
33,56
86,
48.32
90.01
40.94
ReAct (Sun
2021)
95.76
82.98
67.38
45.49
34,06
72.26
Line (Ahn al. 2023)
96.99
14.35
98.76
35.11
92.88
24.93
78,94
61.6
66.33
83.45
72.64
72.53
DICE (San & Li, 2022)
95.01
21.44
86.55
51.66
90.78
36.55
90.48
34.44
78.44
71.04
84.46
52.74
FDBD (Liu Qin, 2023)
97.23
13.86
89.25
50.57
93.24
32.22
92.27
36.87
85.14
65.77
88.71
51.32
ours
96.83
14.63
92.38
29.98
94.61
22.31
96.18
18.57
89.08
44.89
92.63
31.73
al., 2022). There are no overlapping categories between the OOD datasets and the ID dataset. We consider FPR95 and AUROC as performance metrics across both bench- marks. Please refer to the appendix material for more detailed explanation.
general settings, such with a high number of classes in the training set limitations in the model's capacity. Our method addresses this limitation by regularizing prior geometry with real-time features, enabling it to achieve a greater performance improvement over FDBD on CIFAR- 100. This phenomenon shows that our method is more robust under complex environments than FDBD.
5.2. Evaluation on CIFAR benchmark
We compare our method with a diverse set of post-hoc OOD detection approaches, including one probability- bassed method (MSP (Hendrycks & Gimpel, 2017)), three logit- based methods (Energy (Liu et al., 2020), maxLogit (Sun et al., 2022). ODIN (Liang al., 2018)), one density-based method (GEM (Morteza & Li, 2022)), and three distance- based methods (Mahalanobis (Lee et al., 2018), KNN (Sun et al., 2022), FDBD (Liu & Qin, 2023)). Additionally, we compare our method with feature clipping and weight prun- ing techniques such as ReAct (Sun et al., 2021), DICE (Sun Li, 2022), and LINE (Ahn et al., 2023).
5.3. Evaluation on ImageNet Benchmark
In this setting, we compare our method with baseline methods (MSP (Hendryeks & Gimpel, 2017), Energy (Liu et al., 2020), ODIN (Liang et al., 2018), React (Sun et al., 2021) and maxLogit (Basart et al., 2022)), distance- based methods (Malahnobis distance (Lee et al., 2018), KL di- vergence (Basart et al., 2022), KNN (Sun et al., 2022) and FDBD (Liu & Qin, 2023)) and subspace methods (VIM (Wang et al., 2022), Neco (Ammar et al., 2024) and WDiscOOD (Chen et al., 2023))
Table 1 presents the averaged AUROC and FPR95 metrics over six OOD datasets for the CIFAR benchmarks. Our method demonstrates the best AUROC and FPR95 results on average among all comparison methods. In particular, our proposed method reduces the average FPR95 by 9.9% on DenseNet and by 19.6% on WideResNet compared to the second-best method, FDBD.
Table 2 comparative analysis of various OOD detection methods on the ImageNet benchmark. The results show that our method achieves ofate-of-the-art performance on both AUROC and FPR95 metries across all four pre-trained models, highlighting its robustness and adaptability in large- scale OOD detection tasks. Due to NaN scores encountered with the Swin results are based on three models (i.e., ViT, ResNet-50, and DeiT), More detailed results can be viewed in the appendix.
model, the reported WDiscOOD
FDBD is the most relevant comparison method, it detects outlier data by employing distance-bait based score function. However, FDBD's effectiveness depends heavily on the quality of the extracted features, specifically the degree of inter-class separability. This sensitivity can be observed by comparing results on CIFAR- 10 and CIFAR- 100: when we decrease the class separability by increasing the number of classes, the performance of FDBD is significantly impacted, even with the same backbone model. From this, we can infer that FDBD performs better in scenarios where training features are well-clustered and not prone to poor distribution. These conditions are restrictive and difficult to achieve in
As mentioned in (Wang et al., 2022), the residual space method is limited by the feature quality of the original net- work. Moreover, we find that all subspace methods (Wang et al., 2022; Ammar et al., 2024; Chen et al., 2023) meet this problem. This phenomenon can be demonstrated by the weak performance of the subspace methods on ResNet-50 from Table 2. Instead of statically relying on the prior estimated geometry from training samples, our method dynamically refines the geometry to reduce the effect of ill-distributed samples in training distribution. Thus, our
6

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
method achieves significantly better performance compared to subspace methods ResNet-50.
within-class covariance dominates the overall covariance matrix. However, DCM may have inverse effect, the poorly clustered features could result in smoother and more unified distribution.
5.4. Evaluation using DINO
DINO (Caron al., 2021) is a classical self-supervised pre mre-trannng method. The DINO pre- pre-trained ViT model is publicly available, and we follow the same evaluation proto- col as in the ImageNet benchmark. Due to the limitations inherent in self-supervised learning, we restrict our compar- isons to methods that do not require linear classifiers.
ALROCC5)
ALBOC/%)
100.0
06.0
90.0
05.0
90.0
08.0
75.0
dims
dms
As shown in Table 3, our method achieves SOTA perfor- mance DINO in this setting. Notably, we omit WDis- cOOD results because the whitening transformation in WDiscOOD produced NaN values with DINO features, making the evaluation infeasible. Our approach excels in handling the unique characteristics of self-supervised fea- tures, achieving superior OOD detection performance. This highlights our method's adaptability and effectiveness in scenarios involving supervied pre-trained models.
150
250
(a) AUROCC on CIFAR-I0
(b) AUROC on CIFAR- 100
SVHN
LSUN Resize
ISUN
Places365
Texture
LSUN
Figure 4. AUROC on CIFAR-10 and CIFAR-100 pre-trained DenseNet with different residual space dimension
5.6. Performance w.r.t Residual Dimensionality
Figure 4 and 5 show how the performance varies with dif- ferent dimensions of the residual spaces on four pre-trained models: CIFAR-IO and CIFAR-100 pre-trained DenseNet, ImageNet- Ik pre- pre-trained ViT and ResNet50. As shown in Figure 4, the performance does not fluctuate much with different dimensions of residual space. In addition, the best performance on both pre-trained models is achieved at di- mensionalities around 250, indicating the consistency of our method on the same model. The stable trend at the start in Figure shows the robustness of residual space in large- scale OOD detection. The trend in the ViT experiments is consistent with the findings in VIM (Wang et al., 2022).
5.5. Ablation Study
We show the unique functionality of our method by studying three key components of our method: 1) DME: Dynamic Matrix Estimation; 2) RSP: Residual Space Projection for Dynamic Covariance Modeling with within-class covariance. We conducted an ablation study of both CIFAR and ImageNet benchmarks. DenseNet is selected as the backbone network for CIFAR, while ViT and ResNet-50 are exploited as the backbone for ImageNet. This setup provides holistic coverage of small-scale large-scale dataset, and transformer to CNN. We report the FPR95 index in this ablation study in Table 4. The baseline method is the Mahalanobis distance with normalized features.
the real-time adjustment; 3) DCM:
AUROCIT)) 100.0
ALROC(S)
100.0
65.0
35.0
90.0
As shown in the table, the method without considering the real-time features can not perform better than the full method in all scenarios, illustrating the effectiveness of our dynamic estimation of M. In addition, the DCM significantly contributes to the final performance for some backbone models, such as DenseNet and ViT. We assume the large between-class covariance distorts the ge- ometry with which the data can not form a dense manifold. We discuss the benefit of within-class covariance in the appendix. Also, without RSP, the performance can deteri- orate in some situations like ResNet-50, which shows the importance of avoiding prior adjustment in the principal space. Note that RSP can only be computed using the full covariance matrix (within-class + between-class), differing from the RSP in the full model, which specifically opti- mizes the within-class covariance. RSP can be detrimental in this situation due to the large between-class covariance. But it can solety work on poorly clustered scenarios since
95.0
79.0
can find that 10.0
alms
95.0
dims
350 1805
369
(a) AUROC on ResNet-50
(b) AUROC on ViT
Textures
Piace
ImageKet-O O
Opentmage O
SUN
iNeturalist
Figure 5. AUROC on ImageNet- Ik pre-trained ResNet and VIT with different residual space dimensions
5.7. Analyzing p and Values
Theorem 4.2 emphasizes that y and q should be much larger than I to guarantee the solid distance. Since the prior ad- justments are confined to the residual space, p = f Σ fr, where fr = a B is the time feature restricted in resid- ual space, and g = (Ne f/) E (Me - f.-)- Me selected
7

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Table Comparision with different post-hoe OOD detection methods on ImageNet- 1k benchmark. We present AUROC and FPR95 results on VIT. 50s SwinV2-B, and DeiT. We also provide the average results over the 4 pre-trained models. The results of the four pre-pretrained models are averaged over OOD datasets. The detailed results can viewed the appendix. As we can not achieve solid results with WDiscOOD on ImageNet-1k Ik pre-trained Swin V2-B/1. the average results are from the other 3 pre-trained models.
Models
Method
VIT
ResNet-50
Swin-B
Deil
Avg.
AUROC1
FPR95L
AUROCT
FPR95
AUROCT
FPR95]
AUROC1
FPR95]
AUROCT
FPR95]
MSP (Hendrycks & Gimpel, 2017)
88,89
43.46
73.97
70.98
81.29
63.49
79.8
66.97
80.99
61.23
Energy (Liu et ul., 2020)
94.11
27.56
79.53
65.8
80.07
60.35
71.65
72.65
81.34
56.59
ReAct (Sun et al., 2021)
94.07
27.69
83.34
54.81
85.2
53.53
77.16
68.74
84,94
51.19
ODIN (I.iang al., 2018)
93.73
29.68
79.42
65.77
80.68
58.94
76.07
66.43
82.47
55.2
maxLogit (Basurt
2022)
93.73
29.68 68
79.42
65.78
80,94
59.56
76.43
66.38
82.63
55.35
Mahalanobis (Lec et al., 2018)
94.27
27.11
68.36
80.63
87.86
52.07
83,98
73.86
83.62
58.42
KLMarch (Basart al., 2022)
87.8
44.39
76.09
69.93
81.83
63.85
82.68
67.24
82.1
61.35
KNN (Sun et al.. 2022)
92.6
34.38
84.43
57.46
85.08
65,24
82.75
76.24
86.22
58,33
VIM (Wang et al. 2022)
94.23
27.32
83.93
65.92
86.77
51.33
83.91
71.13
87.21
53,93
FDBD (Liu& & Qin, 2023)
93.36
31.71
84.47
60.35
86.57
55.75
82.78
71.84
86.79
54,91
Neco (Ammar et al., 2024)
94.38
27.08
75.15
70.27
81.73
54.74
79.2
62.03
82,61
53,53
WDiscOOD (Chen et al., 2023)
94.41
26.35
70
78.71
83.97
73.83
82.8
59.63
ours
94.27
26.94
87.43
51.77
88.1
51.89
84.97
68.29
88.69
49.72
further how residual projection helps OOD detection by analyzing feature scale. Given that all real-time features are normalized before conducting the prior adjustment, the norm of features before projected into the residual space 1. But in residual space, the norm of a B can be smaller and with higher discriminative power. We visualize the distribution of norms in Figure 7. As shown in the figure, the OOD features tend to have larger norms in residual space for both ViT and ResNet-50 backbones. This distinction in norm distributions confirms that residual projection helps distinguish OOD samples from ID samples
Table 3. Comparison on DINO Comparision with different post- hoc OOD detection methods on DINO. We report the averaged AUROC and FPR95 results over 6 OOD datasets. The detailed results can be viewed in the appendix.
DINO
Method
AUROC1
FPR95
SSD (Schwag al., 2021)
51.5
96.86
Necu (Ammar et al., 2024)
46.97
96.69
KNN (Sun al., 2022)
84.99
63.88
Mahalanobis (Lec al., 2018)
91.57
39.27
ours
91.65
38.23
based on the cluster means {u:} with the minimum Maha- lanobis distance to f. Figure illustrates the value of p and q derived from ImageNet-1k 1k benchmark experiments. As we can see from the figure, both and q consistently exceed 1 by a considerable margin, even with ResNet-50 which produces less compact embedding space. This strongly sup- ports our theoretical assumption. We also present the values of s = f ER (Me f() in the appendix on both VIT and ResNet-50.
froquency
Trequency
D deta
OCD deta
10000
2005
6000
value
value
(a) VIT
(b) ResNet-50
fresuency
frequency
1753e
10 data ..OD data
10 data
Figure 7. The la norms of features extracted by ImageNet- Ik pre- trained ResNet-50 and VIT.
15000
10000
COD cata
12500
8000
5050
4360
6. Conclusion
2530
value
1500 2000 3300
0002
8000
In this paper, we introduce a new perspective for defining distance-basee OOD scores that dynamically refine the dis- torted geometry of the training distribution. Building on this perspective, design OOD detection score that adjusts the covariance in the Mahalanobis distance in real- time. Specifically, we restrict the adjustment to the residual space to avoid unintended influences. Extensive experi- ments demonstrate the effectiveness of our method. We also conduct experiments using the Fuclidean distance and hard
(a) p values on ResNet-50
(b) values in ResNet-50
Figure 6. p and values on ImageNet-Ie Ik pre-trained ResNet-50
5.8. Vector Norms in Residual Space
In addition to examining the effectiveness of projection in residual space based on performance metrics, we investigate
8

---

Improving Out-of-Distribuuiin Detection via Dynamic Covariance Calibration
Table Ablation study on different procedures. The smaller value reported FPR95 means the better performance. DME.
RSP and DCM are short for Dynamic Matrix Estimation, Residual Space Projection and Dynamic Covariance Modeling respectively.
DME
RSP
DCM
CIFAR-10
DenseNet
CIFAR-100
DenseNet
ImageNet-Il 1k ViT
ImageNet- 1k ResNet50
AUROCT
FPR
AUROCT
FPR.
AUROCT
FPR.
AUROCT
FPR
95.19
18.86
89.47
35.84
92.51
33.16
82.49
60.43
95.47
17.51
88.05
36.61
92.65
32.31
81.46
61.77
V
94.08
23.92
85.69
53.70
82.72
86.32
87.31
52.17
96.24
16.38
92.17
30.16
94.24
27.05
81.88
60.49
V
96.83
14.63
92.38
29.98
94.27
26.94
87.43
51.77
OOD detection in the appendix material, illustrating that our method can be applied to various distance metrics. We plan to explore the design of M for vision language models and large language models in future work.
detection with orthogonal projection of gradients. Ad- vances in Neural Information Processing Systems, 36: 38206-38230, 2023.
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference Computer Vision (ICCV), 2021.
Acknowledgements
This work is partially supported by Australian Research Council Project FT230100426.
Chen, J., Li, Y., Wu, X., Liang, Y., and Jha, S. Atom: Robustifying out-of-distribttion detection using outlier mining. Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13-17, 2021, Proceedings, Part III 21, pp. 430-445. Springer, 2021.
Impact Statement
This paper presents a methodooogical enhancement in out-of- distribution (OOD) detection, which is crucial for deploying Al systems in safety-critical environments. This addresses significant risks in sectors like autonomous transportation and medical diagnostics, where unrecognized OOD samples could lead to failures with dire consequences. Our approach, thus, not only advances the technical field of OOD detection but also contributes to societal trust in AI technologies by enhancing their reliability and safety.
Chen, Y., Lin, Y., Xu, R., and Vela, P. A. Wdiscood: Out- of-distribution detection via whitened linear discriminant analysis. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. -298-5307, 2023.
References
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pp. 3606-3601 2014.
Ahn, H., Park, G.-M., and Kim, S. T. Line: Out-of- distribution detection by leveraging important neurons. In 2023 IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 19852-19862. IEEE, 2023.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Ammar, M. B., Belkhir, N., Popeseu, S., Manzan- era, A., and Franchi, G. NECO: NEural collapse based distribdittiiittonnn detection. In The Twelfth In- ternational Conference on Learning Representations, 2024. URL https::/ wwwpeeeiwwwweeweee.eeeeeeeeeeeee net/forunt id=9ROuKblmi7.
DeVries, T. and Taylor, G. W. Learning confidence for out-of-distribution detection neural networks. arXiv preprint 100.:::1822044660 0000000655 2018.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn. D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learning Representations, 2021. URL / openreview n forum? i dit.:iddddrcbd.y........
Basart, S., Mantas, M., Mohammadreza, M., Jacob, S., and Dawn, S. Scaling out-of-distribution detection for real- world settings. In International Conference on Machine Learning, 2022.
Behpour, S., Doan, L., He, W., Gou, L., and Ren, L. Gradorth: simple yet efficient out-o--distribution

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
ElAraby, M., Sahoo, S., Pequignot, Y., Novello, P., and Paull, L. Grood: Gradient-aware out-of-distribution detection in interpolated manifolds. arXiv preprint arXiv:2312.14427, 2023.
Huang, R. and Li, Y. Mos: Towards scaling out-of- distribution detection for large semantic space. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 8710-8719, 2021.
Filos, A., Tigkas, P., McAllister, R., Rhinehart, N., Levine, S., and Gal, Y. Can autonomous vehicles identify, re- cover from, and adapt distribution shifts? In Interna- tional Conference on Machine Learning, pp. 3145-3153, PMLR, 2020.
Huang, R., Geng, A., and Li, Y. On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34: 677-689,10 2021.
Idrees, H., Shah, M., and Surette, R. Enhancing camera surveillance using computer vision: a research note. Polic- ing: An International Journal, 41(2):292-307, 2018.
Fu, W. Chen, Y., Liu, W., Yue, X., and Ma, C. Evidence reconciled neural network for outtt--distritutionn detec- tion in medical images. In Medical Image Computing and Computer Assisted Intervention - -ICCAI 2023, pp. 305-315. Springer Nature Switzerland, 2023. ISBN 978- 3-031-43898-1.
Krasin, I. Duerig, T., Alldrin, N., Ferrari, V., Abu-El-Haija,
S., Kuznetsova, A., Rom, H., Uijlings, J., Popov, S., Veit,
A., Belongie, S., Gomes, V., Gupta, A., Sun, C., Chechik, G., Cai, D., Feng, Z., Narayanan, D., and Murphy, K. Openimages: A public dataset for aarge-scale multi-label and multi-class image classification. Dataset available from https://gituub..oo/oppeeet
Haas, J., Yolland, W. and Rabus, B. Linking neu- ral collapse and 12 normalization with improved out-of- distribution detection in deep neural networks. Transac- tions Machine Learning Research, 2023. ISSN 2835- forum?
2017.
8856. URL https: jkN5Ur2d.....
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Muchine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Han, K., Wang. Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., al. A survey vision transformer. IEEE transactions pattern analysis and machine intelligence, 45(1):87-110, 2022.
Lee, K., Lee, K., Lee, H., and Shin, J. simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information pro- cessing systems, 31, 2018.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Liang, S., Li, Y., and Srikant, R. Enhancing the reliability of ouitti-disttibution image detection in neural networks. In International Conference on Learning Representations, 2018. URL https:: wwwp//pppnweeeeeeeeeeeeeeeeeeeea net /frum?
Hein, M., Andriushchenko, M., and Bitterwolf, J. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEC/CVF conference on computer vision and pattern recognition, pp. 41-50, 2019.
id=H1VGkIxRZ.
Liu, L. and Qin, Y. Fast decision boundary based out-of- distribution detector. arXiv preprint arXiv:2312.11536, 2023.
Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and out-of-distribution examples in neu- ral networks. In International Conference on Learning Representations, 2017. URL https: /ttkLttt////eeeeee//iii //ppnreviee..... net/ htt//forumiii 4TT
Liu, W., Wang, X., Owens, J., and Li, Y. Energy-based out- of-distribution detection. Advances in neural information processing systems, 33:21464 21475, 2020.
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, D. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15262-15271, 2021.
Morteza, P. and Li, Y. Provable guarantees for understand- ing out-o--distributiion detection. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36. pp. 7831-7840, 2022.
Hendrycks, D., Zou, A., Mazeika, M., Tang, I.,, Li, B., Song, D., and Steinhardt, J. Pixmix: Dreamlike pictures com- prehensively improve safety measures. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16783-16792, 2022.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A. Y., et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 4. Granada, 2011.
10

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Pan, T., Xu, F., Yang, X., He, S., Jiang. C.,, Guo, Q.. Qian, F., Zhang, X., Cheng, Y., Yang, L.., et al. Boundary-aware backward-compatin representation via adversarial learn- ing in image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 15201-15210, 2023.
Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8769-8778, 2018.
Vyas, A., Jammalamadaka, N., Zhu, X., Das, D., Kaul, B., and Willke, L. -ustof-distribution detection using an ensemble of self supervised leave-out classifiers. In Proceedings of the European conference on computer vision (ECCV), pp. 550-564, 2018.
Papyan, V., Han, X., and Donoho, D. L. Prevalence of neural collapse during the terminal phase of deep learn- ing training. Proceedings of the National Academy of Sciences, 117440):4465--24663, 2020.
Wang. H., Li, Z., Feng, L., and Zhang, W. Vim: Out-of- distribution with virtual-logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4921 -990,, 2022.
Peng, B., Luo, Y., Zhang, Y., Li, Y., and Fang. Z. Conjnorm:
Tractable density estimation for out-of-distribution de- tection. In The Twelfth International Conference Learning Representations, 2024. URL https:// openreview wwt. 1d-
Wang. Y., Li, B., Che, T., Zhou, K., Liu, Z., and Li, D. Energy-based open-world uncertainty modeling for con- fidence calibration. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 9302- 9311, 2021.
Pinele, J., Costa, S. I., and Strapasson, J. E. On the fisher-rao information metric in the space of normal distributions. In International Conference on Geometric Science of Information, pp. 676-684. Springer, 2019.
Xiao, J., Hays, J., Ehinger. K. A., Oliva, A., and Torralba, A. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3485-3492. IEFE, 2010.
Ren, J., Fort, S., Liu, J., Roy, A. G., Padhy, S., and Lak- shminarayanan, B. A simple fix to mahalanobis dis- tance for improving near-ood detection, arXiv preprint (xXiv22006.09022 2021.
Xu, K., Chen, R., Franchi, G., and Yao, A. Scaling for training time and post-hoc out-of-distribution detection enhancement. In The Twelfth International Conference on Learning Representations, 2024. URL https: openreview net forum? ? dd-DSTjtnaCCg
Sehwag, V., Chiang, M., and Mittal, P. (SSD): A uni- fied framework for self-supervised outlier detection. In International Conference on Learning Representations, 2021. URL httpttts:// http://pppneeeeeeeeeeeeeeee formm id=w5gjXpmR8J.
Xu, P., Ehinger, K. A., Zhang, Y., Finkelstein, A., Kulka- mi, S. R., and Xiao, J. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504 1504.06755 2015.
Sun, Y. and Li, Y. Dice: Leveraging sparsification for out- of-distribution detection. In European Conference on Computer Vision, pp. 691-708. Springer, 2022.
Sun, Y., Guo, C., and Li, Y. React: cut-of-distribution detection with rectified activations. Advances in Neural Information Processing Systems, 34:144-157, 2021.
Yang. J., Wang. P., Zou, D., Zhou, Z., Ding, K., Peng, W., Wang. H., Chen, G., Li, B., Sun, Y.. et al. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems, 35: 32598-32611 2022.
Sun, Y., Ming, Y., Zhu, X., and Li, Y. Out-of-distribution de- tection with deep nearest neighbors. In International Con- ference Machine Learning, pp. 20827-20844 PMLR, 2022.
Yang. J., Zhou, K., Li, Y., and Liu, Z. Generalized out-of- distribution detection: A survey. International Journal of Computer Vision, pp. 1-28, 2024.
Tajwar, F,, Kumar, A., Xie, S. M., and Liang, P. No true state-of-the-art? ood detection methods are inconsistent across datasets. arXiv preprint arXiv:2109.0553
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., and Xiao, J. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint 150..v/550600363 2015.
2021.
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H. Training data-cfficient image transform- ers distillation through attention. In International con- ference on machine learning, pp. 10347-10357. PMLR, 2021.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Tor- ralba, A. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452 1464, 2017.

---

Improving Out-of-Distribation Detection via Dynamic Covariance Calibration
A. Theorical Proof
most cases. However, the improvement is insignificant since other factors, such as geometry in the principal space, affect the performance.
Proof of Theorem 4.2. With Sherman-Morrise Formula, have d(f)= (f a)--aaa---- 'f- a) + ((faa) x-
1-12-11
For = -- < 1, obviously d(f) 2 0.
Table 5. Ablation study on Euclidean distance. We report the performance on ImageNet- pre-trained VIT, ResNet and DINO, and CIFAR-10/100 pre-trained DenseNet.
For p - / > 1, we have
d(f) = - 2s (p-s) 8)²
w adjustment
w/o adjustment
Model
2 0
AUROC1
FPR95
AUROCT
FPR95|
1-1 p
CIFARIO
- p q-- pq - 2s + 2 <
DenseNet
96.443
18.023
96,429
18.101
+ 2 - 2s pq
pq p-9
CIFARI00
- (s 1)² < (p 1)(g - 1).
DenseNet
89.563
39.836
89.217
41.032
ImageNet
ViT
93.993
28.691
93.267
32.817
ResNet-50
87.794
46.162
87.794
46.163
DINO
89
46.425
88.095
49.891
B. Additional Experiments
B.1. Analyzing: s Values
B.3. Near OOD detection
In Section 4.3, we argue that s EA(He f() values can be positive for within-class covariance, where uem is selected based on the class means } with the minimum Mahalanobis distance to real-time features f. Figure 8 shows the distribution of value on ImageNet-Ik pre-trained VIT and ResNet-50. As shown in Figure 8, the s scores are always positive, supporting our assumption that M induces a valid distance.
We conduct near OOD detection on OpenOOD bench- mark (Yang et al., 2022) with ResNet18. We report the experimental results in Table 6. We also compare with the results of the recent distance score and subspace score avail- able in OpenOOD, namely KNN and VIM. All the results are averaged over runs. We also provide the results of RMDS (Ren et al., 2021) with our covariance dynamic ad- justment. RMDS is Mahalanobis distance tailored for near-OOD detection.
froquoncy
frequency
2030
I0 data OOD data
24000
32000
COD cata
10 data
As the table shows, vanilla Mahalanobis distance struggles to perform well on the near OOD detection task, which aligns with observations in previous work (Tajwar al., 2021). In contrast, significantly enhances the effectiveness of the Mahalanobis distance in near OOD detection. In addition, it can also improve the performance of RMDS, reflecting the robustness of our method in different distances in near OOD detection.
1000
e0t0
2010
4000
2000
value
10600 11000 11500 12000 22500 18000
50
100
200
(a) s values on ViT
(b) values on ResNet-50
Figure& 8. The s values on ImageNet- 1k preitrained ResNet-50 and VIT.
C. Algorithm Details
We present the pseudocode for method in Algorithm 1. The calculation of B is similar to the R in residual space method (Wang et al., 2022). We derive the basis matrix B from the eigenspace of within-class covariance Σn.
B.2. Dynamic Adjustment in Euclidean Distance
Our approach can be utilized with various distances which utilize covariance matrix structures. For instance, if we con- sider Σ as the identity matrix I, which corresponds to the Euclidean distance, then M = (I -- We compare the OOD score with and without applying our approach in Euclidean distance in Table 5. All the features normal- ized in the first stage. We conduct experiments on large- scale OOD detection tasks with 3 ImageNet pre-trained models, namely VIT, ResNet-50 and DINO, and CIFAR- 10/100 pret-trained DenseNet. As shown in Table 5, better performance can be achieved with dynamic adjustment in
Why using within-class covariance μRr? We aim for the data to form a manifold, allowing us casily determine its geometry. However, the features from training distribution form separate clusters for different classes in the penultimate layer. The whole data can not form a manifold that requires any local patch to be homomorphic to the Euclidean space. A simple translation in a direction like can not change the separated structure of the data distribution. However, we define the deviation feature with the class mean Pe: the
12

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Table Near OOD detection on OpenOOD benchmark. We conduct experiments under four settings. When using CIFAR- 10 as the seen dataset, we evaluate on CIFAR-I00 100 and Tiny ImageNet as unseen datasets. Conversely, when CIFAR-I00 serves as the seen dataset, CIFAR- TinylmageNet are used as the unseen datasets. Best results are highlighted bold.
CIFARIO-CIFARI00
CIFARIO-Thy FPR|
CIPARI00
CIFAR100-
avg AUROCT
Method
FPR.
AUROC
AUROCT
FPR.
AUROCT
FPR.
AUROC1
FPR.
VIM
52.33
87.03
44.16
88.88
70.79
72.15
54.92
77.73
55.56
81.46
KNN
38,76
89.59
30.89
91.51
72.69
77.1
49.68
83.29
48.01
85.37
Mahalanobis
64.51
79.48
59.06
80.71
88.88
54.6
80.75
60.31
73.30
68.78
RMDS
49.76
88
37.63
90.29
62.90
77.69
49.55
82.60
49.96
84.71
Mahalanobis-dynamii (ours)
61.38
84,73
52.95
86.54
75.24
71.72
60.35
76.34
62.48
79.83
RMDS+dynanic (ours)
38.70
89.42
30.47
91.45
61.70
78.21
48.85
82.81
44.93
85.48
F. Discussion
Table Comparison of runtime and AUROC on CIFAR-10 pre-
trained DenseNet.
OOD detection in Vision-Language model (VLM) and Large language model (LLM) Limited by the computing resource, it is impossible for us evaluate our method on some large foundation models. In addition, the out- of-distribution is hard to define due the large train set, Even if WiscOOD (Chen et al., 2023) evaluates CLIP on ImageNet benchmark, we doubt the validation of the ex- periments since no verification has been stated that there is no overlap between the image-text training set and the OOD datasets the benchmark. However, according to our promising performance on different backbones, we believe our dynamical adjustment can be helpful in OOD detection of VLM and LLM.
Time (s)
AUROC(%
Feature extraction only
218.42
N/A
GradNorm
473.70
92.6
Mahalanobis
221.53
85.9
ours
235.59
96.93
cluster from different classes can be translated to centered as zero point, which is also the mean of the whole data. This allows the score to be calculated over the denser distribution as manifold.
Inference cost The computing complexity of our method is O(n²), where is the feature size. It takes around 0.002 seconds to calculate the score from each feature with 16GB V100 GPU. We provide the detection time cost comparison over 10,000 CIFAR- instances with DenseNet as the back- bone network in Table 7. We note that method has comparable time cost to Mahalanobis distance method. This is because in post-hoc OOD detection, the dominant com- putational cost lies in the feature extraction, which is shared across most methods. In addition, we compare with the time cost of GradNorm (Huang et al., 2021) the table, which shows the effectiveness of Mahanobis distance-basee score functions. So, we believe the inference cost of our method is affordable in practice, especially with more advanced equipment like A100.
Algorithm OOD Score s(-) Calculation
Input: Feature vector f. basis matrix B, within-class co- variance matrix RR. class means {14}
Output: Score function s(-)
Normalize f using the normalizer
a +- worch.einsure
bi f.,
adj- + B TBB aB
dygeo torch. Liw.la..inii............ii.iii adj)
d - (4:)
dists + torch forch.einsum(" (ii bj + b', d, dygeo, d)
score + - ttrchhrchhhcoocc min dists)
return score
D. Detailed Results for CIFAR Benchmarks
G. Visualizaiton of Motivation
Here, we present the detailed results for Table I of the main paper in Table 8 and 9. More details can be viewed in our source code in the supplementary.
Our method built on the assumption that the outlier fea- tures can undermine the effectiveness of the Mahalanobis distance. Figure 9 illustrates our motivation. As the fig- ure shows, when outlier points shift away from the in- distribution (ID) center towards some out-of-distribution (OOD) points, the Mahalanobis distance can be smaller between ID and these OOD points. This will affect the performance of OOD detection.
E. Detailed Results for ImageNet Benchmarks
We present the detailed results for Table and 3 of the main paper, Table 10. More details are in our source code in the supplementary.
13

---

Improving Out-of- Detection via Dynamic Covariance Calibration
Table Comparison on CIFAR pretrained DenseNet. We compare different post-hoc detection methods on CIFAR benchmarks.
The base model is DenseNet.
OOD
Method
SVIN
Texture AUROCT
LSUN Resize
LSUN
ISUN
Placessis
AVG
AUROCT
FPR95]
AUROC1
FPR95
AUROC1
FPR95]
AUROCT
FPR95
AUROC1
FPR95
AUROC1
FPR95
CIFAR-10
MSP
93.57
47:25
88.14
64.33
94.55
42.11
95.54
33.68
94,49
42.54
88.73
62.39
92.5
48.72
94,19
38.88
86.41
3.84
9.58
91
39.75
26.2
maxLogit
94,32
37.95
86.5
$6.06
98. 12
9.45
99.09
4.21
98.05
10.13
91.77
40.37
94.64
26.36
ODIN
Maha MahaVanilla Vanilla
98.11
7.93
92.81
25.66
87.91
51.71
82.62
67.99
88.79
45.52
65.
87.02
85.9
47.64
46.33
ReAct
95.23
33.4
91.29
48
98.4
7.51
99.06
4.63
98.27
8.44
92.3
39,54
95.76
23.59
Line
97.75
11.42
95.11
23.44
99.1
4.12
99.83
0.62
99.02
5,08
91.13
43.85
96.99
14.75
DICE
94,96
27.82
86,96
46.03
99.06
4.23
99.9
0.38
98,99
5.2
90.18
95.01
21,44
FDRD
98.61
6.2
95.94
23.07
98 82
5.56
99.32
3.49
98.77
5.83
91.94
38.97
97.23
13.86
98.41
99.34
4,04
87.84
CIFAR-100
75.18
82.01
71,41
84.8
69.
85.22
85.6
60.5
70.17
86.01
74.71
82.24
74.37
80.13
Energy
$1.3
88.02
71,01
84.33
80.14
70.69
97.43
14.75
78.95
74.6
78.15
78.25
81.17
68.44
maxLogit
81.42
86.26
71.18
83.44
79.77
71.7
97.06
16.98
78.69
75.05
78.22
77.78
81.06
68.53
ODIN
81.43
86.24
71.19
83.42
79.77
71.71
97.06
16.99
78.69
75.03
78.23
71.77
81.06
68.53
Vanilln
88.01
54.26
92.15
28.32
92.08
38.33
44.43
96:47
92.38
36.09
56.29
95.04
77.56
$8.08
KNN
96.35
ReAct
82.77
83.69
77.7
79.79
81.71
71.56
97.13
15.82
81.06
75.19
77.53
78.25
82.98
67.38
23.37
DICE
88.2
39.95
77.13
61.44
83.25
54.93
99.74
0.91
88.52
$2.43
77.45
80.34
86.55
$1.66
FDBD
91.38
49.75
91.59
43.3
88.71
57.75
96.54
19.19
89.43
56.03
77.87
77.39
89.25
50.57
95.53
23.28
95.68
19.8
96.12
22.07
98.78
6.05
96.39
21.11
71.79
87.6
92.38
29.98
Table 9. Comparison on CIFAR pretrained WideResNet. We applied post-hoc OOD detection methods on CIFAR benchmarks. The base model WideResNet.
OOD datasets
Method
SVHN AUROCI
Texture
LSUN_Resize
LSUN
ISUN
Places365
AVG
FPR9S]
AUROCT
FPR95L
AUROC1
FPR951
AUROCT
FPR95]
AUROC1
FPR95,
AUROCT
FPR95L
AUROC1
FPR95]
CIFAR-10
54.89
Energy $8.49
44.99
84.24
58.9
95.47
23.05
99,27
2.88
93.99
30.05
89.69
42.55
91.86
33.74
MahaVanilla
KNN
95.01
31.32
92.26
40.6
95.05
27.53
95.33
26.85
93.93
30.98
90.47
44,06
93.68
33,56
maxLogit
$8.57
44.11
84.41
$7.7
95.38
23.43
99.17
3.25
93.9
30.36
89.62
42.79
91.84
33.61
ReAct
88.63
45.4
86.24
57.45
95.05
24.43
99,23
3.21
93.52
31.62
89.86
42.25
92.09
34,06
ODIN
88.58
44.11
84.42
$7.71
95.38
23.42
99.17
3.29
93.9
30.38
89.62
42.81
91.85
33.62
GEM
97.56
13.26
97.3
15.32
93.16
43.03
84.51
68.43
93.22
37.28
DICE
86.31
45.88
81.07
60.11
95.12
22.8
99.74
0.48
93.71
29.35 36
86.93
48.05
90.48
34.44
Line
71.92
70.2
FDDD
92.8
38.52
89.03
48.55
93.38
32,82
97.65
13.1
91.4
38.94
89.36
49.29
92.27
36.87
OHTS
3.46
CIFAR-100
MSP
70.91
85.24
71.39
85.89
79.2
79.82
88.56
49.1
78.1
81.57
73.41
83.26
76.93
77.48
Energy
70.87
87.59
72.85
85.27
82.62
78.49
96.83
16.74
80.89
82.51
74.94
$1.08
79.83
71.95
Mahulanilla
89.68
44.91
90.03
42.38
91.73
38.28
47.63
99.71
91.16
40.56
65.86
91.94
79.35
59.63
KNN
89.95
43.03
94.32
29.49
56.3
69 69968
85.65
maxLogit
71.22
86.59
73.07
85.07
82.
78.18
96.19
21.5
81.14
81.89
75.13
80.99
79.92
72.37
ReAct
74.61
ODIN
71.23
86.58
73.07
85.09
82.78
78.2
96.19
21.51
81.15
81.93
75.13
80.99
79.93
72.38
90.42
43.98
93.69
92.31
36.98
90.34
57.15
DICE
68.38
90.41
72.4
83.07
78.8
81.68
98.68
4.18
78.2
83.64
74.16
83,27
78.44
71.04
Line
66.08
94.43
70.86
79.59
56.42
97.83
92.64
34.63
58.88
98.33
$3.08
95.88
66.33
83.45
FDBD
83.18
73.01
83.26
71.97
89.75
58.07
91.16
45.85
88.16
63.22
75.31
82.51
85.14
65,77
onrs
92.64
36.07
92.55
34.66
94.04
32.29
87.75
50.83
93.14
33.48
74.35
82
89.08
44.89

---

Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Table 10. Comparison ImageNet-1k 1k pretrained models. We applied post-hoc OOD detection methods ImageNet benchmarks. base models are Vit, ResNet-50, Swin-B, DeiT and DINO.
OOD
Method
Testure AUROC1
SUN
Places
INaturalist
ImageNet-O
OpenImage-
AVG
FPR95]
AUROCT
AUROC1
FPR95].
AUROC1
FPR95].
AUROC1 FPR95_
VIT
MSP
85.42
52.43
86.93
53.22
85.72
57.75
96.97
13.63
85.81
51.75
92.48
31,99
88.89
43.46
Energy
91.25
36.13
93.28
34.44
90.98
42.82
98 99.94
5.6
93.36
30.3
96.87
16.07
94.11
27.56
Vanilanilla
91.71
36.21
92.82
35.35
89.51
46.22
99.77
1.08
94,28
30.1
97.51
13.7
94,27
27.11
92.47
VIM
91.47
37.75
93.3
32.64
89.77
44.44
99,66
1.55
94,09
31,05
97.11
16.48
94.23
27.32
opir
90.86
KLMatch
84,9
51.86
85.11
56.62
83.56
61,49
96.36
13.7
85,16
50.9
91.7
31,76
87.8
44,39
ReAct
91.17
36.35
93.22
34.55
90.83
43.32
98.93
3.63
93.4
30.3
96.88
16.01
94.07
27.69
ODIN
90.86
38.56
92.81
37.46
90.66
44.66
98.81
6.03
92.69
33.55
96.54
17.85
93.73
29.68
FDB
90.54
39.17
92
40.7
89.59
98.76
6.59
92.77
36.6
96.49
19.23
93.36
31.71
Neco
91.92
35:37
42.39
WDiscOOD
91.83
35.87
93.19
33.32
89.77
44.35
99.79
1.03
94.41
29.55
97.49
13.95
94.41
26.35
ours
ResNet-50
68.58
80.63
71.57
52.77
100
83.91
73.97
Energy
86.73
52.29
86.73
58.28
84.13
65.4
90.59
53.95
41.92
100
87.06
64,88
79.53
65,8
MahaVanilla
$9.98
43.48
52.02
97.37
51.68
97.33
63.9
93.78
80.94
66.35
71.63
85.46
68.36
80.63
KNN
97.49
10.85
80,71
69.64
74.86
78.03
86.02
59.55
84.62
62.4
82
64,29
84.43
$7,46
VIM
96.86
14:79
81.1
$2.05
78.42
83.27
87.53
71.33
70.87
85.08
88.81
59.01
83.93
65.92
68
KLMatch
82.82
64.77
80.55
73.51
78.86
75.56
89,91
44.29
38.67
100
85.74
61.48
76.09
69.93
ReAct
ODIN
86.4
$4.31
86.59
59.9
88
65.67
91.14
50.86
40.86
100
87.36
63.88
79.42
65.77
92.1
37.52
86.81
61,25
84.07
67.11
93.71
40,16
100
90.3
56.07
84.47
60.35
Neco
95.62
17.8
66:46
87.25
66.44
88.28
69.17
83.39
75.88
73.3
77.32
71.61
75.15
70.27
WDiscOOD
91.62
38.83
54.64
96.64
53.65
96.82
65.98
91.98
82.16
64.1
71.95
83.9
78.71
98.81
5.48
84
65.53
78.8
75.55
91.62
46.88
82,18
66.1
89.16
$1.08
87.43
$1,77
Swin-H B
83.9
57.87
Energy
83.88
53.37
81.03
62.44
80.18
63.5
88.
43.54
65.04
83.3
82,14
55.97
80.077
60.35
Maha
88.97
47.07
86.78
60.65
85.53
64.62
95.67
21.05
77.13
83.45
93.1
35.58
$2.07
KNN
88.73
48.51
83.2
74.95
81.44
76.69
91.32
51.92
76.32
84.45
89.47
54.93
85.08
65.24
VIM
88.26
45.02
84.35
59.07
80.88
64.07
95.93
22.45
77.89
81.15
93.29
36.24
86.77
$1.33
maxl Logit
83.72
56.1
82.65
60.16
82.04
61.54
39.77
63.54
86.2
84
$3.57
$9,56
KLMatch
83.25
60.28
83.49
69.05
82.61
71.28
90.44
42.61
64.64
84.65
86.54
$5.22
81.83
63.85
ReAct
87.33
ODIN
83.46
$5.66
82.48
59.32
81.83
60.9
89.72
38.65
62.97
85.95
83.61
53.14
80.68
58.94
43.01
82.65
54.5
84.17
53.65
83.35
56.32
93.82
24.96
60.66
92.65
85.73
46.36
81.73
54,74
ours
90.13
41.58
85.14
71.38
83.88
74.77
96.76
13,83
78.87
78.4
93.82
31.36
88.1
$1,89
Dell
MSP
81.59
64.56
80.72
68.35
80,37
70.04
$8.46
51.18
63.24
87
84.39
60.71
79.8
66.97
Energy
66.63
MahaVanilla Vanilla
82.91
78.33
81.93
78.53
$0.88
77.29
92.
55.62
76.26
90.55
89.73
62,82
83.98
73,86
86.57
59.91
76.97
73.98
77.56
87.1
87.8
67.7
VIM
83,3
75.96
81,93
74.79
80.06
74,2
92.9
49.94
75.43
89.7
89.84
62.22
83.91
71.13
maxLogit
80,26
61.38
76.12
68.45
75.44
70.18
85.23
$3.28
61.04
84.7
80.52
60.31
76.43
66.38
KLMatch
$4,14
63.37
82.34
72.91
81.56
74.76
90.8
49.64
69.7
83.5
87.56
59.26
82.68
67.24
ReAct
80.42
64. 18
77.05
700.45
75.86
72.26
84.34
61.11
64.41
82.38
80.86
62.09
77.16
68.74
ODIN
80.01
61.47
FDB
82.66
70.09
81.18
74.8
80
74.58
89.47
62.74
75.16
86.5
88.22
62,32
82.78
71.84
neco
WDiscOOD
82.91
78.24
81.93
78..5
80.87
77.27
92,14
55.59
76.25
90.6
89.74
62.78
83.97
73,83
OuTS
87.15
58.24
78:25
78.85
$4.41
DINO
SSD
50.61
97.34
$2.6
96.71
51.83
96.08
53.84
99.59
47.27
95.15
$2.82
96.27
51.5
96.86
Neco
50.65
93.85
45.33
98.02
46.98
98.06
39.77
98.44
51.56
95.25
47.51
96.52
46.97
96.69
KNN
93.8
26.51
83.82
73.04
79.63
76.29
87.76
65.14
81.32
75.45
83.6
66.84
84.99
63.88
Vanilla
96.84
13.35
84.77
92.71
ours
96.59
14.02
91.11
43.71
87.99
52.13
97.22
12.69
84,22
69.3
92.79
37.55
91.65
38.23
15

---

Improving Dut-of-Distribution Detection via Dynamic Covariance Calibration
Sensitivity with High-Variance Daca
Figure 9. Visualization of the phenomenon that Mahalanobis dis- tance may become less sensitive to OOD samples that align with the directions of high variance in ID data caused by outliers. The bold (solid) line is geodesic line without manually added out- lier points, and the dashed line is the geodesic line with the added outlier points. The two geodesic lines are same level.
16


Reviews:
[{"review_id": "TivuxxBU9S", "reviewer": "Reviewer_WNVA", "summary": "This paper proposes a dynamic covariance calibration approach for OOD detection, addressing the sensitivity of distance-based detectors to outliers in the ID data. While existing methods mitigate this issue by projecting features onto principal dimensions, they risk losing valuable ID information. Instead, the proposed method dynamically adjusts the covariance matrix using real-time input features while preserving the ID structure by only updating the residual subspace.\n\n## Update after rebuttal\n\nMost of my concerns have been answered, especially regarding the novelty of the method, which differs from other subspace approaches due to test time adaptation of the OOD criterion.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive comments and valuable suggestions. We provide detailed responses to each comment in the following.\n### Claims And Evidence\n**C1 Detailed ablation study**:\n\n**C1-Ans**: We include AUROC results in the ablation study table. To show full combinatorial results and enhance the readability, we also reformatted our ablation study table. Note that RSP (Residual Space Projection) and DCM (Dynamic Covariance Modeling) must be built on top of the DME (Dynamic Matrix Estimation) module. DCM is introduced in Section 4.3.\n|DME|RSP|DCM|CIFAR10 DenseNet (AUROC/FPR)|CIFAR100 DenseNet (AUROC/FPR) |ImageNet ViT (AUROC/FPR)| ImageNet ResNet50 (AUROC/FPR)|\n|-|-|-|-|-|-|-|\n|||| 95.19/18.86| 89.47/35.84| 92.51/33.16| 82.49/60.43|\n|✓||| 95.47/17.51| 88.05/36.61|92.65/32.31| 81.46/61.77|\n|✓|✓|| 94.08/23.92| 85.69/53.70|82.72/86.32|87.31/52.17|\n|✓|| ✓| 96.24/16.38|92.17/30.16|94.24/27.05|81.88/60.49|\n|✓| ✓|✓|**96.83**/**14.63**|**92.38**/**29.98**|**94.27**/**26.94**|**87.43**/**51.77**|\n\nAs discussed in Section 5.5, the large between-class covariance distorts the geometry with which the data can not form a dense manifold. Thus, DCM is essential on some backbones, like DenseNet. We also discussed this in detail in Section C of the appendix. As shown in the table, the DME can solely improve the performance on  CIFAR10-DenseNet and ViT. Also, the RSP is essential in ResNet50.\n### Methods And Evaluation Criteria:\n**C2 Experiments on near-OOD detection and iNaturelist**: \n\n**C2-Ans**:We have reported iNaturelist results in Tab. 9 of Appendix E. The near-OOD results can be viewed in **Reviewer YqZR's C4**\n### Experimental Designs Or Analyses:\n**C3 Include NECO and VIM in CIFAR benchmark (Tab. 1) and DINO (Tab. 3)**:\n\n**C3-Ans**: We provide the experimental results of NECO and VIM on CIFAR benchmark and the results of Residual score on DINO in the following.  \n|| CIFAR10 DenseNet (AUROC/FPR) | CIFAR100 DenseNet (AUROC/FPR) |\n|-|-|-|\n|VIM| 93.44/33.66| 87.95/51.39|\n|NECO| 94.51/27.92| 79.79/72.43|\n|Ours| **96.83**/**14.63**| **92.38**/**29.98**|\n\n||DINO(AUROC/FPR)|\n|-|-|\n|Residual (VIM)| 88.25/51.36|\n|Ours| **91.65**/**38.23**|\n\nSince DINO does not have the linear classifier as the supervised pretrained model does, for the DINO experiment, we only evaluate Residual score and discard the energy score in VIM.\n\n### Relation To Broader Scientific Literature:\n**C4**: Detailed differences and relevance of the proposed approach compared to VIM and NECO\n\n**C4-Ans**: Both methods(VIM, NECO) only consider subspace information without direction-specific adjustments, potentially ignoring part of the ID information geometry in the distance-based score. Additionally, these methods do not adaptively utilize test-time information to correct for possible distortions in the training distribution, further limiting their sensitivity to novel or shifted feature directions. We will revise this discussion in Section 3 of the manuscript. The VIM and NECO can be interpreted as the matrix-induced distance-based scores with the formula $d_M(f) = \\sqrt{(f-f_a)M(f-f_a)^\\top }$. From this perspective, NECO and VIM can be explained as these methods to find better geometry of the distance for OOD detection. However, these methods ignore the fact that the matrix $M$ or geometry may be affected by the outliers in ID distribution (Details can be viewed in R2(YqZR)'s C6(Q2)). In our proposed approach, $M$ is related to real time features $f$, denoted as $\\mathcal{M}(f)$. Specifically, $\\mathcal{M}(f) = (Cov-r_f^\\top r_f)^{-1}$, where $Cov$ is the covariance matrix from available features in the train set and $r_f$ is the projected real time features $f$ projected on residual space.  For VIM and NECO, $M = Res(Cov)$ and $M=Pri(Cov)$, with $Res(\\cdot)$ and $Pri(\\cdot)$ being the residual and principal spaces, respectively. This indicates that $M$ remains the same for both methods, regardless of any real-time features. In addition, they may ignore part of the ID information geometry in the distance-based score.\n### Other Strengths And Weaknesses\n**C5**: The name of sub-section 2.2\n\n**C5-Ans**: We have updated the subtitle of section 2.2 to Post-hoc OOD detection methods.\n\n**C6**: The parameters of the OOD criterion are fixed for any given test-sample.\n\n**C6-Ans**: The dynamic component aims to mitigate covariance distortion caused by outliers in real-time features, specifically in Mahalanobis distance. Please view **R2(YqZR)'s C6(Q2)** for clearer motivation for our method. Our method relies on the initial computation of $\\Sigma_R$ and $B$ as necessary initialization steps, while the dynamic component is provided by real-time recalibration of the covariance structure individually tailored for each test sample, significantly enhancing adaptability and robustness.\n### Other Comments Or Suggestions:\n**C7**: Typos \n\n**C7-Ans**:  Thank you for pointing out the typos in our paper. The z in s(z) should be f. We have revised these typos", "claims_and_evidence": "The essential claim of the paper is that OOD detection might be improved by only considering In-Distribution (ID) subspaces. This claim is shared with other recent papers in the literature. Here, however, the ablation shows only FPR95 results but this metric is prone to large variations. AUC is more challenging and could better indicate the impact of each component of the method. Furthermore, the overall table is difficult to read as it does not show the full combinatorial related to each component.", "methods_and_evaluation_criteria": "The benchmarks are only considering far-OOD samples and should be supplemented with near-OOD samples. In particular, CIFAR benchmark could used Tiny-Imagenet as OOD and also add the experiments: C-10 (ID) vs C-100 (OOD) and C-100 (ID) vs C-10 (OOD). For the second benchmark, results on iNaturalist could bring more insights on the method performances for near-OOD detection. For fairer comparison and broader, it would have been interesting to have SOTA results on standard benchmarks such as OpenOOD 1.5.", "theoretical_claims": "I did not find any issue in the theoretical claims.", "experimental_designs_or_analyses": "t is essential that the proposed method is compared with other subspace-based OOD detection methods namely ViM and Neco also on the CIFAR benchmark. Same goes for Tab 3.", "relation_to_broader_scientific_literature": "The proposed approach is particularly similar to ViM, which also operates on the residual subspace of the antepenultimate layer. The essential difference is the OOD score. Moreover, it shares also common ideas with necko. Differences and in particular the relevance of the proposed approach compared to these methods could be discussed in more detail.\n\nViM: Out-Of-Distribution with Virtual-logit Matching, Wang, Haoqi and Li, Zhizhong and Feng, Litong and Zhang, Wayne, CVPR 2022\n\nNECO: NEural Collapse Based Out-of-distribution detection, Mouïn Ben Ammar, Nacim Belkhir, Sebastian Popescu, Antoine Manzanera, Gianni Franchi, ICLR 2024", "essential_references_not_discussed": "Essential works are cited.", "other_strengths_and_weaknesses": "The name of related work sub-section 2.2 ``Test time OOD detection'' is misleading as it might refer to test-time adaptation. Here the authors are more concerned about post-hoc methods. \n\nThe dynamic aspect of the method is difficult to grasp. As $\\bm \\Sigma_R$ and $\\mathcal M(f)$ are computed on the training dataset, the parameters of the OOD criterion are fixed for any given test-sample.", "other_comments_or_suggestions": "* l. 266-268: class is indexed by $i$ then by $c$\n* $s(z)$ does not depends on $z$ in the right side of eq. (3)", "questions_for_authors": "No questions", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JB1w0qsQTF", "reviewer": "Reviewer_YqZR", "summary": "This paper addresses the problem of Out-of-Distribution (OOD) detection, which is critical for ensuring the reliability of AI systems.\nThe authors observe that while existing subspace-based methods use information geometry to detect OOD data, they fail to address the distortion in geometry caused by ill-distributed samples that can arise in training data. \nTo mitigate this issue, the paper proposes a novel approach that dynamically updates the prior covariance matrix using real-time input features.  This update reduces the covariance along the direction of real-time input features and constrains adjustments to the residual space. This preserves essential data characteristics and avoids unintended effects on other directions. \nThe method is evaluated on CIFAR and ImageNet datasets, demonstrating improved OOD detection across various models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's recognition of our work's **novelty**, **improved performance** demonstrated through comprehensive experiments, clear **presentation**, and its **significance** for AI system reliability. Below, we address each comment in detail.\n\n**C1**: gradient-based methods\n\n**C1-Ans**: Since the code of GradNorm[3] is the only implementation available, we compare our method with this gradient-based method. We have added these three works in Section 2\n\nDue to its computational cost, we only conduct the comparison on CIFAR benchmark. In the following, we present the results of GradNorm. \n\n|| CIFAR10 (AUROC/FPR) | CIFAR100 (AUROC/FPR) |\n|-|-|-|\n|GradNorm|92.60/24.83|79.75/63.08|\n|ours|96.83/14.63|92.38/29.98|\n\n**C2**: additional discussion on limitations and potential failure modes of Mahalanobis distance\n\n**C2-Ans**: Previous work has discussed the failure within the dimension corresponding to the smaller eigenvalues of ID covariance [1] and less effective performance when ID and OOD are similar [2]. The strong performance of our method aligns with and supports the findings in [1]. Our approach explicitly adjusts the covariance matrix within the residual space, thereby mitigating the aforementioned vulnerability regarding the discrepancy of Mahalanobis distance in residual space. In addition, our adjustment can also help improve the performance of  Mahalanobis distance in near OOD detection. Details can be viewed in the answer of **C4**. We have added this discussion in our paper.\n\n**C3(Q1)**: Detailed discussion and comparison on computational complexity \n\n**C3(Q1)-Ans**: In the following table, we provide the detection time cost comparison over 10,000 CIFAR10 instances with DenseNet as the backbone network. \n\n||Time(s)|CIFAR10(AUROC)|\n|-|-|-|\n|Feature extraction only|218.42|N/A|\n|Mahala|221.53|85.9|\n|ours|235.59|**96.93**|\n|GradNorm|473.7|92.6|\n\nWe note that our method has a comparable time cost to Mahalanobis distance method. This is because in post-hoc OOD detection, the dominant computational cost lies in the feature extraction, which is shared across most methods. The time complexity we provided is only for the feature-level operation, which is insignificant as $n$ (feature dimension) is small.  In addition, we compare with the time cost of GradNorm in the table\n\n**C4**: Near-OOD on OpenOOD benchmark\n\n**C4-Ans**:  We conduct near OOD detection on OpenOOD benchmark with ResNet18. We report the experimental results in the following table. We also compare with the results of the recent distance score and subspace score available in OpenOOD, namely KNN and VIM. All the results are averaged over 3 runs. We also provide the results of RMDS [1] with our method. RMDS is a Mahalanobis distance tailored for near-OOD detection. As the table shows, our method can effectively improve the performance of Mahalanobis distance in near OOD detection. It can also improve the RMDS, reflecting the robustness of our method in different distances in near OOD detection. \n||CIFAR10-CIFAR100||CIFAR10-Tiny||CIFAR100-CIFAR10|| CIFAR100-Tiny||avg||\n|-|-|-|-|-|-|-|-|-|-|-|\n||FPR|AUROC|FPR|AUROC|FPR|AUROC|FPR|AUROC|FPR|AUROC\n|VIM|52.33|87.03|44.16|88.88|70.79|72.15|54.92|77.73|55.56|81.46|\n|KNN|38.76|**89.59**|30.89|**91.51**|72.69|77.1|49.68 | **83.29**|48.01|85.37|\n|mahala|64.51|79.48|59.06|80.71|88.88|54.6|80.75| 60.31|73.30|68.78|\n|mahala+dynamic(ours)|61.38|84.73|52.95|86.54|75.24|71.72|60.35|76.34|62.48|79.83|\n|RMDS| 49.76| 88.26| 37.63| 90.29| 62.90| 77.69| 49.55| 82.60|49.96|84.71|\n|RMDS+dynamic(ours)|**38.70**|89.42|**30.47**|91.45|**61.7**|**78.21**|**48.85**|82.81|**44.93**|**85.48**|\n\n**C5**: typos\n\n**C5-Ans**: Thank you for pointing out this typo. We have revised it in our paper.\n### Questions\n**C6(Q2)**: Visualization of the phenomenon that Mahalanobis distance may become less sensitive to OOD samples that align with the directions of high variance in ID data caused by outliers\n\n**C6(Q2)-Ans**:  **Visualiztion**: https://i.imgur.com/U4rrTTt.png We visualise this phenomenon in 2D space. Note that, as the features are in the spaces with higher dimensions, the situation may be more complicated than our demonstration. Mahalanobis distance can be considered as a geodesic distance of normal distribution, thus we utilize the geodesic line within the randomly sampled normal distribution (ID data) to demonstrate this phenomenon. As the figure shows, the geodesic line may be made further from the center point by the outlier points.  It is obvious that the OOD points, aligned with outlier points, can be closer to the dashed line than the bold line, which makes the distance between OOD points and ID center smaller. \n\n[1] Ren, Jie, et al. A simple fix to Mahalanobis distance for improving near-ood detection.\n[2] Tajwar, Fahim, et al. No true state-of-the-art? OOD detection methods are inconsistent across datasets.\n[3] Huang, Rui, et al.\"On the importance of gradients for detecting distributional shifts in the wild.", "claims_and_evidence": "The central claim that dynamically adjusting the prior covariance matrix improves Out-of-Distribution (OOD) detection is supported by the experimental results. The authors consistently show improved performance on CIFAR and ImageNet datasets across various models when compared to mahalanobis distance specifically. The ablation studies further reinforce the importance of each component of their method (dynamic adjustment, residual space projection, and deviation features).", "methods_and_evaluation_criteria": "The proposed method is reasonable for the problem of OOD detection. The use of covariance matrix adaptation to capture and adjust information geometry is a sound approach. The evaluation criteria (AUROC and FPR95) and benchmark datasets (CIFAR and ImageNet) are standard and appropriate for assessing OOD detection performance\n\n\n **Experimental Design and Analysis:** The experimental design is comprehensive. The authors compare their method with a wide range of existing OOD detection techniques, providing a strong baseline for evaluation. They also perform ablation studies and analyze the behavior of key variables (p, q, and vector norms). The analysis is generally sound and provides strong support for their claims.", "theoretical_claims": "The primary theoretical claim in the paper is presented in Theorem 4.2.  This theorem provides the conditions under which the proposed dynamic distance metric is valid.    \n\nTheorem 4.2: Given a feature f, a non-zero feature a, and a symmetric positive definite matrix Σ, the theorem defines variables p, q, and s based on these inputs and Σ.  It then states that under certain conditions involving p, q, and s, the proposed distance metric d(f) is greater than or equal to 0.  \n\nThe proof appears to be mathematically sound and follows a logical progression.", "experimental_designs_or_analyses": "**1. Experimental Designs**\n\n* **OOD Detection Benchmarks:** The authors use two standard OOD detection benchmarks: CIFAR (CIFAR-10/CIFAR-100) and ImageNet-1k. These are widely used datasets in the OOD detection community, making the evaluation relevant and comparable to other work. They include a variety of OOD datasets to test the generalization of their method. \n   \n* **Comparison with State-of-the-Art:** The authors compare their method with a diverse set of post-hoc OOD detection approaches, including probability-based, logit-based, density-based, distance-based, and subspace-based methods. This comprehensive comparison provides a strong baseline for evaluating the effectiveness of their proposed approach. The only family of methods that's missing is gradient based methods (for eg. GradNorm[1], GradOrth[2], GROOD[3])\n   \n* **Ablation Study:** The authors conduct a thorough ablation study to analyze the contribution of different components of their method. They systematically remove key components (residual space projection, real-time adjustment, and deviation features) and evaluate the impact on performance. This helps to demonstrate the importance of each component. \n   \n* **Analysis of Residual Dimensionality:** The authors analyze how the performance of their method varies with different dimensions of the residual space. This analysis provides insights into the sensitivity of the method to the choice of residual space dimensionality. \n   \n* **Analysis of p and q Values:** The authors analyze the values of p and q, which are important parameters in their proposed method. This analysis provides empirical support for the theoretical claims made in the paper.\n   \n* **Euclidean Distance Experiments:** The authors conduct additional experiments using Euclidean distance to demonstrate the general applicability of their approach to different distance metrics. \n\n**2. Soundness and Validity**\n\n* The experimental designs appear to be generally sound and well-justified. The authors use appropriate datasets, evaluation metrics, and comparison methods.\n   \n* The ablation studies are particularly strong, providing clear evidence for the contribution of each component of the proposed method.\n   \n* The analyses of residual dimensionality, p and q values, and vector norms provide valuable insights into the behavior of the method and support the theoretical claims.\n   \n* The inclusion of Euclidean distance experiments and hard OOD detection further demonstrates the robustness and generalizability of the approach.\n\nReferences:\n1- Huang, Rui, Andrew Geng, and Yixuan Li. \"On the importance of gradients for detecting distributional shifts in the wild.\" Advances in Neural Information Processing Systems 34 (2021): 677-689.\n2- Behpour, Sima, et al. \"Gradorth: A simple yet efficient out-of-distribution detection with orthogonal projection of gradients.\" Advances in Neural Information Processing Systems 36 (2023): 38206-38230.\n3- ElAraby, Mostafa, et al. \"GROOD: GRadient-aware Out-Of-Distribution detection in interpolated manifolds.\" arXiv preprint arXiv:2312.14427 (2023).", "relation_to_broader_scientific_literature": "The key contribution of this paper is a novel approach to Out-of-Distribution (OOD) detection that dynamically adjusts the prior matrix using real-time input features. This contrasts with prior work that relies on static estimations of geometry from ID data.\n\nSpecifically, the paper builds upon distance-based OOD detection methods, including those using Mahalanobis distance, which utilize a covariance matrix derived from ID data to capture information geometry.. The authors identify a limitation in these methods: they often neglect the distortion of information geometry caused by outlier features in the training data.\n\nThe paper also relates to subspace-based methods that project features onto the subspace of ID data. While these can be seen as using matrix-induced distance scores, the authors argue that simply replacing the covariance matrix may remove important ID information and doesn't provide targeted adjustment for OOD directions. Their dynamic adjustment approach refines the prior information geometry using local information from real-time features. Additionally, they constrain adjustments to the residual space of the training distribution, building on the idea that OOD features may exhibit more energy in this space.", "essential_references_not_discussed": "The paper effectively discusses relevant prior work in OOD detection, covering distance-based, subspace-based, and other categories. However, one area where additional discussion could provide valuable context is the limitations and potential failure modes of Mahalanobis distance. For example, [1] discusses that MSP baseline outperforms Mahalanobis when ID and OOD distributions are very similar can the adjustment help with that problem.\n\nReferences:\n* [1] Ren, Jie, et al. \"A simple fix to mahalanobis distance for improving near-ood detection.\" arXiv preprint arXiv:2106.09022 (2021).\n* [2] Tajwar, Fahim, et al. \"No true state-of-the-art? ood detection methods are inconsistent across datasets.\" arXiv preprint arXiv:2109.05554 (2021).", "other_strengths_and_weaknesses": "**Strengths:**\n\n* **Originality:** The paper introduces a novel perspective to distance-based OOD detection by dynamically adjusting the prior geometry using real-time input features. This approach is a departure from traditional methods that rely on static estimations of the information geometry from the training distribution. The idea of refining the covariance matrix in real-time and constraining adjustments to the residual space to preserve essential ID characteristics is an original contribution.\n   \n* **Significance:** OOD detection is a crucial problem for the trustworthiness and reliability of AI systems, especially in safety-critical applications. The paper addresses a significant limitation of existing methods and proposes a solution that improves OOD detection performance across various models and datasets. This has the potential to contribute to safer and more reliable AI deployments.\n   \n* **Clarity:** The paper is generally well-written and easy to follow. \n\n**Weaknesses:**\n\n* **Computational Complexity:** The paper mentions that the computational complexity of their method is O(n^3), where n is the feature size. While the authors argue that the inference cost is affordable, a more detailed discussion of the computational trade-offs compared to other methods would be beneficial. \n\n* **Evaluation against Near-OOD** The test pairs used for testing against other OOD methods are mostly from Far-OOD, it would be beneficial to see how it performs against near-OOD data as per OpenOOD benchmark.", "other_comments_or_suggestions": "* **Line 110**: I guess we have type in select import neurons should be select improtant neurons", "questions_for_authors": "- The paper mentions that the computational complexity of the method is O(n^3) (line 700). While the authors state that the inference cost is affordable, could they provide a more detailed comparison of the computational cost with other OOD detection methods?\n\n- The paper argues that Mahalanobis distance may become less sensitive to OOD samples that align with the directions of high variance in ID data caused by outliers. Could the authors provide more specific examples or visualizations to illustrate this phenomenon? A clearer illustration would strengthen the motivation for the proposed method.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b2lomLZ5Am", "reviewer": "Reviewer_6R2e", "summary": "To reduce redundant information in the covariance matrix in real-time OOD detection, this work proposes adjusting prior geometry based on the input, enhancing sensitivity to OOD samples while preserving essential information for ID classification.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's positive recognition of our work, particularly regarding the novelty and reasonableness of the idea, strong performance, and clarity of supporting evidence. Below, we provide detailed, point-by-point responses addressing each comment.\n### Other Strengths And Weaknesses:\n**C1**: Why the baseline methods provided in CIFAR benchmark and ImageNet benchmark.\n\n**C1-Ans**:\nThe reason that the compared methods are different in CIFAR and ImageNet benchmarks is that we can not access the hyperparameter settings in the absent methods. To address the author's concern, we reimplement and do hyperparameter search on VIM and NECO in CIFAR benchmark. In addition, we add a gradient-based method in this setting, namely GradeNorm [2]. \n\n|           | CIFAR10 DenseNet (AUROC/FPR) | CIFAR100 DenseNet (AUROC/FPR) |\n|-----------|------------------------------|-------------------------------|\n| GradNorm  | 92.60/24.83                  | 79.75/63.08                   |\n| VIM       | 93.44/33.66                  | 87.95/51.39                   |\n| NECO| 94.51/27.92                  | 79.79/72.43                   |\n| ours      | 96.83/14.63                  | 92.38/29.98                   |\n\n**C2**: Can we make sure this discrepancy gets improved with projection, compared with feature norm without normalization.\n\n**C2-Ans**: Thank you for the insightful question. We have added the discussion about the observation of NAN[1] in our experiment. In the following table, we show the results w and w/o projection to illustrate the effectiveness of the projection.\n|                | ImageNet-ViT(AUROC/FPR) |ImageNet-ResNet50(AUROC/FPR)|\n|-------------------|---------------|----------------|\n| w/o projection| 94.24/27.05        |81.88/60.49|\n| w projection  | 94.27/26.94      |87.43/51.77|\n\nIn addition, we use the norm of the projected feature as the OOD score to answer this question, shown in the following table. The results in the table imply that normalization can help improve the discrepancy. We present our explanation of this phenomenon in the following \n\n|  |ImageNet-ViT(AUROC/FPR) |ImageNet-ResNet50(AUROC/FPR)|\n|-------------------|---------------|----------------|\n| w/o normalizaiton| 92.34/34.80        |62.27/81.95|\n| w normalizaiton  | 92.80/33.40        |83.26/59.74|\n\n\nThe $L_2$ norm of the feature can also be interpreted as the distance from the feature to the zero point. If the features in different semantic distributions have significantly different scales and the center of ID features is far from the zero point, the norm of the features makes it hard to detect the OOD sample very well, since the zero point may also be an OOD point. However, the normalized features are distributed on the sphere with radius 1 and the zero point as center. As long as the class clusters are not distributed too close, the center of normalized features can be easily near the zero points, thus making the norm of the features have geometrical meaning for OOD detection. If the center of the features is the zero point, the center of the projected features is also the zero point. We assume this is the reason that the normalization can help improve the discrepancy and also strengthen the neural collapse phenomenon\n\n[1] Park, Jaewoo, et al. \"Understanding the feature norm for out-of-distribution detection.\" Proceedings of the IEEE/CVF international conference on computer vision. 2023.  \n[2] Huang, Rui, et al.\"On the importance of gradients for detecting distributional shifts in the wild.", "claims_and_evidence": "Most claims are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "I think their method is reasonable, and the evaluation criteria used for OOD detection are appropriate.", "theoretical_claims": "I did not notice any clear errors in their theoretical claims.", "experimental_designs_or_analyses": "The final results show that their method effectively improves OOD detection. However, the experiment in Section 5.8 does not convincingly demonstrate that their method increases the discrepancy in feature norms between ID and OOD samples.", "relation_to_broader_scientific_literature": "Since Mahalanobis distance [Lee et al.] demonstrated the effectiveness of incorporating information geometry in designing distance-based scores, more studies have attempted to capture more precise geometry matrices $M$.  Wang and Ammar integrate information from residual space and principal space to achieve this but remain limited by static geometry. This work introduces input based projection matrix to address this limitation.\n\nLee, Kimin, et al. \"A simple unified framework for detecting out-of-distribution samples and adversarial attacks.\" Advances in neural information processing systems 31 (2018).\n\nWang, Haoqi, et al. \"Vim: Out-of-distribution with virtual-logit matching.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\n\nAmmar, Mouïn Ben, et al. \"Neco: Neural collapse based out-of-distribution detection.\" arXiv preprint arXiv:2310.06823 (2023).", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths:\n1. This work proposes adapting the prior geometry based on the input, improving sensitivity to OOD samples while retaining essential information for ID classification.\n2. Their idea is reasonable, and their implementation is novelty.\n3. Experimental results show that their method outperforms SOTA.\n\nWeakness.\n1. Why the baseline methods provided in CIFAR benchmark and ImageNet benchmark. \n2. The authors give evidence in 5.8 that their projection based on normalized features results in a larger norm for OOD and smaller norm for ID. However, this phenomenon has been widely observed [Park et al.] Can we make sure this discrepancy gets improved with projection, compared with feature norm without normalization.\n\nPark, Jaewoo, et al. \"Understanding the feature norm for out-of-distribution detection.\" Proceedings of the IEEE/CVF international conference on computer vision. 2023.", "other_comments_or_suggestions": "1. STA in Line 67.", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: In the intial review round, this paper received two Weak Accepts and one Weak Reject. Reviewers' opinions were diverse. They recognized several strengths, including novelty (YqZR, 6R2e), effectiveness of the method (YqZR, 6R2e), theoretical basis (YqZR), and clarity of the paper (YqZR). However, they also raised several concerns, such as computational time (YqZR), lack of experiments and comparisons (WNVA, YqZR), lack of analysis (WNVA, YqZR, 6R2e), and lack of novelty (WNVA).

The authors submitted a rebuttal, which was acknowledged by all reviewers. The authors did an excellent job and successfully addressed the concerns mentioned above. In particular, Reviewer WNVA initially questioned the novelty based on the overlap with ViM and NECO. In their rebuttal, the authors emphasized the use of test time inputs. Reviewer WNVA was convinced by the rebuttal and updated the score to Weak Accept.

Reviewer 6R2e was concerned about the small effectiveness of the projection when using ViT. The authors tried to justify this on the grounds that the ViT latent space as a whole already well separates IDs and OODs. However, given that ViT is the standard for classification and other tasks, this may be a limitation that should be discussed. This is also supported by the fact in the ablation study that introducing the projection (RSP) alone does not improve performance, except when ResNet50 is used. In a related matter, when further combined with DCM, the results suddenly improve, which is somewhat difficult to interpret. The authors are encouraged to discuss of these points in the final version.
Decision: Accept (poster)