What If the Input is Expanded in OOD Detection?
Authors: Boxuan Zhang
Jianing Zhu
Zengmao Wang
Tongliang Liu
Bo Du
Bo Han
Venue: NeurIPS 2024 poster (2024)
Keywords: Out-of-distribution Detection
Trustworthy Machine Learning
Primary area: safety_in_machine_learning

Abstract: Out-of-distribution (OOD) detection aims to identify OOD inputs from unknown classes, which is important for the reliable deployment of machine learning models in the open world. Various scoring functions are proposed to distinguish it from in-distribution (ID) data. However, existing methods generally focus on excavating the discriminative information from a single input, which implicitly limits its representation dimension. In this work, we introduce a novel perspective, i.e., employing different common corruptions on the input space, to expand that. We reveal an interesting phenomenon termed *confidence mutation*, where the confidence of OOD data can decrease significantly under the corruptions, while the ID data shows a higher confidence expectation considering the resistance of semantic features. Based on that, we formalize a new scoring method, namely, *Confidence aVerage* (CoVer), which can capture the dynamic differences by simply averaging the scores obtained from different corrupted inputs and the original ones, making the OOD and ID distributions more separable in detection tasks. Extensive experiments and analyses have been conducted to understand and verify the effectiveness of CoVer.

What If the Input is Expanded in OOD Detection?
Boxuan Zhang
Jianing Zhu
Zengmao Wang't
Tongliang Liu
Bo Du'
Bo Han 2,4
School of Computer Science, Wuhan University
TTMLR Group, Department of Computer Science, Hong Kong Baptist University SSydney AI Center, The University of Sydney "RIKEN Center for Advanced Intelligence Project
(boxzhang1005, wangzengmao, dubo)@whu edu. cn {cajnzhu, bhanm1 hkbu. edu.. hk tongliang www.syddeeyy...m edu.
Abstract
Out-of-distributtio (OOD) detection aims identify OOD inputs from unknown classes, which is important for the reliable deployment of machine learning models in the open world. Various scoring functions proposed distinguish it from in-distribution (ID) data. However, existing methods generally focus on excavating the discriminative information from single input, which implicitly limits its repre- sentation dimension. In this work, we introduce a novel perspective, i.e., employing different common corruptions the input space, expand that. We reveal interesting phenomenon termed confidence mutation, where the confidence of OOD data can decrease significantly under the corruptions, while the ID data shows a higher confidence expectation considering the resistance of semantic features. Based on that, we formalize a new scoring method, namely, Confidence aVerrgge (CoVer), which can capture the dynamic differences by simply averaging the scores obtained from different corrupted inputs and the original ones, making the OOD and ID distributions more separable in detection tasks. Extensive experiments and analyses have been conducted understand and verify the effectiveness CoVer. The code is publicly available at: https:// htt///ggtttttt com/tmlre group/CoVer.
Introduction
Out-of-distribution (OOD) detection [23, 28, 44] is important for reliable machine learning model deployment in open-world scenarios, where various samples from unknown classes, i.e., OOD data, are constantly emerging [4]. Deep neural networks [20] (DNNs) are demonstrated to be overconfident about these OOD data, which may result disasters for some safety-critical applications [5, 21]. Traditional OOD detection methods [23, 29, 28, 30, 45, 46, 1, 61] design various scoring functions based on the outputs representations extracted from well-trained models. Recently, some research also extended it into zero-shot setting [3 I]. which leverages the multi-modi information based on vision-language models (VLMs) and requires no further training on distribution (ID) data. A series of methods [48, 36, 52, 26] are proposed for improving OOD detection based on such advances.
Although promising progress has been achieved, existing methods mainly focus excavating the diseriminative information of a single input. For instance, ReAct [45], DICE [46], and ASH [1] integrates the activation regularization reshaping to the forward path of single input in single- modal DNNs: MCM [31] characterizes the confidence of a single input by the similarity between visual features and text representation of ID classes VLMs. However, specializing in single input
"Equal Contribution
*Correpondence to Zengmao Wang (wangzengman @wh..edu.cmm
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Original Input
Corrupted Input
Multiple Inputs (CoVer)
000
OCO
oCo ID
12000
1230
1200
1094
1030
1300
Deniiy
Deniiy
Density
600
ex0
AUS CABUE CEELIO
OOCEOO KAETTE LAGLES CABLEE BERDNEESUS
6 EADIES CASIES BRESS EEIDEEEEEEE SIUTESIIAE
Score
AUROC: 94.63
FPR95: 30.70
AUROC: 91.30
FPR95: 44.85
AUROC: 96.37 1
FPR95: 20.10
Figure I: Comparison of scores distributions and detection results with different inputs for representa- tion dimension expansion. Left panel: results with a single original input; Middle panel: results with a single corrupted input, which perform worse but have mutated scores for some OOD samples (see Figure 2); Right panel: results with multiple inputs (CoVer), which achieve the variance reduction for the ID distribution and perform a better ID-OOD separability (see Figure 3 for more explanations).
may implicitly constrain the representation dimension for detection, leaving some OOD samples with features similar to samples fail to be identified (refer to the distribution overlap in the left panel of Figure 1). Therefore, it naturally motivates the following critical research question: Can we expand the dimension of the input space explore OOD discriminative representations?
In this work, we introduce a novel perspective to investigate that, i.e., employing the common corruttion [22] in the input space. Through systematical comparison, reveal an interesting phe- nomenon termed confidence mutation, where the confidence of OOD data can decrease significantly under the corruptions, while ID data shows higher confidence expectation considering different input dimensions. Specifically, as shown in Figure 1, corrupted inputs result in lower confidence in both OOD and ID data. However, one critical dynamic discovery is that its confidence about overconfident OOD data is changed more than the unconfident ID data under the same corruptions (refer to Figure 2). indicating a natural difference in feature-level resistance of the originally overlapped parts (refer to Figure 3). With the original inputs, we can find that the model is overall more confident in ID data.
Based on the above, we propose new scoring framework, namely, Confidence aVerage Ver), as illustrated in Figure 4. At the high level, we expand the original single representation dimension into multiple ones to excavate discriminative information. In detail, we introduce simple but effective method for identifying OOD data with confidence mutation, which can be formalized as an average of OOD scores (e.g., Eq. (6)) obtained by different corrupted inputs and the original one. With the expectation among multiple input dimensions, CoVer can effectively reflect the knowledge of invariant semantic features that are discriminative from ID data to OOD data. also matches an intuition that ID data can be more likely recognized as high confidence by models considering different input views, especially with the corruptions affecting the non non-semantic figh-frequenyy parts.
We conducted extensive experiments to verify the effectiveness of our proposed method. Since CoVer an input-side design compatible with single-modal and multi-modal networks, we adpot various henchmarks for DNN-based and VL.M-bseed OOD detection tasks. Under extensive evaluations, our Co can achieve the superior performance compared with different baselines. Moreover, CoVer exhibits excellent compatibility, as evidenced by the better performance of some methods combined with Co Ver. Finally, a range of ablation studies of the scoring framework and further discussions from different perspectives are provided. In summary, our main contributions can be listed as follows,
Conceptually, we introduce a novel perspective for identifying OOD inputs by considering the common corruptions expand the representation dimensions. (in Section 3.1)
We reveal an interesting phenomenon termed confidence mulation, where the confidences of OOD data can vary to significantly lower than ID data under corruptions (in Section 3.2).
Technically, we formalize novel scoring method, namely, Confidence aVeruge (Co Ver), simple average of the confidence estimated from extended corrupted inputs and the original one. The corresponding empirical analysis is presented to understand it (in Section 3.3).

---

Origital
Original
Corruptol
Currapted
Color
Coler
Scru Ditribaticn
OGD Soure BierBration
1.20
Contident
1250
Jncertident
3.50
Overcom dent gue
Uncerrident 000
1230
Cangles
12:0
Canalis
Conuated
1.10
1.300
Sport
1:00
1::0
0.00
Camupied
200
400
Index
50
130
400
430
Figure 2: Demonstration about detailed explanations for the discovery illustrated in Figure 1. The ID and OOD data here are divided into four groups, i.e., Confident ID, Unconfident ID, Overconfident OOD, and Unconfident OOD. First Row: the variation of confidence scores for ID and OOD data before and after being corrupted. The critical difference lies in the greater confidence declination for overconfident OOD data compared to unconfident ID data. (see Figure 3 for further discussion). Second Row: scatter maps of confidence scores sampled from the four groups under the same corruption, statistically supporting the findings of the first row. See Appendix C.2 for more details.
Empirically, extensive experiments on both traditional and zero-shot OOD detection bench- marks have verified the effectiveness and compatibility of our CoVer, and we conduct various ablations and further discussions to provide a comprehensive analysis (in Section 4).
Preliminaries
In this section, we briefly introduce the preliminaries of OOD detection on basic setups and the advanced zero-shot setting on VLMs. For related works, we leave detailed discussions in Appendix B.
Problem setups. Let X be the input space and y = (.,,.... VK } be the label space, where K is the number of ID classes. Given the ID random input x; X' and OOD random input x. X, consider the ID marginal distribution DID and those with the OOD marginal distribution DOOD, irrelevant distribution of which the label set has no intersection with ) The goal of OOD detection is to figure out inputs with the OOD distribution DID from those with the ID distribution DOOD, which can be considered OOD detection, given a model f trained on ID) data with logit outputs, a score function S(.) and a threshold X, the detection model g()) can be defined as,
where Doon is defined
binary classification problem. For traditional
(1) where r is detected as ID data if and only if S(x) 2 X: otherwise, is rejected as OOD data that should not be predicted by the model f. Designing practical S(x; f) is crucial for OOD detection.
(x(x)= ID, S(x; f) 2 X; otherwise, (x(x) OOD.
CLIP-based vasion-language models CLIP [42] has shown impressive performance in the zero- shot classification task by profiting from massive amounts of training data and large-size models. Here we briefly review the mechanism of CLIP-based VLMs. A CLIP-based model usually contains an image encoder pimage and text encoder next, Given random input DTD and a label y ), we use fimage and piext to extract the image features h E Rd and the text features ej E Idd as follows:
pimage (x), e,- = (p())) Vj 1,
(2)
where p(-) refers to the prompt template for the input label, d the embedding dimension. The predictions are formulated as the consine similarity between the image features h and text features ej.
arg max (cos(h, e,)),
where ej = p p (p(vs)).
3

---

Raw
PPT
Low Preq
High Freq
Raw
FFT
Low Freq
High
Original
Uncoafilet ID
Original
Ovncnniitee DOD
Corrupted
Corrupted
Corrupted Original
Uncoafiden
Original
Ovvecoafieent DOD
Corrupedd
Orikkel
Conmantd
Figure 3: Visual exploration of random unconfident ID samples and the confidence mutation exempli- fied on random overconfident OOD samples under the same corruption. For each original input and its corrupted variant, we leverage the Fast Fourier Transformation extract their low-frequency and parts. Left panel: visual investigation on unconfident ID samples with ID semantic features low-frequency levels that are resistant to corruptions. Right panel: an intuitive comparison of overconfident OOD samples, whose confidences show significant changes due to the elimination of non-semee non-semantic features the high-frequenge level. See Appendix C.4.2 for more detailed analyses.
Zero-shot OOD detection Different from traditional OOD detection methods based classifier well-trained on single-modal, recent zero-shot OOD detection studies [31, 26] leverage a pre-trained VLM-based model (e.g. CLIP [42]) without any fine-tuning ID training data. The text features from given ID label names (i.e. ID classes) as the blass-wise weights functionally play the same role the classifier. With guaranteed ID classification accuracy, the primary goal of zero-shot OOD) detection in this paper is to distinguish OOD samples that do not belong to any known ID classes
CoVer: Confidence Average
In this section, we formally present our proposed scoring framework, i.c., Confidence aVeragge (CoVer). First, we introduce the motivation of representation dimension expansion and present the notable discovery (Section 3.1). Second, conduct the exploration for the confidence mutation of overconfident OOD data considering the corrupted inputs (Section 3.2). Lastly, we provide the detailed implementation and analysis of our formalized CoVer score (Section 3.3).
3.1 Representation Dimension Expansion
DNNs are demonstrated to be overconfident on those OOD samples, and a series of works [23, 45, 1, 54, 31, 26] are dedicated eliminating the effects through the perspective of feature representation. However, achieving that is demonstrated to be hard as it generally requires careful optimization [54], or additional prior knowledge [36] on the single input. illustrated in the middle panel of Figure adopting some agnostic corruptions on the single input may result in worse separability between the ID and OOD distribution. Specializing in single input seems to implicitly constrain the representation dimension for detection. In this work, we naturally raise the following question,
What if we expand the dimension of representation for the original inputs to enhance OOD discriminative representations?
Using the same corruption adopted in Figure we can conduct the dimension expansion by simulta- neously considering both the corrupted variant and the original input. One notable discovery is that considering multiple inputs can achieve better performance on OOD detection, even though the single corruption transformation brings negative effects identifying OOD samples. The surprising com- parison results attract us to further explore the underlying mechanism of expanding the representation dimension for the original input, especially the dynamics before and after adopting the corruptions.

---

3.2 Confidence Mutation under Corruptions
Although employing corruption on the single inputs leads to worse ID-OOD separability, we can find obvious shifts toward less confidence in both OOD and ID distributions. It is expected for OOD data that corruption can help the model mitigate overconfidence, while the ID data are also affected severely and enlarge the overlap on the single dimension. In contrast, considering the multiple inputs by averaging the confidence scores shows variance reduction for ID distribution, which indicates the distinguishable dynamics between II) and OOD data. To elicit the underlying mechanism, we provide a definition to characterize the change of model confidence on the inputs under corruption.
Definition 3.1 (Confidence Difference). Given well-trained model I and score function S(-) measuring the confidence of f on an input x, we have a basic static to characterize the differences between the original input and that under a corruption c(.): MU,(I, S. f) - (S(x; f) f) S(c(x); f))-
Based on the comparison in Figure I, we divide the ID and OOD data into four groups according to the model confidence on their original inputs, and present an overall comparison of the confidence differences in Figure 2. We reveal the critical dynamic differences in the corrupted variants of ID and OOD data, where both large MU(x, s, f) exist in the data whose natural inputs own higher confidence in each part, demonstrating the model confidence on the overconfident OOD data decrease more than the unconfident ID data under the same corruption. We can get the empirical observation,
Observation 3.2 (Confidence Mutation). Given the overconfidence OOD inputs To DoOD, we can observe more significant differences in the change of confidences under the same corruption c(-) than the ID samples 2; Din with similar model confidence (constrained by e) on the natural inputs,
EnD-D (MU,(Wi, S)) < Bmn-PoOD (MU Sx; S)).
In Figure 3, we further visualize the samples of unconfident ID data and overconfident OOD data. Under the comparison of saliency maps and the Fast Fourier Transformation, we find the confidence mutation reflects the feature level vulnerability of OOD data compared with ID data. Intuitively, the former can be severely affected by the common corruption to eliminate the non-semantic features, which generally exist at the high-free geh-frreuency level. In contrast, the semantic feature of unconfident II) data can maintain confidence as the limited effects of corruption on the low- now-frequency part.
Observation 3.3 (Resistance of ID features in frequency views). Assuming that ID) data owns the ID semantic features existing at the low-frequency level (extract by gg while the OOD data has some non-semantic features at the high-frequency level for activating the high confidence of the model on its prediction, we can observe the following empirical relation on adopting the same corruptions,
(I, x (I.((f(T(((())))IIfFFC
(5)
where I indicates the Fourier transformation. We suggest that common corruptions [22] might act as perturbations of gigh-freueency features within the input representation. For OOD samples, which inherently lack ID semantic features, altering sighffreueecc features could potentially lead to notable changes in model confidence, while the ID data shows relatively better resistance on it (see the left panel of Figure 3). This observation tentatively supports the notion that ID data maintains an overall higher confidence expectation under conditions of expanded representation dimension. To validate its generality, additional results involving various common corruptions are presented in Appendix C.2.2.
3.3 Scoring Function Implementation and Analysis
Based on the previous understanding of confidence mutation, we formalize our CoVer, a new scoring framework for OOD detection. The procedure of Co Ver mainly contains the following four parts as illustrated in Figure 4, and the final averaged multi multi-dimensional scores can be provided as follows,
Scover EEwa(xx, max
E/ e500// d(x; = (a, cx X,
(6)
where ( is the confidence expecation over all scores dimensions, K is the number of ID classes, T is the temperature coefficient of the softmax function. In the following, we detailedly introduce the specific operations obtain the final Scover and the corresponding notations.
To enlarge the dimension of the original single input for confidence average, we introduce various corrupted inputs. In this work, we employ the corruption functions defined in [22], which consists
5

---

Raw Input
Original
Cormupted
<=D
S(x)
N+1
+
Conrzn
Hotoo
Pretrained Model
Score Functions
Masimum Sofimas
Image
Frre Energy Score
CoVer
Chsitier
CLIEN Score
Zace
Seat
NogLabel abe Score
Midde
Foider
Siro)
S(r)
((r))
Score Distribution
Figure 4: Overview of CoVer. Left panel: visualization of the raw input and inputs w.r.t different corruptions; Left- aeft-midde panel: procedures of logit outputs from single-modal modal and multi- modal networks; Right-middle panel: scoring functions that equip each dimensional output with an OOD score; Right panel: realization of CoVer by averaging OOD scores obtained from multiple dimensions.
total of 90 distinct corruptions. We provide the visualization these different corruptions in Appendix C.4.1. Given the input space X' and a of corruption functions C, the corrupted inputs can be formulated (c(x))(x E X, e C) X,, resulting in the multi-dimensional input spaces d(X') X).
Given input image the feature the original dimension ho and features of corrupted dimensions h1 .... y. Then we predict the logit output s(x) for each dimensional feature ha, vd 0, l.... N. For the DNN-based model f, the outputs of these features are denoted as s(x) = (z) = logits,- For the VLM-based model, the outputs are label-wise matching scores based on the cosine similarity: 8y(x) I[k Flle|ll
d(X x), we adopt image encoder with fixed parameters extract
ha-e;
For the logit output s(x) predicted from specific input dimension, assign it with OOD score to implement one dimension of the CoVer score. As shown in the right-middle panel of Figure the OOD score can be formalized by some traditional scoring functions, like the softmax scoring function [23] (refer to Eq. (6)) and the free energy scoring function [30]. In addition, the OOD score can also be formulated by variants of some novel scoring functions, like those in CLIPN [52] and NegLabel [26]. The detailed implementations for alternative scoring functions can be found in Appendix C.3.
4 Experiments
In this section, we present the comprehensive verification of the proposed CoVer in the OOD detection benchmarks. First, we introduce several critical parts experimental setups (in Section 4. 4.1). Second, we provide the performance comparison and compatibility experiments of our CoVer with various DNN-based and VI.M-based OOD detection methods (in Section 4.2). Third, we conduct extensive ablation studies and further discussions to understand the properties of our CoVer (in Section 4.3).
4.1 Experimental Setups
In this part, we present the critical parts of experimental setups and leave more details in Appendix C.
Datasets. Following previous work [1, 31], we adopt the ImageNet- 1K OOD benchmark [24], which uses the ImageNet-1K: [14] as ID data and iNaturalist [49], SUN [55], Places [60], and Textures [7] as OOD data. For each of the OOD datasets, same as MCM [31], we also use subsets of ImageNet- KK for fine-graines analysis, like ImageNet- 10 that mimics the class distribution of CIFAR-10 but with high-resolution images. For hard OOD evaluation, we exploit ImageNet-20 with 20 categories similar to ImageNet- 10 in the semantic space
classes not overlap with the dataset. As

---

Table 1: Comparison with competitive OOD detection baselines based on ResNet-50. The ID data are ImageNet- K. indicates larger values are better and indicates smaller values are better.
OOD Dataset
Method
INaturalist
SUN
Places
Textures
Average
AUROCT
FPR95_
AUROC1
FPR95,
AUROC1
FPR95L
AUROCT
FPR95]
AUROCT
FPR95L
MSP
87.74
54.99
80.86
70.83
79.76
73.99
79.61
68.00
81.99
66.95
ODIN
91.37
41.57
86.89
53,97
84.44
62.15
87.57
45.53
87.57
50.80
Mahalanobis
$2.65
97.00
42.41
98.50
41.79
98.40
85.01
55,80
55.47
87.43
Energy score
89.95
55.72
85.89
59,26
82.86
64,92
85.99
53.72
86.17
58.41
ReAct
96.22
20.38
94.20
24.20
91.58
33.85
89.80
47.30
92.95
31.43
DICE
94.49
25.63
90.83
35.15
87.48
46.49
90.30
31.72
90.77
34.75
DICE+ReAct
96.24
18.64
93.94
25.45
90.67
36.86
92.74
28.07
93.40
27.25
ASH-B
94.25
28.95
90.32
40.21
87.52
49.52
91.53
33.48
90.91
39.04
ASH-B CoVer
97.14
14,04
94.12
25.77
91.05
35,93
91.93
30.39
93.56
26.53
ASH-S
97.88
11.38
94.04
27.96
91.03
39.74
97.62
11.88
95.14
22.74
ASH-S CoVer
98.33
8.73
94.59
26.63
91.47
38.06
97.22
13.92
95.40
21,83
Table 2: Comparison with competitive OOD detection baselines based on CLIP-B/16. The ID data are ImageNet-IK. indicates larger values are better and indicates smaller values are better.
OOD Dataset
Method
INaturalist
SUN
Places
Textures
Average
AUROC1
FPR95
AUROCT
FPR95
AUROCT
FPR951
AUROC1
FPR95
AUROC1
FPR95
Requires training (or fine-tuning)
MSP
87.44
58.36
79.73
73.72
79.67
74.41
79.69
71.93
81.63
69.61
ODIN
94.65
30.22
87.17
54.04
54
55.06
87.85
51.67
88.80
47.75
Energy
95.33
26.12
92.66
35.97
91.41
39.87
86.76
57.61
91.54
39.89
GradNorm
73.70
70.26
79.36
VIM
93.16
32.19
87.19
54.01
83.75
60.67
87.18
53.94
87.82
50.20
KNN
39.61
85.67
Zero-shot (no training required)
Mahulanohis
56.22
99.22
60.89
99.28
68.96
98.31
65.36
98.15
62.86
98.74
Energy
85.54
80.49
84.21
78.75
84.81
72.29
66.63
92.89
80.30
81.11
ZOC
86.09
87,30
81.20
81.51
83.39
73.06
76.46
98.90
81.79
85.19
MCM
37.57
89.77
44.65
CoVer (ours)
95.98
22.55
93.42
32.85
90.27
40.71
90.14
43.39
92.45
34.88
(e.g., dog (ID) wolf (OOD)). To have more experimental comparison, also reproduce one setting from spurious OOD detection [35], whose hard OOD inputs are created to share the same background (i.e., water) ID data but have different object labels (e.g., boat rather than bird). To select the most effective corruption types for each method, we use SVHN [37] as the validation set.
Model Setup. In this paper, implement Co Ver various architectures, including DNN-like ResNet50, and VLM-like CLIP [42], A|CCIIP [6], MetaCLIP [56], Group [57]. Unless otherwise instructed, for VLM-based zero-shot OOD detection, we use CLIP-B/16 which consists of an image encoder based on ViT-B/16 Transformer [15] and a text encoder built with the masked self-attention Transformer [50]. We the algorithmically generated corruptions defined in [22]. Each type of corruption has severity level from to 5, with c = being the least severe and increasing up to By default, we use the CoVer score in the max-softmax form and set
the temperature.
Baseline Methods and Evaluation Metrics. We conpare the proposed method with various competitive methods. Specifically, we adopt Maximum Softmax Probability (MSP) [23], ODIN [29], Mahalanobis [28], Energy [30], ReAct [45], DICE [46] and ASH [1] as traditional OOD detection baseline methods. The VLM-based OOD detection methods we compared with include MCM, method specifically designed for zero-shot OOD detection, as well as some traditional methods including MSP, ODIN, Energy. Mahalanobis, GradNorm [24], ViM [51], KNN [47]. ZOC [18] that were re- re-implemented using a finetuned CLIP ViT-B/16 on the ImageNet- IK, see Appendix A for more details. For a fair comparison, we keep the original hyperparameter setups of the comparative methods and adopt the following metrics to evaluate the OOD detection performance: (1) the false positive rate (FPR95) of the OOD samples when the true positive rate (TPR) [29] of the in-distribution samples is at 95%, (2) the area under the receiver operating characteristic curve (AUROC) [13].

---

Table Compatibility experiments of Ver combined with different OOD detection methods. The ID data are ImageNet- IK. 1 indicates larger values are better and & indicates smaller values are better.
Architacture
INaturaiit
Places
Testures
Average
AURROC1
FPRUS]
AUROOC
FPRRST
AUROCT
FPRU5
AUROCT
FPRO5
ALROCY
FPROS.
ReAct
96.22
20.38
94,20
24.20
91.58
23.85
89.80
47.30
92.95
31.43
ReAa++CoVer
97.58
13.35
95.7
18,91
93.05
29.02
91.55
40.74
94.45
25,51
DICE
94.49
25.63
90183
35.15
57.45
46.49
90.50
31:72
90.77
34.75
ResNet50
DiCeaCoNer
96.8
16.56
93.53
28.52
90.00
40.54
91.14
31,15
92.87
29.19
DICE
92.74
(ReAct)CooVer
ASII-
94.25
28.95
40.21
87.50
19.52
91.53
90.91
39.04
CoVerr
97.14
14.04
94.12
25.77
91.05
35.93
91.93
30.39
93.56
26.53
MCM
94.61
30.95
92.57
37.57
89.77
44.65
85.10
57.77
90.76
42.73
MCM&CoVVr
95.62
24.35
93.48
31.94
91.67
39.74
88.61
50.44
92.10
36.62
LoCoOp
92.77
42.38
92.88
33.09
90.28
11.08
91.07
40.34
91.75
39.22
LuCsOp+CoVee
CLIP-V11
Cl.PN-CoVer
17.13
30.51
23.05
93.63
NegLabsi
99.49
1.95
95.46
20.95
91.58
36.45
89 89.89
12
94.10
26.11
NegLabel-CoVer
99.39
1,15
94.56
28.84
95.01
25.65
92.39
40.39
95.39
24.01
4.2
Main Results
Overall results of OOD detection performance comparison with different baselines. To evaluate the effectiveness of CoVer, we compare it with existing baseline OOD detection methods on the ImageNet- benchmark in two aspects. In Table we present the performance comparison with traditional OOD detection methods using ResNet-50 as the backbone. Our CoVer combined with ASH-S achieve better OOD detection performance, which verifies the effectiveness of our method with an average multi mulli-dimennsonall estimated confidence score. In Table 2, we provide the results compared with VLM- OOD detection methods, which are classified into two groups: fine-tuning methods that require extra data for fine-tuning CLIP and zero-shot methods that require no training. Our method CoVer consistently achieves better performance across the four OOD datasets, aligning with the analysis that OOD and ID data are better distinguished under the expanded dimensions.
Compatibility experiments of CoVer combined with different OOD detection methods. Since it is simple design for representation dimension expansion, CoVer be easily integrated into previous OOD detection methods and achieve performance improvements. In Table 3, we first consider some methods with minor modifications to the architecture of ResNet50, e.g., ReAct, DICE and ASH. While fixing the detection models, we replace each of their OOD scores with our CoVer score. Then we integrate the proposed Co into several VLM-based OOD detection methods. For LoCoOp [36], ELIP-fine-tuninn and report the results with the CoVer's scoring mode. For CLIPN [52] and NegLabel [26] that both introduce a new OOD score, we redesigned them based on the critical idea of confidence average, as detailed in the Appendix C.3. Our CoVer can consistently help these methods gain better performance without specific modality limitations, which shows the algorithmic robustness of our proposed method. More results with other methods and further discussions and analyses are provided in Appendix C.2.
method with the MCM score, we follow its fine-tuning strategy
Zero-shot OOD detection performance comparison hard OOD detection. Following the settings in [31, 26), we explore the superiority of our proposed Co compared with MCM [31] on hard OOD detection tasks, shown in Table Specifically, we alternately use ImageNet- 10 and ImageNet-20. ImageNet-10 10 and ImageNet- 100 as ID and OOD data to simulate the setups presented in [19]. The results demonstrate CoVer has a better distinguished ability than MCM for semantically hard OOD data. We also report the experimental data on zero-shot spurious OOD detection task in the last row of Table 4, which also shows the better performance the proposed CoVer than MCM.
4.3 Ablation and Future Diseussions
In this part, conduct extensive ablation studies and provide thorough understanding of our CoVer. The extra results and discussions (e.g., impact and limitations) are provided in Appendix C.2.
Ablation on diverse VLMM architectures. We compare our Co score with the MCM score on different VLM architectures and the results are reported in Table 5, The first part shows the superiority
8

---

Table Zero-shot OOD detection performance comparison on hard OOD detection tasks. All methods are based on CLIP-B/16.
Table 5: Zero-shot OOD detection performance with different VLM architectures' representa- tions on (oggeNet-1K((I)))
ID Dataset
OOD Dataset
Mathod
AUROC1
FPR95,
Architecture
Backbone
Method
AUROC1
FPR95]
ImageNet-10
ImageNet-20
MCM
CoVer
98. 98.60
98.68
4,10
6.00
ResNet50
CoVer
MCM
89.98
88.99
46.18
29.79
ImageNet-20
ImageNet-10
MCM
97.69
17.07
CLIP
VIT-R/52
MCM
CoVer
90.21
89.82
45.75
44.78
ImageNet-10
ImageNet-100
MCM
CoVer
97.60
99,30
14.58
2.30
VIT-L/M
MCM
CoVer
92.61
91.49
32.97
38.17
CoVer
99. 28
1.92
ARCLIP
VIT-L114
MCM
CoVer
91.54
93.03
32.15
40.74
ImageNet-100 100
ImageNet-10 10
MCM
CoVer
86.38
86.50
65.55
66.18
MetaCLIP
VIT-B/16 qUT-B/l
MCM
CoVer
87.57
88.64
55.68
58.97
WaterBirds
Spurious OOD
CoVer
MCM
90.52
90.31
35.66
33:17
Group VIT
GroupVIT
CoVer
MCM
86.94
85.10
57.85
51.19
Orgin
Origin
Severity Severity
EXler
Muss-oee
Sevaty
Seruity
FRRO5
624
FFROS
224
20
Comruptiion Type
Exparided Nums
Comuutiin Type
Mothods
(a) Multi-Dim Strength
(b) Expanded Dimensions
(c) Severity Level
(d) w/w.o Comparison
Figure 5: Ablation Study. (a) superiority of the multi-dimensional scoring framework; (b) exploration of different quantity of expanded input dimensions; (c) using different severity levels of a specific corruption type; (d) comparison with different realizations for each dimensional confidence score.
of our Co on different backbones of CLIP, and the larger backbone boosts the performance of OOD detection. The second part shows that CoVer can generalize better to various VLM architectures compared to MCM. More fine-grained results on different OOD sets can be found in Appendix C.2.1.
Superiority of multi-dimensional scoring framework. To verify the superiority of our CoVer with multi-dimmesional scoring framework compared with the uni-dimensional one, report the performance comparison using different corruption types in Figure 5(a). Here, the uni-dimensional framework is denoted as using a single corrupted input for confidence estimation. The results indicate that extended dimensions provide valuable clues for enhancing OOD discriminative representations, thus enlarging the separability between ID and OOD data. We leave the fine-grained results and further discussions in Appendix C.2.2, which provide a better understanding of the improvement.
Significance of the number of expanded measuring dimensions. As a critical aspect of our CoVer, the number of the extended confidence measuring dimensions will control the performance enhancement for OOD detection. In Figure 5(b), we present results for varying the number of expanded dimensions using various corruptions with the same severity level. The results demonstrate that increasing number of expanded representation dimensions gradually improves the performance then probably declines, while consistently outperforming the baseline. This indicates that the addition of measuring dimensions prioritizes enhancing the distinction of OOD data with mutated confidence while ID data shows resistance. More detailed results and analyses are provided in Appendix C.2.3.
Comparison of the variation trends in different corruption severity levels. In Figure 5(c), we show the performance by varying the severity level c for each specific corruption style. We can observe that three types of trends emerge with an increasing level of severity, i.e., up, down, and up then down. This phenomenon indicates that an appropriate level of corruption critical for the optimization of CoVer. One possible reason may be that the threshold maximizing the distinction between ID and OOD data varies from different types of corruption. To further explain this observation, we provide more results on the among various corruption styles and more discussions in Appendix C.2.4.

---

Generality of integrating with different OOD detection schemes. Since the proposed CoVer is general scoring framework with an average of confidence scores measured from multiple dimensions, the specific realization for each dimensional score have multiple choices. In Figure 5(d), we present the comparison with different realizations integrated w./w.o. CoVer (e.g., ResNet-50 based DICE [46] and ASH-B [1], CLIP-B/16 based MCM [31] and NegLabel [26]), where they have different performance improvements compared with the original baseline without constraints for the modality.
5
Discussion
5.1 Broader Impact
OOD detection is crucial for deploying reliable deep learning systems in real-world applications [21], ensuring models can effectively identify data that differ significantly from the training distribution. This ability is vital in safety-critical critical areas [5]. where incorrect predictions due to unexpected inputs can lead to severe consequences. For instance, in the field of autonomous driving, OOD detection helps the system recognize and react appropriately to novel scenarios not covered during training, such as new road signs or altered traffic conditions due to construction. This is imperative as prevents autonomous vehicles from making potentially hazardous decisions based on learned but now irrelevant data, thereby enhancing their safety and robustness in dynamic environments.
Our research highlights fundamental yet overlooked challenge in existing OOD detection methods, which often specialize in a single input type. This specialization may inadvertently limit the repre- sentational dimensions for detection, complicating the identification of subtle OOD samples. For effective OOD detection, is crucial to not only improve empirical performance through enhanced OOD discriminative representations but also address this pervasive issue within the general scoring framework. Our new scoring framework leverages expanded input dimensions and utilizes a confi- dence score expectation to address these concerns, which also shares similar intuitions with some related work [43] in adversarial defense via random transformation. Comprehensive experiments demonstrate its effectiveness and compatibility, suggesting that our method is potentially new generalized framework and provides new insights into OOD detection from a different perspective.
5.2 Limitation
While our method introduces promising framework for OOD detection and provides unique insights through the use of corrupted images to enhance representational dimensions, it does face certain challenges. First, our method indeed faces several failure cases. When Co Ver utilizes certain severe corruption types (e.g., Spatter, Elastic transform), its performance is worse than with single input. This is because these types are more severe compared to others, leading to excessive damages to semantic features. Effective corruption types are those only perturb non-semantic features, which generally exist at the high-frequency level, resulting in different confidence variations between ID and OOD data. Notably, except for leveraging the validation set, our approach lacks a standardized criterion for selecting the types and intensities of corruptions, which is essential for uniform effectiveness across various scenarios. Additionally, the expansion of input dimensions, though beneficial for detection accuracy, may lead to increased evaluation times. These limitations highlight areas for potential improvement, particularly in balancing detection capabilities with computational efficiency. Despite these challenges, the enhanced detection capabilities method introduces mark significant step forward in the reliability of machine learning models against OOD inputs.
Conclusion
In this paper, we introduce a novel perspective for OOD detection, i.c., expanding the representation dimensions. With the different common corruptions, we reveal an interesting phenomenon termed confidence mutation, where the confidence values of some overconfident OOD samples can vary significantly compared with the original inputs. To this end, we propose a new scoring framework. namely, Confidence a aVerage (CoVer), which simultaneously considers the original and expanded input dimensions. Adopting simple but effective average operation, Co can capture the dynamical discrimination of OOD samples and better enhance the separability of ID and OOD distributions. We have conducted extensive experiments to present its effectiveness and compatibility with different methods. We hope our work can draw new insights from different view on OOD detection.
10

---

7 Acknowledgement
BXZ, ZMW and BD were supported by the National Key Research and Development Program of China 2023YFC2705700, National Natural Science Foundation of China under Grants 62225113 13, 62271357, Natural Science Foundation of Hubei Province under Grants 2023BAB072, the Innovative Research Group Project of Hubei Province under Grants 2024AFA017, the Fundamental Research Funds for the Central Universities under Grants 2042023kf01 334, Wuhan Natural Science Foundation 01440408010202230 and the numerical calculations in this paper have been done on the supercom- puting system the Supercomputing Center of Wuhan University. JNZ. and BH were supported by NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation and 2024A 2399, RIKEN Collaborative Research Fund, HKBU
Nos. 1022A1515011 Faculty Niche Research Areas No. RC-FNRA-1G/22223SC0 Incentive Scheme.
and HKBU CSD Departmental
References
[1] Djurisic Andrija, Bozanic Nebojsa, Ashok Arjun, and Liu Rosanne. Extremely simple activation shaping for out-of-distributin detection. In ICLR, 2023.
[2] Haoyue Bai, Gregory Canal, Xuefeng Du, Jeongyeol Kwon, Robert D Nowak, and Yixuan Li. Feed two birds with one scone: Exploiting wild data for both out-of-distribution generalization and detection. In /CMI., 2023.
[3] Julian Bitterwolf, Alexander Meinke, Maximilian Augustin, and Matthias Hein. Breaking down outtif-distributiin detection: Many methods based on OOD training data estimate combination of the same core quantities. In ICML, 2022.
[4] Julian Bitterwolf, Maximilian Mueller, and Matthias Hein. In or out? fixing imagenet out-of- distribution detection evaluation. In ICML, 2023.
[5] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bossclut, Emma Brunskill, et al. On the opportunities and risks of foundation models. In arXiv, 2021.
[6] Zhongzhi Chen, Guang Liu, Bo-Was Bo-Wen Zhang. Fulong Ye, Qinghong Yang, and Ledell Wu. Altelip: Altering the language encoder in clip for extended language capabilities. In arXiv, 2022.
[7] Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, 2014.
[8] Pierre Colombo, Eduardo Dadalto, Guillaume Staerman, Nathan Noiry, and Pablo Piantanida. Beyond mahalanobis distance for textual ood detection. In NeurlPS, 2022.
[9] Pierre Colombo, Marine Picot, Federica Granese, Marco Romanelli, Francisco Messina, and Pablo Piant. A halfspace-mmas depth-based method for adversarial attack detection. TMLR, 2023.
[10] Pierre Colombo, Marine Picot, Nathan Noiry, Guillaume Staerman, and Pablo Piantanida. Toward stronger textual attack detectors. In arXiv, 2023.
[11] Maxime Darrin, Pablo Piantanida, and Pierre Colombo. Rainproof: An umbrella to shield text generators from out-of-distribut data. In arXiv, 2022.
[12] Maxime Darrin, Guillaume Staerman, Eduardo Dadalto Cámara Gomes, Jackie CK Cheung, Pablo Piantanida, and Pierre Colombo. Unsupervised layer-wise score aggregation for textual ood detection. In AAAI, 2024.
[13] Jesse Davis and Mark Goadrich. The relationship between precision-recal and ROC curves. In ICML, 2006.
[14] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In CVPR, 2009.

---

[15] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers for image recognition at scale. In arXiv, 2020.
[16] Xuefeng Du, Yiyou Sun, Jenry Zhu, and Yixuan Li. Dream the impossible: Outlier imagination with diffusion models. In NearIPS, 2023.
[17] Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. VOS: learning what you don't 't know by virtual outlier synthesis. In ICLR, 2022.
[18] Sepideh Esmaeilpour, Bing Liu, Eric Robertson, and Lei Shu. Zero-shot out-of-distribution detection based on the pre-trained model clip. In AAAI, 2022.
[19] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-offddisrribtion detection. In NeurlPS, 2021.
[20] Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning. MIT Press, 2016.
[21] Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joseph Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribttion detection for real- world settings. In ICML,, 2022.
[22] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In ICIR, 2019.
[23] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and auttof-diitribution examples in neural networks. In ICLR, 2017.
[24] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts the wild. In NeurlPS, 2021.
[25] Wenyu Jiang, Hao Cheng, Mingcai Chen, Chongjun Wang. and Hongxin Wei. Dos: Diverse outlier sampling for out-of-distribution detection. In arXiv, 2023.
[26] Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, and Bo Han. Negative label guided ood detection with pretrained vision-language models. In ICLR, 2024.
[27] Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang. A tutorial on energy-based learning. In Predicting structured data, 2006.
[28] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurlPS, 2018.
[29] Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of diittii-iitiiitt image detection in neural networks. In ICLR, 2018
[30] Weitang Liu, Xiaoyun Wang, John D Owens, and Yixuan Li. Energy- based out-of distrribbtt detection. In NeurIPS, 2020.
[31] Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of-distribution detection with vision-language representations. In NeurlPS, 2022.
[32] Yifei Ming, Ying Fan, and Yixuan Li. Poem: sampling. ICML, 2022.
detection with posterior
[33] Yifei Ming and Yixuan Li. How does fine-tuning impact out-of-distribution detection for vision-language models? IJC,, 132(2):596-609 609, 2024.
[34] Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Li. How to exploit hyperspherical embed- dings for out-of-distribution detection? In arXiv, 2022.
[35] Yifei Ming, Hang Yin, and Yixuan Li. On the impact of spurious correlation for out-of- distribution detection. In AAAI, 2022.
12

---

[36] Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa. Locoop: Few-shot detection via prompt learning. In NeurIPS, 2023.
[37] Yuval Netzer, Tao Wang. Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NeurIPS Workshop. 2011.
[38] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
[39] Jaewoo Park, Yoon Gyo Jung, and Andrew Beng Jin Teoh. Nearest neighbor guidance for out-of-distribution detection. In CVPR, 2023.
[40] Marine Picot, Nathan Noiry, Pablo Piantanida, and Pierre Colombo. Adversarial attack detection under realistic constraints. 2022.
[41] Marine Picot, Guillaume Staerman, Federica Granese, Nathan Noiry, Francisco Messina, Pablo Piantanida, and Pierre Colombo. A simple unsupervised data depth-based based method to detect adversarial images. 2023.
[42] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML., 2021.
[43] Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Barrage of random transforms for adversarially robust defense. In CVPR, 2019.
[44] Jie Ren, Peter Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of of-distributiion detection. In NeurIPS, 2019.
[45] Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of futtof-distributtio detection with rectified activations. In NeurlPS, 2021.
[46] Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribation detection. In ECCV, 2022.
[47] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribttion detection with deep nearest neighbors. In ICML, 2022.
[48] Leitian Tao, Xuefeng Du, Xiaojin Zhu, and Yixuan Li. Non-parametric outlier synthesis. In ICLR, 2023.
[49] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In CVPR, 2018.
[50] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurIPS. 2017.
[51] Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual-logit matching. In CVPR, 2022.
[52] Hualiang Wang, Yi Li, Huifeng Yao, and Xiaomeng Li. Clipn for zero-shot ood detection: Teaching clip to say no. In ICCV, 2023.
[53] Qizhou Wang. Zhen Fang Yonggang Zhang, Feng Liu, Yixuan Li, and Bo Han. Learning to augment distributions for out-of-distrriution detection. In NewrIPS, 2023.
[54] Qizhou Wang, Feng Liu, Yonggang Zhang, Jing Zhang, Chen Gong, Tongliang Liu, and Bo Han. Watermarking for -uitt--disttibution detection. NeurIPS, 2022.
[55] Jianxiong Xiao, James Hays, Krista A Fhinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 2010.
13

---

[56] Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po- Yao Huang. Russell Howes, Vasu Sharma, Shang- Wen Li, Gargi Ghosh, Luke Zettlemoyer. and Christoph Feichtenhofer. Demystifying clip data. In arXiv, 2023.
[57] Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, 2022.
[58] Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng. Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, al. Openood: Benchmarking generalized out-of-distribution detection. In NeurIPS Datasets and Benchmarks Truck, 2022.
[59] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of out-of-distribution detection: A survey. In arXiv, 2021.
[60] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. TPAMI, 40(6):1452-1444 1452-1464, 2017.
[61] Jianing Zhu, Yu Geng, Jiangchao Yao, Tongliang Liu, Gang Niu, Masashi Sugiyama, and Bo Han. Diversified outlier exposure for out-to-distribution detection via informative extrapola- tion. In NeurlPS, 2023.
[62] Jianing Zhu, Hengzhuang Li, Jiangchao Yao, Tongliang Liu, Jianliang Xu, and Bo Han. Un- leashing mask: Explore the intrinsic out-of-distribution detection capability. In /CML, 2023.
14

---

Appendix for CoVer
The whole Appendix is organized follows. In Appendix we present the detailed definitions and implementation of baseline methods that are considered in our exploration. In Appendix B, we provide detailed discussions about related works. In Appendix C, we provide extra experimental details and more comprehensive results with further discussion on the underlying implications. In Appendix we provide some preliminary statistical analysis about CoVer. Finally, in Appendix E, we provide the further analysis for better understanding of our work.
Reproducibility Statement
To ensure the reproducibility of experimental results, we provide the source code at https:// // gi github. wwwmtooooooo//eeeo http:/ttlrggrup///eeee group/CoVee. Below summarize several important aspects to facilitate reproducible results:
Datasets. The datasets we used are all publicly accessible, which is introduced in Section Following MCM [31], we also use subsets of ImageNet-1K 1K for fine-grained analysis, like ImageNet-10, 10. For hard OOD evaluation, we exploit ImageNet-20 with 20 categories similar to ImageNet-10. 10. We also reproduce the spurious OOD detection [35] with 0.9, which determines relative size majority vs. minority groups.
Assumption. We set our main experiments CLIP-like model on the original classification task is available [42]. Under this assumption, CoVer detects OOD samples in parallel with the zero-shot classification task and has no impact on ID classification performance.
zero-shot scenario where a well-trained
Open source. The source code available at https: //t: /gthtb...b... http//mllrrrou//// We provide backbone for our experiments as well several auxiliary components, such as score estimation.
Environment. All experiments are conducted on NVIDIA GeForce RTX 3090 GPUs with Python 3.10 and PyTorch 2.2.
A Details about Considered Baselines and Metrics
In this section, we provide details about the baselines for the scoring functions, as well as the corresponding preerpparaeeters and other related metrics that are considered in our work.
Maximum Softmax Probability (MSP). [23] proposes to use maximum softmax probability to discriminate ID and OOD samples. The score is defined as follows,
SMsp (2; f) max Ply = c/a; f) max sof tmax (ff
where ff represents the given trainnrai model and is one of the classes y = fl...., larger softmax score indicates the larger probability for sample be ID data, reflecting the model's confidence on the sample.
.il...... C}. The
ODIN. [29] designed the ODIN score, leveraging the temperature scaling and tiny perturbations to widen the gap between the distributions of ID and OOD samples. The ODIN score is defined as follows.
f(t) T
SODIN (x; f) = max Ply = c|;:; f) = max sof tmax
(8)
where 7 represents the perturbed samples (controled by e), T represents the temperature. For fair comparison, we adopt the suggested hyperparameters [29]: 1.4 x 10-3 3 T 1.0 x 10a.
Mahalanobis. [28] introduces a Mahalanobis distance-based confidence score, exploiting the feature space of the neural networks by inspecting the class conditional Gaussian distributions. The Mahalanobis distance score is defined as follows,
Sahalanobbis (x;; max - (())) -cc)
(9)
15

---

where ce represents the estimated mean of multivariate Gaussian distribution of class Σ represents the estimated tied covariance of the C' class-conditional Gaussian distributions.
Energy. [30] proposes to use the Energy of the predicted logits to distinguish the ID and OOD samples. The Energy score is defined follows,
Sinergy (z; f) = -T log
Male/
(10)
where T represents the temperature parameter. As theoretically illustrated in [30], lower Energy score indicates a higher probability for a sample to be ID. Following [30], we fix the T to 1.0 throughout all experiments.
ASH. [1] designs a extremely simple, post-hoc method called Activation SHaping for OOD de- tection. It removes large portion of an input's activation at late layer and adjusts the rest of the activation values by scaling them up or assigning them a constant value. The simplified representation is then passed throughout the rest of the network. The logit output is used classify ID samples and calculate scores for OOD detection as usual. For ASH-B version, we adopt the MSP score and implement it with the hyperparameter p 65; For ASH-S version, apply it with energy score and the hyperparameter p = 90. Both settings are suggested by [1].
MCM. [31] proposes Maximum Concept Matching (MCM), simple yet effective zero-shot OOD detection method based on aligning visual features with textual concepts. Formally, the MCM score can be defined as:
(x)))//+
SMCM (x';)mn T,I)= max
(11)
K 2j=1 e0/(x)/T
CLIPN. [52] proposes a novel CLIP architecture, which equips CLIP with a "no" logic via the learnable "no" prompts and "no" text encoder. Specifically, CLIPN proposes two novel inference algorithms to perform OOD detection via using negation semantics, where the algorithm named gereeing-to-differ (ATD) is more effective experimental results. The ATD form of the CLIPN score can be formulated as follows,
SCLIIN = = (xcsla)/T cooll
(12)
Di c*)N(w)/T
where C is the number of classes, (x) and sx) are denoted as the inner product the image feature and the corresponding text feature
(T) =< pmage (z), P (p(1))) >,
$ig (z) =< pimage (x)) 1
(p(B)
>
(13)
where fext is the "no" text encoder and p(v)") the text with "no" logic.
NegLabel. [26] proposes novel post hoc OOD detection method, called NegLabel, which takes vast number of negative labels from extensive corpus databases and designs a novel scheme for the OOD score collaborated with negative labels. NegLabel score can be formulated as
(x) S* (sim(x, Y), sim (x)
(14)
where S* (+.-) represents fusion function that combines the similarity of a sample with ID labels sim(x,3) and the similarity of the sample with negative labels sim (x, --). The sum-softmax form of NegLabel score is defined as follows,
S ahell a
(15)
ΣR Ei=1 eS((n)/T EM (T)TT
where is the number ID labels, T the temperature coefficient of the softmax function and M is the number of negative labels, (xx) and o, ( are formulated as the cosine similarity, defined as follows,
(z cos( c (z), e.), 8, (xx) cos imagge (x), E,,
(16)
16

---

B Detailed Discussion with Related Works
In this section, we provide detailed discussions about related works.
Traditional OOD Detection. There has been an increasing interest in OOD detection since the phenomenon of overconfidence in OOD data was first discovered in [38]. As a formal benchmark for OOD detection, [23] proposed using softmax prediction probability
conventional baseline method. Afterward, numerous approaches have been developed to address visual OOD detection, which can be classified into two categories, i.e., post hoc scoring mechanism and training-time regularization [59, 58]. Post hoc methods are dedicated to exploring a better OOD score by freezing the model's parameters. ODIN [29] improves the previous MSP [23] by scaling with the temperature and slightly perturbing the inputs. Mahalanobis introduces the Mahalanobis distance in the feature space to measure the confidence score. Energy [30] exploits the energy function [27] to distinguish ID and OOD data. Both ReAct [45] and DICE [46] are improved from Energy, ReAct by feature clipping, and DICE by discarding the most prominent weights in the fully connected layer. ASH [1] designs an extremely simple method that removes large portion of an input's activation and adjusts the rest. On the other hand, training-time -tme regularization methods exploit the potential access to partial OOD data during model training. MOS [24] groups all classes and introduces an extra class to each group to reformulate the loss function during training. VOS [17] enhances the quality of the energy score by creating synthetic virtual anomalies. CIDER [34] exploits KNN [47] to boost OOD detection performance through the optimization of contrastive loss. DAOL [53] alleviates the OOD distribution discrepancy by crafting an OOD distribution set that contains all distributions in a Wasserstein ball centered on the auxiliary OOD distribution. The presence of outliers leads to superior performance compared to training without outliers, evidenced by numerous previous studies [3, 25, 2, 16, 62].
OOD Detection with vision-language representations. With the rapid development of multi- modal large language models (MLLMs), such as CLIP [42], much attention has been paid to OOD detection with vision-languad representations [33]. MCM [31] proposed the first zero-shot OOD detection framework that combines the temperature scaling strategy and maximum softmax probability as the OOD score. Following MCM, some works fine-tuned CLIP's image encoder for visual OOD detection [48. 36.. NPOS [48] utilized generated OOD data optimize the ID-OOD decision boundary. LoCoOp exploited the portions of CLIP's local features as OOD features to realize OOD regularization. Some latest methods [52, 26] boosted OOD detection by adding extra clues obtained from negative textual information. CLIPN [52] equipped CLIP with a "no" logic via a text encoder that can understand negative prompts. NegLabel [26] introduced numerous negative labels and distinguished OOD samples by examining their affinities between ID and negative labels.
Data depths and information projections. Computing OOD scores on the embedding output of the last layer of the encoder is not the best choice for textual OOD detection. To address this, [12) proposed aggregating OOD scores across all layers and introduced an extended text OOD classification benchmark, MILTOOD-C. In similar vein, RainProof [11] introduced a relative infor- mation projection framework and a new benchmark called LOFTER on text generators, considering both OOD performance and task-specific metrics. Building on the idea of information projection, REFEREE [40] leveraged I- I-projection to extract relevant information from the softmax outputs of a network for black-box adversarial attack detection. On the other hand, APPROVED [41] proposed to compute a similarity score between an input sample and the training distribution using the statistical notion of data depth at the logit layer. HAMPER [9] introduced a method to detect adversarial examples by utilizing the concept of data depths, particularly the halfspace-mass (HM) depth, known for its attractive properties and -n-ddifferentiaiiitt.. Furthermore, TRUSTED [8] relied on the information available across all hidden layers of network, leveraging novel similarity score based on the Integrate Rank- Weighted depth for textual OOD detection. LAROUSSE [10] employed a new
anomaly score built on the HM depth to detect textual adversarial attacks in an unsupervised manner.
C Additional Experimental Results and Further Discussion
In this section, we provide additional experimental results from different perspectives to verify the effectiveness our proposed CoVer. First, we introduce the additional experimental setups for the empirical verification in previous figures and implement details about our method. Second, we
17

---

provide comprehensive results with further discussions of ablations. Finally, extensive visualization analyses are provided for better understanding of Co
C.1 Additional Experiment Setups
Implement Details. Unless otherwise specified (e.g. Table 1 and Table 3), we conduct the major experiments based on pre-trained CLIP-B/16 for zero-shot OOD detection, following the previous research work [31, 26]. Furthermore, the primary form of CoVer is based on the maximun-softtma score function, as defined in Section 3.3. For the extended corrupted inputs, We utilize the SVHN dataset as the validation set to determine the most effective corruption types for each method in all experiments. The detailed adopted corruption types for each method are provided in Table 6, where Corruption Type(X) denote the corruption at X severity level.
Table 6: Adopted corruption types and corresponding severity levels when Co integrated with other methods.
Method
Expanded Coruuption Types
CoVer
Comtrusi(3)
DICE + Co
Brightmesa(1. (1 2). Gaussian Blur(1. 2), Saturate(1. 2), and 1.
DICE (ReAct) CaVer
Brightness(L.
ASH-B ASH.S + CoVer
Brighaness(1 2)
Brightners(1, 2), Gouszian Blur1,
Blur(1, 2), Suturate(1, 2), Defocus Blur(1, 2), and Fog(1,
LoCoOp + CoVer
Brightmesot 1, 2), Goussian Blur(), 2), Motion Rhur(), 2), Soturate(l, 2), Defocus Rlur(1, 2), and Fog(), (1 2)
CLIPN CoNer
Brightness( 1) and Saturate())
Nhel CoVer
Brighmesst I, 2) and Saturatet 1)
Figure 1. In Figure compare the score distributions and detection results with different input modes. Specifically, we use the corruption type of Contrast(4) as an example and report the results on the iNaturalist dataset of the ImageNet-1K 1K benchmark. In the right panel of realize (Coptrus())))))((4) ()) inputs.
CoVer by averaging the confidence scores obtained by the original and
Figure 2. In Figure we conduct experiments to demonstrate the detailed explanations for the discovery illustrated in Figure 1. For the fair comparison, all samples here, including confident ID, unconfident ID, overconfident OOD, and unconfident OOD samples, are randomly sampled from the ID and OOD distributions in the left and middle panels of Figure 1.
Figure 3. Figure 3, we visualize several samples for further understanding of the confidence mutation. Specifically, we use the Fast Fourier transform (FFT) to obtain the owwfrequency and high-frequency portions of the image with the radius of circular filter r 0.6. Same as Figure I and Figure 2, unconfident ID data and overconfident OOD data are randomly sampled from the corresponding part of ImageNet and iNaturalist datasets, respectively. We continue to adopt Contrast(4) for corruption.
Figure 5. For Figure 5(a) to Figure 5(d), we provide discussions about the detailed experimental settings and their fine-grained results in Appendix C.2.
C.2
Full Results of Ablations
C.2.1 Ablation on VLM Architectures.
The detailed results are shown in Table 7. It is evidenced that in addition to CLIP Vit-B/16, our CoVer can boost the OOD detection performance on various kinds of VLM architectures compared MCM, including the CLIP architecture based on different backbone networks (e.g., ResNet50 and Vit-L/14) and other types of VLM architectures (e.g., AItCLIP, MetaCLIP, and GiTupViT).
C.2.2 Superiority of Multi- Talti-Dimensional Scoring Framework.
In Figure 5(a), we present some representative results to verify the superiority of the multi-dimensional scoring framework. Here, we report the fine-grained results using different types of corruptions with severity level of 5, as shown in Table 8. We can find that there is constantly bad OOD detection
18

---

Table Compared to MCM with different VLM architectures on ImageNet- K(ID) All values are percentages. f indicates larger values are better and indicates smaller values are better.
UOD Dutaset
Architectrre
Roclhone
Mathed
INatoralist
SUN
Avaraoo
AUROCT
[PR95]
AUROCI
FPR95]
AUROCT
AUROCT
FPR95]
AUROCT
FPRO5]
ResNer50
MCM
52.16
46.21
60.11
55.99
49.29
VIT-R/32
MCM
33,92
41.79
25.64
84.67
19.82
VIT-R/16
30.95
37.57
MTLL/14
94.95
24.58
38.17
AICLIP
CoYer
96.00
95.17
24.35
92.04
34.61
58.93
47.43
93.04
52.15
MeaCLIP
quickgeess
MCM
CoYer
89.30
68
64.97
61.07
91.09
46.76
99.07
54.13
60.16
N7.57
83.64
55.08
GoopAI
VIT-111
CoVer
91.65
59.50
37.08
87.67
55.57
53.1
83.74
69.43
84.7
$4.15
58.92
86.94
51.19
57.85
performance under the single-dimensional scoring framework with a single corrupted image as the input sample. However, considering the multiple inputs, i.e., both original and corrupted inputs, averaging their confidence scores shows a huge performance enhancement. For example, comparing the uni-dimensional and multi-dimensional results when the corruption type is Contrast, the incorpo- ration of the extended dimension provides + +1..57% AUROC (from 74.20 74.20% to 91.77%) and 52.02% FPR95 (from 88.61 % 36.95%), indicating that corrupted inputs require the combination of original inputs to reveal its ability to enhance the distinguishabiliyy of OOD samples at the feature level. For more detailed experiments regarding CoVer expanded by corrupted inputs at other severity levels. please refer to Appendix C.2.4.
To have a more intuitive understanding on the superiority of multi nulti-dimeenional inputs, we provide a comparison with score distributions and detection results w/w.o. an expanded input dimension transformed by Brighiness. Fog, Motion blur, and Speckle noise in Figure 6. From the middle column, we notice that the OOD samples are more difficult to detect by the model confidence from single corrupted inputs. This is mainly because the confidence of the II) sample, which was originally high, drops drastically when it is corrupted thereby interfering with the model's judgment of the OOD sample. In contrast, through a simple but critical average operation, CoVer generally achieves better ID-OOD separability. This phenomenon can be attributed to two main reasons. First, the ID samples have an overall higher confidence expectation, eliminating the originally confident ID interference present under the single corrupted input. Secondly, as illustrated in Figure and Figure 3, the data lies in the originally overlapped part of the ID and OOD distributions, i.e., unconfident ID data and overconfident OOD data, demonstrate significant differences in the variation of confidences under the same corruption. Specifically, the ID data shows resistance while the OOD data shows vulnerability, thus better exposing the OOD samples to be rejected.
C.2.3 Imapact of the Number of Expanded Measuring Dimensions.
In Figure 5(b), we report the OOD detection performance variations evaluated on different numbers of expanded measuring dimensions. However, we have not specifically analyzed the impact of employing different types of corruptions under each number of extended representation dimensions. In Table 9, we further present the detailed results for different numbers of extended dimensions under corruption severity levels 1 and 2, with each number we enumerate two different kinds of combinations of corruption types. The experimental results demonstrate that considering confidence estimation on both original input and expanded variant dimension constantly enhances the OOD detection performance across four OOD datasets.
However, as the expanded dimension gives priority to the phenomenon of confidence mutation that more diseriminative in OOD data than that is in ID data, the newly added representation dimension sometimes leads to slight decline in performance. For instance, the best performance three expanded dimensions that grouped by Defocus blur, Motion blur and Fog inputs under the corruption severity level 1; the combination of five types of corruptions including Defocus blur, Motion blur, Gaussion blur, Fog, and Saturate achieve better OOD detection performance when the variants are generated by corruptions at severity level 2. This phenomenon suggests that, while
19

---

Table Fine-grained results of Co using different types of corruptions with a severity level of based on CLIP-B/16. The experiments are conducted on ImageNet- 1k benchmark
OOD Dataset
Cormuttion Type
Mode
iNaturalist
SUN
Places
Textures
Average
AUROC1
FPR95L
AUROCT
FPR95]
AUROCT
FTR951
AUROC1
FPR95L
AUROCT
FPR95]
Brightness
Mulii
95.11
41.42
28.29
90.17
92.76
36.84
50.25
90.05
87.35
51.42
43.08
$8.08
86.18
55,21
52.3
89.21
91.5
40.13
50.33
Fog
Mulli
Uni
88.42
95.61
60.34
24.19
84,71
93.19
75.11
33.77
90.15
81.34
40.96
78.46
88.11
82.0
65,48
48.72
91.77
84,12
36.91
69.85
Contrast
Multi
Uni
96.41
82.0
87,94
19.6
71.36
92.46
36,05
89.12
89.17
67.68
91.01
44.07
89.05
75.77
86.06
48,07
91.77
74,2
36,95
88.61
Blur
Mulii
94,98
71.66
26.13
72.48
91.4
42.02
87.78
67.88
50.18
87.24
76.12
50.92
90.35
74.65
42.31
Defocus Blur
Multi
Uni
80.75
94,04
65.67
36.8
92.12
77.1
39.58
80.25
89.09
74.12
83.31
45.25
88.32
79.03
49.8
70.0
90.89
77.75
41.36
74,81
Gaussian Blur
Multi
Uni
77.49
93.52
73:47
33.3
73.84
91.59
83.98
42.03
71.13
88.64
86.05
47.39
77.61
88.01
50.25
69.86
75.02
90.44
43.24
78.34
Spatter
Multi
75.45
92.06
44.98
81,57
92.3
40.42
89.46
79.15
83.43
46.1
83.89
71.27
85.
62.36
76.
89.43
84.88
48.47
Uni
88.48
57.4
86.37
64.72
84.76
66.84
83.03
64.63
85.66
63.4
Saturate
Multi
94.24
31.27
92.11
38.41
89.9
43,08
87.23
53.37
90.87
41.53
Elastic Transform
Uni
51.9
98.67
65.13
95.6
65 08
91.5
47:27
97. 97.09
58.1
96.17
Mulli
90.57
53.15
91.12
45.97
88.52
49.36
77.98
72.84
87.05
55.33
Compression
Uni
79.43
86.23
83.88
75.1
80.99
75.62
77.05
81,49
80.34
79.61
93.08
39.52
92,95
36.21
89.97
42.17
86.27
57.34
90.57
43.81
Pixelate
Uni
82.88
74.39
78.18
85.36
75.9
88.07
73.27
83.65
77.56
82.87
Mulli
94.56
29.86
91.79
39,89
89.11
45,89
85,86
$8.48
90.33
43,53
Speckle Noise
80.59
69.26
67.8
68.17
92.07
72.19
90.79
Mulii
96.49
19.3
91.51
44.35
88.92
49.75
85.44
56.95
90.59
42.59
Glass Blur
Mulli
Uni
94.51
76.1
30.81
82.53
74.05
92.23
37.47
84.98
70.49
89.12
87.97
45.22
82.46
62.0
61.58
86.05
89.58
70.66
85,38
43.77
Guussian Noise
Uni
73.7
87.88
56.11
97.37
$5.99
96.32
59.85
95.58
61.41
94.28
Multi
95.47
23.58
90.74
47.27
88.36
$2,06
85.4
$7.82
89.97
45.18
Shoc Noise
58.25
97.29
58.1
60.3
96.1
93.75
Mulii
95.97
21.26
90.85
46.16
88.44
$1.13
85.18
$8.09
90.11
44.16
Zoom Blur
Uni
67.78
93.04
69.32
91,24
66.3
92.24
66
90.94
67.25
91. 86
Multi
92.35
90.93
42.52
87.93
49.05
84.46
59.27
88.92
47.81
Snow
Multi
Uni
85.35
95.35
74.35
26.59
79.24
91.94
39.16
85.5
$8.76
75.88
87.31
45.93
76.94
86.6
75.55
53.4
79.35
90.66
41.27
80.78
Impulse Noise
74.25
90.69
54.37
86
94.18
61.52
Mulii
95.75
22.7
90.05
50.09
87.73
$3.49
86.06
$4.91
89.9
45.3
extended dimensions preferentially provide additional clues that make the OOD samples more salient, excessive extended dimensions or the inclusion of input dimensions transformed from some specific uncommon corruptions may result in greater degree of interference in the ID samples, leading to slight declination of the performance.
C.2.4 Full Results of Ver with Variants Corrupted at Different Severity Levels.
In Figure 5(c), report three representative variation trends of performance with an increasing level of the corruption severity. In Table 10 and Table 11, we present the full results of the proposed CoVer with 18 different types of corruptions at severity levels. It is worth noting that the experimental results here are all based on the average of the model confidence scores measured on one original input and one extended corrupted input. Furthermore, we present the performance changing trends of our Cover based on all 18 corruption styles in Figure 7. It can be seen that the performance of CoVer is more sensitive to the severity levels of corruptions compared to the number of extended representation dimensions analyzed in Appendix C.2.3. Specifically, as shown in Figure 7, some common corruption types from categories including weather (e.g., Brightness and Fog) and blur (e.g., Motion blur and Defocus blur) can achieve lower FPR95 values. We further observe that the commonality of these better-performin more mild compared to other types (e.g.. Spatter and Elastic transform from the digital class, Impulse noise from the noise category), and they generally do not excessively corrupt the semantic features. In Appendix C.4, we will more intuitively demonstrate the distinctions between these various types of corruptions through the visualizations of corrupted ID and OOD samples, illustrating the varying degrees of enhancement or attenuation they impart on the performance of Co
comruptio types is that their perturbations to image features
20

---

Table 9: Comparison with different numbers of expanded representation dimensions at corruption severity level I and 2 based on CLIP-B/IE The ID data are ImageNet- IK. For each number of expansions, we provide two choices for the corruption types combination.
0OO0 Dasset
Severity Lovel Expasded Nama
lspal
Nanualit
SUN
Places
Textuess
Average
AUROCTT
F99055]
AUROCT
F19055
AUROCCT
F1905
AUTOCCT
FFR96)
AUROCCT
FFR05]
Original
55.77
orggial Sonmast
W4,0
90.36
V1.77
37.34
19: 99.99
43.95
87.08
55.11
91.11
41.64
original defocas blar
94.80
20.36
93.57
31.87
98.68
39.19
81.74
52.22
91.70
38.13
Orginul
brighness
origiall delinae hiher
meticn bur
9493
25.15
93.49
32.17
90.65
39.53
87.98
$3.19
91.66
36.26
hrighmess
91.32
origiaal definaue bllr
motion but fng
05.18
26.54
93.40
32.72
90.62
39:05
$5.08
51.88
91.82
37.77
Load
hrightec caurii
9481
28.3
97.89
35.99
90.10
43.8
$7.16
55.04
91.90
40.75
gaussion blur
otiginal
27.92
90.55
52.71
91.67
orgginll scmmat
brighmess satuat
05
27.40
92.96
35.84
90.29
42.98
87.86
53.78
91.49
40.04
pussioon
original blu
molion bir
95.18
26.49
93.33
32,89
90.57
40.27
87.01
52.8
91.75
38.11
gasedan bar snunic
oéginul
pproiinn bler
26655
34.8
87.70
metion bur
onigiial delomue biir
motion
tnighnnse
original brishnses
94.63
20.73
92.67
36,69
90:09
43.55
87.00
54.65
91.11
41.16
orggiial delisuss biir
94.35
90.00
11.37
98.57
39.60
$8.11
56.25
91.81
original
continll
28.57
36.55
87.81
53.33
original defocus Mar
metion barr
27.81
93.52
31.61
90.38
39.85
87.08
51.37
91.08
37.66
costrist saurate
hrightnees
24.50
mesius gass
original defocus bur
94.44
36
93.35
33.05
40.99
19.04
brightnee
Level
conmrst saurate
M574
23,68
92.83
36.66
90,11
44.00
85.26
51.06
91.76
38.87
lay
orgiial
original brishnses
contmal saunts
2414
93.35
34.8
90.32
42.45
50.12
37.90
fog gassiion
original definaee blur
rociot blir gasssoo bler
95.54
25,07
M3.33
33.74
M0.35
40
55.00
50.23
91.95
37.24
+ fig oourat
origial Brightnces
fog gassioo metion Hur
95
2412
91.75
33.70
90.40
13.56
50
50446
91.99
17.66
ofiginal defocus blr
fig fig-oolmms contratt
33.21
90.25
50,09
21

---

Original Input
Brightness Input
Multiple Inputs (CoVer)
2000
5400
14600
000
900
1000
3000
10000
100
1003
4090
0
C
8000
Score
Score
Score
AUROC: 94.63 FPR95: 30.70
AUROC: 92.84
FPR95: 41.42
AUROC: 95.11 f FPR95: 28.29
Original Input
Fog Input
Multiple Inputs (CoVer)
DO0
DOG
oep
1200
1350
1202
DD
1300
Dendty
6006
8000
8003
CCEDO
CCESOS
Score
Score
Score
AUROC: 94.63
FPR95: 30.70
AUROC: 88.42
FPR95: 60.34
AUROC: 95.61
FPR95: 24.19
Original Input
Motion Blur Input
Multiple Inputs (CoVer)
3005
3003
3000
Ealu 23122 POLLO 20100
<211 SUEES ERAUD EE0LE 80033 2010 000000
2 2210 10005 23002 p0i B000 200000
AUROC: 94.63 FPR95: 30.70
AUROC: 82.12
FPR95: 71.66
AUROC: 94.98
FPR95: 26.13
Original Input
Speckle Noise Input
Multiple Inputs (CoVer)
1000
1006
1500
14000
3400
3400
1200
1290
1280
1286-
1280
bersit
1285
LABUE MOUTERI Score
GALOS CLETNS CLELE
Score
Score
AUROC: 94.63
FPR95: 30.70
AUROC: 83.54
FPR95: 80.59
AUROC: 96.49
FPR95: 19.30
Figure 6: Comparison of scores distributions and detection results with different inputs for represen- tation dimension expansion under various corruptions.
22

---

Table 10: Full results Co Ver under one extended input with 18 alternative types corruptions at severity levels based CLIP-B/16. The ID data are ImageNet-IK
Corruption Type
Severity Level
INatoralist
SUN
OOD Dataset
Places
Textures
Average
AUROC
FPR95
AUROC
EPROS
AUROCI
FPROS]
AUIROCI
FPR
AUROOCT
FPR9S]
94.63
29.73
92.67
36.69
90.09
43.55
57.07
54.65
91.11
41.16
Brightess
94.67
30.24
92.63
37.17
90.00
43.53
87.51
53.67
91.20
41.15
94
29.52
92.67
37.06
90.04
43.86
52.61
40.76
95.11
28.29
92.76
36.84
90.05
43.08
52.30
91.50
95.
26.09
92. 76
37.00
90.12
42.95
57.67
52.68
91.41
39 68
V5. 42
25,2
92.
36,71
90.13
43.54
25.00
51.
91.59
39.
Fog
95.49
24,68
93.04
90.15
41.64
83.17
49.17
91.71
37.52
95.61
24.19
93.19
33.77
90.15
40.96
$8.11
48.72
91.77
36.91
94.60
30,26
92.77
37.24
$9.99
43.93
$7.08
55.11
91.11
41 14
28.37
Contrast
95.
25.55
93.04
35.94
90.11
42.90
$3.57
50.23
91.80
38.65
96.37
20.10
92.95
35.18
$9.85
42.13
90.13
43.90
92.32
35.33
96.41
19.60
92. 46
36,05
44.07
48.077
91,27
36.95
95.34
26.04
93.41
32.15
90.44
40.28
87.17
53.35
91.59
37.95
Meticn Hlur
95. 16
26.14
93.06
32,88
89.75
41.62
87.26
51.58
91.31
38.
94.97
26,03
92.
37.97
47.20
87.26
90.74
40.58
26.13
42.02
87.78
50.18
87.24
30.92
901.3
42.31
94. 30
29. 26
93.57
31.87
90.68
39.19
87.74
$2.22
91.70
38, 13
94.85
28.64
93,60
31.77
90.57
39.60
88.21
50.25
91.81
37.56
94.25
30.52
92.
36,40
66
42.88
49:40
91.27
39
94.04
30.80
92.12
39,58
$9.09
45.25
$8.32
49.50
90.89
41.36
94.66
30.49
92.97
35.25
90.39
41.49
$6.90
55.05
91.23
40.57
Gaussian Blur
94.41
30.95
93.16
33.84
90.08
41.50
$7.99
$1.38
91.41
39.42
93.94
33.01
92.65
36.77
$9.59
43.91
$5.03
$0.90
91.05
41.15
93.52
33.3
42.03
88.64
47.39
$0.25
90.44
43.24
94 25
32,70
93.00
35.24
90.22
41.99
85.46
5% 64
9U1.3
42 16
Sputter
94
34.14
93.01
36,06
90.01
42.80
84.68
61.05
96147
43.51
92.25
44,55
92.39
39.15
89.66
44.66
84.29
61.74
47.52
92.06
22.30
46.10
48.47
95. 14
27.49
92.75
36.52
90.00
43.86
86.74
56.31
91.16
41 05
96.06
22.22
92.38
38.12
$9.74
41.80
$7.15
51.27
91.33
39.85
92.
94.58
30.42
92. 44
38.03
39.86
44.05
$7.20
54.11
91.02
41.65
94.24
31.27
92.11
38.11
$9.90
43.08
$7.23
$3.37
90.87
11.53
94.71
29.15
92.69
36.08
90.09
42.86
87.12
$4.79
91.15
40.72
Elastic Transform
93.81
34,98
92.17
39.52
39.42
45.50
84.66
61.08
90.02
45.27
93.
38,06
91 80
41,82
$8.94
47.06
82.71
644.72
89.15
47.91
90.57
53.15
91.12
45.97
85.52
49,36
77.98
72.84
$7.05
$5.33
93.39
38.20
99.99
35.26
90.57
41.91
87.04
55 55.00
90.95
42 22.82
JPEG Compression
93. 26
38.71
93.06
34.75
90.37
41.36
87.14
55.50
90.96
42.58
39 86
93.08
92.95
36.21
80.97
42.17
90.57
43.81
944
33.18
92.07
39.90
89.59
45.36
86.50
$7.44
SULSK
43
94.32
31.99
92.07
39.41
89.56
45.51
$6.69
56.79
90.66
43.42
Piselate
944
94.66
29.32
91.42
41.85
47.21
85.64
59.54
90.15
44.49
94..6
29.86
91.79
39.89
$9.11
45.89
85.86
$8.118
90.33
13.53
94.36
33.35
92.40
38.66
89.68
45.02
$7.16
$5.50
90,90
13.13
31.45
Speckle Noise
95.44
26.28
92.03
41.94
39.27
47.41
85.60
56.37
90.84
43.00
96.03
22.62
91.70
43.94
$3.95
49.31
$6.00
$7.23
94.67
43.27
96.49
19.30
91.51
44.35
83.92
49.75
85,44
56.95
90.59
42.59
94.79
29.74
92.95
35.19
89 89
42.51
85.70
$8.92
S0L83
41.59
Glass Blur
94 30
30.71
91.97
40.13
$8.76
46.88
$2.40
63.01
89.36
45.18
30.81
92.23
94.12
34.13
92.23
39.59
89.65
45.07
86.95
$5.76
90.74
43.64
94.34
32.43
92.18
39.25
89.44
45.64
87.27
53.53
90.81
42.71
Gaussian Noise
28.69
39.77
94.57
28.32
91.22
44.79
$3.50
49.26
86.40
56.28
90.25
14.66
95.47
23,58
90.74
47.27
$3.26
$2.06
85.40
$7.82
89.97
45.1
23

---

Table 11: (Extension of Table 10) Full results CoVer under one extended imput with 18 alternative types of corruptions at 5 severity levels based on ELI6-B/16. The ID data are ImageNet-1K
Comuption Type
Severity Level
OOD Dainset
iNaturalist
SUN
Places
Textures
Average
AUROC1
FPR95]
AUROC1
FPR95
ITP955,
94.37
32.83
92.34
39.34
89.59
45.12
87.14
55.73
90.86
43.25
94.82
29.23
92.29
38.92
59.19
44.90
87.18
55.21
90.95
42,07
Sbot Noise
26.53
91.81
42.03
47.28
86.77
55.67
90.71
42.88
95.80
22.83
91.08
46.01
$8.56
50.64
85.74
57.64
90.30
44.28
95.97
21.26
90.85
46.16
88.44
51.13
85.18
90.11
44.16
94.63
29.75
93.16
32.44
90.20
40.27
87.18
$2.57
91.29
38.76
2
94.08
33.05
92.70
35.13
$9.62
42,70
87.00
53.03
90.85
40.98
Zoom Blur
35.87
37.36
45.09
85.95
55,66
90.15
43,49
92.95
37, 97.98
91.65
39.51
$8.48
46.92
85.63
57.04
68
45,36
92.35
40.41
90.93
42.52
$7.93
49.05
84.46
59.27
88.92
47.81
94.35
42.80
94.63
30.66
92.23
37.93
$9.18
44.87
86.15
56.47
90.55
Snow
94.57
30.42
92.18
37.944
89.18
44,
86.20
55.80
901.3
42.23
95.35
26.59
91.94
39.16
$8.76
45.93
$6.60
$3.40
90.66
41.27
93.33
41.46
92.61
38.58
89.70
44.57
86.37
57.32
90.50
45,
Impulse Noise
94.43
31.89
91.41
45.29
88.63
49.90 99.98
86.05
55.76
90.13
45.73
95.11
26.64
90.73
47.75
88.04
$2,06
86.01
54,56
89.97
45.25
53.49
55.0
Severity
52.5
Severity
50.0
Severity
Severity
FPR95
47.5
Severity
45.0-
42.5
40.0
37.5
35.0-
Brightness
Fog
Contrast
Motion Bur
Defocus Blur Gaussian Blur Corruption Type
Spatter
Saturate
Elastic
Origin
Severity
52.5
Severity
50.0
Severity
Severity
FPR95
47.5
Severity
42.5
37.5
Gaussian Corruption Type
Figure 7: Performance variation trends of CoVer by varying severity levels under different types of corruptions.

---

C.3 Exploration on the Formulation of CoVer
In Section 3.3, we discuss about the scheme of designing the OOD score with extra corrupted inputs:
Scover = - _x (S(x)), d(x, == {x, (() x E &, CRECECC C},,
(17)
The critical mechanism of ScoVer can be summarized as follows: Scover is an confidence expectation with respect to the OOD score estimated from natural input and its corrupted variant.
To have a closer look at our proposed CoVer, we further separately explain the implementation mechanism of Co at the input level and the output level. At the input level, CoVer an extension of the input representation dimension, which introduces additional variant feature inputs into the model. On the other hand, Ver is an extension of the confidence assessment dimension at the output level, which enables the integration of multi-dimmnnsioal confidence by averaging the confidence under every representation dimension. Such a design criterion leaves a further exploration on the specific formulation of Co Ver. In other words, the realization of the estimated confidence score under each evaluation dimension can have multiple choices.
Table 12 shows the results of different implementation schemes for the CoVer score function. We also provide the original score function formulations of each method and their performance for comparison. All experiments in this table are evaluated on ImageNet-IK IK benchmark and all methods based CLIP-B/16. The results indicate that these scoring methods can achieve more comparable performances under the setting of the CoVer mode, as analyzed in Section 4.2,
Turning the attention to the implementation form of the proposed Co we divide it into two categories: traditional scoring functions (from row to row 4) and newly designed scoring functions based negative textual logic (from row 5 to row 8). For traditional maximum-softnax and free energy scoring functions, we simply employ the expectation of them on the expanded multiple representation dimensions realize our Co as demonstrated in Eq. (17). For novel CLIPN and NegLabel scoring functions that exploit extra textual information, we apply the CoVer's 's mechanism the calculation of the similarity between the image input and the negative text input. Formally can be summarized as follows:
Scover (x) =
St (sim(x. y), sim (z,y
(18)
where x ~ d(x) is denoted as the input image sampled from original input space, and X ~ d(X', 2) refers to the imput image sampled from expanded input space. and X are involved in calculating the similarity to the positive textual concept y and negative textual concept y respectively. S is the realization form of CLIPN and NegLabel, which can be found in row 6 and row 8 of Table 12.
Table 12: Comparison with the original and Co Ve''s's modified form of different scoring functions based on CLIP-B/16. All experiments are evaluated on ImageNet- IK benchmark.
GOO Daraet
Metaol
Mide
Funtiins
SUN
Touerze
Ansrage
AUBOCC
19899
AUROCT
11995)
AUBECC
TVRNS
AUROUTI
[7895)
AURDCT
Oiiginal
$4.51
30.95
90.57
37.57
95.77
44.55
80.10
57.37
90.70
42.73
MSP
Culla
95.98
22.55
93.42
32.85
90.27
46.7
90.14
43.39
92.45
34,38
Diigiaal
bang
CW
Ey Je attee
56.28
$6.17
92.11
Otiiinal
95A3
21.62
92.15
30.51
MNL34
93.35
29.06
CLIPN
CaVer
95-41
25.34
M5.72
17.15
$4300
25005
10.99
113
36/91
Onginal
59.43
1.55
95.46
28.55
91.55
26.45
50.85
45.12
94.10
36.11
Neglabel
Cakee
JAMN
99.39
1.15
14.26
25
35JI1
25,55
$2.19
95.30
24/11
25

---

C.4 Visualization Analysis
C.4.1 Visualization of Corrupted ID and OOD Samples
In Figure 8, we provide the visualized examples under 18 different types of algorithmically generated corruptions which are officially introduced in [22]. Furthermore, there are 5 severity levels for each corruption style, ranging from inconsequential devastating corruption of the original clean image as shown in Figure 9. By synthesizing these 18 types and severity levels, we generate a comprehensive and diverse set of 90 distinct forms corrupted inputs, which can leveraged to enhance dimensionality of input representation.
Spatter
Saturate
Elastic
Snow
Fog
Brightness
Gaussian Noise
Speekle Noise
Shot Noise
Impulie Noise
Glass Blur
Motion Blur
Zoom Blur
Defocus Blur
Gaussia Blur
Original Sample
Corrupted Samples
Figure 8: Visualization of an original sample and its corrupted instances under each corruption type officially defined [22]
Clean
Severity
Severity
Severity
Severity
Severity
Figure 9: Visualization of varying severity levels, with Impulse Noise, Snow, and Glass Blur (all introduced in [22]) modestly markedly corrupting the natural clean image.
26

---

C.4.2 Comparison with Salient Maps and Corresponding Confidence Variations.
To better illustrate the effectiveness of Co which primarily stems from the differing confidence variations between ID and OOD samples under identical corruption conditions, we provide some visualization results of ID and OOD images, as shown in Figure 10 and Figure 11. All the images are picked from the datasets in the ImageNet-IK OOD detection benchmark. Each subfigure shows the feature maps of the original image and its corrupted variants, including Contrast (Severity Level 4) and Defocus Blur (Severity Level 2). For comparison, we also provide the corresponding confidence variations between the original (red bar), the corrupted (blue bar), and the averaged (yellow bar) one. The confidence scores are based on the form of maximum-sofftaal scoring function given by CLIP-B/16. We continue to divide the data into four categories, denoted as confident ID data (refer to row 1 to row 2 in Figure 10), unconfident ID data (refer to row to row 6 in Figure 10), overconfident OOD data (refer to row 1 to row 4 in Figure 11), and unconfident OOD data (refer to row 5 to row 6 in Figure 11). Here we focus on the differences between unconfident ID data and overconfident OOD verify the analysis claimed in Section 3.2.
In Figure 10, it obvious that ID images have more significant ID-semantic feature activations (see the foreground salient responses) due to the knowledge of the ID label space. Firstly, for confident ID) data, the changes in confidence post-coruppion can manifest as either minor or abrupt. For example, in the image in row 1, column 1 (ILSVRC2012. val 00020025), the corruption results in only negligible loss of the semantics of the ID category present in the foreground. Conversely, for the image in row 2, column 1 (ILSVRC2012. val mal_0004440), the corruption enhances disturbances from non-semantic areas, leading to a loss of model confidence. Nevertheless, by averaging the original and corrupted confidence scores, the confident ID data remains stable within higher confidence interval, demonstrating the superiority of Corer's mechanism. Secondly, for unconfident ID data, due the presence of ID semantic features, the model exhibits resilience in its confidence when subjected to corruption. For instance, the image in row 4, column (ILSVRC2012, 2 ___al__rr_ mll_0202)53) shows the model's resilience, it continues focus the foreground regions belonging to the II) category despite various degrees of corruption, thus maintaining its confidence score unaffected. The same results can also be seen in row 3, column I (ILSVRC2012 val. 00048997) and row column (ILSVRC2012_wa) val_a002201199
In Figure 11, particularly in the case of overconfident OOD images, there is excessive reaction to the Corruption. This phenomenon is apparent because, unlike ID data, OOD images inherently lack semantic information about ID categories, leading to the disappearance of areas with high feature activation under the same type and severity level of corruption. For instance, the image in row 2, column (f_ fororoll__aaa _araall_aan_____e gawgggarddddodeeee 00003688) demonstrates that regions highly responsive in their original state become irrelevant after corruption, resulting in the confidence mutation. This shift further confirms that the model's 'sss overconfidence them primarily due to misleading non-semantic features, such as textures and styles. Similar results can also be found in other OOD datasets, such as the image in row 3, column 1 (sun arsrlxiznzlekfvv). and the image in row 4. column (wrinkled_ __070). However, the confidence of unconfident ID data, comparable to that of these overconfident OOD samples, remains stable. Consequently, by averaging the confidence scores before and after corruption, CoVer effectively captures the distinctions between OOD and ID data that initially overlap, thus enhancing the separability of ID and OOD distributions. Furthermore, for unconfident OOD data, due to their overall low relevance to ID semantics, the confidence scores consistently remain in lower range (like the image in row 5, column 1, named 7153886677 5eddea4444655111266 5ed7766
27

---

Name: 1S50R22012_ 00000025
Nome: 15VRC2012 0001
Drggn
Contilerze
Orign
Contre
Confidenee Variiloo
140
20m
5.0
Datauet imagelet
Octesst Name: imagetest
Drign
Contrist
Defocua Blur
Contidence Varation
Orign
Cantrest
Delccus Blar
130
Confidenze Variaton
1.4
1.00
300
Dosacet Nome: ImageNet
Docasct Name: Inapelles LSVRC2002
Drign
Content
Defrous Rlvr
Confiderce Varoatonn
Origin
Contintt
Delinus Bllrr
Convidience Vistatie
Conuciet
Comptiot
300
1.80
100
Comruption 7ypm
Campppiin
Battuue Nome: Imagelier
Datowet Nome: imapettee 9VB03812
Name: 1SSRE20022 00007931
Rg home
Defocus
Corfdennee
Comras
Convettr
200
200
Compu plaon 7p
Compptinn 7pp
Ditases
Name: LL9992202_ N0000055
Fig Name: LSV9C2012,>> 00031446
Origin
Variation only
Oriin
Confiidenne Ubristion
Averge
Aveice
200
1.09
00020767
LSVIC1012
Origin
Contost
Defocus Blur
Confidence Variaiio
Orgin
Contraxt
BeRicos Har
Confidenze Variatoon
Onigne
1.04
300
3.00
Figure 10: Case visualization of ID images. The left part of each subfigure contains the original image (with the dataset name and filename) and its corruptions with their feature maps. The right part shows the confidence variations corresponding each corruption.
28

---

Nome: 500ceStccatt 2dd75822ee 15240/5 1524ab75e
Drigin
Defoou
Contilerce
Orign
Cuntrse
Miler
Confidence Variatoo
Detzact Name:
Drign
Contrist
Defocua Blrr
Corfidencs Varation
Orgnn
Cantrest
Delccus Bayr
Confitence Verision
1.09
200
Dusaset Nome: SUN ant.arrriiik
Octosst Name. SUN sin, www.ioffrrrrrooottoo
Drign
Contmet
Defoous Rax
Confiderce Varitton
Orignn
Contrntt
Delonus Blir
Contidienc Vistarie
Camapte
200
100
190
Compaptin
Compptiin
Detastt Mome: Todtyoss Rg home gr12
Dataoet Kome: Tratunes
Name: rided 9000
Defocus
Corfeeenc
Comapsel
4.30
Ca Compption
Comrpptin
Fig Name: MW2.artee1550000944455555
Fig Namii Nameis,kk,, s.iky_ 0000234
Origi
Confidences Variation
Orgin
Confiidenze Unvistion
ong set Avarge
1.30
1.00
Compption
Origin
Contrott
Defocus Blur
Corfidence Variaion
Orgin
Contrax
DeRicus Har
Confilence Veriatoo
Oriendl
L33
1.00
Figure 11: Case visualization of OOD images. The left part of each subfigure contains the original image (with the dataset name and filename) and its corruptions with their feature maps. The right part shows the confidence variations corresponding each corruption.
29

---

D Preliminary Statistical Analysis
In this section, we present the statistical implications with detailed definitions and assumptions. The primary objective is to show that CoVer can lead to an increase in the separability of the distributions of ID and OOD by introducing confidence expectations under the extended representation dimension. In the following parts, we first introduce the considering performance metric and some preliminary setups for the analyses.
The separability between ID and OOD data can be reflected by the FPRx, which is the performance metric of an OOD detector, defined as follows,
Metric.
FPR Faut (A)
(19)
where Pm and Fou represent the cumulative distribution functions (CDFs) corresponding to the confidence scores obtained by ID and OOD samples, respectively. À [0. 1] is denoted as the true positive rate (TPR), indicating the proportion of samples that are correctly classified as one of the ID categories. The metric FPRA quantifies the overlap degree between the scores that the OOD detector assigns to ID and OOD samples, with lower values indicative of superior performance.
Preliminary setups. Following previous works [32, 31], owing to the robust representational capabilities of pre-trained DNNs and the consistent alignment between cosine similarity scores and labels observed in CLIP-like models, we assume that the features extracted from DNNs or the cosine similarity scores in CLIP-like models approximately conform to Gaussian Mixture Model (GMM) with equal class priors: (2-N (Hpp, Tps) + 1 (ppos pp))). where ppi and ppo are the means of the ID and OOD distribution, while pii and a po the corresponding standard deviations.. Specifically, we use DID = N (Jip,, api) and DOOD = N (#'o: pp) denote the ID marginal distribution and the OOD marginal distribution, respectively.
Assumptions. Refer to Figure 1 and Figure 6, the comparisons of score distributions obtained by different input modes, we can derive a series assumptions about the variation relationships between Myi and ppo. and between dpi and ppo Empirical exploration can be found in Figure 12.
Assumption D.1 (Variation of Pep and Hyo).. Let Ap and Appo represent the changes in the means of ID and OOD distributions, respectively, after corruption and averaging. We assume that AApii
Ap resulting in narrowing gap between Pli and Pro.
This assumption predicated on the observation that the means of ID distributions, Mps. decrease more significantly under identical corruption levels compared to OOD distributions, Hyo- The generally higher initial confidence scores of ID samples make their means more susceptible to substantial decreases (refer to the left panel of Figure 12). This reduction is greater than that experienced by OOD samples, thereby significantly narrowing the gap between Uyd and plpo. For an intuitive example, the gap between the means of confident ID data (left panel of Figure 12) and overconfident OOD data (right-middle panel of Figure 12) is closer. This illustrates the pronounced impact of the averaging operation on ID distributions compared to OOD distributions.
Assumption D.2 (Variation of ps and poo) We define the changes in the variances of ID and OOD distributions as p and Adpas respectively. We postulate that the reduction variance for ID distributions, Aopis i is greater than that for OOD distributions, Ao A AA
This assumption is supported by the observation that high-confidence ID samples, due to their higher initial confidence levels, experience larger and more abrupt drops in confidence upon corruption (see the left panel of Figure 12). Consequently, the averaging process post-corruption results in a significantly greater reduction in the variance Tpi in ID distributions compared to Tno in OOD distributions. This marked decrease in variability within ID confidence scores, relative to the OOD ones, underscores the efficacy of the averaging operation in dramatically stabilizing the ID confidence scores more than the adjustments observed in OOD confidence scores.
Given the preliminaries and assumptions above, we can derive the following extended lemma to demonstrate the superior performance of CoVer.
Lemma D.3 (Declination of FPR2). Assuming the variation relationships between pei and plpn and between yi and po. CoVer enables a lower FPR
30

---

Overco
organ
orgenal
cogin
Corlidecc Sopre (1-3)
Confirece Scote
Cont Centounce Soore
1:30
Comvenoe
Gnso Iuaaae
Uocatfieess Awaage
desrge
Contideece Soore 13:33
Contidence Soure 11.
Scges
Spore
1:100
Figure 12:
Empirical exploration evidence the proposed assumptions. Based Figure 2, further present scatter maps of averaged confidence scores for comparison.
Proof of Lemma D.3. We aim to investigate the relationship between the FPRA metric and the variations in ppis Mpos Opis and po. The FPR, metric can be reformulated as follows:
FPRA = Φ ( (A; (Ai opi): Ppo ppo)
+ (Mps + Qpi 4 +1(()=1 (A);; ppos po))
Mpi + ppi ② 2 U4o
(20)
po
where & is he cumulative distribution function the Gaussian distribution, and Φ is its inverse function. Considering the differences before applying CoVer:
Aμ P'ps P'po
Ao api Tp
(21)
With the assumptions that Aμ and Ao are affected by the averaging process in CoVer, we express these changes as:
A Annww
= 11ps po
Arwew pr o'
(22)
where p'g and plp the new means Ver, d'r and p are the variances post-Co Ver. Given that Aynew is reduced and Annew indicates significant contraction, particularly a reduction in d'u relative "no- the numerator in (Jyi p - 11po)//po- decreases while the denominator basically unchanged. This results in the argument of becoming smaller. Since & is monotonically increasing, decrease in the argument directly translates to lower value of FPRx, thereby reducing the probability of falsely classifying OOD samples as ID.
This analysis underscores the benefit of CoVer, particularly through its influence on expanding representation dimensions and optimizing the detection framework. By narrowing the gap between the mean scores and significantly reducing the variance in ID distributions relative OOD, Vever enhances the model's 's discriminative capability and improves its robustness, ultimately leading to more reliable OOD detection.
E
Future Analysis
E.1
Discussion about Extra Runtime
Considering the addition of corrupted inputs, CoVer would result in non-negligible additional runtime. there are N expanded dimensions, it will take N times the duration single input to implement CoVer. However, our CoVer is only applied in the inference phase of OOD detection, and it is generally fast, as shown in Table 13. We believe that the performance improvements offered by Co Ver are well- worth the extra few minutes of runtime.
31

---

Table 13: Inference runtime of single input using single RTX 3090 GPU based on CLIP-B/16.
Dataset
Тype
Inference time (8)
ImageNet
ID
121 (#11)
iNaturalist
OOD
45 (=
SUN
0OD
41 (=12)
Places
0OD
33 (188
Textures
0OD
16 (+1)
E.2 Exploring the Impact of Corruption Types and Severity Levels
Appropriate types of corruptions and corresponding severity levels are crucial for the optimization of our CoVer. all experiments, we use the SVHN dataset as the validation set to select the most effective corruption types for each method. Specific examples of selections are provided in Table 14. For the types of corrupted inputs and their corresponding severity levels, we have conducted related explorations (e.g., Tables 10 and 11, Figures 6 and 7 of original submission) for performance references. Some specific corruptions (e.g., Brightness, Fog, Contrast, Motion Blur, Defocus Blur) generally improve the OOD detection performance, as the corruptions semantic level of the input, instead of damaging the semantic features too much like the other types. Empirically, refer Table 15, can use the same type of corruption as the expanded input (e.g.. here Brightness with severity 1 used) to perform better than the original version. This provides the verification of the previous intuition about the general guidance for choosing appropriate corruption types, and understanding dimension expansion for OOD detection.
mainly the non-
Table 14: Examples of corruption types selections in the utilized validation set SVHN based on ResNe150 Selected?
Dan
Deat
Method
Expanded Type
AUROCT
FPR95
ImageNet-IK
SVHN
MSP
97.45
13.80
ImageNet-1K
SVHN
MSP
Brightness
98.81
6.36
ImageNet-1K IK
SVHN
MSP
Fog 98.83
6.52
ImageNet-1K K
SVHN
MSP
Motion Blur
98.62
7.69
X ImageNet-1K
SVHN
MSP
Snow
89.43
56.47
ImageNet-1K
SVHN
MSP
Impulse Noise
94.71
29.74
ImageNet-IK
SVHN
MSP
Spatter
95.87
28.03
Table 15: CoVer combined with each method using the same expanded corruption type.
Architecture
Din
Method
Expanded Type
AUROCT
FPR95.L
ResNet50
ImageNet-1e
ReAct
92.95
31.43
ResNet50
ImageNct-11 1K
ReAct CoVer
Brightness(1)
93.94
28.10
ResNet50
ImageNet-IK
DICE
90.77
34.75
ResNet50
ImageNet-1K
DICE Co Ver
Brightness(1)
91.96
31.66
ResNet50
ImageNet-IK
ASH-B
90.91
39.04
ResNet50
ImageNet-1K
ASH-B CoVer
Brightness(1)
92.24
30.55
CLIP-B/16
ImageNet-1K
MCM
90.76
42.73
CLIP-B/16
ImageNet-IK
MCM CoVer
Brightness(1)
91.05
41.49
CLIP-B/16
ImageNet-IK
CLIPN
93.35
29,66
CLIP-B/16
ImageNet-IK
CLIPN + CoVer
Brightness(1)
93.47
27.82
CLIP-B/16
ImageNet-IK
NegLabel
94.10
26.11
CLIP-B/16
ImageNet-1K
NegLabel + CoVer
Brightness(1)
95.15
24.99
32

---

E.3 Exploration on the harder OOD dataset
NINCO [4] has proposed three OOD datasets with no categorical contamination which include NINCO, OOD unit-tests, and NINCO popular OOD datasets subsamples, which are demonstrated to be harder than common OOD detection benchmarks. Here, we evaluate the effectiveness of CoVer on these datasets in 16. The results demonstrate that CoVer, when combined with ASH, consistently achieves better performance across the three NINCO OOD datasets.
Table 16: The overall results of Ver on three NINCO OOD datasets without leveraging VLMs/CLIP. The ID data are ImageNet-1K.
Architecture
Din
Dout NINCO NINCO NINCO unit-tests
Method
AUROCT
FPR95
ASH
82.26
69.22
AHS + CoVer
82.80
68.59
ASH
99.13
4.85
ResNet50
ImageNet-IK
NINCO unit-tests
AHS + Co
99.49
2.12
NINCO subsamples
ASH
82.07
56.10
NINCO subsamples
AHS + CoVer
82.67
54.44
E.4
Comparison with other Competitive Methods
Comparison with NNGuide and MaxLogit. To provide comprehensive comparisons, have conducted comparison experiments with NNGuide [39] and MaxLogit [21] to enrich our analysis in Table 17. First, experimental results show that CoVer outperforms these competitive post-hoc methods on the ResNet50) architecture. Second, the performance of these post-doc methods, especially NNGuide, encounter significant drop when conducted on CLIP-B/16 architecture. We believe the reason for the poor performance is the difference in training data. Many pos-hoc methods are designed on ImageNet pre-trained networks, where only ID data are used during training. In contrast, when training CLIP, both ID datas and OOD datas are used. This leads to different activations of OOD data. Another reason that the pos-hoc method relies heavily on the choice of hyperparameters. The hyperparameters of NNGuide need to be re-selected on different models. Despite these issues, our CoVer can still perform better than these methods. Furthermore, we also combine our CoVer with MaxLogit and NNGuide and report the results in Table 18, which further demonstrates the effectiveness and compatibility of our method.
Table 17: Comparison with NNGuide and MaxLogit based on ResNet50 and CLIP-B/16. The ID data are ImageNet-IK.
OOD
Architettore
Method
INuturalist
Textares
Averge
AUERCC
ALEOCC
[PRO5]
FPR95]
MaxLogit
50.91
86.59
57.70
NNGuiderk=1)
93.13
34.06
90.41
38.86
88.06
91.67
29.59
37.57
RecNet50
XNGuiderk=10)
91.33
29.27
91.23
36.4
88.71
46.2
92.93
26.31
91,30
34.55
XNGuidzrk-100
95.10
26.06
91.44
36.86
88.63
47.64
93.61
24.17
92.19
33.68
CoVer (ours)
97.14
14.04
94.12
25.77
91.05
35.93
91.93
30.39
93.56
36.53
MasLopit
89.31
61 66
87.43
64.39
85.95
63.67
71.68
56.61
83.59
09.08
XNGonderk=l)
65.05
38
68.35 19.56
97.27
72.19
93.51
66.06
98.49
67.97
97. 16
CLIP-B/16
1NG
60.95
99 68
68.06
06
71.65
94.83
62.61
98. 00
65.83
97 89
(NNuideck=100)
51.54
99 85
64 84
68. 74.74
96 40
$3.26
09.63
59
06. 70
CoNer lours)
95.08
22.55
93.42
32.85
90.27
40.71
98.14
43.30
92.45
34.88
Table 18: Compatibility experiments of CoVer combined with NNGuide and MaxLogit based ResNe150 and CLIP-B/16. The ID data are ImageNet-
OOD Duinact
Archinecture
Method
(Natoralis:
SUN
Places
Testures
Average
AUROCT
[P599]]
AURRC!
[18951
AUROCT
[PR95]
AUROC"
JFR95..
AUROCT
FPR955
MacLogit
CoN
MaxLogi
91.13
50.91
36.59
50.87
86.18
65.58
86
56.36
67.00
$7.70
CoNoe
92.85
42.19
$7.19
SU.17
8LU7
63.84
$6.59
$4.10
87.90
$4.38
NNGuiddrtee))
93113
34.06
90.41
38,56
85.05
47.46
01.67
20.30
90.51
37.57
NNGande CWeee
NNGoldde Corre
91.91
31.31
NNGusadd (103)
95.10
26.06
91 44
36.95
08.63
47.64
93.61
24.11
92.19
33.65
NNGulde (k- Ca
96-42
19.96
92.20
$4,43
89.39
44.34
93.79
23.58
92.95
30.45
33

---

Comparison with Watermarking. In addition, Watermarking [54] is another competitive methods needed be considered. We have added the analysis about the Watermarking method with our Co in the following two aspects.
Conceptually, we have noticted that Watermarking utilizes a well-trained mask to help the original images be distinguishable from the OOD data. However, Watermarking is still trying to excavate the useful feature representation in a single-input perspective. In contrast, the critical distinguishable point and also the advantage of our CoVer method lies in input dimension expansion with the corrupt variants, which instead provide a extra dimension explore the confidence mutation to better identify the OOD samples.
Experimentally, we have conducted the comparison and report the results in Table 19. The results show that, on the one hand, training an optimized watermarking for effectively distinguishing between ID and OOD samples is a time-consuming process. On the other hand, Co achieves this by introducing corrupted inputs to capture the confidence variations between ID and OOD data during the test phase, which is simpler, faster, and more effective.
Table 19: Comparison with competitive OOD detection method Watermark based on ResNet50. The ID data are ImageNet-IK.
OOD Danset
Architecture
Misiuad
iNstunaii
SUN
Plases
Toxtanes
Avcrage
AUROC1
FFROEL
AUROC1
F99995)
AUROCT
F9R95]
AUBOCT
FPR95)
AUROCT
FPR95,
Runtine
ResNed00
MSPsCoVVer
71.43
67.53
3.
Comparison with Data Depth, Information Projection, and Isolation Forest. We have also conducted comparison experiments between our CoVer and baselines methods from data depths, information projections, and isolation forest, as detailed in Table 20.
Due to the large scale of the ImageNet training set, we sampled 50 samples per class to construct a subset from the training data to represent the training distribution, recommended by the similar work named NNGuide [39]. For data depths, we reimplemented APPROVED [41] for comparison. For information projections, we reproduced REFEREE [40] for comparison. For Isolation Forest, we use logits as the input to detect the anomaly logits in ID and OOD samples.
The results indicate that AD and textual OOD detection methods, such as Data Depth and Information Projection, may not suit for visual OOD detection tasks, view also mentioned in related surveys [8, 9I. Similarly, classical ML methods for AD, such as Isolation Forest, seem to be failed to excavate discriminative representations when applied to image OOD detection. However, since these methods are insightful in distinguishing the outliers, we believe it is worth further efforts in the future to adopt the critical intuition into the OOD detection problem.
Table 20: Comparison with competitive anomaly detection and textual OOD detection baselines based on ResNet50. The selected methods' types are Data Depth, Information Projection and Isolation Forest, respectively. The ID data are Ie.
OOD Dataset
Architecture Method
NNtumalist
SUN
Places
Testures
Average
AUROCT
FPRUS
AUROCT
FPRUS]
AUROCT
FPRUS]
AUROCT
FPRUS]
AUROCT
FPRUK]
APPROVED
$3.25
95.90
60.01
94.63
56.77
94.25
70.21
78. 30
60.05
90.77
ResNor50
Isolation Fouest
REFERBE
70.76
79.77
85:94
94.76
73.28
59.55
94.78
96.91
60.27
72.9
03.91
96.80
65,89
74.01
81.37
94.08
64.12
74.99
80,001
95.664
CoVer
90.81
44.90
82.51
66.38
81.57
60.34
31.00
65.43
83.97
61.41
E.5
Compatibility with each DNN-based mehtods
It is worth to note that, in Table 1, we only reported the results of Ver combined with ASH because best demonstrates the excellence of CoVer. In Table 3, we also show the results of Co combined with DICE and ReAct, and Co can also provide performance gains for them. Here in Table 21, we further report the comparison of Ver combined with each mentioned DNN-based methods (adding MSP. ODIN, and Energy score), which strongly demonstrates its superiority.
34

---

Table 21: Compatibility experiments of Cover combined with each mentioned DNN- OOD detection method. The ID data are ImageNet- IK
Architectune
Mathed
INuturalist
SUN
Places
Textures
AURUCT
AUROCT
FPR95
FP895]
AURUC"
HFR55]
AURUCT
HPR95]
MSP
70.83
73.99
66.00
66.15
YoPe
44.49
66.35
69.34
81.00
65.43
83.97
ODIN
91.37
41.57
53.97
45.53
20.80
ODIN+CoVer
93,06
31.56
51.47
85.98
58. 69
87.97
44.77
$8.94
46.62
59.85
55.72
55,89
59.2
82.56
54.52
53.72
58.41
ResNe150
KeAcT
KoVer
92.23
96.22
20.35
46.67
87.42
94.20
24.20
$6.50
$4.98
91 11.58
63.16
33.35
$6.99
59.80
47.30
51.70
87.91
92.95
54,51
31.45
RoArt&CoVer
97.58
13.35
95.7
18.91
93.08
29.82
91.55
40.74
94.48
25.51
DICE
94.49
25.
90.83
35.15
87.45
$6.49
90.30
31.72
90.77
14.75
DiCE-CoVver
96.8
16.56
93.53
28.53
90.00
40.54
91.14
31.15
92.87
29.19
ASII-D
94.225
28.95
90.32
40.21
87:53
29.5 52
91.53
13.18
90.91
19.04
ASHI-B+CoNer
97.14
14,04
94.12
25.77
91.05
35.93
91.93
30.39
93.56
26.53
NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have clearly stated the main contributions made in the paper and important assumptions and limitations in both the Abstract and the Introduction (Section 1).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
In Section 5.2, we have discussed the limitations of this work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
35

---

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include new theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have presented a reproducibility statement at the beginning of the Appendix. In addition, we have provided the code and data in the supplementary material reproduce the main experimental results of the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
36

---

(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b)
If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g.. a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described supplemental material?
Answer: [Yes]
Justification: We have provided open access to the code and data with sufficient instructions in the supplementary material to faithfully reproduce the main experimental results of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips. wtwps.cccpubllli.. cc/ppblic/gguiiiccccceeeeooppeel wwwedeeeeeooebmmieeeeeee
weriisssssopiiiioii for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Section 4.1, we have briefly introduced the experimental setups about datasets, model setups, and evaluation metrics. Furthermore, we have provided full details experimental settings in Appendix C.I.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
37

---

The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In the third ablation study about the number of expanded measuring dimensions of our proposed method, the results are accompanied by error bars, and the factors of variability are clearly stated (refer to Section 4.3). In Appendix C.2.3, we have provided the full results of this experiment.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided sufficient information on the computer resources, such as the type, amount, and memory of compute workers GPU required for the experiments the reproducibility statement at the beginning of the Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: /t/n/nnuripp... httppp/.c.c/pp..ttttcccceeeeeeee
Answer: [Yes]
38

---

The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics, and the authors remain anonymous.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In Section 5.1, we have discussed the potential broader impact of this paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
39

---

12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All used assets in the paper properly credited, with explicit mentions of their licenses and terms use, in full compliance with the recommended guidelines.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi wwwoee..c........ com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
40

---

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
41


Reviews:
[{"review_id": "BzKmpicrNX", "reviewer": "Reviewer_LFoY", "summary": "The paper aims to identify Out-Of-Distribution (OOD) samples by applying common image corruptions (noise, blur, etc.) to the input. The  phenomenon is referred to as confidence mutation, where original inputs, along with corruptions, increase the confidence of in-distribution (ID) data, while the confidence of OOD data decreases. Also, a new scoring function is proposed, Confidence aVerage (CoVer), which averages the OOD scores of different corrupted inputs and the original one.\n\nExperiments are on 2 benchmarks (1) traditional OOD - ImageNet-1K as In-Distribution (ID) and iNaturalist, SUN, Places, and Textures as OOD (2) zero-shot OOD - mixture of ImageNet-10, ImageNet-20, ImageNet-100, WaterBirds, and Spurious. The metrics used for evaluation are AUROC and FPR95.\n\nMethods exhibit better performance with the addition of CoVer.", "questions": "In Table 1, why is CoVer only combined with ASH and not other methods? \n\nTable 6 in the appendix demonstrates that CoVer is integrated with other methods using different types of corruptions and severity levels for each method. This raises questions about the comparisons. Why are the types and levels of corruptions not the same across all methods?\n\nSince the paper has different ablations (various corruption types, number of dimensions/corrupted inputs, severity levels) and the experiments make an explicit choice on each one, was a validation set ever used? I did not see anything in the code or the paper, indicating choices were tuned on the test set.\n\nConsidering that other methods only use one input, how much additional runtime does CoVer add? For example, given the size of ImageNet-1K, using CoVer would necessitate extracting 6 million features (1.2 million * 5 corruptions).\n\nHow effective would CoVer be if it encountered much more difficult OOD data without leveraging VLMs/CLIP?", "limitations": "The authors discussed limitations and broader impact in Section Appendix E.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The use of corruptions to improve OOD performance is creative. The implementation seems straightforward and does not require extra datasets. Also, CoVer seems to improve every method it's applied to.\n\nThe experiments utilize similar backbones/datasets as in the literature. This is helpful when comparing with other methods.\n\nAll figures in the paper, including Fig. 1-4, are intuitive and easy to follow. The captions are detailed, and the overview (Fig. 4) makes understanding CoVer very straightforward.\n\nThe paper is well-written and clear. It effectively discusses the problem, presents the idea, and describes the experiments in a coherent manner. Additionally, all equations are clearly explained.", "weaknesses": "The experimentation/idea is somewhat weak and seems more like an application rather than significant knowledge advancement. For example, CoVer is not standalone but applied to an existing method like ASH (see Table 1) or on top of a VLM, thereby leveraging the performance of other methods. Additionally, the claimed contribution of a novel scoring method doesn't seem entirely novel, as averaging is straightforward and an existing scoring function like Softmax/Energy is subsequently employed. \n\nComparisons in Table 1 with different baselines is a little unfair because CoVer makes use of \"extra\" data (corrupted images), while others simply use a single input/image.\n\nEvaluating OOD performance when using VLM/CLIP models is problematic. Such models utilize a vast amount of data which makes it near impossible to know what images/classes have been seen before and are considered \"in-distribution\". While the community continues to use them in various settings, their use with OOD problems should be a clear limitation.\n\nI find it unclear how to determine which corruption types or severity levels to use if I implemented CoVer. The results suggests variability/inconsistency without a clear guideline for decision-making.\n\nThe runtime of CoVer is not discussed. From my understanding, one would need to pass every corrupted input and extract its features, which is time consuming and uses more space.\n\nThere is no discussion of failure cases. When using corrupted inputs, I assume there are instances when CoVer labeled an ID image as OOD.\n\nSince the OOD datasets used in Table 1 differ significantly from ImageNet, it would be intriguing to observe CoVer's performance on more challenging ones such as NINCO [1] (ensures no categorical contamination) without leveraging CLIP.\n\nThe experiments do not utilize or compare with NNGuide [2] or MaxLogit [3], more recent and better performing methods than those in the paper.\n\n[1] Bitterwolf, Julian, Maximilian Müller, and Matthias Hein. \"In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation.\" International Conference on Machine Learning. PMLR, 2023.\n\n[2] Park, Jaewoo, Yoon Gyo Jung, and Andrew Beng Jin Teoh. \"Nearest Neighbor Guidance for Out-of-Distribution Detection.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[3] Hendrycks, Dan, et al. \"Scaling Out-of-Distribution Detection for Real-World Settings.\" International Conference on Machine Learning. PMLR, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time devoted to reviewing this paper and your constructive suggestions. Here are our detailed replies to your questions.\n\n>**W1:** About Novelty\n\nWe would like to reclaim that our work is novel in introducing a new perspective, i.e., expanding the dimension of input with corrupted variants, for OOD detection. **This has not been explored in previous methods [1, 2]**, and has both **conceptual and techniqual distinguishes** from them with significant knowledge advancement.\n\n**Conceptually,** we are the first to expand the raw input with the corrupted ones and identify the phenomenon of confidence mutation, and reveal the underlying mechanism with high-frequency feature change. **The multiple input dimensions introduced in our framework** is the critical advance compared with previous ones focusing on single-input.\n\n**Techniqually,** CoVer captures the intrinsic differences of ID and OOD samples using the averaging operation, as founded to be **simple yet highly effective** and evidenced by our experiments in **Table 1,2,3 of the original submission**. As our idea provides a different perspective, the combination with these SOTA methods aims to demonstrating that **our CoVer reveals the additional enhancement not covered by the basis**. Similar to previous methods (ASH, DICE, ReAct) originate from Softmax/Energy, we believe our work brings new insights on this problem.\n\n> **W2:** About \"extra data\"\n\nThank you for the question. We would like to clarify that **it is our novelty to use extra corrupted inputs to expand the input space, and it is not unfair.** While previous OOD detection methods have distinguished between ID and OOD data based on a single input, **we are the first to identify the challenges in this paradigm and propose expanding it to multiple inputs**, measuring their confidences respectively. With these novel insights, CoVer **provides a new perspective** for the OOD detection problem.\n\n> **W3:** About using VLMs/CLIP\n\nThank you for the insightful comments.  \n\nFirst, we have conducted various experiments of CoVer on the **DNNs architecture (ResNet50)**. The results demonstrate performance improvements and **exhibit the same trend as those on VLMs** (**refer to Table 1 and Table 3 of the original submission**).\n\nSecond, we would like to explain that **the zero-shot settings in our work follow the MCM [3]**, which is the pioneering work in zero-shot OOD detection with VLM. As in MCM, the in-distribution classes in zero-shot OOD detection are the classification task of interest, which is defined by a set of class labels/names $Y_{in}$ instead of the classes used in pre-training. Accordingly, OOD is defined w.r.t. the ID classes, not the data distribution during pre-training. Hence, we only utilize the powerful image encoder to extract visual features and the text encoder to extract textual embeddings as ID concept prototypes, without the prior knowledge for distinguishing ID and OOD classes. We will add this part of explanation to our paper.\n\n> **W4, Q2:** About which corruption types or severity levels to use\n\nThanks for your constructive comments. \n\nFirst, in all experiments, we use the **SVHN dataset as the validation set** to select the most effective corruption types for each method. **We will clearly explain the usage of the validation set in our main paper**.\n\nFor the types of corrupted inputs and their corresponding severity levels, we have conducted related explorations (**e.g.,  Tables 10 and 11, Figures 6 and 7 of our original submission**) for performance references. Some specific corruptions (**e.g., Brightness, Fog, Contrast, Motion Blur, Defocus Blur)** can generally improve the OOD detection performance, as the corruptions are mainly on the non-semantic level of the input, instead of damaging the semantic features too much like the other types.\n\nEmpirically, refer to **Table 3 in attached PDF**, we can use **the same type of corruption** as the expanded input (e.g., here Brightness with severity 1 used) to perform better than the original version. This provides the verification of the previous intuition about the general guidance for choosing appropriate corruption types, and understanding dimension expansion for OOD detection.\n\n> **W5, Q4:** About runtime\n\nThank you for your valuable question. Runtime is indeed an issue we didn't consider. As you mentioned, if there are $N$ expanded dimensions, it will take $N$​ times the duration of a single input to implement CoVer. However, **our CoVer is only applied in the inference phase of OOD detection**, and it is **generally fast**. As shown in **Table 4 in attached PDF**, we report the inference time of each single input on ID and OOD datasets. **We will clearly discuss the runtime issue in the revised version.**\n\n> **W6:** About failure cases\n\nThank you for the constructive comment. We indeed identified some failure cases and reported them in **Figure 7 of the original submission**. When CoVer utilizes certain severe corruption types (e.g., Spatter, Elastic transform), its performance is worse than with single input. This is because these types are more severe compared to others, leading to **excessive damages to semantic features**. Effective corruption types are those **only perturb non-semantic features**, which generally exist at the high-frequency level, resulting in different confidence variations between ID and OOD data. We will incorporate these discussions into the revised version and make it clearer with our empirical results.\n\n**References**:\n\n[1] Yang J, Wang P, Zou D, et al. Openood: Benchmarking generalized out-of-distribution detection. In NeurIPS, 2022.\n\n[2] Yang J, Zhou K, Li Y, et al. Generalized out-of-distribution detection: A survey. In IJCV, 2024.\n\n[3] Ming Y, Cai Z, Gu J, et al. Delving into out-of-distribution detection with vision-language representations. In NeurIPS, 2022.\n\n> **W7, Q5, W8, Q1, Q3**\n\nDue to the space limit, we place our answer in the general response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SoqcQHlef3", "reviewer": "Reviewer_HCvU", "summary": "Authors ntroduce a new approach to out-of-distribution (OOD) detection by expanding input representation dimensions using common corruptions. Traditional methods focus on single-input representations, limiting their effectiveness. This work identifies \"confidence mutation,\" where OOD samples' confidence levels drop significantly under corruptions, while in-distribution (ID) data maintains higher confidence due to resistant semantic features.\n\nThe proposed method, Confidence aVerage (CoVer), averages confidence scores from original and corrupted inputs to improve OOD detection. Extensive experiments show that CoVer enhances performance across various benchmarks. Key contributions include:\n\n1. Expanding input representation dimensions for better OOD detection.\n2. Introducing confidence mutation to distinguish OOD data.\n3. Proposing the CoVer scoring method for improved separability.\n4. Validating CoVer's effectiveness through extensive experiments\n\nThis is a nice tricks that are many connection to others fields such a s NLP.", "questions": "1. Why were data depths, which project onto all possible directions and are naturally suited for OOD detection, not explored in your related work?\n2. Why were classical methods like Isolation Forest, which are well-known for their anomaly detection capabilities, not considered?\n3. Can you compare your results against the previously introduced baselines involving data depths, information projections, and classical methods like Isolation Forest?\n\nTo me the paper is incomplete and the positioning as well as the choice of the methods (react, dice) is poor and deserve the paper, whereas classical ML tools are appropriated. \n\nI encourage the authors to revise.", "limitations": "See above", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Nice idea which have not been applied to OOD detection but is well known on other fields such as NLP (e.g . retrieval)", "weaknesses": "**Omission of Data Depths and Information Projections in Related Work**\n\nData depths and information projections have shown significant promise in the OOD detection community due to their ability to provide robust and high-dimensional representations of data. Data depths, in particular, are well-suited for this problem as they project in all possible directions, capturing the entire distribution's structure and providing a natural fit for OOD detection tasks. Despite their relevance, these approaches are notably absent from the related work section of this paper. Including them could provide a more comprehensive overview of the field and strengthen the contextual foundation of the study. Relevant works that should be considered include:\n\nM. Darrin. \"Unsupervised Layer-wise Score Aggregation for Textual OOD Detection.\"\n\nM. Picot. \"A Halfspace-Mass Depth-Based Method for Adversarial Attack Detection.\" TMLR 2023.\n\nP. Colombo. \"Beyond Mahalanobis Distance for Textual OOD Detection.\" NeurIPS 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time devoted to reviewing this paper and your constructive suggestions. Here are our detailed replies to your questions.\n\n> **W1:** Omission of Data Depths and Information Projections in Related Work \n\nThanks for your valuable suggestions and bringing us those insightful works of data depths and information projections. Here, **we have concluded a related discussion about these studies** in the following. \n\nComputing OOD scores on the embedding output of the last layer of the encoder is not the best choice for textual OOD detection. To address this, [1] proposed aggregating OOD scores across all layers and introduced an extended text OOD classification benchmark, MILTOOD-C. In a similar vein, RainProof [2] introduced a relative information projection framework and a new benchmark called LOFTER on text generators, considering both OOD performance and task-specific metrics. Building on the idea of information projection, REFEREE [3] leveraged I-projection to extract relevant information from the softmax outputs of a network for black-box adversarial attack detection. On the other hand, APPROVED [4] proposed to compute a similarity score between an input sample and the training distribution using the statistical notion of data depth at the logit layer. HAMPER [5] introduced a method to detect adversarial examples by utilizing the concept of data depths, particularly the halfspace-mass (HM) depth, known for its attractive properties and non-differentiability. Furthermore, TRUSTED [6] relied on the information available across all hidden layers of a network, leveraging a novel similarity score based on the Integrate Rank-Weighted depth for textual OOD detection. LAROUSSE [7] employed a new anomaly score built on the HM depth to detect textual adversarial attacks in an unsupervised manner.\n\n**We will add the above to the related work section for providing a comprehensive overview, and also add the formal citations for reference in our revised version.** \n\n> **Q1:** About data depths\n\nThanks for your question. **We have discussed the related work on data depths in the previous response and will add it to the related work section in our revised version**. Since the discussion of the related work in our original submission mainly refers to recent representative reviews of visual OOD detection, like [8] and [9], we conduct limited exploration on this direction. Although our CoVer has conceptual distinguishes with exploring data depths for OOD detection, it is notable that **both provide new perspective for exploring discrimnative features** between the ID and OOD samples beyond the raw inputs.\n\n> **Q2:** About Isolation Forest\n\nThanks for the question. We have noticed that Isolation Forest, though a classical method of anomaly detection (AD), is not a common baseline considered in previous literatures [8, 9] for OOD detection. **In fact, AD is quite different from OOD detection, as AD treats ID samples as a whole [9]**. This means that regardless of the number of classes (or statistical modalities) in ID data, AD does not require differentiation within the ID samples (while OOD detection does). OOD detection considers the knowledge for all the known classes, while AD mainly learns the normal patterns from the majority of data and identifies the anomalies. In our work, **we select the baseline methods following previous well-recognized studies**, such as DNN-based methods (e.g., ReAct, ASH, DICE) and VLM-based methods (e.g., MCM, CLIPN, NegLabel).\n\n> **Q3:** Comparison with data depths, information projections, and Isolation Forest\n\nThank you for the question. **We have conducted comparison experiments between our CoVer and baselines as you mentioned**, as detailed **in Table 8 in attached PDF**. \n\nDue to the large scale of the ImageNet training set, we sampled 50 samples per class to construct a subset from the training data to represent the training distribution, as recommended by the similar work named NNGuide [10]. For data depths, we reimplemented APPROVED [4] for comparison. For information projections, we reproduced REFEREE [3] for comparison. For Isolation Forest, we use logits as the input to detect the anomaly logits in ID and OOD samples. \n\nThe results indicate that AD and textual OOD detection methods, such as Data Depth and Information Projection, may not suit for visual OOD detection tasks, a view also mentioned in related surveys [8, 9]. Similarly, classical ML methods for AD, such as Isolation Forest, seem to be failed to excavate discriminative representations when applied to image OOD detection. However, since these methods are insightful in distinguishing the outliers, we believe **it is worth further efforts in the future to adopt the critical intuition into the OOD detection problem.**\n\n**References:**\n\n [1] Darrin M, Staerman G, Gomes E D C, et al. Unsupervised Layer-wise Score Aggregation for Textual OOD Detection. In AAAI, 2024.\n\n[2] Darrin M, Piantanida P, Colombo P. ainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data. In Arxiv, 2022.\n\n[3] Picot M, Noiry N, Piantanida P, et al. Adversarial Attack Detection Under Realistic Constraints. 2022.\n\n[4] Picot M, Staerman G, Granese F, et al. A Simple Unsupervised Data Depth-based Method to Detect Adversarial Images. 2023.\n\n[5] Colombo P, Picot M, Granese F, et al.  A Halfspace-Mass Depth-Based Method for Adversarial Attack Detection. TMLR, 2023.\n\n[6] Colombo P, Dadalto E, Staerman G, et al. Beyond Mahalanobis Distance for Textual OOD Detection. In NeurIPS, 2022.\n\n[7] Colombo P, Picot M, Noiry N, et al. Toward stronger textual attack detectors. In Arxiv, 2023.\n\n[8] Yang J, Wang P, Zou D, et al. Openood: Benchmarking generalized out-of-distribution detection. In NeurIPS, 2022.\n\n[9] Yang J, Zhou K, Li Y, et al. Generalized out-of-distribution detection: A survey. In IJCV, 2024.\n\n[10] Park J, Jung Y G, Teoh A B J. Nearest neighbor guidance for out-of-distribution detection. In ICCV, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IAbsHGMGb7", "reviewer": "Reviewer_NZqw", "summary": "This paper first identifies the shortcoming of previous out-of-distribution (OOD) detection methods: the single-input paradigm limits the representation dimension for extracting valid information. To address this issue, a novel method CoVer that expands the original input space with multiple types of corruption is proposed. Based on the phenomenon of confidence mutation, CoVer achieves better performance by averaging the confidence scores across different dimensions. The proposed method achieves the best performance in the extensive experiments.", "questions": "1. Since the proposed method is a framework to improve the OOD performance, I wonder how does the proposed CoVer method improve each method? Why the proposed method is effective to improve different types of the method should be further explaiend.\n2. For the proposed method, it mainly focuses on expanding the inputs. While for the different method, they usually focus on different knowledge. I wonder how to decide what types of the inputs that can improve the performance of different methods.", "limitations": "Yes", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation of the paper is easy to follow. The idea of using common corrupted images to expand the input space is both novel and reasonable. While previous methods based on single input often struggle with detecting hard-to-distinguish OOD samples, the proposed method offers a practical approach to improving the performance of OOD detection.\n\n2. The extensive experiments demonstrates the solid contributions of the proposed work and significant improvements have ahieved for different methods. Using the ImageNet-1K benchmark, the experiments validate the compabability of the proposed CoVer method effectively.\n\n3. The proposed method is designed with robust theoretical foundation of confidence mutation and the proposed score function. The analysis is presented clearly and accessibly for the novel insights, making it easy to understand the mechanism of the proposed framework.", "weaknesses": "1. Although the proposed method can improve the performance of the SOTA methods, whether they face the same challenges and how does the proposed method improve each method are not clearly explaiend.\n\n2. For the different methods, what types of the inputs can improve them effectively lacks of deeply analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time devoted to reviewing this paper and your constructive suggestions. Here are our detailed replies to your questions.\n\n> **W1, Q1:** Although the proposed method can improve the performance of the SOTA methods, whether they face the same challenges and how does the proposed method improve each method are not clearly explained. Since the proposed method is a framework to improve the OOD performance, I wonder how does the proposed CoVer method improve each method? Why the proposed method is effective to improve different types of the method should be further explained.\n\nThanks for your constructive comments. **First,** we would like to state that almost all the SOTA methods considered in our work face **the same challenge on excavating discriminative representation for OOD detection.** Although the SOTA methods are designed with different advanced techniques (e.g., ReAct, DICE, and ASH integrates the activation regularization or reshaping to the forward path of a single input in DNNs; MCM, LoCoOp, CLIPN, and NegLabel explore the CLIP's representation of a single input to detect OOD samples in VLMs), we notice that **the single input may implicitly constrain the representation dimension** for detection as the discriminative features are not infinite to utilize. \n\n**Second,** our CoVer introduce the extra dimensions with corruptions to **reveal the intrinsic distinguishes of ID and OOD samples.**  In the multi-inputs space, ID data maintains an overall higher confidence expectation, whereas OOD data encounters notable changes in model confidence since the high-frequency features are altered as verified in our analysis. Leveraging the difference trends between ID and OOD samples by a simple but effective averaging operation, our CoVer can further improve the ID-OOD separability when combined with those SOTA methods (i.e., adding the corrupted input dimension for scores averaging).\n\n> **W2, Q2:** For the different methods, what types of the inputs can improve them effectively lacks of deeply analysis. For the proposed method, it mainly focuses on expanding the inputs. While for the different method, they usually focus on different knowledge. I wonder how to decide what types of the inputs that can improve the performance of different methods.\n\nThank you for your thoughtful question. For the types of corrupted inputs and their corresponding severity levels, we have conducted some related explorations (**e.g.,  Tables 10 and 11, Figures 6 and 7**) in the Appendix of our original submission for performance references. We have noticed that some specific corruptions (**e.g., Brightness, Fog, Contrast, Motion Blur, Defocus Blur**) can generally improve the OOD detection performance. It can be found **those types provide corruptions on the non-semantic level of the input**, instead of damaging the semantic feature too much like the other types. It is also empirically verified in our previous analysis about confidence mutation, are induced by the high-frequency feature changes. However, to further efficiently **determine which corruption types to choose** for expanding the input dimension, it is recommended to rely on an additional validation set (e.g., the SVHN dataset used in our exploration) as the selection basis, similar to the hyperparameter selection process in previous works.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b4PIsWQM9i", "reviewer": "Reviewer_hcf7", "summary": "In previous OOD detection methods, extracting discriminative information from OOD data relative to ID data is challenging with the representation of a single input. This paper provides a novel perspective for out-of-distribution (OOD) detection by leveraging multiple types of corruptions to expand the original single input space. Utilizing the interesting phenomenon of confidence mutation, the authors introduce a new scoring method termed CoVer, which averages the confidence scores measured from multiple input spaces, achieving better separability between ID and OOD distributions. Extensive experimentation underscores that the CoVer approach not only outperforms previous methods in both DNN-based and VLM-based OOD detection benchmarks, but also exhibits commendable compatibility when combined with different OOD detection methods. Moreover, the CoVer technique demonstrates exemplary adaptability across diverse VLM architectures.", "questions": "1.Watermaking[1] is a very popular and SOTA method for OOD detection. Compared with watermarking and so on, what are the advantages of using corrupted images to expand the input space. Since the corrupted images are the important data in the proposed method, it is necessary to present the advantages for OOD with corrupted images more clearly.\n[1]Wang Q, Liu F, Zhang Y, et al. Watermarking for out-of-distribution detection. Advances in Neural Information Processing Systems, 2022, 35.\n\n2. The proposed method can enhance most of the OOD methods and improve their performance. However, these methods usually designed with different techniques, such as fine-tuning and zero-shot methods. Hence, I wonder whether there exsit differences to improve these methods with CoVer and how to guarantee the effectiveness when combine CoVer with these methods.", "limitations": "Yes", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Well written and technically sound. This paper is well organized and written. Both the motivation and the main contributions of the proposed method are easy to follow. Sufficient experiments have been conducted to demonstrate the effectiveness of the proposed method.\n\n2.Novelty. The challenges of the previous methods are well summarized in OOD detection, i.e., single input constrains the representation ability for detection. The proposed method expands the input space to aware the ID and OOD distribution effectively. \n\n3. Extensive experiments and ablations. The paper conducts extensive experiments, including OOD detection using different backbones, hard OOD detection, and various aspects of ablation studies. The performance improvements strongly verify the effectiveness of the proposed method.\n\n4. Clear justification and theoretical analysis. The paper provides a clear and intuitive justification for the proposed method by a thorough theoretical analysis. This analysis clearly elucidates the mechanisms beyond CoVer, making the paper convincing and compelling.", "weaknesses": "1.Lack explaination and comparison. Different types of corruptions are used in the method, which is similar to the idea of Watermarking (Wang, et al. 2022), where a generic watermark is trained to distinguish between ID and OOD data. The advantages are not deeply analyzed.\n\n2.Less clarification. The CoVer approach is flexible and compatible and can be combined with different methods to achieve performance gains. The combination is not clearly presented for different methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time devoted to reviewing this paper and your constructive suggestions. Here are our detailed replies to your questions.\n\n> **W1, Q1:** Lack explaination and comparison. Different types of corruptions are used in the method, which is similar to the idea of Watermarking, where a generic watermark is trained to distinguish between ID and OOD data. The advantages are not deeply analyzed. Watermaking is a very popular and SOTA method for OOD detection. Compared with watermarking and so on, what are the advantages of using corrupted images to expand the input space. Since the corrupted images are the important data in the proposed method, it is necessary to present the advantages for OOD with corrupted images more clearly.\n\nThank you for your constructive comments. **We have added the analysis about the Watermarking method with our CoVer in the following two aspects.**\n\nConceptually, we have noticted that Watermarking utilizes a well-trained mask to help the original images be distinguishable from the OOD data. However, **Watermarking is still trying to excavate the useful feature representation in a single-input perspective**. In contrast, the critical distinguishable point and also the advantage of our CoVer method lies in input dimension expansion with the corrupt variants, which instead provide a extra dimension to explore the confidence mutation to better identify the OOD samples.\n\nExperimentally, we have conducted the comparison and report the results in **Table 9 in attached PDF**. The results show that, on the one hand, training an optimized watermarking for effectively distinguishing between ID and OOD samples is a **time-consuming process**. On the other hand, CoVer achieves this by introducing corrupted inputs to capture the confidence variations between ID and OOD data during the test phase, which is **simpler**, **faster**, and **more effective**. We will add this part in our revision and discuss them in detail in our appendix.\n\n> **W2, Q2:** Less clarification. The CoVer approach is flexible and compatible and can be combined with different methods to achieve performance gains. The combination is not clearly presented for different methods. The proposed method can enhance most of the OOD methods and improve their performance. However, these methods usually designed with different techniques, such as fine-tuning and zero-shot methods. Hence, I wonder whether there exsit differences to improve these methods with CoVer and how to guarantee the effectiveness when combine CoVer with these methods.\n\nThank you for your thoughtful question. The fine-tuning methods designed for OOD detection focus on **utilizing extra auxiliary outliers** to regularize the model to be better aware of OOD data, while the zero-shot methods are focusing on **scoring function design** of excavating the original distinguisable feature that can better represent the differences. Although their focus are conceptually different, **our CoVer provides the similar enhancement as the adapation is on the input side** (i.e., adding the corrupted input dimension for score averaging). It sheds new light on leveraging raw input in an extra dimension, **but is not bound to either fine-tuning or zero-shot framework** that targets on utlizing or enhancing the single-input representativeness for OOD detection task. Thus, we have found **limited significant differences** when CoVer combined with these two methods, both in their high-level adapation and in our empirical results. \n\n**To guarantee the effectiveness of CoVer**, we recommend to use an additional validation set (e.g., the SVHN dataset used in our exploration) as the selection basis for choosing the effective corruption types and severity level, similar to previous works like Watermarking to learn a optimal mask using the validation set.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## General Response

We appreciate all the reviewers for their thoughtful comments and suggestions on our paper.

We are very glad to see that the reviewers find **our focused problem is important** (R3,R4) within the OOD detection research, and the method is **novel** (R1,R2,R3,R4) and **simple but adaptable** (R1,R2,R3,R4) to various other techniques, and the **experiments are good, comprehensive** and demonstrate the **general effectiveness** of our CoVer (R1,R2,R3,R4). We are also pleased that the reviewers find our writing and figures is **very clear** and **easy to understand** (R1,R4).

We have tried our best to address the reviewers' comments and concerns in **individual responses to each reviewer** with comprehensive experimental justification. The reviews allowed us to improve our draft and the contents added in the revised version and **the attached PDF** are summarized below:

**From Reviewer LFoY**

- Clarify and Discuss our main novelty and settings of CoVer. (see Section 1,  3.1, and 3.2 in the original submission, will highlight in these sections)
- Summarize and add results for expanded types and runtime issues (see Table 3,4 in PDF, will add in Appendix F)
- Discuss the failure cases of CoVer (will add and highlight in the limitation part of Appendix E)
- Add the experimental verification on NINCO dataset and compare with NNGuide and MaxLogit (see Table 1,2,5 in PDF, will add in Appendix F)
- Conduct comparison experiments on each mentioned DNN-based methods (see Table 6 in PDF, will add in Appendix F)
- Clarify and Explain the usage of validation set in CoVer (see Table 7 in PDF, will add and highlight in Section 4.1).

**From Reviewer HCvU**

- Discuss and add the reference for methods in related fields. (will add in Appendix B)

- Explain the differences of data depths, information projections and Isolation Forest,  and their challenges in visual OOD detection. (will add in Appendix F)


- Add the experimental comparison with newly considered methods. (see Table 8 in PDF, will add in Appendix F)

**From Reviewer NZqw**

- Discuss and revise the statement for our critical observations and add more explanation (will revise in Section 1 and 3.1) 

- Show our results about the effects of expanded types and provide some insights on choosing corruption types. (will add and highlight in Section 4.1)


**From Reviewer hcf7**

- Add the experimental comparison with Watermarking. (see Table 9 in PDF, will add in Appendix F)

- Discuss the generalization and effectiveness of CoVer. (see Section 3.1 in the original submission)

**We appreciate your comments and time!** We have tried our best to address your concerns and revised the paper following the suggestions. **Would you mind checking it and confirming if there are any unclear parts?**

---

### Some rest answers:

For **Reviewer LFoY**

> **W7, Q5:** Experiments on NINCO

Thank you for your valuable question. NINCO has proposed three OOD datasets with no categorical contamination which include NINCO, OOD unit-tests, and NINCO popular OOD datasets subsamples. Here, we evaluate the effectiveness of CoVer on these datasets in **Table 5 in attached PDF**. The results demonstrate that CoVer, when combined with ASH, consistently achieves better performance across the three NINCO OOD datasets.

> **W8:** Comparison with NNGuide and MaxLogit

Thanks for your valuable suggestion. We have conducted comparison experiments with NNGuide and MaxLogit to enrich our analysis in **Table 1 in attached PDF**.

First, our experimental results show that CoVer outperforms these competitive post-hoc methods on the ResNet50 architecture.

Second, the performance of these post-doc methods, especially NNGuide, **encounter significant drop when conducted on CLIP-B/16 architecture**. We believe the reason for the poor performance is **the difference in training data**. Many pos-hoc methods are designed on ImageNet pre-trained networks, where only ID data are used during training. In contrast, when training CLIP, both ID datas and OOD datas are used. This leads to different activations of OOD data. Another reason is that the pos-hoc method relies heavily on **the choice of hyperparameters**. The hyperparameters of NNGuide need to be re-selected on different models. Despite these issues, our CoVer can still perform better than these methods.

Furthermore, we also **combine our CoVer with MaxLogit and NNGuide** and report the results in **Table 2 in attached PDF**, which further demonstrates the effectiveness and compatibility of our method.

> **Q1:** why only combined with ASH, not the others?

Thank you for your question. In Table 1 in the original submission, we only reported the results of CoVer combined with ASH **because it best demonstrates the excellence of CoVer**. In **Table 3 in the original submission**, we also show the results of CoVer combined with DICE and ReAct, and CoVer can also provide performance gains for them. Here in **Table 6 in attached PDF, we further report the comparison of CoVer combined with each mentioned DNN-based methods** (adding MSP, ODIN, and Energy score), which strongly demonstrates its superiority.

> **Q3:** About validation set

Thank you for your insightful questions. We utilized **the SVHN dataset as the validation set** to determine the most effective corruption types for each method in all experiments. Specific examples of selections are provided in **Table 7 in attached PDF. We will clearly explain the usage of the validation set in our main paper**. We would like to explain that our submitted code is developed from the MCM code repository, and methods such as MCM and NegLabel did not explicitly employ a validation set in their implementation. Furthermore, the main function realization of the submitted code focuses on the critical implementation steps of CoVer for the usage, omitting the validation set part. **We will definitely add relevant code in the revised version and clearly explain it**.

Meta-review: After reading the paper and the reviews, with significant reviewer discussion, I am voting to accept the paper. The use of multiple realizations of the input via random transformations being used at once is a bit counter-intuitive but has been shown to provide a good lift to improve OOD detection, and is applicable to multiple different OOD detectors. A large number of datasets are tested with different models, showing consistent improvement in most cases. While the AUC improvements are modest, the FPR gains are larger - which implies that the nature of the improvement comes more from calibration than new capability. Though calibration can be done with a secondary set, in my view, there is a non-trivial value to calibration absent explicit secondary mechanisms like a calibration set as it simplifies development/deployment. While there is some related work with the same inspiration for adversarial attacks[1], the approach is more than sufficiently different. The largest detractor brought by some reviewers are concerns around novelty, and related work not included.  Data augmentations are not new, and the specific augmentations are not new - but the method of applying them is. Given that two reviewers voted to accept with high scores, I'm inclined to believe that more than one future reader will be interested in using/leveraging this paper and its approach. I don't see any reviewer concerns that the approach itself is invalid/had some evaluation flaw, but I think the content of this meta-review and the feedback given by the reviewers should be carefully included in the camera ready to ensure an appropriate level of claim is made. Specifically, the claims of being "the first to expand" and offering a "new perspective" should be tempered to the reality of the paper and shouldn't be a distraction from the otherwise good writing and interesting results. Several related works were noted that should also be included in the camera-ready version. 

1. Barrage of Random Transforms for Adversarially Robust Defense. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019
Decision: Accept (poster)