An Information-theoretical Framework for Understanding Out-of-distribution Detection with Pretrained Vision-Language Models
Authors: Bo Peng
Jie Lu
Guangquan Zhang
Zhen Fang
Venue: NeurIPS 2025 poster ()
Keywords: Out-of-distribution Detection
Pretrained Vision-Language Models
Point-wise Mutual Information
Primary area: other

Abstract: Out-of-distribution (OOD) detection, recognized for its ability to identify samples of unknown classes, provides solid advantages in ensuring the reliability of machine learning models. 
Among existing OOD detection methods, pre-trained vision-language models have emerged as powerful post-hoc OOD detectors by leveraging textual and visual information. 
Despite the empirical success, there still remains a lack of research on a formal understanding of their effectiveness. 
This paper bridges the gap by theoretically demonstrating that existing CLIP-based post-hoc methods effectively perform a stochastic estimation of the point-wise mutual information (PMI) between the input image and each in-distribution label. This estimation is then utilized to construct energy functions for modeling in-distribution distributions.
Different from prior methods that inherently consider PMI estimation as a whole task, we, motivated by the divide-and-conquer philosophy, decompose PMI estimation into multiple easier sub-tasks by applying the chain rule of PMI, which not only reduces the estimation complexity but also provably increases the estimation upper bound to reduce the underestimation bias. 
Extensive evaluations across mainstream benchmarks empirically manifest that our method establishes a new state-of-the-art in a variety of OOD detection setups.

An Information-theoorticall Framework for Understanding Out-of-distribution Detection with Pretrained Vision-Language Models
Bo Peng, Jie Lu, Guangquan Zhang, Zhen Fang" University of Technology Sydney
Abstract
Nut-of-distribuue (OOD) detection, recognized for its ability to identify samples unknown classes, provides solid advantages in ensuring the reliability of machine learning models. Among existing OOD) detection methods, pre-trained vision- language models have emerged as powerful post-hoe OOD detectors by leveraging textual and visual information. Despite the empirical success, there still remains a lack of research on formal understanding of their effectiveness. This paper bridges the gap by theoretically demonstrating that existing CLIP-based post- hoc methods effectively perform a stochastic estimation of the point-wise mutual information (PMI) between the input image and each in-distribution label. This estimation is then utilized to construct energy functions for modeling in-distribution distributions. Different from prior methods that inherently consider PMI estimation as whole task, we, motivated by the divide-am sondee-nd-conqueee philosophy. decompose PMI estimation into multiple easier sub-tasks by applying the chain rule of PMI, which not only reduces the estimation complexity but also provably increases the estimation upper bound reduce the underestimation bias. Extensive evaluations across mainstream benchmarks empirically manifest that method establishes a new state-ofthh-ane in a variety OOD detection setups.
1 Introduction
Despite the significant progress in machine learning that has facilitated a broad spectrum clas- sification tasks 12] 79 All. models often operate under closed-world scenario, where test data stems from the same distribution as the training data. However, real-world applications often entail open-world scenarios in which deployed models may encounter unseen classes of samples during training, giving rise to what is known as aatoo--distributinn (OOD) data. These OOD instances potentially undermine model's stability and, in certain cases, inflict severe damage its performance. Accordingly, a reliable discrininative model should not only correctly classify known in-distribution (ID) samples but also flag any OOD inputs unknown. This directly motivates OOI) detection 55 72|288 which ensures the safety of decision-critical applications 25 87].
This paper focuses post-hoc OOD detection, which are more practical than learning-base based methods that require resource-intensive retraining. Earlier studies 220 34 61 36) primarily utilized the single modality of pre-trained models, but the success of contrastive language-image pre-training
(CLIP) [57] has recently shifted research toward expanding post-hoc OOD detection from single- modal to multi-modal methods. Researchers have since explored ways better leverage models to enhance the performance and upplicability of post-hoc OOD detection. A notable method is MCM 42,, which defines textual features concept prototypes for each IDD class and uses the scaled distance between visual features and the closest ID prototype to measure OOD uncertainty. This
*Correspondence Zhen Fang (zhen.fang edu.s.)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

method has paved the way for using pre-trained vision- models (VLMs) in post-hoc OOD detection. However, MCM relies only on textual information from the ID label space, leaving VLMs' text interpretation capabilities underutilized. To address this, Nagl [26] introduces numerous negative labels, allowing the model to hetter distinguish OOD samples. heuristic grouping strategy in NegLabel is also proposed to further enhance OOD detection performance. Despite its promising potential, it is worth noting that a formalized understanding of CLIP- post-hoc OOD detection remains significantly lacking in the field. This prompts the research question underlying this work:
How to theoretically justify the empirical effectiveness of CLIP-based post-hoc OOD detection?
Theoretical Significance. To address this challenge, we draw inspiration from information theory and propose an information- information-theorrtical density-based framework. In this framework, ID data is modeled as an energy- model, where the point-wise mutual information (PMI) [6] between the input image and each ID label forms the energy functions. We argue that this analytical framework is well-suited for studying OOD detection, OOD data, by definition, inherently diverges from ID data in terms of their underlying density distributions. Guided by this framework, we show that representative CLIP-based post-hoc OOD detection methods 42] 26 can be interpreted as stochastic Monte Carlo estimations of PMI. Furthermore, we theoretically establish the following key points: 1) introducing negative labels increases the estimation upper bound, thereby mitigating underestimation bias; and 2) the grouping strategy effectively approximates the expectation through multiple sampling, reducing estimation variance.
Algorithmic Contribution. To further facilitate PMI estimation for OOD scoring, the starting point of our method is to decompose PMI as a sum of terms by applying the chain rule on PMI. In addition to reduce the overall estimation complexity according to the siide-aan-conquee philosophy. we prove that the decomposed PMI estimation can further increases the estimation upper bound to reduce underestimation bias without explicitly introduce a corresponding number of negative labels. Notably, NegLabel (66 has empirically found that introducing excessive negative label would degrade OOD detection performance.
2 Related work
The core of CLiP-based OOD detection lies in how to leverage texture supervision with pre-trained VLMs to assist OOD detection on the visual domain. On the one hand, the pioneering work, MCM 42,, , defines textual features as concept proto- types for each ID class and uses the scaled distance between visual features and the closest ID prototype to measure OOD uncertainty. Intead of relying only textual information from the ID label space, Lagel (26) incorporates additional negative class names mined from available data sources, such as WordNet, as negative proxies. To mitigate the nonalignment between target visual OOD distribution and the generated negative textual OOD distribution, AdaNeg (75) leverages the benefits of test-time adaptation to generate adaptive proxies by exploring potential OOD images during testing. On the other hand, CLIPPbased OOD detection can also be improved by prompt representation learning. In particular, LoCoOp 43 learns ID text prompts by pushing them away from the portions of CLIP local features that have ID-irrelevant nuisances (e.g., backgrounds). CLIPN (88 and LSN [6) design learnable "no" prompt and "no" text encoder to capture negation semantics within images. Differently, LAPT (76) initializes prompts with negative labels (26). followed by tuning prompts with cross-modal and cross-distribution mixing. Due to limited space, related works on traditional OOD detection are discussed in Appendix[A]
3 Preliminary
Notations. Let X' and y represent the input space and the label space, respectively. Considering two
random variables X E and Y E V, we represent pxy and px as the probability density functions of the joint distribution Pxy and the marginal distribution Px. respectively. Similarly, py and py
denote the mass functions of marginal and conditional distributions Py and Pylx. respectively. We
write P Px,y as the joint ID distribution defined over X x 3/. where } A (yI...., ツK} C. y is the

---

space for known ID labels. During testing, there are some unknown OOD joint distributions P PX.YY. defined over X' x Vo where V. Y 31 presents the space of unknown OOD labels.
Post-hoc Detection Strategy. Concurrently, OOD detection follows training-free scoring mecha- nism, i..,, given a pre-trained ID classification model parameterized by 0, and a scoring function S, then detected ID data if and only if ((x; 0) X, for some given threshold A:
(xx ID, ((x: 0) 2 X; otherwise, g (xx) OOD,
(1)
where À is chosen correctly classify high fraction of ID data (e.g.. 95%).
CLIP-based Models. Given any visual input x E X and any label y E y, we extract features of x and y using an arbitrary CLIP--based model f that consists an image encoder f(:; 0mmg) and an text encoder f(;; Qeei) as follows:
f(x; mmg) E Sd-11
f(2(1); O Ed-
where 2(-) is the text prompt template, s0-1 {r E ||||2 = =1]. and 0 = {Oing feeu).
CLiP-based OOD Detectors Studied. CLIP-based models, which are initially proposed for zero- shot ID classification, have recently been extended to zero-shot OOD detection where there is no need to train on ID samples. The pioneering work, MCM (42), treats the prompt of ID labels as concept prototypes and measures the ID-ness of the input image by comparing the similarity between the input image and the concept prototypes in the feature space learned by CLIP-based models, i...
SMCM 0) A max exp(z T I/T)
DE
ZEED exp(2 (sp(2Tr/T)) ryyr)'
where T >> 0 is temperature hyper-parameter. Unlike MCM that only employs information from the ID label space, NegLabel (26) introduces a L-sized set of negative label [ (yK+1.... -K++ sourced from lexical databases, followed by randomly grouping the selected I. negative labels into T non-overlapping subsets G1, ....... Gr, i.e., G: ngj = 0, Vi + j:
T
exp(z T/TT)
SNegLabel (x; 0)
Σ
Σ DEVi
T
Σ E9,u0
exp(z rT/TT)
[=1
4 Theoretical Analysis
While both MCM and NegLabel have empirically emerged to be effective post-hoc OOD detectors, their inherent connections and theoretical understandings are largely lacking. To the best of our knowledge, there is limited prior work providing provable guarantees for Cl.ee-based post-hoc OOD detection methods from a rigorous mathematical point of view. In this section, we provide theoretical justification for CLIP-based post-hoc OOD detection from the perspective of inffommation-theoreiiie density estimation. In particular, due to the fact that OOD data, by definition, inherently diverges from ID data by means of their data density distributions, we, following advanced density-based OOD detection methods [51] 44 36), render the ID density function as an ideal metric for ID-OOD discrimination. Inspired by prior works (36), we consider modeling the unknown true ID density function px, of ID input marginal distribution Fx, by resorting to the baaed model 22 [9]
(x) cxp [E(x)] exp [E(x)]
E(x) log Σ exp [aW v)l.
1
(4)
Z
DED1
where Z exp [x(x) dx is input-independent normalization function,
0 is hyper- parameter and W(x, y) is the point-wise mutual information (PMI) [6] that explicitly measures the y Y. As implied by the following definition of PMI, the formulation in Eq. (4) implicitly assumes that the modelled ID density 9x, (x) is induced by an underlying unknown joint distribution Pxy defined over X y
association between the input x E X and the label
In accordance to Jiang et al. (26) labels E Vd, that have lower affinities with ID images compared to 00D images are considered as negative labels
3

---

Definition 1. The PMI between two observations x E A' and y E 3 is defined as follows:
Pxy y)
Py Pyix (x)x)
W(x, y) _ log
Px py Py (z) = log
(5)
Pr (v)
Note that directly calculating W(x, y) in Eq. (5). which is built upon the conditional distribution Pyix and the marginal distribution Py, can be computationally intractable since the two underlying distributions unknown in nearly all practical applications. In response this challenge, our key idea is to replace the unknown mass function py x (y|x) with the estimated one Pyzx (y)x; 0) using the pre-trained CLIP-based model parameters 0 for tractable estimator of the modeled ID data density function Pxx (x) in Eq. ( i.c.,
px, (x; 0) exp [x)
Ex(x) = log Σ exp [avi(x,
X exp [xo()))
(6)
Zo
BE
where Zo exp [x)] dx and W(x, y; 0) log (x1x()) is the estimator of the true PMI py())
W(x. :). the following, we demonstrate that MCM and NegLabel,, despite their seemingly distinct scoring functions, can be interpreted methods for stochastically estimating PMI and therefore the
energy function that effectively replicates the behavior of the ID density.
4.1 Towards Understanding MCM
To tractably derive (u|x; 0) in Eq. (6). we, motivated by prior works [6 assume that x (yl x; 0) belongs to energy-based variational family that uses a critic he parameterized by and is scaled by the marginal mass function py, i.e.,
py (v) exp ha (x, y) Pa (y)x 0) = Egey PY P) exp (x.
py (V) exp (x,
(7)
where (x Ey-By ( 2) is the normalization function. Following Peng et al. (51), one can consider a Monte-Carlo method to construct a simple and analytically tractable estimator of Do (x by sampling a N-sized set of i.i.d. samples NN (u..... NNN p. i....
Tx(x) N
cx 0j),
which implies that
Pvix (y)x(y/x;0)
N (x
log
log ΣØ= exp ho (x. ú5)
(9)
Py (y)
If we set (x, v) = z r/Tr and PNN = V1 such that N = K in Eq. 19) in the extreme case where
> +00, combining Eq. (9) and Eq. (6) implies that
Prix (1/x; 0) py
lim Ex(x) lim log Σ exp t0+00
log
a+000
MEJI
PyIx(11x;0)
max log
py (1)
K exp ho (x, v)
(10)
R max log yEli
-= exp ho (x)),)
exp(z T I/T)
log max DEmi
log K const
j=1 exp(z 12/T)
SMCM
Since the logarithm function is monotonically increasing, Eq. 100 implies that SMCM 0) in Eq.
(2) can be understood as a stochastic estimator of lim,-++0 Eg (x) (up to constant). Theorem
provides provable guarantees of how SxCM 0) correctly recovers the true energy function E(x) in
Eq. 4 when + +oc.
As we will demonstrate later, the mass function py can cancel out during the calculation of W(xx, 0) so that there is need to estimate py-

---

Theorem 1. Let (x. z r/T and N = K. we, following prior works 156 47y, assume that hx(x, y) = log Prix(u|x) + c(x) with c(x) as a constant term depending on x, in the extreme case where a +00, then have the following
(y)
lim E(x) = lim Fo(x) = lim log SMCM((x; 0).
(11)
a-++00
N
4.2 Towards Understanding NegLabel
To study NegLabel theoretically, we make the ID data density estimation depend on multiple samples.
In particular, given a set of i.i.d. samples ýN (ún,,.., -.. DNN IP.y. can rewrite py xx (y) x; 0) as:
py x (ylx; 0) = - [prix (y/x, 0)]
(12)
where YN = (Yi, Y2,..., Yv) is an N v-dimensional random variable with each Y; an i.i.d. copy of the random variable Y. Motivated by Poole et al. [56]. we then model the term D (y/x,
in Eq. 12 as follows
py (y) exp (x. y) Vo(x,U,yN)//N + 1)²
(13)
where
N exp ho (x. 0z).
Fo(x.8.8N1 hx y)
j=1
we set (x v/ = z r/T. K + L/T and a = 1, combining Eq. 12 and Eq.
with Eq.
66p directly results in the following:
(y|x(vlx;0)
Ex(x) log
Σ
PY
yEVi
expha(x,))
= log
Σ
"N-IIPP
V y. 2
+ log N
(14)
VEDi
log
Σ VEVi
1
Σ exp(z r//T) + log(K K + L.TT.
exp(z 1//T)
T
Vs U
const
SxxLLa((x;8)
Since the logarithm function is monotonically increasing, Eq. (4) implies that SNegLabel (x: 0) can
be interpreted as another Monte Carlo estimator of Eo (up constant) by sampling ýN-1 from P T times with N K + L/T. where, for each y Vi. samples from G. U \ {v)
instantiated as N the t-th round of sampling (1 < f S T Theorem 2 provides provable
guarantees of how SNegLabel xx 0) correctly recover F(x) in Eq.
Theorem 2. Let he(x, y) = z I/T, a = 1, and N = K + L/T. . we, following following prior
works 56 471, assume that ho h (x,y) log PYix (v/x) c(x) with c(x) as a constant term depending py
on X, then we have the following
E(x) = lim Ex(x)= lim log(K + SNattaae) 0).
(15)
N
T--oc
L/T-+00
Remark. To reveal how negative labels benefit OOD detection, let us looking to Eq. (14) where the estimated energy function (x) is upper bounded by log N with as the number of labels drawn from Fy. By introducing negative labels to take capture at most log( K + L/T") nats of Exx). which is strictly larger than log K of SMCM (x; 0) in Eq.
K L/T, the estimator (x) are allowed to
On the other hand, Theorem 2 states that the recovery of (xx) requires N K + L/T + +oc,
which theoretically justified the use of negative labels in OOD scoring.
We detail the derivation in Appendix B
We justify this formulation in Appendix
"More details can be found in Step 3 of Appendix D
We detail the derivation in Appendix D
5

---

Methodology
Based on the thorectical analysis in Section 4 one may conclude that MCM and NegLabel can be regarded to formulate the estimation of PMI as a whole task. Differently, inspired by the divide-and- conquer philosophy, we conjecture that PMI estimation could be simplified as well as improved by decoupling the task into multiple earlier sub-tasks. Central to our method, we introduce an auxiliary Of of the random variable E X' with T as transformation function. To be specific, given the input image, we can create sub-view x by randomly either 1) occluding some of the pixels in x with T as Cutout [10] or 2) cropping a random portion of with T Random Cropping. In the rest of this paper, we consider the latter case as the default setting.
random variable X = T(X) E X. whose realization is denoted by X, to represent sub-views
an
Theorem 3. For any x E X and y E y; ler x = T(x) be sub-view of the input x, W(x, y) can be decomposed into the following Two term 31
W(x,8) y) = W(& v) + ((x,y1x)
(16)
where W(x, y/ x)), i.e. the PMI between and y conditioned on &, is defined follows:
Px (x, y/x)
W(x,y(&) A log
Pxxx Pr)
(17)
Xy (V X, x)
=log
Py (1)x)
Let W(X, y; 0) and Wxx, y/x: 0) be the estimator of ( y) and W(x, y/x) with the pre-trained
parameters 0, respectively, Theorem 3 directly implies that we can rewrite W(x, y; 0) as follows:
vi(x, y: 0) W(x, yy 0) + W(x 0)
(18)
Parameterizing W(x y: 0). Let Prix (y/ Xx; 0) denote the estimator of Py (yl x), according Eq. parameterize W y: 0)
Prix (vlx::)
Wix y; 0) log
(19)
py (y)
Given ýN-1 = (ir,...,
1N-13 ~ pV following Eq.
12 and Eq.
13
py (y/s;; 0) takes the
following form:
py (y) exp ho (8x v)
pr (y) xc; 0) E S
(20)
Vo X y, N
Similar to Eq.
14 let = K + L/T and ho 3) = r/T with = f(::; mmng). we can arrive at
the following Monte-Carke estimator of W(x, 0) given by
N exp/
W y: 0) = log E. PNN-I
(0 VN
A(X, y; A)
(21)
cxp(2 T r/T)
log Σ DEMi
+ -logo + L/T).
exp(ă rr,r))
Parameterizing W(x y/ 0). Let p (2/x,, x; 0) be the estimator of Prix (y)xx(///xxx according to Eq. 47 we can parameterize W(x 8/|x; 0) as:
P (x|xx(( W y/ x: 0) log
(22)
x (v|x;0)
SSubvviews are those derived from the original view without introducing any external information We detail the derivation in Appendix E

---

Under the assumption of a similar energy-bate based variational family to Eq. 77. we can formulate
Xx|xx (1/xx x; 0) follows:
Py (1/x; exp (x x,
(x)xx(0)xxx:00)
Σ Prix (0) exp h x,
(23)
BEy
Combining Eq. 23 with Eq.
22
we have the following:
exp expla (X, x
W(x, y x 0) log Σ JEY pr (0/x:: 0) exp he (x, x V)
exp ho (X. X, y)
log
(24)
Binuy [n( 0) exp ha (x, X, 0)
exp ho (X. x, y)
log
+ log( K + L),
K+L Σ j=1 n/(3, 1) exp ho (x, v3)
where 17 9) Py [0)x 0)/py (i). We note that it is suffice follow Eq. 6 derive the last step of Eq. (24). where, as suggested by Theorem [] both negative labels and ID labels are leveraged for the Monte-Carle estimation of the expectation. Recalling that, according to Eq. (19). (X. 0) = exp W(X, J: 0), connecting Eq. (2) to Eq. (18) results in reformulating the estimated
energy function Ea(x x) in Eq. (6) with a = as follows:
Ex(x) = log Σ exp [V y: 0) + y 0)
yEJ
log
E yE
cxp
[ y; 0) + hx & v)] exp [W ÿū 0) + ho (x, x, 1)]
E EnpPy
log Sours (x; 0) + log( + L),
(25)
where, inspired by Tsai et al. [63], we define (x x; y) - r T[32 + (1 - 1B)z)/n with (0, and > O as hyper-parameters, and
exp [ x y: 0) + hot x x, y)]
Sours (x; = -
Σ NEJ1
(26)
K+L 2==1 exp A(X yj; 0) no X uj)]
Similarly, present the following theorem to reveal the provable guarantee of how Sours (x: 0)
correctly recovers the true energy function E(x) in Eq. (4).
Theorem Let ho x, y) A T [8 (1 -1 3/z\/N, ho (0x 7T TTT, a= and N L K/T.
If we, following prior works (56 38 assume that he (xx, = log Prix(ul8) + c(x) with c(x) as
py (v)
constant term depending on X, and that ho ho x. log Prixx (vix.8)
Prix (v/&) c(x, with c(x, x)
constant term depending on x and ix then we have the following
E(x) lim (x(x) = lim log(K + L) 0).
(27)
N
T-+00
Remark. Comparing Eq. (55 with Eq. (10) and Eq. (14), one can find that Sours (x; 0) capture at most log( K + L) nats of the true E(x). which is strictly larger than log K in SMCM 0) and log( K + T. in SNegtabel [x; 0). Although this upper bound, i.e., log( + L.). can be achieved by SNeglahel (x: 0) in Eq. (14) by either 1) introducing TL negative labels or 2) fixing T 1, we note that 1) NegLabel [26] has empirically observed the degeneration of OOD detection performance caused by excessive negative labels, and that 2) decreasing T can be in conflict with Theorem 2 where the recovery of E(x) with SNegtabel 0) explicitly requires T be sufficiently large.
We detail the derivation in Appendix F

---

Table I: OOD detection results the ImageNet- 1K dataset. 1 indicates larger values are better and vice versa. The best results in the last two columns are shown in bold.
Method
INaturalist
SUN
Places
Textures
Average
AUROCT
FPR95
AUROCT
FPR95}
AUROCT
FPR95
AUROCT
FPR95
AUROCT
FPR95,
Methods
requiring training (or fine-tuning)
MSP
87.44
58.36
79.73
73.72
79.67
74.41
79.69
71.93
81.63
69.61
ODIN
94.65
30.22
87.17
54.04
Energy
95.33
26.12
92.66
35.97
91.41
39.87
86.76
57.61
91.54
39,89
GradNorm
72.56
81.50
72.86
82,00
73.70
80.41
70.26
79.36
72.35
80.82
VIM
93.16
32.19
87.19
54.01
83.75
87.18
53.94
87.82
50.20
KNN
94.52
29.17
92.67
35.62
91.02
39.61
85.67
64.35
90.97
42.19
VOS
94.62
28.99
92.57
36.88
91.23
38.39
86.33
61.02
91.19
41.32
NPOS
96.19
16.58
90.44
43.77
89.44
45.27
88.90
46.12
91.22
37.93
LSN
95.83
21.56
94.35
26.32
34.48
38.54
30.22
CLIPN
95.27
23.94
93.93
26.17
92.28
33.45
90.93
40.83
93.10
31,10
LoCoOp
96.86
16.05
95.07
23.44
91.98
32.87
90.19
42.28
93.52
28.66
LAPT
99.63
1.16
96.01
19.12
NegPro
98.73
6.32
95.55
22.89
93.34
27.60
91.60
35.21
94.81
23.01
HFTT
93.27
27.44
95,28
19.24
90.26
43.54
88.23
43.08
91.76
33.33
Zero-Shot Training-free Methods
ZOC
86,09
87.30
81.20
$1.51
$3.39
73.06
76.46
98.90
81.79
85.19
MCM
94.59
32.20
92.25
38.80
90.31
46.20
86.12
58.50
90.82
43.93
NegLabel
99.49
1.91
95.49
20.53
91.64
35.59
90.22
43.56
94.21
25.40
Ours (Median)
99.70
1.04
96.16
16.06
93.37
26.92
91.01
40.78
95.07
21.20
(Mean)
99.64
1.04
96.32
18.45
31.15
38.79
22.36
Experiments
Evaluation Metries. The performance of OOD detection is evaluated via two widely used metries: 1) the false positive rate of OOD data is measured when the true positive rate of ID data reaches 95% (FPR95); 2) the area under the receiver operating characteristic curve (AUROC) computed to quantify the probability of the ID case receiving a higher score than the OOD case.
Baseline Methods. We compare
method with MSP (20), ODIN (34, Energy
36
KNN
61 LSN
Gradnorm [24], Vim [66]. VOS [12] NPOS [62] ZOC 13
CLIPN 68
LoCoOp
43
46
LAPT 76 NegPro 33 HFTT
32
MCM
4
NegLabel
26 and AdaNeg 75
Implementation Details. Unless otherwise specified, we employ the CLIP ViT-B/16 model as the pre-trained VLM. We use the same NegMinnin algorithm NegLabel (26) to extract top 15% dissimilar words to ID labels from WordNet as negative labels, followed by separating the negative labels into T = 10 groups for OOD scoring. Following NegLabel [26). adopt the text prompt of "The nice <label>... We apply the random cropping augmentation on each test-time image x with the scale range (A, 1.0) produce the sub-view x, followed by resizing X 224 224. Regarding hyper-parameters in main results, we set T = 0.02, x 0.08, X 0.55, a 0.8 and 3 0.3. To reduce variance of random cropping, the final OOD scoring function is averaged over V randomly cropped sub-views. All experiments are conducted with a single Tesla A100 GPU. Source codes for reproduction be found in Supplementary Materials.
INaturalist
SUN
Places
Textures
100.0
100.0
100.0
96.0
AUROC (5)
96.0
96.0
AUROC
94.0
94.0
90.0
0.001
0.005
0.01
0.02
0.03
90.0
0.01
0.04
0.08
0.1
0.2
90.0
0.4
0.5
0.6
0.7
0.8
Figure 1: Ablation study on ImageNet- IK w.n.t hyper-parameters 7 (left), (middle) and (right)

---

Table 2: Evaluation Commain-generaizan OOD detection. indicates larger values are better and vice versa. The best results in the last two columns are shown in bold per ID dataset.
ID Dataset
Method
INaturalist
SUN
Places
Textures
Average
AUROCT
FPR95]
AUROCT
FTR95]
AUROCT
FPR95]
AUROC1
FPR95]
AUROC1
FPR95]
MCM
87.74
63.06
85.35
67.24
81.19
70.64
74.77
79.59
82.26
70.13
ImageNet-S
NegLabel
99.34
2.24
94.93
22.73
90.78
38.62
89.29
46.10
93.59
27.42
Ours
99.51
1.62
96.02
20.17
93.89
34.69
91.35
42.94
95.52
25.11
MCM
79.50
76.85
76.19
79.78
70.95
80.51
61.98
86.3 37
80.88
72.16
ImageNet-A
NeyLabel
98.80
4.09
89.83
44.38
82.88
60.10
80.25
64.34
87.94
43,25
Ours
99.16
3.58
91.64
39.63
86.25
55.64
87.43
76
91.23
39.40
6.1 Main Results
We conduct experiments on the ImageNet dataset, demonstrating the scalability of our method. Specifically, we inherit the setup from prior work [42
26 75), where the ID dataset is ImageNet-
1K [9] and OOD datasets include iNaturalist 644, SUN 77 Places365 80], and Textures 77]. At
test time, all images are resized to 224x224 Table I presents the performance of our approach
and existing competitive baselines, where the proposed approach significantly outperforms existing methods. Specifically, advanced post-hoc methods generally perform better than learning-bass methods especially when the SUN dataset acts as the OOD data without requiring additional train- ing. Besides, compared with the state-of-the-arr NegL.abel our method reveals 3.16% and 2.21% % averaged improvement w.r.t FPR95 and AUROC on the ImageNet dataset. For advanced works, i.c. NegLabel+ AdaNeg, that additionally consider visual negative proxies in OOD scoring, improved version, i.e., AdaNe, performs better on all four OOD datasets.
6.2 Ablation Study
We analyze the
most essential to our algorithmic design, including the minimum crop scale À and two scaling temperatures and K. The corresponding results are plotted in Figure On the one hand, having large or small value of the two scaling temperatures does not necessarily improve the OOD detection performance while art NegLabel when the value of r and i varies from 0.05 to 0.02 and from 0.04 to 0.1 respectively. On the other hand, it can be found that the aggressive cropping strategy, which corresponding that the value À is small, can deteriorate the OOD detection. We suspect that this is because aggressive cropping may hurt the semanties of the original image.
method consistently outperforms the state-of the-
6.3 Extensions
Domain-generalizable OOD Detection. We consider domain generalizable OOD detection scenarios, where domain shifts occur in ID data. With ImageNet- K as the ID data, we, following NegLabel (26). consider ImageNet-S [65] and ImageNet-A (23) as ID data receptively. The performance gain in
immlies the more robustness of our method to domain shift.
Table 3: OOD detection results on the ImageNet-IK IK with various learned prompts, i.c., NegPro 33 and LAPT (76), respectively. Following Zhang & Zhang 775 the performance is measured by FPR95 J. The best results are shown in bold.
Method
iNaturalist NegPro LAPT
SUN
Places
Textures
Average
NegPro
LAPT
NegPro
LAPT
NegPro
LAPT
NegPro
LAPT
Acdlaahel-AdaNeg
3.87
0.58
11.35
9.98
25.45
30.47
29.79
25.25
17.62
16.32
Ours+AdaNeg
4.16
0.63
9.47
8.39
23.79
25.64
26.42
25.76
15.96
15.11
OOD Detection with Learned Prompt. While this paper, following Neglabel (26.. to use a pre- defined prompts for ID label, we show that our method can be made stronger with the mostly recent technology of prompt learning. Empirically, we compare the results in Table 5 by using the prompts leamned by either Negpro B3 or LAPT [76].

---

7 Conclusion
This paper presents cetemmtttoo-thhoretti framework characterizes and unifies the theoretical understanding of post-hoc OOD detection with pre-trained VLMs. In particular, by modeling the ID data with an energy-based model with the PMI between the input image and each ID label as energy functions, We show that representative CLIP-based post-hoe OOD detection methods implicitly work as stochastic Monte Carlo estimations of PMI for density estimation. Motivated by the sondee-nddconquee philosophy, we decompose the original PMI into a sum of conditional and unconditional PMI terms to facilitate OOD detection, which demonstrates both theoretical and empirical superiority.
Acknowledgement
This work supported by Australian Research Council Discovery Early Career Researcher Award (DE250100363) and Australian Laureate Fellowship (FL190100149).
References
[1] Mouin Ben Ammar, Nacim Belkhir, Sebastian Popescu, Antoine Manzanera, and Gianni Franchi. Neco: Neural collapse based tut-of-distribution detection. In The Twelfth International Conference Learning Representations. 2024.
[2] Rich Caruana and Alexandru Niculescu-Mizil, An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning, pp. 161-168, 2006.
[3] Chunchun Chen, Wenjie Zhu, and Bo Peng. Differentiated graph regularized non non-negative matrix factor- ization for semi-supervised community detection. Physica A: Statistical Mechanics and its Applications, 604:127692, 2022.
[4] Chunchun Chen, Wenjie Zhu, Bo Peng, and Huijuan Lu. Towards robust community detection via extreme adversarial attacks. In 2022 26th International Conference on Pottern Recognition (ICPR), pp. 2231-2237. JEEE, 2022.
[5] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607 PmLR, 2020.
[6] Kenneth Church and Patrick Hanks. Word association norms, mutual information, and lexicography. Computational linguistics, 16(1):22-29 1990.
[7] Mircea Cimpoi, Subhransu Maji. Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3606-3613, 2014.
[8] Chris Cremer, Quaid Morris, and David Duvenaud. Reinterpreting importance--eigghted autoencoders. arXiv preprint arXiv: /704002900 2017.
[9] Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database, 2009 IEEE conference on computer vision and pattern recognition, pp. 248 225.. Icce, 2009.
[10] Terrance Vriress Improved regularization of convolutional neural networks with cutout, arXiv preprint arXiv:1708.0 04552, 2017.
[11] Kien Do. Truyen Tran, and Svetha Venkatesh. Clustering by maximizing mutual information across views. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9928 9938, 2021.
[12] Xuefeng Du, Zhaoning Wang, Mu Cai, and Sharon Li. Towards unknown- learning with virtual outlier synthesis. In International Conference on Learning Representations, volume I, pp. 5, 2022.
[13] Xuefeng Du Zhen Fang, Ilas Diakonikolas, and Yixuan Li. How does wild data provably help ood detection? In The Twelfth International Conference on Learning Reprexentations, 2024.
[14] Xuefeng Du Yiyou Sun, and Yixuan Li. When and how does in in-distribution label help -iit-iiiiiiiitiionn detection? arXiv preprint arxiv:2405.18633 2024.
10

---

[15] Sepideh Esmaeilpour, Bing Liu, Eric Robertson, and Lei Shu. Zero-shot out-of-distribution detection based the pre-trained model clip. In Proceedings of the AAAI conference artificial intelligence, volume 36, pp. 6568-6576 2022.
[16] Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Han, and Feng Liu. Is outtof-distribution detection learnable? In NeurlPS, 2022.
[17] Zhen Fang, Yixuan Li, Feng Liu, Han, and Jie Lu, On the learnability of out-of--distriuttio detection. Journal of Machine Learning Research, 25, 2024.
[18] Ruiyuan Gao, Chenchen Zhao, Lanqing Hong. and Qiang Xu. Diffguard: Semantic mismatch- nuismachh-uuided out- of-distribution detection using pre-trained diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1579-1589, 2023.
[19] Will Grathwohl, Kuan-Chiel Wang. Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:19/2.03263, 2019.
[20] Dan Hendryeks and Kevin Cimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv: 1610.02 133, 2016.
[21] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint 181.:::6// 2018.
[22] Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distributie detection for real-world settings. arXiv preprint arXiv:1911.11132. 2019.
[23] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15262-15271, 2021.
[24] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34:677-689, 2021.
[25] Xiaowei Huang, Daniel Kroening, Wenjie Ruan, James Sharp, Youcheng Sun, Emese Thamo, Min Wu, and Xinping Yi. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretabillity. Computer Science Review, 37: 37:100270, 2020.
[26] Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, and Bo Han. Negative label guided aod detection with pretrained vision-language models. arXiv preprint arXiv:2403.2000 2024.
[27] Julian Kamulls, Julia B Nakhleh, Robert Nowak, and Yixuan Li. Training ood detectors in their natural habitats. In International Conference on Machine Learning, pp. 10865. PMLR, 2022.
[28] Hao Lang, Yinhe Zheng, Yixuan Li, Jian Sun, Fei Huang, and Yongbin Li. A survey on out-of-distributie detection in nip. arXiv preprint w.xii:3305.032.55 2023.
[29] Yann LeCun, Sumit Chopra, Raia Hadsell. M Ranzato, Fujie Huang, al. tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
[30] Kimin Lec, Honglak Lec, Kibok Lec, and Jinwoo Shin. Training canfidence-calibraaed classifiers for detecting
samples. arXiv preprint arXiv: 1.09255, 2017.
[31] Kimin Lee, Kibok Lee, Honglak Lec, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. Advances in neurul information processing systems, 31, 2018.
[32] Sachyung Lee, Jisoo Mok. Sangha Park, Yongho Shin, Dahuin Jung. and Sungroh Yoon. Textual training for the hassle-free removal of unwanted visual data: case studies on ood and hateful image detection. Advances in Neural Information Processing Systems,
2024.
[33] Tianqi Li, Guansong Pang. Xiao Bai, Wenjun Miao, and Jin Zheng. Learning transferable negative prompts for aut-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 17584-17594 2024.
[34] Shiyu Liang, Yixuan Li, and Rayadurgam Srikant, Enhancing the reliability of out-of of-distribttion image detection in neural networks. arXiv preprint arXiv:1766.00 02690, 2017.
11

---

[35] Luping Liu, Yi Ren, Xize Cheng, and Zhou Zhao. Out-of-distribution detection with Luffusion-based neighborhood.
[36] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances neural information processing systems, 33:21464-21477 2020.
[37] Xixi Liu, Yaroslava Lochman, and Christopher Zach. Gen: Pushing the limits of softmax- out-of- distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter Recognition, pp. 23946-23955, 2023.
[38] Zhuang Ma and Michael Collins. Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency, arXiv preprint arXiv:1809.0182. 2018.
[39] Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks, Advances in neural information processing systems, 31, 2018.
[40] Andrey Malinin and Mark Gales. Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness. Advances in Neural Information Processing Systems, 32, 2019.
[41] Mare Masana, Xialei Liu, Bartlomiej Twardowski. Mikel Menta, Andrew D Bagdanov, and Joost Van Weijer. caassinncremenal learning: survey and performance evaluation on image classification. JEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5513-555) 2022.
[42] Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of-distribution detection with vision- langugge representations. Advances in neural information processing systems, 35: 35087-35102 2022.
[43] Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa. Locoop: Few-shot out-of-distribution detection via prompt learning. Advances Neural Information Processing Systems, 36, 2024.
[44] Peyman Morteza and Yixuan Li. Provable guarantees for understanding out-of- out-of-diitributiin detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 7831-7840, 2022.
[45] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 427-436, 2015.
[46] Jun Nie, Yonggang Zhang, Zhen Fang. Tongliang Liu, Han, and Xinmei Tian. Out-of-distribution detection with negative prompts. In The Twelfth International Conference on Learning Representations, 2024.
[47] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.03744 2018
[48] Bo Peng and Wenjic Zhu. Deep structural contrastive subspace clustering. In Asian Conference Machine Learning, pp. 1145-1160. PMLR, 2021.
[49] Bo Peng, Wenjie Zhu and Xiuhui Wang. Deep residual matrix factorization for gait recognition. In Proceedings of the 2020 12th International Conference on Machine Learning and Computing, pp. 330-334, 2020.
[50] Bo Peng, Zhen Fang, Guangquan Zhang, and Jie Lu. Knowledge distillation with auxiliary variable. In Forty-first International Conference on Machine Learning, 2024.
[51] Peng. Yadan Luo, Yonggang Zhang, Yixuan Li, and Zhen Fang. Conjuorm: Tractable density estimation for out-of-distribation detection. In The Twelfth International Conference on Learning Representations, 2024.
[52] Bo Peng, Yadan Luo, Yonggang Zhang, Yixuan Li, and 7hen Fang. Conjnorm: Tractable density estimation for out-of-distiibutio detection. arXiv preprint arXiv:2402.11888. 2024.
[53] Bo Peng, Jie Lu, Guangquan Zhang, and Zhen Fang. On the provable importance of gradicnts for language-assisted image clustering. arxiv preprint arXiv:2510. 163355,, 2025.
[54] Bo Peng. Jie Lu, Guangquan Zhang, and Zhen Fang. On the provable importance of gradients for autonomous aanguage-assister image clustering, In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 19805 19815, 2025.
12

---

[55] Bo Peng. Jie Lu, Yonggang Zhang, Guangquan Zhang, and Zhen Fang. Distributional prototype learning for oul-of-distriuttinn detection. In Proceedings the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. I. pp. 1104-1114, 2025.
[56] Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pp. 5171-5180. PMLR, 2019.
[57] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
[58] Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Rohhan, and Mohammad Sahokrou. A unified survey on anomaly, novelty, open-set, and out-of-distribation detection: Solutions and future challenges. arXiv preprint arXiv:21 10.1051, 2021.
[59]
Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribation examples with gram matrices. In International Conference on Machine Learning, pp. 8491-8501. PMLR, 2020.
[60] Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. theoretical analysis of contrastive unsupervised representation learning. In International Conference on Machine Learning, pp. 5628-5637. PMLR, 2019.
[61] Yiyou Sun, Yifei Ming. Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference Machine Learning, pp. 20827-20840. PMLR, 2022.
[62] Leitian Tao, Xuefeng Du, Xiaojin Zhu, and Yixuan Li. Non-parametric outlier synthesis. arXiv preprint arXiv:2303.0296 2023.
[63] Tsung Wei Tsai, Chongxuan Li, and Jun Zhu, Mice: Mixture of contrastive experts for unsupervised image clustering. In International conference on learning representations, 2020.
[64] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pictro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8769-8778, 2018
[65] Haohan Wang, Songwei Ge, Zachary Lipton, and Erie P Xing. Learning robust global representations by penalizing local predictive power. Advonces in Neural Information Processing Systems, 32, 2019.
[66] Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: distrr--dittibttion with virtual- logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12200, 2022.
[67] Haoran Wang, Weitang Liu, Alex Bocchieri, and Yixuan Li. Can multi-label classification networks know what they don't know? Advances Neural Information Processing Systems, 34:29074 29007777799977701 2021.
[68] Hualiang Wang. Li, Huifeng Yao, and Xiaomeng Li. Clipa for zero-shot ood detection: Teaching clip to say no. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1802-1812, 2023.
[69] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference machine learning, pp. 9129999299999 9939. PMLR, 2020.
[70] Jianxiong Xiao, James Hays, Krista A Ehinger. Aude Oliva, and Antonio Torralba. Sun database: Large- scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3492. IEEE, 2010.
[71] Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang- Groupvit: Semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18134-18144, 2022.
[72] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized -istt--distiibutionn detection: A survey. arXiv preprint
2021.
[73] Yijun Yang. Ruiyuan Gao, and Qiang Xu. masking. In European Conference on Computer Vision, pp. 373-390 Springer, 2022.
detection with semantic mismatch under
13

---

[74] Yeonguk Yu, Sungho Shin, Seongju Lee. Changhyun Jun, and Kyoobin Lee. Block selection method for using feature norm in out-of-distribution defection. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 15701-15711, 2023.
[75] Yabin Zhang and Lei Zhang. Adaneg: Adaptive negative proxy guided ood detection with vision-language models. arXiv preprint urXiv:2410.20149, 2024.
[76] Yabin Zhang, Wenjie Zhu, Chenhang He, and Lei Zhang. Lapt: Label-driven automated prompt tuning for ood detection with lisoun-laguuge models. European Conference on Computer Vision, pp. 271-288. Springer, 2025.
[77] Yonggang Zhang, Jie Lu. Bo Peng, Zhen Fang, Yiu-ming Cheung. Learning shape in-distribution feature space for out-of-distribution detection. Advances in Neural Information Processing Systems, 37: 99384-49402, 2024.
[78] Zihan Zhang and Xiang Xiang. Decoupling maxlogit for out-of-distriuution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3388 0397,3337700 2023.
[79] Zhong- Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu. Object detection with deep learning: A review. IEEE transactions on neural networks and learning systems, 30(11):3212-33
2019.
[80] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on puttern analyvis and machine intelligence, 40 16:444 2017.
[81] Qinli Zhou, Wenjie Zhu, Hao Chen, and Bo Peng. Community detection in multiplex networks by deep structure preseeving non-negative matrix factorization. Applied Intelligence, 55(1):26, 2025.
[82] Wenjie Zhu, Bo Peng. Han Wu, and Binhao Wang. Query set centered sparse projection learning for set based image classification. Applied Intelligence, 50(10):3400 -4111, 2020.
[83] Wenjie Zhu, Bo Peng. and Chunchun Chen. Self-supervised embedding for subspace clustering. In Proceedings the 30th ACM International Conference Information Knowledge Management, pp. 3687-3691, 2021.
[84] Wenjie Zhu, Chunchun Chen, and Bo Peng. Unified robust network embedding framework for community detection via extreme adversarial attacks. Information Sciences, 643:119200 2023.
[85] Wenjie Zhu, Bo Peng. Chunchun Chen, and Hao Chen. Deep discriminative dictionary pair learning for image classification. Applied Intelligence, 53(19))22017-2233 2023.
[86] Wenjie Zhu, Bo Peng, and Wei Qi Yan. Deep inductive and scalable subspace clustering via nonlocal contrastive self-distillation. IEEE Transactions on Circuits and Systems for Video Technology. 2025.
[87] David Zimmerer, Peter M Full, Fabian Isensee, Paul Jäger, Tim Adler, Jens Petersen, Gregor Köhler, Tobias Ross, Annika Reinke, Antanas Kascenas, et al. Mood 2020: A public benchmark out-of-distribution detection and localization on medical images. IEEE Transactions on Medical Imaging, 41(10):2728 2738, 2022.
14

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly illustrate the focused problems and state our contribution method- ologically, theoretically and empirically in the abstract and introduction.n
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation of our method in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
assumptions are given in the main content and see appendix for full proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide implementation details of our method in the Experiment section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
Source codes for reproduction can be found in Supplementary Materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuiu wwwiiicgguie//eeeeeeee CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide implementation details of our method the Experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have provided the standard deviation of our method in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have included the number and type of used GPU in the Experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http::///errissieeeo http:///..ccpp...ttttcccsstteee.
Answer:
[Yes]
Justification: I have read the ethics review guidelines before conducting research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the societal impact of our method in the appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
18

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: This paper does not involve the use of any pretrained language models, image generators, or scraped datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: we have cited the original paper that produced the code package or dataset and the used datasets in this paper are properly licensed.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi www.swtthhhce...c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
19

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The datasets and models used in this paper are open-source.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The datasets are not with crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The datasets are not with crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact core methodology, scientific rigorousness, originality of the research, declaration is not required.
20

---

Answer:
[No]
Justification:
No LLM is used in this paper.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to LLM policy ptotps://neuriin /ttp:///////////rrooor/o////oorr for what should or should not be described.
A Related Works on Traditional OOD Detection
On the theoretical side, there are various attempts to explore the theoretical understanding of OOD detection. Fang et al. [7] study the generalization of OOD detection by PAC learning and find necessary condition for the learnability of OOD detection. Morteza & Li 44 provides provable understanding of the OOD detection result by modelling the feature embedding space as mixture of multivariate Gaussian distributions. Du al. studies the impact of ID labels on OOD detection.
On the practical side, the popularity of OOD detection is motivated by the empirical observation 45 that neural networks tend be over-confident in OOD data despite the remarkable achievement
in applications [83] 488 82
85
84
49
50
8 86 53 54 One line of work performs OOD
detection by devising post- hoc scoring functions, including confidence-biee basee methods 337
22
energy-based methods [36] 67]. distance-based approaches 31 59 74 I 522
77
55 gradient-
based approaches (24], generative approaches 773 18 35], and Bayesian approaches
40 Another
line of work addresses OOD detection by fine-tuning pre-trained discrimination model with training-time time regularizations that help the model learn ID/OOD discrepancy following the guideline of outlier exposure [21] For instance, the discriminative model is regularized to produce lower confidence (30 33) or higher energy (36) for outlier points, More recently, some works consider a more practical but challenging scenario where auxiliary outliers are contaminated with unlabelled ID counterparts. WOOD (27 formulates learning with noisy OOD data as constrained optimization
problem while SAL [13 separates candidate outliers from the unlabeled data and trains binary
classifier using the candidate outliers and the labelled ID data.
B Proof of Theorem 1
As reminder, Theorem is stated follows:
Theorem Let he (x. y) z r/TT and N K. we, following prior works (56 47J, assume that hx y) log prix(u|x) + c(x) with c(x) as constant term depending in the extreme
py (y]
where a +00, we then have the following:
lim E(x)
lim Ex(x)= = lim log N SMCM((:; 0). t000000
(28)
a 400
N -000000
Proof. Step 1:
pxix(ylx;0) 0) (y
lim
Eo(x) lim log Σ exp
a log
a a-ooc
a+++0 a yE
lim 1
exp 1o (x,
log
exp
log
EnjYY lexp ho 0)
yEDi
Pyix (vlx)
(29)
exp (xx)
lim log
exp
a log
(v)
a
&
Ex~Py
Pylx(Mlx PY
exp c(x)
yE3i
(r(x(y|x)
lim
log
exp
a log
lim E(x),
a
Py (v)
yE
21

---

where the penultimate step of Eq.
29 is derived based on the fact that
py]x (2)x(
Pr (
Σ
pr
-prxx(01x) =1
(30)
py (0)
pr
DEV
$EY
Step
Given that hx(x, y) 2 2/r//T, and that, as implied by the law of large numbers,
-im Σø= exp expho(xx (x 8,) = Eg-Ey (exp (x, 0) (for all yy )), have the following:
N exp(2z T1/T)
lim log SMCM((x;)) N-++00
lim N No+00
log max
yEJi
Σj=1 exp(z r//TT)
N
log max exp(z T r/T) yeMi
lim
log
exp(z ry/T)
0000
N
==1
= log max exp(z T r/T) log lim yEli
exp(z T r//T)
N N
N
j=1
log max exp(( r/T) log Egwpy [exp(z (oppz(r/1)) r/T)] yEm
exp(z Tr/7)
(31)
log max yEJi
Ey-rr (exp(z r/Tr]
exp(z T T//T)
max log yEli
Ejnty (exp(z r/T)]
1
exp 2 I//T)
lim
log Σ exp a log
Ei (exp(z iz 1//))
DEDi
cxp (x, y)
= lim
log
exp a log
a a
Ei-Fy fexp he (x, 0)
DEM1
lim (x) a-++20
Step and Step 2 imply this result.
C Justification of (1)x, 0)
As reminer, py x (1/x,, &x: 0) is given as follows:
Py (v) exp ho v)
(32)
expho + Σi= expho (x. 05)
22

---

(v)xx(0/x --
16 NI-IN
DEY
yEy
pr(y) exp (xp y)
=
Σ yEY
B SN-INR
exp y) + ブラ= exp (x D9))
exp (x)
Σ yEY
Py (1)LE 8N- &N-IN
N
exphooxx + Σラ exp ho (x, 15))
exp ha (x, 11
N = BywBy Bg 1
(33)
expho((x 1) Σタ=1 exp ha (x, 09)]
exp px UN)
NNgg N
j=1 exp ho (x, 0j)
exp he (x, v1))
E BYNNPY
j=1 exp ho 0j)
exp ho [x. 91)
Byx
Eド= exp h2g (x, 9T)
Please refer Section 5.2 in Cremer et al.
[8] for more details of the derivation.
D Proof of Theorem 2
As reminder, Theorem 2 is stated as follows:
Theorem Let (x y) = T I/T, a and = K followwgn prior works 56
assume that ho( (x. y) log Prix (y/x) py (v) + c(x) with c(x) as constant term depending x, then
have the following:
F(x)
lim N
Ex(x)=
lim T
log(K + (.TT) x: 0).
(34)
L/T-++0
Proof.
Given that a = 1, we have
prix (v/x))
Prix (y)x)
F(x) log Σ
exp [M w)) = log Σ
exp
log
log
(35)
py (v)
Py (v)
MEJi
yEJ1
VEJi
PVix (v/ 0)
Ex(x) log Σ
exp
log Wxx w)
log Σ
(36)
(y
yEVi
VE
Step 1:
Given that ho v) = log the following
Py (v/x) (v)
+ c(x), when N + +00, the Law of Large Numbers implies
N-1 expho(xxv)+ +
lim No0000
Σ
exp ho 91)
N
==1
N-1 Σ
1
= lim N
expho(x,y) + lim
lim
exp ho ( 03)
(37)
N
No+00
N
No+00
N-1
j=1
-BanBy (exp ho (M(()))) 9)]
(rix((jxx)
EggNBy
expc(x) exp c(x)
py (i)
Step 2:
23

---

This implies that
N. expxho(xx))
lim Fo(x) = lim
log
B II
N-+000
N
explo(Xx,1) + Σラ= exp (x, 1g)
MEJM
())x(11x)
lim
log
By ByN-I-PN-
(38)
N N-++00
py (v)
MEJ1
Prix (v/x) E(x)
log
PY
vEDi
Step Let 11 be the i-th sample drawn the 1-th round, the law of large numbers implies
( 0)
N exp ho (x. y)
EgN 1-
(y)
exp he (x, y) + Σラ=1 exp he ( (x, Dt))
T
N exx ho (x,,)
lim
(39)
--
T
exp (x v) + N-
exp ho (x, //
N
lim
Mx, y, t),
T--x
T
where, for each f I,...,T,
exp /1 (x, 3)
L(x, exp ho y) + Σソ N-1
(40)
exp (x, 0/0)
(0(-)) is sampled from Ip ii.d, can build İ(z, t) for any of y. If we consider a valid case where each label in G. U y/ \ (v) is exactly sampled to constitute y y when calculating [(x, y, t) in Eq. (39). we have = K + L/T such that N + too requires L/T - toc,
As (1)
which results in rewriting Eq. (39) as follows:
N
lim Ex(x)
lim log
Σ DEA1
(
N N0
T4-00
T
N
1=1
lim log Σ
K
+ T
L/T
exp y)
(41)
74--00
Σ Egyeauss exp (x ú,)
N
DEti
lim log(K + SNggaNeeL (x;
T T-++00 L/T-+00
Step 2 and Step 3 implies this result.
Remarks. We note that the same ideas of the derivation in Step 3 have been witnessed in contrastive learning that known for minimizing InfoNCE [69] 600 of the form =x y) where
exp (o
R(x, y) = Ey v eFyy
log exp (x. y) + E exp ho (x. 0z)
(42)
Recalling the batch-wise empirical loss of contrastive learners such as SimCLR 5 (x and y share a same modality) and CLIP (x and are with different modalities). i.e.,
exp he he H.) log exp (2,,),)))))))) v:) + Σÿfe exp hxpph
B
where {(r,, B is the current training batch, one can check that each given R(x, y) is estimated by exactly sampling yN as {ı,( 81) B and y # y}.
24

---

E Proof of Theorem 3
Definition 2 (Mutual Information (MT)). Given random variables X and Y, the MI between X and Y is the Kullback-Leibler (KL) divergence between the joint distribution Pxy and the product of marginal distributions PxPy, i.e.,
log (xy(x,3) px pxx()ppr())
IX & Oxx(Pxy|IPxPF) (xxx))xFxx
B(x.))-Px [W(x, v)]
(43)
where W(x. y) is the PMI defined in the main paper.
Lemma 1. Given three random variables X, Y. and X, the mutual information ((X; can be
decomposed into the following two ways:
(X,X;V) = I(X;V) + ))) = 1(x;V[x)=
1(x
(44)
where (X;Y|X) Le., the MI between X and Y conditioned on X, is defined as follows:
x
I(X; A E (xxx,)))xPxx log PxI (x x)xxppyxx' (v/
= Pxxx [xx, (45)
As reminder, Theorem is stated again as follows:
Theorem 3. For any x E and y E y, let x = (xx be sub-view of the input x, W(x, y) can be
decomposed into the following two terms:
W(x, = W(x, v) + W(x, y/x),
(46)
where W(x, 3) X), ie., the PMI between x and y conditioned on 8, is defined as follows:
Pxvxx (x, y/x)
W(x,8(*) log
PxIx (x(x) (y1x)
(47)
Prxx x)
=log
Prix (3/3x)
Proof. For any X E X' and y E ), let x = = T(x) be a sub- view of the input x,the data processing inequality implies that [(x; y/ x) = 0, which means that T(X;Y) = T(X;Y) + ((X;) Given that
log (x,
(2, v) Px (x)x)y (y)
ΣΣ P log
Pxy Pxr (x, v) p& (x)
B(R.))~FXY
x
[log (x v)
Σ PXxxXXx Ex
px (x)xx (v)
&
(48)
Pxv (x, log
P (x,,,,
Px (x)py (v)
*
(x, v)
Exx.y,))_Pvvs
log Dx (X
and
log (x v) px(x))p (v)
logg pxr(x,)) y)
E(xx))Fxx
ΣΣ pxy
px
[1og pxy(x,))
Σ Exxxx(x,118)
(x(x))p(r))
& 4
(49)
pxr(x,
(x
log px (x)x))
log (x,
px(x)pp(
25

---

we have the following:
-x;Y)= I(X;1) ((x;)
log (x, y)
pxy (8. 4)
Px 8/
log
+ Exx.u.))
log
px px (y)
Px (x(x)pr(1)
Pxx (x(x)Py (
log pxx y) px px(x)pr (y)
log Pxy (8, 8)
pxy (x. y/x)
B(x.):R))Pxxs Xxx
+ ((x. Xxx
log
px (x)py (1))
(x
pxv pxy(x, v) log
px (x)py (v) loog Pxy (X, M)
PXVIX v18)
=P(x,,,8)=FXXX XXX
log
Px(x)py (y)
Px (x (x
(B(x,,.))) x [W(x)) v) - W(x, 9) - (xx, y/x) = 0
*>(x,, y) W(x,y) y) W(x, s/xx
(50)
F Proof of Theorem 4
As reminder, Theorem 4 is stated follows:
Theorem 4. Let us define ho(x, x. y) A r [ (1 3)2)/ x. and (xx y) = 2 r/T. If we,
Pvj & (u12) + c(x) with c(x)
following prior works [56 38) respectively, assume that he y) log
(v)
(vlxx)
as a constant term depending on &, and that ho(x, &, y) = log Prix (ul8) + c(x, x) with c(x, x) as a constant term depending on x and &, we then have the following for a
E(x) = lim Ex(x)= = lim
log(K + L)S (x; 0).
(51)
N-+000 L/T-++00
T-+000
Proof. Step 1: Given that a we have
E(x) log Σ
1
exp (aW( log Σ
exp
log
Pylx (y|x)
-log
Prix (vlx(
py py( (1)
DEli py (3)
yE
yE
(52)
Py (v18) (Y + c(x), we have
Given that ho (x. y) = llog
lim W(x = lim log Pr (y1 0) N
N
py (v)
N e-xxp ho (x)
lim
log B B
(53)
co y) + - exp h 11)
Prix (v)x) W(x. v)
log
py py())
Step 2:
26

---

Since hx(x, v) = log (v/x,8
(218) + o(x, x), by closely following Eq. (37) and Eq. (38 Eq. (53)
implies that
exp [ y; 0) ho &, v)] BEni E exp [V((x D; 0) ho h x Ø)]]
lim
Eo(x)= = lim
log
N N-+00
N -
exp [W(x 1/ ho X X, w)]
= log
Bgnpy
exp
[W(X) 5) + ho x, &
yEJ4
exp log Py (v) pv
+ log
PrIX& x.)
Prix (y/x) + c(x, x)]
log Σ
=
yEMi
Rinny
exp llo
Prix(
log
Prixs (0)x8)
Pr (7)) + c(x, x)]]
(54)
P (u/x,8)
= log
Σ
(1))
Prixx (8)x..) pr (8)
yEJi Bawpy
PYIX (1/x, x)
log
E(x)
py (V)
MEJ
Step 3:
Let (0 be the i-th sample drawn at the 1-th round, the Law of Large Numbers implies
N exp 11 (8, v)
w(x,0)
lim log
E Eyx-
N +
exp ho (8, v) + Σ 2j=1
exp ho (2, 0z)
(55)
T
N exp exexp ho (x. v)
lim lim 40+ T-0+00
(x, + EØ= exp ho x 01/0)
(t) N-1
Similarly, by taking G. J1 \ {v) as the examplar of y/ J Jim for W(X v) in Eq. (55), we have
N + L/T such that N - 400 requires L/T - to0, which results in rewriting Eq. (55 as follows:
exp(xTr/7) exp(2 r,/T)
W(x,y) lim
log
Σ
loog K + L/T)
T-+
T
Σ
1/T---0 BE
t=1
(56)
lim A(X,y)
T T
4/T--000
Given that N = T L/T + to0 when L/T too and + too, as implied by the Law of Large
Number, combining Eq.
54 with Eq, (6) arrives the following:
lim
log( K + Loun
T T-+000
L/T
exp [A y: 0) + ho & v)]
lim
log Σ
log(( + L)
T-+x
DEti j=1 exp A yı; 0) + 1x v5)
K+L
1./T-+00
(57)
exp [W y) + hog (x, x 4)]
lim
log
+ +log(K + L)
L
DEai j=1 exp W(X, y) + h0 X, x,313]
exp [ y) + 1x & v)]
= log Σ
F(())
EgsPy exp [W(X, 0) + ho X, S)]
yEJi
Step 2 and Step 3 imply this result.
27

---

Table 4: Comparison with different VLM architectures ImageNet- 1K (ID).AIl All values percent- ages. f indicates larger values are better and vice versa. The best results in the last two columns are shown in bold per ID dataset. Results averaged over 5 independent runs.
ID Dataset
Method
INaturalist
SUN
Places
Textures
Average
AUROC1
FPR95,
AUROC1
FPR35]
AUROCY
FPR95]
AUROCT
FPR95,
AUROC1
FPR95]
GiT
71
Neglaael
98.07
8.60
91.52
35.12
88.85
41.63
88.45
47.06
91.72
33.10
Ours
99.52
6.86
95.52
25.08
92.24
37.44
91.28
46.15
94.70
29.08
G Additional Experiments
G.1
More Backbones
G.2 Cropping vs Cutout
Table 5: Ablation study on ImageNet-1k IK w.r.t the choice of obtaining subviews. 1 indicates larger values are better and vice versa. Results are averaged over 5 independent runs.
Method
INaturalist
SUN
Places
Textures
Average
AUROCT
FPR95
AUROC1
FPR95
AUROC1
FPR95
AUROCT
FPR95]
AUROCT
FPR95
Cropping
99.64
1.04
96.32
18.45
95.81
31.15
92.15
38.79
96.00
22.36
Cutout
99.52
1.26
95.08
19.08
95.46
32.44
91.50
39.38
95.39
23.04
H Limitations
This paper directly uses the off tffethe-sheef negative labels mined by NegLabel for PMI estimation. will be exciting to what makes good negative labels for OOD detection with pre-trained VLMs.
I Broader impacts
Our project aims improve the reliability and safety of modern machine learning models, which leads to benefits and societal impacts, particularly for safety-critich applications such as autonomous driving. Our study does not involve any human subjects or violation of legal compliance. do not anticipate any potentially harmful consequences our work.
J Stability
To verify that our method consistently provides strong performance, we run with 10 independent seeds for ImageNet- 1K and report the average and standard deviation of FPR95 and AUROC as follows.
Table Ablation on stability. OOD detection performance based on CLIP-B/16 of our method on ImageNet- IK. Results are averaged over 5 independent runs.
iNaturalist
SUN
Places
Textures
FPR954
AUROCT
FPR95
AUROCT
FPR95
AUROC1
FPR95
AUROCT
0.0
0.1
1.2
0.5
3.0
1.0
1.6
0.6
28


Reviews:
[{"review_id": "CxFCtONvhu", "reviewer": "Reviewer_p925", "summary": "This paper provides a theoretical perspective on CLIP-based OOD detection by interpreting it as stochastic PMI estimation, and enhances performance and robustness through a divide-and-conquer approach using the chain rule of PMI.", "questions": "- Evaluate the effect of different sub-view generation methods on OOD detection performance to assess how sensitive the chain-rule decomposition is to the type of augmentation used.\n- Test the proposed method on OOD benchmarks from domains other than natural images, such as medical, satellite (e.g., EuroSAT), or sketch datasets, to evaluate whether heuristic sub-view generation still holds in these settings.\n- Provide quantitative analysis on inference time and GPU memory usage (e.g., FLOPs, runtime per image) compared to baselines like MCM or NegLabel to demonstrate practical viability of the proposed decomposition strategy.\n- Include comparisons with recent OOD detectors (HFTT (NeurIPS 2024), NPOS (ICLR 2023), ZOC (AAAI 2022), etc.) to better position the proposed method in the broader OOD detection landscape.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The proposed method leverages the chain rule of PMI to decompose the original estimation into simpler conditional and unconditional components. This divide-and-conquer approach not only reduces estimation complexity but also provably increases the upper bound.\n- The method achieves state-of-the-art performance across multiple OOD benchmarks, including ImageNet-S, ImageNet-A, and hard OOD tasks like SSB-hard and NINCO. Notably, it shows improvements in both AUROC and FPR95.\n\nWeaknesses\n- The decomposition of PMI using the chain rule depends on heuristic-based transformations such as Cutout or Random Crop to generate sub-views. These augmentations, while effective, lack theoretical justification and may not generalize well across different domains or modalities.\n- The decomposition strategy introduces additional computation for PMI estimation, yet the paper does not discuss the impact on inference latency or memory overhead, which is particularly important in real-world post-hoc detection settings.\n- While the paper includes many CLIP-based and post-hoc baselines (e.g., MCM, NegLabel, AdaNeg), the overall scope of comparison feels limited.", "was_revised": true, "final_justification": "While I still consider the paper to be borderline (in some points), after reviewing the authors’ rebuttal (additional results), I have decided to raise my score.", "rebuttal": "We appreciate the insightful comments provided by Reviewer p925. Please see our responses to your concerns below.\n\n## 1. Evaluate the effect of different sub-view generation methods\n\n Indeed, we have conducted studies in Appendix H.3 of the manuscript with regards to different sub-view generation methods including cutout and cropping (the our best knowledge, they are the only two options for image data). For your convenvience, we report the detailed results in the below, where it can be found that cutout and cropping achieve comparable OOD detection perfomance while significantly outperforming NegLabel. This implies the flexibility of our method.\n\n|FPR95|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|Cutout|1.26|19.08|32.44|39.38|23.04|\n|Cropping|1.04|18.45|31.15|38.79|22.36|\n|NegLabel|1.91|20.53|35.59|43.56|25.40|\n\n|AUROC|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|Cutout|99.52|95.08|95.46|91.50|95.39|\n|Cropping|99.64|96.32|95.81|92.15|96.00|\n|NegLabel|99.64|95.49|91.64|90.22|94.21|94.21|\n\n## 2. Test the proposed method on OOD benchmarks from domains other than natural images\n\nIn fact, we have conducted experiments on the ImageNet-sketch dataset in Table 2 where ImageNet-sketch is abbreviated as ImageNet-S.  For your convenvience, we provide the results in the below.\n|FPR95|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|Ours|1.62|20.17|34.69|42.94|25.11\n|NegLabel|2.24|22.73|38.62|46.10|27.42\n\n|AUROC|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|Ours|99.51|96.02|93.89|91.35|95.52|\n|NegLabel|99.34|94.93|90.78|89.29|93.59|\n\n\nAs per your advice, we additionally conduct experiments on the BIMCV-COVID19+ dataset [a], following the same setup as AdaNeg [b]. Specifically, we select BIMCV as the ID dataset, which includes chest X-ray images CXR (CR, DX) of COVID-19 patients and healthy individuals. For the OOD datasets, we, following AdaNeg [b], use CT-SCAN and X-Ray-Bone datasets. The CT-SCAN dataset includes computed tomography (CT) images of COVID-19 patients and healthy individuals, while the X-Ray-Bone dataset contains X-ray images of hands. We report AUROC as follows:\n|AUROC|CT-SCAN|X-Ray-Bone|AVG|\n|-|-|-|-|\n|ours|74.63|99.82|87.26|\n|NegLabel|63.53|99.68|81.61|\n|ours+AdaNeg|95.16|99.99|97.58|\n|NegLabel+AdaNeg|93.48|99.99|96.74|\n\n## 3. Provide quantitative analysis on inference time\n\nRegarding the computational cost for inference, the table below reports the inference speed, where, same as AdaNeg [b] the inference speed is measure by Frame Per Second (FPS) with a NVIDIA V100 GPU. It can be found that our method achieves the better trade-off between efficiency and effectiveness.\n\n||FPS|FPR95|FPR95|\n|-|-|-|-|\n|Ours|558|22.36|96.00|\n|NegLabel|592|25.40|94.21|\n|AdaNeg+NegLabel|476|18.92|96.66|\n\n##  4. Include comparisons with recent OOD detectors\n\nWe thank the reviewer for bringing HFTT, NPOS and ZOC into our eyes. Finding that these works are interesting and make a solid contribution to OOD detection, we are greatly glad to compare them. \n\nIn particular, while both HFTT, NPOS and ZOC offer creative directions for OOD detection using vision-language models, our work provides a fundamentally deeper contribution by establishing a unified information-theoretical framework that explains and generalizes existing CLIP-based post-hoc methods. Unlike HFTT, which relies on a specific training scheme and synthetic textual data, or ZOC/NPOS, which introduces an additional generation module requiring supervised training, our method is training-free, fully post-hoc, and compatible with any pretrained CLIP-like model. More importantly, we offer provable theoretical insights into why and how exsiting representative CLIP-based OOD scoring function scores can be interpreted as estimators of point-wise mutual information (PMI), and we propose a novel PMI decomposition strategy that improves estimation fidelity without introducing additional negative labels. This level of theoretical rigor and generality not only bridges gaps in the current literature.\n\nThe empirical comparsion below shows that our method achives significantly better performance than others, which empirically highlights the superority of our method.\n|FPR95|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|ZOC|87.30|81.5|73.06|98.90|85.19|\n|NPOS|16.58|43.77|45.27|46.12|37.93|\n|HFTT|27.44|19.24|43.54|43.08|33.33|\n|Ours|1.04|18.45|31.15|38.79|22.36|\n\n|AUROC|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|ZOC|87.30|81.51|73.06|98.90|85.19|\n|NPOS|96.19|90.44|89.44|88.80|91.22|\n|HFTT|93.27|95.28|90.26|88.23|91.76|\n|Ours|99.64|96.32|95.81|92.15|96.00|\n\n[a] Bimcv covid-19+: a large annotated dataset of rx and ct images from covid-19 patients\n\n[b] AdaNeg: Adaptive Negative Proxy Guided OODDetection with Vision-Language Models"}, {"review_id": "jhiNg7eZ6U", "reviewer": "Reviewer_X4sW", "summary": "This paper provides a theoretical view of post-hoc CLIP-based OOD detection methods from a point-wise mutual information perspective. It demonstrates that two representative methods in this area, MCM and NegLabel, can both be interpreted as Monte Carlo estimators of the energy function, and suggests theoretical motivation of incorporating negative labels. Then, following the framework, the paper proposes to introduce the use of subviews by randomly masking or cropping parts of an image, to enhance OOD detection performance. Empirical results show that the proposed approach brings consistent performance gain over existing baselines.", "questions": "* It is not entirely clear how this paper’s proposed theoretical framework is specific to CLIP-based post-hoc OOD detection. While MCM and NegLabel consider the CLIP-based setting due to its generalizability and emergent properties, their core ideas often mirror traditional OOD detection methods. For instance, MCM can be considered as EnergyScore [1] but with CLIP replacing standard image classifiers, and NegLabel can also in some sense be viewed as a variant of contrastive/outlier exposure based approach. It appears that in this paper’s theoretical framework, CLIP is not specifically relevant except for setting it as the critic. Thus, it would be helpful to provide clarifications on the motivation behind focusing specifically on CLIP-based post-hoc OOD detection, instead of considering a more general OOD detection setting. \n\n* The current theoretical formulation, especially for NegLabel, is somewhat coarse-grained, in the sense that it primarily models the incorporation of non-ID prompts, but does not account for their specific properties or \"negativeness\". In practice, the selection of negative prompts has a significant impact on performance. However, from the theoretical perspective, it seems that incorporating any kind of non-ID prompt would help over MCM. It would be helpful to see more clarification or discussion on this point.\n\n[1] Liu et al. \"Energy-based Out-of-distribution Detection.\" Neurips 2020.", "limitations": "Yes the authors have discussed potential limitations.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n* The paper provides a theoretical framework for post-hoc CLIP-based OOD detection, providing a systematic way to analyze methods and complementing the often empirical nature of research in this direction. \n* The experiments are extensive, covering challenging settings and compatibility with both training free and prompt learning methods. Results show that the proposed method shows strong performance and brings consistent improvement over the baselines.\n\nWeaknesses:\n* It is not entirely clear how the proposed theoretical framework is specifically connected to CLIP-based OOD detection besides the fact that CLIP is used as the estimator. It would be helpful to provide clarifications on the motivation behind focusing specifically on CLIP-based post-hoc OOD detection, instead of examining a more general OOD detection setting.  (See Questions section below.)\n\n* Aspects of the theoretical derivation are built upon various existing works cited in the paper, so there is some contribution offset in this sense.", "was_revised": true, "final_justification": "Thank the authors for the rebuttal. I am satisfied with the response and decide to keep my score of borderline accept.", "rebuttal": "We appreciate the insightful comments provided by Reviewer X4sW. Please see our responses to your questions below.\n\n## 1. It would be helpful to provide clarifications on the motivation behind focusing specifically on CLIP-based post-hoc OOD detection\n\nIndeed, we argue that our thorectical framework is CLIP-specific. In the following, we will show this point by proving that a pretrained CLIP is naturally a PMI estimator.\n\n> We start from the following following inequality [a]\n>\n>$$F=\\mathbb{E}\\_{(\\mathbf{x},y)\\sim\\mathbb{P}\\_{XY}}[\\mathcal{W}(\\mathbf{x},y)]=\\mathbb{E}\\_{(\\mathbf{x},y)\\sim\\mathbb{P}\\_{XY}}[\\log\\frac{p_{Y|X}(y|\\mathbf{x})}{p_{Y}(y)}]\\geq\\mathbb{E}\\_{(\\mathbf{x},y)\\sim\\mathbb{P}\\_{XY}}[\\log\\frac{\\hat{p}\\_{Y|X}(y|\\mathbf{x};\\boldsymbol{\\theta}')}{p\\_{Y}(y)}]=F_{\\boldsymbol{\\theta}'} $$ \n>\n>Clearly, the inequality holds with inequality if and only if $\\hat{p}\\_{Y|X}(y|\\mathbf{x};\\boldsymbol{\\theta}')=p_{Y|X}(y|\\mathbf{x})$.\n\n> Recalling a CLIP training dataset $\\mathcal{D}=${$(\\mathbf{x}\\_1,y\\_1),\\ldots,(\\mathbf{x}\\_N,y\\_N)$ } $\\sim\\mathbb{P}^N\\_{XY}$ and CLIP learning objective\n>$$L^{CLIP}\\_{\\boldsymbol{\\theta}'}=\\frac{1}{N}\\sum\\_{i=1}^N\\log\\frac{\\exp h\\_{\\boldsymbol{\\theta}}(\\mathbf{x}_i,y_i)}{\\sum\\_{j=1}^N \\exp h\\_{\\boldsymbol{\\theta}}(\\mathbf{x}\\_j,y\\_j)}$$\n\n> For the MCM case, by parameterizing $\\hat{p}\\_{Y|X}(y|\\mathbf{x};\\boldsymbol{\\theta}')$ via Eq. (7), we have\n>$$F_{\\boldsymbol{\\theta}'}^{MCM}=\\mathbb{E}\\_{(\\mathbf{x},y)\\sim\\mathbb{P}\\_{XY}}\\left [\\log\\frac{\\exp h\\_{\\boldsymbol{\\theta}}(\\mathbf{x},y)}{\\mathbb{E}\\_{\\hat{y} \\sim \\mathbb{P}\\_{Y}}\\left [\\exp h\\_{\\boldsymbol{\\theta}}(\\mathbf{x},\\hat{y})\\right ]} \\right]$$\n> One can easily check that $L^{CLIP}\\_{\\boldsymbol{\\theta}'}$ is an empricial version of $F_{\\boldsymbol{\\theta}'}^{MCM}$ (by the law of large number)\n\n> For the NegLabel case, by parameterizing $\\hat{p}\\_{Y|X}(y|\\mathbf{x};\\boldsymbol{\\theta}')$ via Eq. (12) and Eq. (13), we have\n>$$F_{\\boldsymbol{\\theta}'}^{NegLabel }=\\mathbb{E}\\_{(\\mathbf{x},y)\\sim\\mathbb{P}\\_{XY}}\\bigg[\\mathbb{E}\\_{\\hat{\\mathbf{y}}\\_{N-1}\\sim\\mathbb{P}\\_Y^{N-1}}\\bigg[\\log\\frac{\\exp{h_{\\boldsymbol{\\theta}}(\\mathbf{x},y)}}{\\exp{h_{\\boldsymbol{\\theta}}(\\mathbf{x},y)}+\\sum\\_{j=1}^{N-1}\\exp{ h\\_{\\boldsymbol{\\theta}}(\\mathbf{x},\\hat{y}\\_j)}} \\bigg] \\bigg]+\\log N$$\n>$F\\_{\\boldsymbol{\\theta}'}^{NegLabel }$ is known as InfoNCE [b]. It has been widely acknowledge in the literature that contrastive learning optimizes InfoNCE.\n\n> Based on the discussion, one may conclude that maximizing $L^{CLIP}\\_{\\boldsymbol{\\theta}'}$ is equivalent to maximizing $F_{\\boldsymbol{\\theta}'}$ no matter whether $\\hat{p}\\_{Y|X}(y|\\mathbf{x};\\boldsymbol{\\theta}')$ is parameterized via Eq.(7) or Eq. (12).\n>\n> Thanks to the large-scale nature of the CLIP training dataset and the CLIP model itself, it is reasonable to approximate $F$ with $F_{\\boldsymbol{\\theta}}$ where $\\boldsymbol{\\theta}=\\arg\\max_{\\boldsymbol{\\theta}'}L^{CLIP}\\_{\\boldsymbol{\\theta}'}$.\n>\n> This implies that we can approximate the PMI $\\mathcal{W}(\\mathbf{x},y)=\\log\\frac{{p}\\_{Y|X}(y|\\mathbf{x})}{p\\_{Y}(y)}$ with $\\log\\frac{\\hat{p}\\_{Y|X}(y|\\mathbf{x};\\boldsymbol{\\theta})}{p\\_{Y}(y)}$\n\n\n## 2. It seems that incorporating any kind of non-ID prompt would help over MCM.\n\nYes. Empirical evidence confirms that any non-ID prompts (including randomly selected negative labels) significantly enhance OOD detection performance compared to MCM [c]. As demonstrated in Table 7 (Appendix A.4) of NegLabel [d], using 10,000 randomly chosen labels from WordNet yields superior results:\n\n|FPR95|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|MCM|32.20|38.80|46.20|58.50|43.93|\n|NegLabel (random selection)|9.11|28.67|45.10|55.87|34.69|\n\n|AUROC|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|MCM|94.59|92.25|90.31|86.12|90.82|\n|NegLabel (random selection)|97.96|93.93|89.54|86.62|92.01|\n\nEven though, we never deny that the quality of negative labels has a significant impact on performance, which can be attributed to semantic consistency in representation space, i.e., inputs with similar semantics cluster closer in the embedding space. In other words, Introducing lower-discriminability negative labels (i.e., those semantically similar to ID classes) focuses the sampling distribution on a narrower region, therefore resulting in a more biased Monte-Carlo estimation.\n\n\n## 3. the theoretical derivation are built upon existing works cited in the paper\n\nWhile we agree that our theoretical analysis is partially inspired by foundational principles and estimation techniques from prior work, we respectfully argue that our contribution lies in how we extend these ideas to provide novel insights tailored to OOD detection with vision-language models (VLMs), which is not incremental as it solves the lack of theoretical grounding in CLIP-based OOD detection and directly enables our state-of-the-art method.\n\n- To our best knowledge, we are the first to propose a PMI decomposition via sub-views (Theorem 3) for OOD detection. This decomposed formulation is not only theoretically elegant but also leads to provable benefits in bias reduction (Theorem 4).\n\n- We provide the first formal information-theoretic explanation that unifies MCM and NegLabel as stochastic estimators of PMI (Theorems 1 and 2), bridging previously disconnected empirical techniques under a energy-based view (Section 3.1 & 3.2).\n\n- We argue that it is non-trivial to extend energy-based scoring function from single-modal to multi-modal settings. Clearly, a natural approch to define the energy based on the CLIP-based zero-shot classifier logits, i.e., cosine similarity. However, as thorectically justified in [e], it is PMI not cosine similarity that correspondes to the optimal similarity measure in the CLIP's uni-modal embedding space. This is also empirically supported by empirical observations in [d], i.e., Energy (zero-shot) achieves significantly worse performance (79.57 AUROC and 82.21 FPR95 in average) than MCM and NegLabel, both of which, as proved in the manuscript, can be understood as different PMI estimators.\n\n[a] The im algorithm: A variational approach to information maximization\n\n[b] Representation learning with contrastive predictivecoding\n\n[c] Delving into out-of-distribution detection with vision-language representations\n\n[d] Negative label guided ood detection with pretrained vision-language models\n\n[e] Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metric, ICLR25"}, {"review_id": "IAKRtKmoJA", "reviewer": "Reviewer_oce3", "summary": "This paper proposed an information theory-based method to solve the OOD detection via the VLM. The main motivation for this paper is that point-wise mutual information (PMI) could clearly differentiate the (in-distribution) ID and OOD samples. The problem here is how to calculate the PMI. This paper proposes an approximation method to estimate the PMI. The experimental results show that PMI could achieve the better performance on OOD detection.", "questions": "No other question, please refer Weaknesses.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The writing for this paper is good.\n\n2. The proposed theory is solid.\n\n\nWeaknesses:\n\n1. Limited novelty. The motivation of the paper is to compute the OOD score using an energy function. However, this idea has already been proposed in \"Energy-based Out-of-Distribution Detection.\" The authors suggest that VLMs (Visual-Language Models) can estimate PMI to derive the energy function, but this innovation seems insufficient, as the method does not fundamentally differ from existing energy-based methods. The authors have not clearly distinguished their approach from the one presented in \"Energy-based Out-of-Distribution Detection,\" which weakens the novelty of the proposed work.\n\n2. Lacking an ablation study about the computation cost. The proposed method requires sampling multiple sub-views for Monte Carlo estimation. However, there is no discussion on the computational overhead incurred by this process. It would be useful to include experiments or comparisons to assess whether this sampling step leads to a significant increase in computational cost.\n\n3. Lacking the ablation study about the different backbones. Since the paper aims to provide a theoretical framework for VLMs, it is surprising that the experiments focus solely on CLIP as the backbone. Including comparisons with other VLMs would strengthen the validity of the claims and provide a more comprehensive evaluation of the proposed method\n\nTo sum up, this paper leverages the information theory by connecting the PMI and energy function to detect OOD samples, which is interesting. However, due to the weaknesses, the contribution of this paper is limited. Meanwhile, this paper lacks the necessary ablation study. Thus, I rate it as borderline reject. If the author could address these concerns, I am willing to increase my score.", "was_revised": true, "final_justification": "This paper establishes a connection between information and CLIP in OOD detection through PMI. The initial concerns include unclear contribution and a lack of the necessary ablation study. After rebuttal, the author addresses these concerns. However, the problem is that the proposed PMI belongs to the theory framework of energy-based functions, thereby weakening its contribution to be significant. Thus, I think borderline accept is suitable for this paper.", "rebuttal": "We appreciate the insightful comments provided by Reviewer oce3. Please see our responses to your concerns below.\n\n## 1. The authors have not clearly distinguished their approach from [a]\n\nWe respectfully point out key distinctions that demonstrate the novelty of our work:\n\n- Multi-Modal Energy as PMI Estimation\n\n[a] defines energy over unimodal pre-trained classifier logits. In stark contrast, we derive an information-theoretic energy function directly from the Pointwise Mutual Information (PMI) between images and text embeddings in pre-trained VLMs (e.g., CLIP). This provides the first unified framework explaining established CLIP-based scores (MCM, NegLabel) as PMI estimators, establishing a novel theoretical grounding for post-hoc VLM-based OOD detection and directly motivates our method.\n\nIt is non-trivial to extend energy-based scoring function from single-modal to multi-modal settings. Clearly, a natural approch to define the energy based on the CLIP-based zero-shot classifier logits, i.e., cosine similarity. However, as thorectically justified in [e], it is PMI not cosine similarity that correspondes to the optimal similarity measure in the CLIP's uni-modal embedding space. This is also empirically supported by empirical observations in [b], i.e., Energy (zero-shot) achieves significantly worse performance (79.57 AUROC and 82.21 FPR95 in average) than MCM and NegLabel, both of which, as proved in the manuscript, can be understood as different PMI estimators. \n\n\n- Divide-and-Conquer Chain Rule Decomposition\n\nRather than estimating PMI in one shot, we decompose it via the chain rule into two simpler terms，this provably raises the estimation upper bound and therefore reduces underestimation bias while keeping the number of selected negative labels unchanged. We note that This decomposition is unique to our framework and not possible in prior energy methods.\n\nOur contributions are:\n\n(i) A new PMI-theoretic foundation for multi-modal energy\n\n(ii) A novel chain-rule decomposition mitigating PMI estimation bias\n\nThis represents a clear conceptual and practical advancement beyond [a] and related works. We thank the reviewer for the opportunity to clarify our novel contributions.\n\n## 2. Lacking an ablation study about the computation cost. \n\nThanks for your constructive comments. Since, same as NegLabel [b], this paper focuses on post-hoc OOD detection, our method have no computational cost for training. Regarding the computational cost for inference, the table below reports the inference speed, where the inference speed is measure by Frame Per Second (FPS) with a NVIDIA V100 GPU. It can be found that our method achieves the better trade-off between efficiency and effectiveness.\n\n||FPS|FPR95|AUROC|\n|-|-|-|-|\n|Ours|558|22.36|96.00|\n|NegLabel|592|25.40|94.21|\n|AdaNeg+NegLabel|476|18.92|96.66|\n\n##  3. Including comparisons with other VLMs would strengthen the validity of the claims.\n\nThanks your insightful comments. In Appendix H.2, following NegLabel [b], we evalute our method with GroupViT [c]. For your convenvience, the OOD detection results are reported as follows, where our method consistently outperforms NegLabel beyond CLIP-based models .\n\n|FPR95|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|NegLabel|8.60|35.12|41.63|47.06|33.10|\n|Ours|6.86|25.08|37.44|46.15|29.08|\n\n|AUROC|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|NegLabel|98.07|91.52|88.85|88.45|91.72|\n|Ours|99.22|95.52|92.24|91.28|94.70|\n\nBesides, we conduct extra experiments using ALIGN [f], following NegLabel [b]. We provide the results in the below, where our method keeps outperforming NegLabel.\n\n|FPR95|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|NegLabel|4.55|43.18|61.01|49.89|39.66|\n|Ours|3.24|40.21|57.32|45.79|36.64|\n\n|AUROC|iNaturalist|SUN|Places|Textures|AVG|\n|-|-|-|-|-|-|\n|NegLabel|98.86|90.93|85.03|88.24|90.76|\n|Ours|99.15|92.11|88.17|90.46|92.47|\n\n[a] Energy-based out-of-distribution detection\n\n[b] Negative label guided ood detection with pretrained vision-language models\n\n[c] Groupvit: Semantic segmentation emerges from text supervision\n\n[d] AdaNeg: Adaptive Negative Proxy Guided OODDetection with Vision-Language Models\n\n[e]  Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metric, ICLR25\n\n[f] Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisio"}, {"review_id": "wwcIrlyh7P", "reviewer": "Reviewer_jjLB", "summary": "This paper introduces an information theory-based approach to enhance OOD detection using pretrained vision-language models such as CLIP. The method involves stochastic estimation of pointwise mutual information (PMI) and its decomposition via the chain rule. The authors validate the effectiveness of the proposed method through experiments on several classification benchmarks.", "questions": "What is the underlying intuition of the proposed method? Are there any illustrations or pseudocode to help convey the core idea?\nIt is important to link the evaluation results to the underlying effects of the proposed method. Rather than simply showing that it outperforms existing approaches on standard benchmarks, I’m more interested in understanding why and how it improves OOD detection", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nShows promising results on standard OOD detection tasks\n\nWeakness:\n\n1. The intuition behind the proposed method is not clearly presented, and the absence of illustrations or pseudocode makes the results difficult to reproduce\n2. The evaluations do not clearly address why and how the proposed method enhances OOD detection\n3. The evaluations are primarily conducted on standard classification benchmarks to demonstrate the effectiveness of the proposed method; including benchmarks from real-world applications would further strengthen the results", "was_revised": true, "final_justification": "Thank you to the authors for providing further explanations and conducting additional experiments that effectively addressed my main concerns. Based on these clarifications and new results, I have decided to raise my score. I hope that these insights and the outcomes of the new experiments will be incorporated into the final version of the paper, should it be accepted.", "rebuttal": "We thank Reviewer jjLB for valuable comments. As to the questions and suggestions you raise, we took them seriously. Our response is as follows.\n\n## 1. What is the underlying intuition of the proposed method and why and how the proposed method enhances OOD detection\n\nThe core intuition of our method starts from the thorectical analysis that representative post-hoc CLIP-based OOD detectors (MCM and NegLabel) can be understand as stochastic estimators of Pointwise Mutual Information (PMI) estimator. We, motivated by thedivide-and-conquer philosophy, propose to decompose PMI into two subterms, i.e., $\\mathcal{W}(\\mathbf{x},y)=\\mathcal{W}(\\tilde{\\mathbf{x}},y)+\\mathcal{W}(\\mathbf{x},y|\\tilde{\\mathbf{x}})$ so as to transfer the estimation of $\\mathcal{W}(\\mathbf{x},y)$ into separately estimating $\\mathcal{W}(\\tilde{\\mathbf{x}},y)$ and $\\mathcal{W}(\\mathbf{x},y|\\tilde{\\mathbf{x}})$.\n\nThe theoretical benefits of our PMI decomposition to OOD detection can be summarized in two aspects:\n\n- Our PMI decompositon reduces the complexity of estimating $\\mathcal{W}(\\mathbf{x},y)$. This is becuase\n\na) estimating $\\mathcal{W}(\\tilde{\\mathbf{x}},y)$ involves estimating $p(\\tilde{\\mathbf{x}},y)$, which is simpler than estimating $p({\\mathbf{x}},y)$ as $\\tilde{\\mathbf{x}}$ is a subview of $\\mathbf{x}$ to make the space of possible $\\tilde{\\mathbf{x}}$ and $y$ lower-dimensional.\n\nb) estimating $\\mathcal{W}(\\mathbf{x},y|\\tilde{\\mathbf{x}})$ involves estimating $p(\\mathbf{x},y|\\tilde{\\mathbf{x}})$, which is simpler than estimating $p({\\mathbf{x}},y)$ as $\\tilde{\\mathbf{x}}$ is a subview of $\\mathbf{x}$ to make the conditioned space of possible $\\mathbf{x}$ and $y$ lower-dimensional.\n\n- We show in Theorem 4 that our PMI decompositon provably increases the upper bound of the the estimated PMI to $\\log(K+L)$ from NegLabel’s $\\log(K + L/T)$ or MCM's $\\log K$, therefore reducing the underestimation bias.\n\n## 2. Are there any illustrations or pseudocode to help convey the core idea?\n\nThe pseudo algorithm of our method is given as follows:\n>**Input**: input image $\\mathbf{x}$, ID labels $\\mathcal{Y}\\_{I}$, Negative labels $\\mathcal{Y}\\_{N}$, pre-trained CLIP model parameters $\\theta$, sub-view generator $\\mathcal{T}$, pre-defined critics $h\\_{\\boldsymbol{\\theta}}(\\mathbf{x},\\hat{y})$ and $h\\_{\\boldsymbol{\\theta}}(\\mathbf{x},\\tilde{\\mathbf{x}},\\hat{y})$ \n>\n>**Output:** Our proposed OOD scoring function $S\\_{ours}(\\mathbf{x})$\n\n>Generate a subview via $\\tilde{\\mathbf{x}}=\\mathcal{T}(\\mathbf{x})$\n>\n>Randomly divide $\\mathcal{Y}\\_{N}$ into $T$ non-overlapping groups, i.e., $\\mathcal{Y}\\_{N}=\\cup_{i=1}^L \\mathcal{G}_i$\n>\n> **For** each $y$ in $\\mathcal{Y}\\_{I}$ **do**\n>> $\\hat{\\mathcal{W}}(\\tilde{\\mathbf{x}},y;\\boldsymbol{\\theta})\\leftarrow\\log \\frac{1}{T}\\sum\\_{t=1}^T\\frac{ \\exp h_{\\boldsymbol{\\theta}}(\\mathbf{x},y)}{\\underset{{y\\_j\\in\\mathcal{G}\\_t\\cup\\mathcal{Y}\\_I}}{\\sum}\\exp h_{\\boldsymbol{\\theta}}(\\mathbf{x},y_j)}+\\log(K+L/T)$\n>\n> **end For**\n>\n>$S\\_{ours}(\\mathbf{x})\\leftarrow\\sum\\_{y\\in\\mathcal{Y}\\_{I}}\\frac{\\exp \\big [\\hat{\\mathcal{W}}(\\tilde{\\mathbf{x}},y;\\boldsymbol{\\theta})+h\\_{\\boldsymbol{\\theta}}(\\mathbf{x},\\tilde{\\mathbf{x}},y)\\big ]}{\\underset{{y\\_j\\in\\mathcal{G}\\_t\\cup\\mathcal{Y}\\_I}}{\\sum} \\exp \\big [\\hat{\\mathcal{W}}(\\tilde{\\mathbf{x}},\\hat{y};\\boldsymbol{\\theta})+h\\_{\\boldsymbol{\\theta}}(\\mathbf{x},\\tilde{\\mathbf{x}},\\hat{y})\\big ]}$\n\n## 3. Including real-world benchmarks would further strengthen the results.\n\nThanks for your constructive. As per your advice, we conduct experiments on the BIMCV-COVID19+ dataset [a], following the same setup as AdaNeg [b]. Specifically, we select BIMCV as the ID dataset, which includes chest X-ray images CXR (CR, DX) of COVID-19 patients and healthy individuals. For the OOD datasets, we, following AdaNeg [b], use CT-SCAN and X-Ray-Bone datasets. The CT-SCAN dataset includes computed tomography (CT) images of COVID-19 patients and healthy individuals, while the X-Ray-Bone dataset contains X-ray images of hands. We report AUROC as follows:\n|AUROC|CT-SCAN|X-Ray-Bone|AVG|\n|-|-|-|-|\n|ours|74.63|99.82|87.26|\n|NegLabel|63.53|99.68|81.61|\n|ours+AdaNeg|95.16|99.99|97.58|\n|NegLabel+AdaNeg|93.48|99.99|96.74|\n\n[a] Bimcv covid-19+: a large annotated dataset of rx and ct images from covid-19 patients\n\n[b] AdaNeg: Adaptive Negative Proxy Guided OODDetection with Vision-Language Models"}]

Author rebuttal:

Meta-review: Reviewers agreed on several strengths of the paper, including its solid theoretical grounding (p925, X4sW, oce3) and strong empirical performance across benchmarks (p925, X4sW, jjLB). At the same time, concerns were raised in the initial reviews: computational cost (p925, oce3), limited breadth of experiments and analysis (p925, oce3, jjLB), and lack of clarity regarding the theoretical novelty (X4sW, oce3, jjLB). However, the authors' rebuttal and subsequent author-reviewer discussion were generally effective in addressing these points. In their final justifications, all reviewers indicated that their major concerns had been resolved.

Taking all reviews and discussions into account, while the theory in this work explicitly covers only energy-based post-hoc methods, energy-based scoring remains one of the dominant approaches in OOD detection. On this basis, I find that the paper makes a meaningful and timely contribution by offering a unifying theoretical view of energy-based CLIP methods and demonstrating practical performance gains.
Decision: Accept (poster)