An Online Statistical Framework for Out-of-Distribution Detection
Authors: Xinsong Ma
Xin Zou
Weiwei Liu
Venue: ICML 2025 poster (2025)
Keywords: Out-of-distribution
Hypothesis testing
False positive rate
P-value
Primary area: general_machine_learning

Abstract: Out-of-distribution (OOD) detection  task  is significant   in  reliable and safety-critical applications.  Existing approaches primarily focus on developing  the powerful score function, but overlook the design of decision-making rules based on these score function. In contrast to prior studies, we rethink the OOD detection task from an perspective of online multiple hypothesis testing. We then propose a novel generalized  LOND (g-LOND) algorithm to solve the above problem. Theoretically, the g-LOND algorithm  controls false discovery rate  (FDR) at pre-specified level without the consideration for the dependence between the p-values. Furthermore, we prove that the false positive rate (FPR) of the g-LOND algorithm converges to zero in probability based on the  generalized Gaussian-like distribution family. Finally, the extensive experimental results verify the effectiveness of g-LOND algorithm for OOD detection.

An Online Statistical Framework for Out-of-Distribution Detection
Xinsong Ma Xin Zou
Weiwei Liu
Abstract
OOD detection has garnered considerable attention recently and an extensive collection of literature has been developed to tackle this issue (Liu et al., 2020; Yang et al., 2021; Wang et al., 2022; Liu et al., 2023), which mainly devote to ob- taining powerful score function. However, these methods neglect to design a decision rule based on their score func- tion, and directly use traditional threshold- rule, where the threshold is determined such that the true positive rate (TPR) on ID validation set is 95% before test- ing (Sun et al., 2022; Wei al., 2022). But the traditional decision rule is empirical and thus its outputs lack rigorous statistical guarantee. Consequently, a natural question arise: How to design an OOD detection method with rigorous the- oretical guarantee and well empirical performance? This paper aims to systematically study this question.
(OOD) detection task is sig- nificant in reliable and safety-critical applications. Existing approaches primarily focus develop- ing the powerful score function, but overlook the design of decision- pocciion-mmaing rules based on these score function. In contrast to prior studies, we re- think the OOD detection task from an perspective of online multiple hypothesis testing. We then propose a novel generalized LOND (g-LOND) algorithm to solve the above problem. Theoret- ically, the g-LOND algorithm controls false dis- covery rate (FDR) at pre-specified level without the consideration for the dependence between the p-values. Furthermore, we prove that the false positive rate (FPR) of the g-L.OND algorithm con- verges to zero probability based on the general- ized Gaussian-like distribution family. Finally, the extensive experimental results verify the effective- ness of g- g-LOND algorithm for OOD detection.
decision
Different from previous OOD detection literature, in- vestigate the OOD detection problem from statistical per- spective. Note that in some sensitive applications such as financial transactions (Özbayoglu et al., 2020) and self- driving (Huang et al., 2020), the testing examples arrive sequentially in a stream, and the machine learning model must make decisions in time. Hence, we frame the OOD detection task an online multiple hypothesis testing prob- lem. (Javanmard & Montanari, 2015) proposes the LOND algorithm to address this testing problem, which has been heavily studied in various applications (Javanmard & Mon- tanari, 2015; Robertson & Wason, 2018; Robertson et al., 2022; Liou et al., 2023). Theoretically, to control the false discovery rate (FDR)* 2, the LOND algorithm demands that the p-values corresponding to different null hypotheses are mutually independent, which easily leads to a contradic- tion to real-world applications. We then propose novel generalized LOND (g-LOND) algorithm to eliminate these restrictions on p-values for FDR control. Moreover, we de- rive the asymptotic false positive rate (FPR) of the g-LOND algorithm based on the generalized Gaussian- distribu- tion family. Finally, extensive experiments demonstrate the promising performance of the g-LOND algorithm on OOD detection. We summarize contributions below:
1. Introduction
The ability to detect Out-of-Distribution (OOD) inputs is fundamental requirement for deploying deep learning models in safety-critical applications (Liang al., 2018: Sastry& & Oore, 2020; Ma et al., 2022). Although DNNs have demonstrated impressive capabilities across a range of complex tasks, such as speech recognition and machine translation (He et al., 2016; Amodei et al., 2016; Dankers et al., 2022), they are inherently vulnerable to OOD exam- ples that deviate from the distribution of training data (i.e., in-distribution (ID) data) ). This vulnerability can lead to overconfident yet incorrect predictions, which is particularly problematic in domains such as computer vision (Sun et al., 2022; Wei et al., 2022).
'School of Computer Science National Engineering Research Center for Multimedia Software. Institute of Artificial Intelligence and Hubei Key Laboratory of Multimedia and Network Communi- cation Engineering, Wuhan University, Wuhan, China. Correspon- dence to: Weiwei Liu <liuweiwei863@ @ gmail.com>.
(1) We frame the OOD detection task an online multi-
Proceedings of the nnnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).
2FDR is related to the concept of Precision, and can be con- sidered the generalization of type-I error for single hypothesis testing. See the motivation of controlling FDR Section 3.2.

---

An Online Statistical Framework for Out-of-Distribution Detection
ple hypothesis testing problem and propose novel g- LOND algorithm solve it. Our method is distribution- free, and does not rely on the extra information of OOD data.
Javanmard & Montanari, 2018) develop this work and make this type of methods more powerful. (Javanmard & Mon- tanari, 2015) propose the LOND and LORD algorithms to control FDR and mFDR. The LOND algorithm has been heavily studied in various applications (Javanmard & Mon- tanari, 2015; Robertson & Wason, 2018; Robertson et al., 2022; Liou et al., 2023). Note that these works mainly focus on the situation where p-values are mutually independent or satisfy the special dependence such as conditional superuni- formity. Unlike these literature, this paper aims to propose novel algorithm to control FDR without the consideration of dependence between p-values.
(2) Theoretically, the g-LOND algorithm enables to control the FDR at a pre-specified level. Besides, we demon- strate that the FPR for the g-LOND algorithm converges to zero in probability based on the generalized Gaussian- like distribution family.
(3) Finally, we conduct extensive experiments demon- strate the effectiveness of the g-LOND algorithm for OOD detection. The results show that our method en- ables to achieve better tradeoff between TPR and FPR compared with previous methods.
3. Preliminaries
3.1. cnttof-Distributionn Detection
We donote by X S Rd the feature space and y {1, ................................ K the label space with the joint distribution D on Z = X x V. Besides, X' has marginal distribution Din. Let (æz, y) be the feature-label pair, where instance I E X' and label y E V. We denote the transpose of vec- tor/matrix by the superscript'. ' Denote by fo : X - y the classifier (or hypothesis) which is parameterized by 0 E O where θ is the parameter space and maps x to a certain label y. In addition, we denote by [(fo(r), y) the certain loss function.
2. Related Work
OOD Detection. Existing efforts on OOD detection mainly focus on classification-based methods, density-based meth- ods and distance-based methods. The classification- baased methods directly obtain confidence from the classifier which integrates some specific modifications for loss functions or classifier architectures (Lee et al., 2018; Liu et al., 2020; Huang et al., 2021; Liu al., 2023; Djurisic et al., 2023) with some beneficial design to OOD detection. These de- signs mainly focus on the loss function (Liu et al., 2020; Huang et al., 2021; Liu et al., 2023), classifier architecture (Lee et al., 2018; Djurisic et al., 2023), and some post-hoc processing techniques (Hendrycks & Gimpel, 2017; Liang et al., 2018). The density- based methods characterize the in-distribution distribution using probabilistic models (Abati al., 2019; Ren al., 2019). The distance-based methods usually compute distance in the high-dimension space such as feature space and gradient space to distinguish ID and OOD example (Hendrycks et al.. 2022; Sun et al., 2022). Ad- ditionally, minor methods involve introducing some OOD samples to train the new networks (Yang et al., 2021; Ming et al., 2022). The above methods focus on designing or training a powerful score function and then apply empirical decision rule to identify the OOD examples directly. How- ever, this decision rule is empirical, thus the outputs of it lack rigorous theoretical guarantee. This paper aims to solve this problem.
In the canonical classification setting, the primary goal is to maximize the standard accuracy of models on the unseen examples from the underlying distribution Dg. Concretely, we hope to find the classifier f()) with the smallest risk:
R(f)
min B B(sa))D((fo(t)..))) dE
During the prediction phase, the test examples are usually assumed to come from the same distribution Di., as the training set. Nevertheless, practical situations may introduce some inputs from unfamiliar distributions, with label space potentially lacking any intersection with y These inputs are referred to as the OOD data and should not be predicted.
The goal of OOD detection is to identify the OOD examples in testing set. In the previous literature, the OOD detection task is formulated as the following binary decision problem:
ID, if sf 2 s* <(x) LOOD,
FDR Control Similar to control type-I error in single hypothesis testing, how to control FDR the core prob- lem for multiple hypothesis testing algorithms (Benjamini & Hochberg, 1995; Blanchard & Roquain, 2008; Yu al., 2023; Ma et al., 2024; 2025). Online multiple hypothesis testing is first introduced by (Foster& & Stine, 2008), who propose a alpha investing method to control mFDR for the streamed variable selection problem in high dimensional space. Many following researches (Aharoni & Rosset, 2014;
(1)
if < s*
where s(-) denotes the score function and the threshold s* is empirically selected so that the ture positive rate (TPR) on ID validation set is 95% before testing (Sun et al., 2022; Wei et al., 2022).
The previous studies about OOD detection mainly devote to obtaining a powerful score function which captures the
2

---

An Online Statistical Framework for Out-of-Distributoon Detection
discriminative information in ID data (Hendrycks & Gimpel, 2017; Liu et al., 2020; Djurisic et al., 2023; Liu et al., 2023). However, the question of how to make decisions based the score functions is ender-explored for online OOD detection. In contrast to the previous literature, we investigate the OOD detection problem starting from the decision rule.
In single hypothesis testing, if the p-value is below a pre- scribed significance level a, then the null hypothesis is rejected. While this ritual enables to control the probability of type-I error, in the case of multiple hypotheses testing we need to adjust the significance levels to control other metries, in which false discovery rate (FDR) proposed by (Benjamini & Hochberg, 1995) is widely used. The sta- tistical advantages of FDR have been detailedly discussed in (Benjamini & Hochberg, 1995; Benjamini & Yekutieli, 2001).
3.2. A Statistical Framework for OOD detection
Unlike the existing literature, we provide a new insight for the OOD detection problem from an online hypothesis testing perspective. Specifically, for a testing set Treat {Xtest, .x ...... OOD detection task is formulated as the following hypothesis testing problem:
Given a sequence of the null hypotheses (H we let R(n) be the set of indices of the rejected hypothe- ses for the first n null hypotheses ( HH1 H2.0. ... .. Similarly, denote by Ho(n) and (n) the set of indices for the true null hypotheses and false null hypothesis for {H1,,, H2.0. Hn. J. respectively. Let nu [n)(n)] be the number of true null hypotheses. In statistics, if one null hypothesis is rejected, it is said to make a discovery. FDR is the expected proportion of false discoveries among the rejected hypotheses.
H1 Xteat ~ Dinn
H1 Xiett x Din
H2:0 Xgeat ~ Dins
H2.1 Xteat ~ Din
Hn:: : Xtest Din
Hng1 : Xtest de Din
Definition 3.3 (FejjaBeimiini & Hochberg, 1995)). Let V denote the number of true null hypotheses rejected; more- over, let R be the number of rejected hypotheses, the false discovery proportion (FDP) for the first n null hypotheses is defined as:
where H&D and Ha1 are called null hypothesis and alternative/non-null hypothesis, respectively. If Hoo is re- jected, we declare that X/est is OOD.
To support or reject the claim in null hypothesis, commonly we use the concept of p-value as the measure of statistical significance. Its definition is as follows.
FDP(n) = V/R, R 0,
0,
otherwise.
Definition 3.1. (p-value (Casella & Berger, 2001)] Given a sample X= 3, A statistic p(X) is called p-value correspond- ing to the null hypothesis Ho, if p(X) satisfies
The expectation of FDP(n) is called the FDR, namely
[R(n)) ) n Ho(m))] max [m())
Plp(X)= < 8 Hol S t
(3)
FDR(n) ((DP(n))) E
for every 0 1.
Obviously, if p(X) follows the uniform distribution on (0, 1) under the null hypothesis, then p(X) is valid p- value. A small p-value usually provides strong evidence against the null hypothesis. It is noteworthy that the p-value has clear statistical interpretation. For example, suppose the p-value of a OOD testing example X/est is 0.01. This means that for any coming testing example Xfeat, the prob- ability that Xtst is more similar to OOD data than Xtest is 0.01. In other words, it is extremely difficult to find example that is less like OOD data than Xteest Hence, we are highly confident that X/est is OOD.
The FDR can be considered as the generalization of the probability of type-I error for multiple hypothesis testing. Therefore, multiple testing algorithms should control its FDR first. According to Ma et al. (2024), the FDP is closely related with the concept of Precision. Denote by (n) the complement of set R(n). Using the confusion matrix notations the FDP can be also expressed as
FN
FDP(n)
FN + TN
The Precision is defined as
Remark 3.2. In statistics, the following terminology char- acterizes the distribution of null p-values: if P P))((X) S t [[ob] = f, the p-value p(X) is called exact or uniform; if P[p(X)= < H|H < t. p(X) is called conservative. Com-
TP
(R))) n Ho(n)) max(|R=(n) 1}
Precision
TP+FP
Thus, the FDP is the "dual" concept of the Precision.
pared an exact p-value, a conservative one tends to under- state the evidence against the null Hypothesis.
*Based on the notations of R(n), Ho(n) and H1 we have the following relations: TP Ho(n)), FP = (R'(m) n Hi (n) and TN = RR(n) (n))
FN = (R())
'A sample means a sequence of examples.
3

---

An Online Statistical Framcwork for Out-of-Distribatoon Detection
3.3. LOND Algorithm
applications, which is a nonparametric estimation method for the p-value p p(Xtest). In statistics, let X1; X2, Xmm. be a sample of observations (i.e., random variable). For testing example X, the empirical p-value of X is defined as
To control FDR for the testing problem in (2), a direct idea is to apply the LOND algorithm, which has been heavily studied in various applications (Javanmard & Montanari, 2015; Robertson & Wason, 2018; Robertson et al., 2022; Liou et al., 2023).
プドー j== =
1(x; - x
+1
m + 1
Definition 3,4 (LOND (Javanmard & Montanari, 2015)).
Given a sequence of null hypotheses H1 H2.0.- H2.0. ... which arrive sequentially in a stream, with corresponding p-values pi, P2..... For a prescribed level a choose any sequence { such that Σ Σ== %
Note that X is one-dimensional random variable. Never- theless, OOD detection, the inputs are high-dimensional images. Therefore, utilizing the score function designed for OOD detection, we reduce each high-dimensional im- ages to the univariate score, and then compute the empir- ical p-values. Specifically, given a calibrated set Teal {XTal, X._i.... ........... } consisting of ID data, for testing example X!est, the empirical p- p-value p; corresponding to null hypothesis H&p is expressed as
of positive numbers 7 and define
0 a 1),
where ((0) = 0 and D - - 1(ps S ox). Then, the null hypothesis Ha is rejected if its corresponding p-value P; satisfies p3 0z.
p: - -() Kí E [m] : 8(X S ((X
H +
In statistics, is commonly specified 0.05. Similar to the type-I error, in multiple hypothesis testing, testing algorithm for problem 2 should make as many discoveries as possible (low FPR) while maintaining the FDR at a prescribed level. Javanmard & Montanari (2015) prove that p-values corresponding to all null hypotheses are available and mutually independent, then LOND algorithm controls FDR at pre-specified level.
m
where 8(-) is a certain score function. According to (Arlot et al., 2010), we can easily verify that empirical p-value in Eq. (5) satisfies Definition 3.1. Note that the empirical p-values for different testing examples are not independent since they rely on a common calibrated set Teal, Hence, we cannot directly apply LOND algorithm with empirical p-values to control FDR.
In most multiple hypothesis testing literature (Benjamini & Hochberg, 1995; Benjamini & Yekutieli, 2001; Blanchard &
3.4. Generalized LOND Algorithm
Roquain, 2008; Delattre& & Roquain, 2015; Cao et al., 2022), the p-values or the distribution of the testing statistic are assumed to be known. Denote by F(+) the cumulative dis- tribution function of s(X) where s(-) is the score function and X ~ Dz. Then, for given example Xtest, its p-value can be expressed as
To control FDR, we modify the LOND algorithm and then propose novel generalized LOND (g-LOND) algorithm to remove the constraints on the dependence between p-values. We denote f+ lim fz). and define two function classes:
p(xteat) (s(D)(((X) S s(Xtest)) =((((X)eat)
F (f(x) (0) = f(x) 2 1),
(4)
FB = (f(x): : (4 0, f(x) > 0, L'' f(x)
dr $1}
According the Definition 2, under the Hy (Xtest is the ID data), we have
loga E Fz. The g-LOND algorithm is defined as follows.
for x. E 10. It is easy to verify that -1
IP (F((((Xt )) 5z) P PP < ( (x)) = F (x)) = x,
Definition 3.5 (g-LOND). Given sequence of null hy- potheses H1 Hz,0. ... which arrive sequentially in a stream, with corresponding p- p-values Pl, P2: .... For a pre- scribed level a choose a function f(-) E Fi U F2 and sequence of positive numbers 7 = (2 such that
where (-) is the inverse function of F(-). Therefore, the random variable F(x(X (teet)) follows the uniform distribu- tion on (0, 1), namely, p(X is valid p-value and is exact. Obviously, small score s(xtest) leads to small p- value, consistent with the classical setting of OOD detection in (1) and the meaning of the p-value.
Di= 7/ 1, define
a, = =n 1),
However, in the context of the OOD detection, often we have litle prior information about underlying distribution F(.). Hence, the empirical p-value often used in real-world
k 1(f(p) S aa). Then, where D(0) = 0 and =i
the null hypothesis H,,0 is rejected if its corresponding p- value ps satisfies f(p:)) S ac.
4

---

An Online Statistical Framcwork for Out-of-Distribation Detection
Algorithm I Practical g-LOND Algorithm
(pi.pp....,,P. is PRDS on Ho(n) as well.
1:
Input:
Training set T. {Xpml, Xgat ..... X testing set Tlest (Xtest X2 2 3.... Xtest ,.".,, prescribed
calibrated set
2. For any E Ho(n), the function
Teat
P((11: .......PP)) E A Pi S
level a E (0, 1) and sequence (n
2:
Train the score function on T:
is increasing in a for any increasing set A.
S
((x) = max
Lemme 4.3 indicates that the PRDS is invariant for mono- tonic transformation. Then, our core theorem is presented as follows.
=E|K] ffll
where f is the logit Output. 3: Calculate the p-value corresponding to Xtest:
Theorem 4.4. Given prescribed level E (0, 1), a sequence of p-values P1,P2,"' corresponding to H2...... and f(-) E F1. If P2....... Pr. are mu- tually independent or PRDS, then the FDR of g-LOND al- gorithm satisfies
H E [m] s(X5al) S ((X:est) Pi
fest }
m +
4: Compute the significance level corresponding to Xtest
FDRG-LOND a.
0: = a - 1),
5: Output: Declare that Xteat is OOD if (p:) ac.
The proof of Theorem 4.4 is presented in Appendix A. Ob- viously, the empirical p-values are not independent, since they depend on the same trained socre function and cali- brated set. (Yu et al., 2023) demonstrates that the empirical p-values conditionally PRDS under some assumptions. This paper aims conduct in-depth study of the g-LOND algorithm under weaker conditions for FDR control. Then, we have following theoretical result.
The practical g-LOND algorithm with empirical p-values presented in Algorithm 3.4. Inspired by (Hendrycks al., 2022), we use maximum normalied logit s(x) 91 ligil our score function.
Theorem 4.5. Given prescribed level a E (0, 1), se- quence of empirical p-values pL. P2, ... corresponding to H H,o, H2.0.... and f(.) E F2. Then the FDR of g-LOND algorithm satisfies
4. FDR Control of g-LOND Algorithm
We first investigate the theoretical properties of the g- LOND algorithm about FDR. is well known that if p- values PI. are mutually independent or PRDS, the BH procedure can control FDR at level a (Benjamini & Hochberg, 1995; Benjamini & Yekutieli, 2001). Factually, the g-LOND algorithm also enjoys this theoretical result. We begin with the following definitions.
FDRy-LOND a.
The proof of Theorem 4.5 is presented at Appendix A.2. Theorem 4.5 indicates that the g-LOND algorithm controls FDR at a prescribed level regardless of the dependence be- tween the empirical v-auuuss Obviously, such argument invalid for many popular methods such LOND) algorithm and LORD algorithm. which broadens the applicability scope of the g-LOND algorithm.
Definition 4.1 (Increasing Set). A subset A C IR" is said to be increasing if for all x E A, x S implies y E A, where the comparison of x and y is component-wist
Then, PRDS property is defined as follows.
Definition 4.2 (PRDS(Benjamini & Yekutieli, 2001)). A family of random variables {X1, X2,... .,,, is said to be PRDS on subset C 21, ..,,,............... n) if for all E Io. the function (2, Xx,) D|X, x) is increasing function in x for any increasing subset A.
Remark 4.6. Motivated by (Benjamini & Yekutieli, 2001), (Javanmard & Montanari, 2015) show that the LOND al- gorithm also control FDR for dependent p- p-values by choosing special sequence řy where
7
To derive our theoretical results, we first introduce the fol- lowing lemma.
2km k
Obviously, such a algorithm is too conservative and almost accepts all null hypotheses for large resulting in a high FPR.
Lemma 4.3. ((Ma et al., 2024)) Suppose that the p-values P1,P2, .2...., are PRDS on Ho)
'Note that > & is monotonically increasing w.r.t. i and tends oo - co. Therefore, for large i, have di (Da( - 1) + 1) +
1. Denote P = f(p)) for all i E (1,2,....o) n} where f() is strictly increasing or decreasing. Then
5

---

An Online Statistical Framcwork for Out-of-Distribution Detection
5. Asymptotic FPR of g-LOND Algorithm
where f 0 is allowed to vary with the number of hy- potheses Eq. (6) shows that the non-null hypothesis is distinguished frome null hypothesis by positive mean shift
False positive rate (FPR) a significant evaluation criterion for OOD detection. Using the notations and Hj. the FPR can be expressed as
μ. In (Donoho & Jin, 2004), μ is set to /2r log(n). In this section, is parameterized as
FP FP+TN
(n) n (a(()) (1(r))
# (Ar 1/A
FPR
(7)
where r > 0. Following (Donoho & Jin, 2004; 2006; Jin & Ke, 2016), we focus on the sparse region in which the number of true null hypothesis larger than that of the true alternative hypothesis. In this case, we have
Although FDR control has been widely studied, relatively little is known about the theoretical properties of FPR. In this section, we investigate the asymptotic behavior of FPR for the g-LOND algorithm. Our analytical framework is similar to that of (Donoho Jin, 2004; Neuvial & Roquain, 2012).
[7((m)) = n 1-8
n1
where 3 < < 1. Besides, we denote
n1
n2
5.1. Analytical Framework
In the field of multiple hypothesis testing, many theoreti- cal studies (Javanmard & Montanari, 2015; 2018) assume that p-values are available, equivalently, the distribution of test statistic for each hypothesis is known. In this case, p- values can be expressed as P; = (I,)), where (-) is the survival function of test statistic and Tj is the observation of test statistic corresponding to H,. For mathematical con- venience, we impose that Ti. .e...... .......... Tn are independent continuous random variables. Reasonably, working with p-values pt, P2, ... ................................ vations T2, ... Tn- We first define function class as follows.
5.2. Asymptotic Property of FPR for g-LOND Algorithm
For simplicity, let 7/8 for 1, where
C
8
C
Denote т as the time of k-th rejection (with 0). Based on the analytical framework in Section 5.1, we first establish the upper bound of Th expectation 6.
, Pn is equivalent to working with obser-
Theorem 5.2. Given the first n hypotheses H2, H,, ,H,. and suppose that the correspond- ing observations Ti, "1,T2,....7 Tn satisfy the condition (6). Then, the expectation of Tk A satisfies
F= (t()): lim log (t) A
1 1
B(T A n1) S 3k ni for all 0 < <knm,
In this section, we describe the distribution of the obser- vation T; in terms of the generalized Gaussian-like family, which is a variant of the generalized Gaussian distribution.
where n1 17/
The proof of Theorem 5.2 is presented at Appendix A.3. Theorem 5.2 shows that the number of rejections is impacted by the sequence {7x ] and the parameter 33. For example, small leads to small ((TAAAn)) meaning that the g-LOND al algorithm tends to make more rejections. Based on Theorem 5.2, we obtain the following core result.
Definition 5.1 (Generalized Gaussian-like Distribution Fam- ily). A random variable X is said to follow the generalized Gaussian- distribution family with the location ja and the degree X, denote X G(H, A). if the survival function (-) of X satisfies (t n1) F.
meaning
Theorem 5.3. Given the first n hypotheses H1, Hz,.... H. and suppose that the correspond- ing observations Tı, T2..... Tn satisfy the condition (6) with (0 1/2 Iʃ < 3 and > B + +o - 1, then the FPR of g-LOND algorithm satisfies
is easy to verify that the popular Gaussian distribution and Laplace distribution belong to the generalized Gaussian-like distribution family, which are often used in the theoretical analysis.
Our inspiration for considering the tail generalized Gaussian distribution comes from the previous works (Donoho & Jin, 2004; Ingster & Suslina, 2012) on global testing. terms of the notation G(p, A), we assume that the observation T; is distributed as
FP
+
in probability.
FP+TN
The proof of Theorem 5.3 is presented at Appendix A.4. Theorems 4.5 and 5.3 show that the FPR of gLoL algo- rithm tends to zero probability while maintaining its FDR at the prescribed level.
2) if Ha(n) G(H,))
T
(6)
if E (1)))
c A means min(a, b).
6

---

An Online Statistical Framework for Out-of-Distribution Detection
Table The experimental results on CIEAR-100 as ID data. We use R as the backbone for each method. The best results are highlighted in bold.
Data
CIFAR-10
TinylnggaNet
SVHN
Texture
Place365
Practical
TPR
FPR
F1
TPR
FPR
IPR
FPR
F1
TPR
FPR
F1
TPR
FPR
F1
ASH
94.46
67.10
94.46
73.92
76.16
94.46
63.47
50.07
72.40
78.78
77.58
38.91
Cider
93.27
86,80
64,39
93.27
74.16
75.51
93.27
42.80
61.00
93.27
70,30
78.60
93.27
76.63
38.79
Energy
94.52
78.83
67.01
94.52
72.15
76.58
94.52
74.79
46.06
94,52
78.85
77.50
94.52
78.83
38.56
KLM
80.56
56.06
65.53
80.56
57.54
71.55
80.56
43.77
52.53
80.56
53.57
77.51
80.56
45.63
42.85
MSP
94.68
79.28
66.96
94,68
73.58
76.34
94.68
78.69
44.84
94.68
80.24
77.30
94.68
78.69
38.65
RankFeat
95.15
63.23
95.15
91.77
72.72
95.15
92.94
41.02
89.87
75.68
95.15
94,00
34.73
SHE
94.38
78,23
67.10
94.38
76.37
75.58
94.38
69.00)
48.07
94.18
80.04
77.19
94.38
80.40
38,05
VIM
94.15
85.82
65.04
94,15
81.77
74.30
94.15
71.29
47.26
94,15
57.50
81.80
94.15
81.84
37,57
DICE
93.95
77.78
67.02
93.95
75.55
75.54
93.95
64.88
49.33
93.95
76.43
77.70
93.95
78.92
38.35
PALM
94.55
78.29
67.17
94.55
72.23
76.58
94.55
75.10
45.94
94.55
79.20
77.44
94.55
77.52
38.96
Gram
94.61
96.67
62.65
94,61
93.23
72.16
94.61
35.54
63.71
94,61
70.47
79,25
94.61
96.32
34.03
KNN
94.61
79,32
66.92
94,61
71.67
76.74
94.61
63.47
$0.05
94.61
64.64
77.48
94.61
76.02
39.43
ODIN
94.65
80.20
66.71
94.65
74.17
76.20
94.65
86.16
42.65
94.65
74.37
78.46
94.65
77.79
38.91
B-LOND
9I.4
"4..19
68,06
"I.33
553777"
78.36
DE.08
"25.76"
69,54
44.82"
818
".2.8
500.82
52.87
Classical
FPR95
AUC
AUPR
FPR95
AUC
AUPR
FPR95
AUC
AUPR
FPR95
AUC
AUPR
FPR95
AUC
AUPR
ASH
80.06
75.66
75.80
79.92
65.89
85.60
75.89
74.36
80.72
78.76
56.57
Cider
90.49
65.68
63.26
80.32
77.26
83.43
65.52
72.91
64.34
74.70
78.23
86.45
81.84
73.37
49.24
Energy
80.40
79.05
79.75
74,09
82.76
87.96
76.66
82.03
69.84
80.14
78.35
85.79
80.46
79.52
60.62
KLM
84.26
73.91
68.97
77.80
79,22
83.02
78.43
79.34
66.69
76.28
75.77
81.21
77.35
75.70
42.46
MSP
80.36
78.47
79.60
74.63
82.07
87.81
79.66
78.42
64,92
81.26
77.32
85.42
79.64
79.22
61.13
RankFeat
95.01
60.17
91.37
92.50
72.14
62.89
69.40
93.76
63.82
40.54
SHE
80.60
78.15
78.98
78.97
79.74
85.71
71.54
80.97
65.99
82.00
73.64
81.42
82.63
76.30
$4,20
VIM
87.70
72.21
72.88
83.85
77.76
85:21
74.28
83.14
74.22
60.00
85.91
91.22
75.85
56.35
DICE
81.36
78.04
78.74
79.69
80,72
86.65
69.35
84.22
72.94
79.44
77.63
85.07
82.52
78.33
59.08
PALM
79.97
79.38
80,09
74.18
83,25
88.47
76.98
81.41
68.64
80.74
78.74
86.25
79.15
80.28
62.10
Gran
96.99
49.41
49.19
93.71
53.91
62.23
22.92
95.55
90.06
71.16
70.79
74.62
96.60
46,38
20.21
KNN
80.68
77.02
74.63
73.23
82,54
88.39
64,91
84.15
72.22
66.02
83.66
89.46
77.27
79.43
58.57
ODIN
81.31
78.18
78.88
75.61
87.16
58.41
75.59
79.33
79.05
79.45
59.24
B-LOND
7E,r
778.92
S0.18
66.61
"83,29"
88.66
34.74
"8..22"
76.77
36.25
89,229
68.51
"1.39
65.67
6. Experiment
2011), and Places365 (Zhou et al., 2018) as OOD data. For ImageNet-200, we use SSB-hard (Vaze et al., 2022; Zhang et al., 2023b), NINCO (Bitterwolf et al., 2023), iNatural- ist (Horn et al., 2018), Textures (Cimpoi et al., 2014), and Openlmage-O O (Wang et al., 2022) as OOD data.
In this section, we aim to validate the effectiveness of our proposed g-LOND algorithm. The evaluation criteria in- clude the practical metrics TPR. FPR and Fl-score, and the classical metrics FPR95, AUROC and AUPR. Exten- sive experimental results demonstrate the superiority of our method.
Metries. In this paper, report the practical and classical metrics. The practical metrics include TPR, FPR and F1- score. The classical metrics include FPR95, AUROC and AUPR. In this paper, we regard ID as positive?. 7.
6.1. Experimental Settings
We mainly follow the experimental settings in (Yang et al., 2022; Zhang et al., 2023b), and our codes are based on (Zhang et al., 2023b). We next introduce some necessary settings in experiments.
Model. When the CIFAR-100 is used as the ID data, we use the ResNet10 as the backbone for each method. When ImageNer200 is used as ID data, we use the ResNet50 as the backbone for each method. We follow the experimental implementation in Yang et al. (2022); Zhang et al. (2023b). More details can be found in Yang et al. (2022); Zhang et al. (2023b).
Baselines. We choose some popular OOD detection meth- ods as our baselines, including MSP (Hendrycks& & Gimpel, 2017), ODIN (Liang et al., 2018). Gram (Sastry & Oore, 2020), Energy (Liu et al., 2020), VIM (Wang et al., 2022), KNN (Sun et al., 2022), KLM (Hendrycks et al., 2022), RankFeat (Song et al., 2022), DICE (Sun & Li, 2022),
6.2. Experimental Results
The experimental results on CIFAR-100 as ID data are pre- sented in Tables and the Results on ImageNet200 ad the ID data are presented Table 2. We first analyze the ex- perimental Results of the g-LOND algorithm in terms of the practical metries. As the Table I shown, the g-LOND dramatically improves the detection performance in terms
ASH (Djurisic et al., 2023), Cider (Ming et al., 2023), SHE
(Zhang et al., 2023a) and PALM (Lu et al., 2024).
Benchmarks. We use CIFAR-100 (Krizhevsky, 2009) and ImageNet-200 (Deng et al., 2009) as ID data. For CIFAR-100, we use 10, TinylmageNer (Krizhevsky et al., 2017), SVHN (Netzer et al., 1), Texture (Kylberg,
IIn the code of (Zhang al., 2023b), OOD is to be positive.
7

---

An Online Statistical Framework for Out-of-Distribution Detection
Table The experimental results on ImageNet200 as ID data, We use ResNet50 as the backbone for each method. The best results are highlighted in bold.
Data
SSB hard
NINCO
iNaturalist
OpenImage.0
Texture
Practical
IPR
FPR
TPR
FPR
F1
TPR
FPR
TPR
FPR
F1
TPR
FPR
ASH
95.53
86.71
28 28.62
95.53
83.21
76.46
95.53
73.53
95.53
49.53
85.33
Cider
95.06
78.43
30.56
95.06
62.92
80.50
95.06
26.16
84,82
95.06
22.58
90.40
95.06
43.81
69.81
Energy
95.54
72.07
32.50
95.54
59.36
77.43
95.54
45.69
77.58
95.54
41.83
80.04
95.54
54.43
65.55
KLM
93.28
67.06
37.87
93.28
53 .4
75.
93.28
41,49
78.13
93.28
39.86
86.73
93.28
49.53
67.63
MSP
95.36
70.78
32.84
95.36
62.35
80.78
95.36
40.85
79.22
95.36
47.69
85.64
95.36
53.66
65.77
RankFeal
94.68
71.07
27.42
70.54
94.68
55.59
75.91
54.50
SHI
95.44
70.71
32.89
95.44
65.94
80.03
95.44
45.46
77.62
95.44
41.57
87.05
95.44
55.60
65.04
VIM
94.75
80.27
29.99
94.75
66.44
79.57
94.75
33.47
81.71
94,75
32.15
83.94
94.75
47.30
DICE
94.80
72.11
32.28
94.80
64.34
80.06
94.80
45.36
77.35
94,80
35.21
88.19
94,80
53.80
65.46
PALM
95.69
71.90
32.60
95.69
63.13
80.77
95.69
38.36
80.31
95.69
45.50
86.29
95.69
53.83
65.86
Gram
94.11
87.20
28.14
76.93
77.03
94.11
87.76
64.55
94,11
53.69
83.70
94.11
79.58
56.28
KNN
94.79
70.14
32.88
94.79
$7.09
81.69
94,79
25,59
84,93
94,79
24.73
90.19
94.79
42.82
70.14
ODIN
95.00
76.77
31.00
95.00
63.91
80.25
95.00
33.19
81.94
95.00
30.56
87.11
95.00
B- 3-LONDiour))
92.48
53.2
35.51
52.89
A45.88
'8,336"
90.007
20.388
86.73
93.29
17.38
91.56
53716
35.16
74.36
Classical
FPR95
ALC
AUPR
FPR95
AUC
AUPR
FPR95
AUC
AUPR
FPR95
AUC
AUPR
FPR95
AUC
AUPR
ASH
68.86
79.52
42.16
67.52
85.2
89.10
29.53
94.79
34.74
91.77
95.61
87.79
Cider
79.52
77.94
39.06
68.48
86.
91.32
45,99
92.87
93.81
35.20
92.36
95.21
56.57
90,22
87.9
Energy
73.01
79.83
43.78
65.70
85.1
90.08
44.31
92,55
93.28
41.06
90.79
93.78
56.64
89.23
85.91
KLM
74.61
77.56
37.28
71.11
83.96
88,70
46.65
91,80
91,90
67.66
86.13
90.46
64.37
87.66
82,42
MSP
72.88
80.38
46.66
64.48
86.29
91.26
41.94
92.8
93.44
49.94
88.36
92.36
56.98
89.24
86.19 19
RankFeat
91.50
58.74
55.10
64.34
98,26
29.10
50.19
37,26
SHI
71.07
78.30
41.10
67.47
82.07
87.40
42,95
91.43
91.51
38.20
90.51
93.41
$5.58
87.49
82.39
VIM
85.44
74.04
75.67
83.32
89.81
56.03
90.96
92.49
38.85
92.61
96.94
DICE
72.89
79.06
42.6
65.29
84,49
89.53
46,53
91.81
92.42
35.89
91.53
94.11
54.75
89.06
85.32
PALM
72.57
80.75
44.19
64.25
86.38
91.03
39.21
93.7
94.38
43.18
90.25
93.41
$5.34
90.13
87.38
Gram
89.25
65.95
26.50
79.45
69.40
74.67
89.88
65.30
62.30
57.01
80,54
83.35
81.78
67.72
$1.52
KNN
77.09
77.03
38.57
63.83
86.10
91.07
34.01
93.99
94.45
39.44
91.29
92.19
$2.35
90.19
87.54
ODIN
76.84
77.19
39.81
63.90
83.34
88.02
33,33
40.35
90.65
93.78
50.33
B-LONDiouS)
"3.49"
80.61
43..27
54,43
86,59
91.39
20.88
96.72
35.47
33.57
9283
994,89
3198
91.88
88.69
of FPR and Fl-score for all OOD data, despite a slight decrease in TPR. For example, using CIFAR-100 as ID data and Place365 OOD data, our proposed method re- duces the FPR from 45.63% to 30.62%, and improves the Fl-score from 42.85% to 52.87% compared with the best baseline, a direct improvement of 15.01 and 10.02 % at the cost of less than 3% decrease in TPR. Moreover, this improvement still exists in Table 2 in terms of the FPR and F1-score. For instance, using Iea2e as ID data and SSB hard as OOD data, our proposed method reduces the FPR from 67. 17.06% to 53.72%, and improves the F1 Fc-sc from 37.87% to 49.31% compared with the best baseline, a direct improvement of 13.34% % and 11.44% at the cost of 3. 21% decrease in TPR. These experimental results indicate that the g-LOND tends to classify more testing example as OOD while controlling the number of falsely classify- ing the ID as the OOD. Hence, our method achieves the smaller FPR and the larger F1 F1-score than baselines. The above analysis indicates that our method achieves a better tradeoff between TPR and FPR, which is consistent with our motivation of controlling FDR.
and the NINCO as the OOD data, our proposed g-LOND outperforms the best baseline by 9.4% in terms of FPR95 and by 0.21 % in terms of AUROC. This conclusion still holds for the different OOD data in Table 1. Overall, proposed g-LOND algorithm achieves promising OOD de- tection performance.
7. Conclusion
In this paper, we systematically investigate OOD detection problem from statistical perspective. Concretely, formu- late the OOD detection task as online multiple hypothesis testing problem and propose a novel g-LOND algorithm to solve it. Theoretically, the g-LOND algorithm controls FDR for dependent p- p-values. Besides, we derive the asymp- totic FPR of the g-I.OND algorithm under the generalized Gaussian-like distribution family, indicating that the FPR of g-LOND tends to 0 in probability. Finally, the exten- sive experiments demonstrate the effectiveness of g-LOND algorithm.
Then, we analyze the results of our method in terms of the classical metrics. From the Tables 2, we find that compared to the baselines, the AUROC and AUPR of the g-LOND algorithm achieve certain degree of improvement. What's even more exciting, the FPR95 of g-LOND is reduced obvi- ously. For example, using the ImageNet200 as the ID data
A Acknowledgment
This work is supported by the National Natural Science Foundation of China under Grant 624B2106, the Key R&DD Program of Hubei Province under Grant 2024BAB038, the National Key R&D Program of China under Grant 2023YFC3604702, the Fundamental Research Funds for
8

---

An Online Statistical Framework for Out-of-Distribution Detection
the Central Universities under Grant 2042025kf0045.
Casella, G. and Berger, R. L.. Statistical inference. Cengage Learning. 2001.
Impact Statement
Chen, Y. and Liu, W. A theory of transfer-based black-box Explanation and implications. In Neur!PS, 2023.
To our best knowledge, this work has no negative social im- pact. This work mainly provides a solid theoretical support for the field of the OOD detection. Hence, our work may promote the development of the related applications.
attacks:
Cimpoi, M., Maji, S., Kokkinos, Mohamed, S., and Vedaldi, A. Describing textures in the wild. In CVPR, pp. 3606-3613, 2014.
References
Dankers, V., Bruni, E., and Hupkes, D. The paradox of the compositionality of natural language: A neural machine translation case study. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), ACI., pp. 4154-4175, 2022.
Abati, D., Porrello, A., Calderara, S., and Cucchiara, R. Latent space autoregression for novelty detection. In CVPR, pp. 481-490, 2019.
Delattre, S. and Roquain, E. New procedures controlling the false discovery proportion via rolano-wolf's heuristic. Annals of Statistics, 43(3): 1 141-1177, 2015.
Aharoni, E. and Rosset, S. Generalized c-investing: def- initions, optimality results and application to public databases. Journal of the Royal Statistical Society Se- ries B: Statistical Methodology, 76(4):771-794, 2014.
Deng, J., Dong, W. Socher, R., Li. L., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248-24488 255, 2009.
Amodei, D., Ananthanarayanan, S., Anubhai, R., Bai, J., Battenberg, E., Case, C., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., Elsen, E., Engel, J. H., Fan, L., Fougner, C., Hannun, A. Y., Jun, B., Han, T., LeGresley, P., Li, X., Lin, L., Narang, S., Ng. A. Y., Ozair, S., Prenger, R., Qian, S., Raiman, J., Satheesh, S., Seetapun, D., Sengupta, S., Wang, C., Wang, Y., Wang. 7., Xiao, B., Xie, Y., Yogatama, D., Zhan, J., and Zhu, Z. Deep speech 2 End-to-end speech recognition in english and mandarin. ICML, volume 48
Djurisic, A., Bozanic, N., Ashok, A., and Liu, R. Extremely simple activation shaping for cuttt--distriuutio detection. In ICLR, 2023.
Donoho, D. and Jin, J. Higher criticism for detecting sparse heterogeneous mixtures. Annals of statistics, 32(3):962- 994, 2004.
Donoho, D. and Jin, Asymptotic minimaxity of false discovery rate thresholding for sparse exponential data. Annals of Statistics, pp. 2980-3018. 2006.
pp. 172-182, 2016.
Arlot, S., Blanchard, G., and Roquain, E. Some nonasymp- totic results resampling in high dimension, ii: Multiple tests. The Annals of Statistics, pp. 83-99, 2010.
Foster, D. P. and Stine, R. A. a-investing: a procedure for sequential control of expected false discoveries. Jour- nal of the Royal Statistical Society Series B: Statistical Methodology, 70(2):429 444, 2008.
Benjamini, Y. and Hochberg, Y. Controlling the false dis- covery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57(1):289-38) 300, 1995.
Gong, X., Yuan, D., and Bao, W. Understanding partial multi-label learning via mutual information. In Ran- zato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), NeurlPS, pp. 4147-4156, 2021.
Benjamini, Y. and Yekuticli, D. The control of the false dis- covery rate in multiple testing under dependency. Annals of statistics, pp. 1165-1188, 2001.
Gong, X., Yuan, D., and Bao, W. Partial label learning via label influence function. In ICML, volume 162, pp. 7665-7678, 2022.
Bitterwolf, J., Müller, M., and Hein, M. In out? fixing im- agenet nut-of-distribuun detection evaluation. In ICML., volume 202, pp. 2471-2500 2023.
Gong, X., Yuan, D., and Bao, W. Discriminative metric learning for partial label learning. IEEE Transactions on Neural Networks and Learning Systems, 34(8):4428- 4439, 2023a.
Blanchard, G. and Roquain, E. Two simple sufficient condi- tions for fdr control. Electronic Journal of Statistics, 2: 963-992, 2008
Gong, X., Yuan, D., Bao, W., and Luo, F. A unifying probabilistic framework for partially labeled data learn- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):8036-88)) 8048, 2023b.
Cao, H., Chen, J., and Zhang, X. Optimal false discovery rate control for large scale multiple testing with auxiliary information. Annals of statistics, 50(2):807, 2022.

---

An Online Statistical Framcwork for Out-of-Distribution Detection
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, pp. 770-778, 2016.
Liang, S., Li, Y., and Srikant, R. Enhancing the reliability of out-of-distribttion image detection in neural networks. In ICLR, 2018.
Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and outtrf--dstributtion examples in neural networks. In ICIR,, 2017.
Liou, L., Hornburg, M., and Robertson, D. S. Global fdr con- trol across multiple maseq experiments. Bioinformatics 39( 19())btac718, 2023.
Hendrycks, D., Basart, S., Mazeika, M., Zou, A., Kwon, J., Mostajabi, M., Steinhardt, J., and Song, D. Scaling cittof-ddistibuttio detection for real-world settings. In ICML,, volume 162, pp. $759-8773, 2022.
Liu, W., Shen, X., Du, B., Tsang, I. W., Zhang, W., and Lin, X. Hyperspectral imagery classification via stochastic hhsvms. IEEE Transactions on Image Processing, 28(2): 577-588, 2019.
Horn, G. V., Aodha, O. M., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S. J. The inaturalist species classification and detection dataset. In CVPR, pp. 8769-8778, 2018.
Liu, W., Wang, X., Owens, J. D., and Li, Y. Energy-based -uitt--distributtin detection. In NeurIPS, 2020.
Liu, X., Lochman, Y., and Zach, C. GEN: pushing the limits of softmax-based out-o--distribution detection. In CVPR, pp. 23946-23953 2023.
Huang, R., Geng, A., and Li, Y. On the importance of gradients for detecting distributional shifts in the wild. In NeurlPS, pp. 677-689, 2021.
Lu, H., Gong, D., Wang, S., Xue, J., Yao, L., and Moore, K. Learning with mixture of prototypes for out-of- distribution detection. In ICLR, 2024.
Huang, X., Kroening, D., Ruan, W., Sharp. J., Sun, Y., Thamo, E., Wu, M., and Yi, X. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretabil- ity. Computer Science Review, 37:100270, 2020.
Ma. X., Wang, Z., and Liu, W. On the tradeoff between robustness and faimess. In NeurlPS, 2022.
Ma, X., Zou, X., and Liu, W. A provable decision rule for out-of- distribbttii detection. In ICML, 2024.
Ingster, Y. and Suslina, I. A. Nonparametric goodness-of-ft testing under Gaussian models, volume 169. Springer Science & Business Media, 2012.
Ma, X., Wu, J., and Liu, W. SAC-BL: A hypothesis testing framework for unsupervised visual anomaly detection and location. Neural Networks, 185:107147, 2025.
Javanmard, A. and Montanari, A. On online control of false discovery rate. arXiv preprini arXiv: TrXXiv:502.066 1502.06197 2015.
Ming. Y., Fan, Y., and Li, Y. POEM: out-of--istribution detection with posterior sampling. In ICML, volume 162, pp. 15650-15665, 2022.
Javanmard, A. and Montanari, A. Online rules for control of false discovery rate and false discovery exceedance. The Annals of Statistics, 46(2): 5544, 2018.
Ming, Y., Sun, Y., Dia, O., and Li, Y. How to exploit hyper- spherical embeddings for out-of- detection? In ICLR, 2023.
Jin, J. and Ke, Z. T. Rare and weak effects in turge-scale inference: methods and phase diagrams. Statistica Sinica, pp. 1-34, 2016.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, Y. Reading digits in natural images with unsupervised feature learning. 2011.
Krizhevsky, A. Learning multiple layers of features from tiny images. Master's thesis, University of Tront, 2009.
Neuvial, P. and Roquain, F. On false discovery rate thresh- olding for classification under sparsity. Annals of Statis- tics, 40(5):2572-20500 2600, 2012.
Krizhevsky, A., Sutskever, I.. and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Commun. ACM, 60(6):84-99) 90, 2017.
Özbayoglu, A. M., Gudelek, M. U., and Sezer, O. B. Deep learning for financial applications A survey. Applied Soft Computing, 93: 90:3444 2020.
Kylberg, G. Kylberg texture dataset v. 1.0. Centre for Image Analysis, Swedish University of Agricultural Sciences, 2011.
Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R., DePristo, M. A., Dillon, J. V., and Lakshminarayanan, B. Likeli- hood ratios for out-of -ittr--distribution detection. In NeurIPS, 2019.
Lee, K., Lee, K., Lee, H., and Shin, J. A simple unified framework for detecting fiitr--disttitution samples and adversarial attacks. In NewrIPS, pp. 7167-7177, 2018.
10

---

An Online Statistical Framcwork for Out-of-Distribation Detection
Robertson, D. S. and Wason, J. Online control of the false discovery rate in biomedical research. arXiv preprint arXiv:1809.07299 2018.
Zhang, J., Yang, J., Wang, P., Wang, H., Lin, Y., Zhang, H., Sun, Y., Du, X., Zhou, K., Zhang, W., Li, Y., Liu, Z.,, Chen, Y., and Li, H. Openood v1.5: Enhanced benchmark for -ftto-i-iitttutionn detection. CoRR, abs/2306.09301, 2023b.
Robertson, D. S., Wason, J., König, F., Posch, M., and Jaki, T. Online error control for platform trials, arXiv preprint arXiv:2202.03383, 2022.
Zhang, S., Zhou, C., Zhang, P., Liu, Y., Li, Z.,, and Chen, H. Multiple hypothesis testing for anomaly detection in multi-type event sequences. In ICDM, pp. 808-817, 817, 2023c.
Sastry, C. S. and Oore, S. Detecting out-of-distributoon examples with Gram matrices. In ICML, volume 119, pp. 8491-8501, 2020.
Zhang, S., Zhou, C., Liu, Y., Zhang, P., Lin, X., and Pan, S. Conformal anomaly detection in event sequences. In ICML, 2025.
Song, Y., Sebe, N., and Wang, W. Rankfeat: Rank- feature removal for out-of-distribution detection. In NeurIPS, 2022.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. Places: 10 million image database for scene recog- nition. IEEE Trans. Pattern Anal. Mach. Intell., 40(6): 1452-1464, 2018.
Sun, Y. and Li, Y. DICE: leveraging sparsification for out- of-distribution detection. In ECCV, volume 13684, pp. 691-708, 2022.
Sun, Y., Ming, Y., Zhu, X., and Li, Y. Out-of-distribution detection with deep nearest neighbors. In ICML, volume 162, pp. 20827-20840 2022.
Zou, X. and Liu, W. On the adversarial robustness of out-of- distribution generalization models. In NeurIPS, 2023a.
Zou, X. and Liu, W. Generalization bounds for adversar- ial contrastive learning. Journal of Machine Learning Research, 24:1 14:1-114:54, 2023b.
Vaze, S., Han, K., Vedaldi, A., and Zisserman, A. Open-set recognition: good closed-set classifier is all you need. In ICIR, 2022.
Wang, H., Li, Z., Feng. L., and Zhang, W. Vim: Out-of- distribution with virtual-logit matching, In CVPR, 4911-4920, 2022.
Wei, H., Xie, R., Cheng, H., Feng, L., An, B., and Li, Y. Mitigating neural network overconfidence with logit normalization. In ICML, volume 162, pp. 23631-23644 2022.
Xu, J. and Liu, On robust multiclass learnability. NeurIPS, 2022.
Yang. J., Wang, H., Feng, L., Yan, X., Zheng, H., Zhang, W. and Liu, Z. Semantically coherent out-of-distribution detection, In ICCV, pp. 8281-8289, 2021.
Yang, J., Wang. P., Zou, D., Zhou, Z., Ding, K., Peng, W., Wang, H., Chen, G., Li, B., Sun, Y., Du, X., Zhou, K., Zhang, W., Hendrycks, D., Li, Y., and Liu, Z. Openood: Benchmarking generalized In NewrlPS, 2022.
detection.
Yu, C., Ma, X., and Liu, W. Delving into noisy label detec- tion with clean data. In ICML, volume 202, pp. 40290- 40305, 2023.
Zhang, J., Fu, Q., Chen, X., Du, L., Li, Z., Wang, G., Liu, X., Han, S., and Zhang, D. Out-of diitti--iitiitttinn detection based on in-distribution data patterns memorization with modern hopfield energy. ICLR, 2023a.

---

An Online Statistical Framcwork for Detection
A. Proofs
A.1. Proof of Theorems 4.4
Proof. We will prove these two theorems in three cases.
Case 1: p-values are mutually independent. Note that
[n) n (T)
1(f(p.) & a E Ho) 11
FDR(n) =E
max 1, [R()]]]
1(f(p:) S ag E Ho)
a2 max (nn).
E
00
According to the definition of Dnn,, max D(n), 2 ((1) for i E [n]. Then, we obtain
0₁ max - D(i 101-11a +1
a
0% (D( 1) D(i - 1)
07/
If f() Fi, then for any p-vuuue pa S f(p)). Hence, have
E(1(J(p:) S an Ho) 1)) < B(1(p: Sa,iE Ei) 1))
< B[a; - 1)) = az.
It follows that
(f(()) S a,,i E Ho)
1(p S a;, i E (Ha)
D - 11)]
E
E E
a₁
a₁
B(IID(i - 1)) =
The above analysis demonstrates that
8 FDR(n) S > E(1 on)) a.
n
# = a.
Case 2: p-values are PRDS.
Recall that f(-) is strictly increasing. By Lemma 4.3, we know that ((2)) -.. ,f(p) are PRDS. The proof in case 1 shows that
1(f(p.) S ar)
FDR(n) S
Σ
an E
Q;
(xHo(m)
Without loss of generality, we assume a; E (0, 1). For a positive number (0, 1), we choose positive integer m such that a; e. Denote sy = +1-j for j [m + 1]. Note that
PP (f(()) < a.) P (f(p)) S as a4 E Uff 8j+1))
12

---

An Online Statistical Framework for Out-of-Distribation Detection
Then, for i E Ho. the following chain of inequities hold:
(f(x)) S oc)
m P(((p)) S 8341, E (8j, 83+1])
E
04
8j
m P(f(P)) $1411) P(a; E (s), spalll((p))) s 8541). P(f(p)) s 884+1)
P([(()) S 884+1)
8j
m
P(as E (8), sy+1ll((mi) s 85+1) 8541
8j
m
Se-
(P(a; S s)+1lf(p)) s - P(a, S s,(f(pi)) S {z++1))
j=1
m-1
Σ (P(a: S 8/+1 ((x) S 85+1) P(a, S s/+1ls(pi) s 8)+t))
/=1
+ + (P(a; S sm (p:)) S sm+1) - P(a; S s1 splf(p;) S $2))
< P(ay S (p S sm+1)
(by Lemma 4.3)
Letting c + and applying the monotone convergence theorem, we have
(f(p.) S az)
E
1
Oii
It follows that
FDR(n) S Σ
an 15 a.
ic (a)
A.2. Prrof of Theore 4.5
Proof. Note that D(n) 2 D(i - 1) + 1(f(p)) S as) and
1(((p) sa, =1 1(15 ai
< ai
f(p))
f(p))
Then, we have
1(f(p) S as)
FDR(1) < Σ
E
(m))
isn
Σ
E
1(f(p)) S as)
1(f(p.) s sai]
ign,iEHo
ai/[(p)
Σ
E
-1((m)) 2 D(i 1) +1)
mm)(D(m),1]
ign,iEHo
as/S(pi)
Σ
E
D(i-1)+t 1)+1 1(D(n) 2 D(i - 1) +1)
ion
(D 1) 1(D(n) 2 D(i 1) m
Σ isn
E
(D(i-1)+1)f(p)
am
E
[(p:)]
H
13

---

An Online Statistical Framcwork for
Detection
Let random variable U be uniformly distributed on (0, For any / (0, 1), we have
P(p: 2 P(U t).
If 1, then Pp:p S = P(U < = 1. Hence, P(p: S () - PU () for any 2 0. Since s()) is positive and strictly increasing, it follows that
x) = P(p: S S
P f(p.)
σ)
[(U)
for any x 2 0. According to the following property of expectation:
E(X) = / 2f(x) dr /
P(X 2 x) dx
(8)
for any non-negative random variable X, then we obtain
P( 1 [(U) 1)
E
P
1) J.
f(p:)
J
[mm)
E(
= J [(u)
du S 1.
f(U)
Therefore, obtain
FDR(n) 1
Σ E at
Σ
0% E
[(ps)
fpp)
isn
ign
0% a.
A.3. Proof of Theorem 5.2
Proof. For notational convenience, we denote by Xtest Xiest, Xtest the OOD examples in testing set. For f in the g-LOND algorithm, we have fxx) 2 and f (x) < x. Besides, denote by tb = - T the cumulative distribution function (CDF) of testing statistic T' under the null hypothesis. For simplicity, we assume 4(0) 1/2. Denote by Fi (t) the CDF of the p-values under alternative hypothesis. Then, F(t) can be expressed as
((( V (1)
where - the inverse function of Under the null hypothesis, the CDF of the p value Fo(() t. Then the CDF the p-values can be expressed as
F(t) (1 - F) + =F)(t) (1 E)t + ev 4 1 t)).
Denote L = 1- Fi and A = Φ (1 t). Because V/ F, we have
== き (1 - t) ~ (A log(1 111
According to the definition of g-LOND algorithm, for > 0, and m 2 Th + we have
P(T++1 > m Tx)
II P(f(p)) > (k 11m) = II (1 ((++ 1)x)))
(m +1++
s expp=
Σ F(+
i=
14

---

An Online Statistical Framcwork for Out-of-Distribation Detection
Denote 5.- Th n1. For Th < M1,, we have
E E
[k)
Th
+1
Σ
P( > m (kk)
Th +
Σ
exp1
Σ
((++
Tk
expl
Σ
(k++
1=
Define t" satisfying + 6- 1111---1-1 f-)((t))) = μ. Note that V E F. Then, for t 2 t", we have
(t) = (1 Φ (1- (t)) 2 V(p Φ5 1 f (t))) 1/2.
Define M1 == +( 1)C/E)) If < M1, we have Fi(f ((k: + 1>h)) 2 1/2.
Without loss of generality, we assume M1 2 n. Since Fx < m & M1, we have that
Σ
expl-
Σ
((++ <
Σ
m=++++1
1=t+11
m=
81 F
exp( -mE/mm/
< 2
=2n
m=1
follows that
E(fe+1 (x)) S Fr + 1 2m
Therefore, we have
E(T Am) & 3km
(10)
which completes the proof of Theorem 5.2
A.4. Proof of Theorem 5.3
Proof. For any 8 >> define M2 = [n8/8], /8). Since < < 1.2 for large n, have Ma < E where 11
(71 n
For M2 i < n1, define 0g == i n get
PID(1)) < =
= PPTTo > s P(Tpo An 21
<3. [0: [0:Ing
(0; +)ng
3
i
9
38 3 n2 68.
According to the definition of g-LOND algorithm, we have
E (a))
= Ef
+ < <E
Hence, the expectation of FPR for L-ON satisfies
E(PRR) = n1 (x))(N)
Σ
E(ū(f- - Σ Ev(
(11)
(EHi(n)
For sig M2.
1
Σ
ENU (T < M₂
m1
15

---

An Online Statistical Framework for Out-of-Distribution Detection
For M2 1 ssisi 11 <.
(100))))))0000100000010000001000 - 66 (()
(((()))))))(())))))))) 2 0.)
= +5 ((())()))))
follows that
M2 + in M2 (60 + ((h0)()
Σ [(x
iE (m)
(7/2 $ 76 + t (f 0n))
= 75 + w( - kn):
where Kn = -1- f- fn)) Since v +
r and f (nn S nn as n - co, then we have
- Ki 2 (r 1/a - + + 3 - 1) (Alogn) 00
and further (H - Kn) + 0 as n - oc. Then we have
sup E(FPR) S 78. 220000
(12)
Note that Eq. (12) holds for any 5 > 0, thus E(FPR) - as + co. By Markov's inequality, for any />>> 0, we have
P(FPRy. LOND 21) B(FPR, LOND) - 0
t
Hence, we conclude that
0 in probability,
16


Reviews:
[{"review_id": "yt038wUwar", "reviewer": "Reviewer_sP8g", "summary": "This work investigates out-of-distribution (OOD) detection from the perspective of online multiple hypothesis testing. This paper proposes a generalized LOND algorithm that controls the false discovery rate even under dependent p-values. This work also derives the asymptotic false positive rate of the g-LOND algorithm under a generalized Gaussian-like distribution family. Experiments demonstrate the effectiveness of g-LOND.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Q1__: about the reference.\n\n__Ans-Q1__: Thank you for providing these meaningful references [1]-[3]. We will discuss these papers in related work section.\n\n\n\n__Weakness 1__. The proposed method may be sensitive to both the size and quality of the calibration set.\n\n__Ans-w1__. In practice, we just need to randomly sample a small number of examples from the ID training set as the calibrated set, without any special operations. For example, when using CIFAR-10 as ID data, the calibrated set contains 2000 ID examples. According to your comments, we use CIFAR-10 as ID data, and  SVHN and Place365 as OOD data to study the performance of TPR, FPR and F1-score with variant size of calibrated set. The experimental results are presented in Figure 1 of PDF (see https://anonymous.4open.science/r/gLOND-BBCE/Experimental%20Results%20for%20Rebuttal.pdf). The results show that the performance of our method does not significantly vary as the size of the calibration set increases.\n\n\n\n\n__Weakness 2__. Evaluation is limited to ResNet models; exploring and validating the approach on more advanced architectures (e.g., Transformers, CLIP) would be beneficial.\n\n__Ans-w2__. Because of the limitation of time, we just choose four methods based on CLIP architecture as our baselines, including MCM[4], GLMCM[5], SeTAR-MCM and SeTAR-GLMCM[6]. We use ImageNet-1k as ID data, and use iNaturalist, Places, Sun and Texture as the OOD data. Our code is based on [6]. The experimental results are presented in Table 1 of PDF (see https://anonymous.4open.science/r/gLOND-BBCE/Experimental%20Results%20for%20Rebuttal.pdf), which demonstrate the superiority of our proposed g-LOND algorithm over these CLIP-based methods in [4]-[6].\n\n\n[1] GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection, CVPR 2023\n\n[2] POEM: Out-of-Distribution Detection with Posterior Sampling, ICML 2022\n\n[3] How Does Unlabeled Data Provably Help Out-of-Distribution Detection? ICLR 2024\n\n[4] Delving into out-of-distribution detection with vision-language representations.\n\n[5] Zero-shot in-distribution detection in multi-object settings using vision-language foundation models.\n\n[6] SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation.", "claims_and_evidence": "The main claim that the generalized LOND algorithm controls the false discovery rate even under dependent p-values is supported by formal theorem and detailed proofs in the appendix. Furthermore, extensive experiments demonstrate its empirical improvements.", "methods_and_evaluation_criteria": "The proposed method frames OOD detection as an online multiple hypothesis testing problem, leveraging FDR-control procedures to handle dependence among p-values, which is a reasonable approach for OOD tasks. The use of public benchmarks such as TinyImageNet and SVHN is standard, and TPR, FPR, and F1 provides appropriate metrics for evaluation.", "theoretical_claims": "I check the theoretical claims and proofs, which looks logically consistent. The generalized Gaussian assumption may not match the complexity of real-world data.", "experimental_designs_or_analyses": "The experimental design, which uses commonly accepted OOD datasets like SVHN, Places365, and iNaturalist, supports the paper’s claims. The authors rely on ResNet18 and ResNet50 as backbone models. However, using more advanced architectures such as CLIP could support a broader analysis.", "relation_to_broader_scientific_literature": "This work connects the existing statistical framework of LOND to OOD detection, which is an online multiple hypothesis testing method. This provides a rigorous theoretical underpinning for OOD methods, which are important for ensuring AI safety.", "essential_references_not_discussed": "A few works appear essential for understanding the paper’s key contributions but are not currently discussed in the paper. For example:\n\n[1] GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection, CVPR 2023\n\n[2] POEM: Out-of-Distribution Detection with Posterior Sampling, ICML 2022\n\n[3] How Does Unlabeled Data Provably Help Out-of-Distribution Detection? ICLR 2024", "other_strengths_and_weaknesses": "Strengths\n\n1. The paper leverages FDR control and provides a theoretical analysis of the false positive rate.\n\n2. Extensive experiments on several public benchmarks demonstrate the method’s effectiveness.\n\nWeaknesses\n\n1. The proposed method may be sensitive to both the size and quality of the calibration set.\n\n2. Evaluation is limited to ResNet models; exploring and validating the approach on more advanced architectures (e.g., Transformers, CLIP) would be beneficial.", "other_comments_or_suggestions": "None", "questions_for_authors": "Refer to the Weaknesses section for the detailed questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hNczH40rSt", "reviewer": "Reviewer_UdXp", "summary": "This paper studies the OOD detection task as an online multiple hypothesis testing problem. It presents a new algorithm, called the generalized LOND algorithm (g-LOND), built upon the well-known LOND algorithm. They provide theoretical results about the false discovery rate (FDR) and false positive rate (FPR) for their algorithm. The paper also provides many experiments on OOD detection, comparing to several baselines on several datasets. Overall, the proposed method offers a systematic and theoretically grounded solution to the OOD detection problem, enhancing the reliability of sensitive applications.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Q1__. While the paper rigorously develops its theoretical framework, it would be beneficial to outline any underlying assumptions made in this paper.\n\n__Ans-Q1__. In Theorem 4.5 and 4.6, we have no underlying assumptions. In Theorem 5.3, we just assume that the testing statistic follows generalized Gaussian-like distribution family in Definition 5.1.\n\n\n__Q2__.  There are some typos in the paper in Appendix. The authors should check the text.\n\n__Ans-Q2__. Thanks for your careful review. We have checked the typos in Appendix. We will fix these typos in new version.", "claims_and_evidence": "Yes. The authors conduct extensive experiments to demonstrate the effectiveness of the g-LOND algorithm.", "methods_and_evaluation_criteria": "Yes. The proposed g-LOND algorithm makes sense for the OOD detection problem.", "theoretical_claims": "Yes. I have checked the correctness of the proofs in Appendix, which are both clear and solid.", "experimental_designs_or_analyses": "Yes. The experimental designs and analyses are sound. This paper evaluates the proposed method using practical and classical criteria, making its conclusions convincing.", "relation_to_broader_scientific_literature": "This paper clearly explains how it is related to previous work. The authors establish the connection between OOD detection and online multiple hypothesis testing, and then proposes novel g-LOND algorithm with strong statistical guarantee.", "essential_references_not_discussed": "To the best of my knowledge, the essential and relevant references are discussed", "other_strengths_and_weaknesses": "(1) Overall, this paper is well-motivated and has a clear organization. Besides, its notations and the definitions are clear, and the ideas are easy to follow.\n\n(2) Different from previous literature which mainly focuses on designing or training score function, this paper study the OOD detection problem based hypothesis testing framework and proposes a novel g-LOND algorithm to solve it.\n\n(3) Under some conditions, the authors establish the asymptotic theories about FPR, which  remains underexplored in previous literature.\n\n(4) Extensive experimental results on multiple benchmarks (including large-scale and high-resolution ImageNet) can support the proposed method.\n\n Weaknesses: I have not identified major weaknesses of this paper, while I do have some minor concerns that are listed in the \"Questions For Authors\" part.", "other_comments_or_suggestions": "No", "questions_for_authors": "(1) While the paper rigorously develops its theoretical framework, it would be beneficial to outline any underlying assumptions made in this paper.\n\n(2) There are some typos in the paper in Appendix. The authors should check the text.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ebuq3k8BwU", "reviewer": "Reviewer_eu3U", "summary": "this paper thinks the OOD detection task from an perspective of online multiple hypothesis testing.\n\nthe g-LOND algorithm controls false discovery rate (FDR) at pre-specified level without the consideration for the dependence between the p-values.\n\nAlong with thorecticla analysis, the empirical effectiveness of g-LOND is evluated on cifar-100 and imagenet-200.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Q1__: about the reference.\n\n__Ans-Q1__: Thank you for providing these meaningful references [a]-[i]. We will discuss these papers in related work section.\n\n\n__W1,2__. the evaluiation on CIFAR-10 and ImageNet-1k lacks. the comparision with the mostly recent baselines [a,b,c,d,e,f,g,h,i] lacks.\n\n__Ans-W1,2__. According to your suggestions, we conduct corresponding experiments on CIFAR-10 and ImageNet-1k using the methods  LINe[b], NECO[d],VAR[f], NAC[h] and CONJNORM[i]. We use the code of OpenOOD [j]. \nIt should be noted that reference [g] in your Review is missing. Besides, the methods SHE in [a] and ASH in [c] have been used our baselines (see Section 6.1 baselines). Since [e] does not open their code and time of rebuttal period is limited, we do not implement the method in [e]. The experimental results are presented in Tables 2-5 of PDF (see https://anonymous.4open.science/r/gLOND-BBCE/Experimental%20Results%20for%20Rebuttal.pdf ), which demonstrate the superiority of our proposed g-LOND algorithm over these methods in [a][b][c][d][f][h][i].\n\n\n\n\n\n__W3__. Due to the rapid development of neural networks, CLIP-based models should be also considered.\n\n__Ans-W3__. Because of the limitation of time, we just choose four methods based on CLIP architecture as our baselines, including MCM[k], GLMCM[l], SeTAR-MCM and SeTAR-GLMCM[m]. We use ImageNet-1k as ID data, and use iNaturalist, Places, Sun and Texture as the OOD data. Our code is based on [m]. The experimental results are presented in Table 1 of PDF (see https://anonymous.4open.science/r/gLOND-BBCE/Experimental%20Results%20for%20Rebuttal.pdf), which demonstrate the superiority of our proposed g-LOND algorithm over these CLIP-based methods in [k]-[m].\n\n\n\n[a] Out-of-distribution detection based on in-distribution data patterns memorization with modern hopfield energy\n\n[b] LINe: Out-of-Distribution Detection by Leveraging Important Neurons\n\n[c] Extremely simple activation shaping for out-of-distribution detection\n\n[d] NECO: NEural Collapse Based Out-of-distribution detection\n\n[e] Optimal Parameter and Neuron Pruning for Out-of-Distribution Detection\n\n[f] VRA: Variational Rectified Activation for Out-of-distribution Detection\n\n[h] Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization\n\n[i] Tractable Density Estimation for Out-of-Distribution Detection\n\n[j] OpenOOD: Benchmarking Generalized\nOut-of-Distribution Detection\n\n[k] Delving into out-of-distribution detection with vision-language representations.\n\n[l] Zero-shot in-distribution detection in multi-object settings using vision-language foundation models.\n\n[m] SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "yes", "experimental_designs_or_analyses": "yes", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "[a] Out-of-distribution detection based on in-distribution data patterns memorization with modern hopfield energy\n\n[b] LINe: Out-of-Distribution Detection by Leveraging Important Neurons\n\n[c] Extremely simple activation shaping for out-of-distribution detection\n\n[d] NECO: NEural Collapse Based Out-of-distribution detection\n\n[e] Optimal Parameter and Neuron Pruning for Out-of-Distribution Detection\n\n[f] VRA: Variational Rectified Activation for Out-of-distribution Detection\n\n[h] Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization\n\n[i] Tractable Density Estimation for Out-of-Distribution Detection", "other_strengths_and_weaknesses": "**Strength**\n1. this paper investigates OOD detection from a fresh perspective\n2. this paper is well written\n3. the evalution, which is based on 6 metrics, is comprehensive\n\n**Weakness**\n1. the evaluiation on CIFAR-10 and ImageNet-1k lacks\n2. the comparision with the mostly recent baselines [a,b,c,d,e,f,g,h,i] lacks\n3. due to the rapid development of neural networks, CLIP-based models should be also considered.\n\n*While I am happy to rasie my rating if the authors can address my concerns, i will also consider reviews from other reviewers regarding the technological nolvety of hypothesis testing used in this paper before making my final decison.*", "other_comments_or_suggestions": "see weakness", "questions_for_authors": "no", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L6urmbfJhZ", "reviewer": "Reviewer_RpMq", "summary": "The paper focuses on the out-of-distribution (OOD) detection task. Unlike previous research that primarily focuses on designing powerful score functions, this paper introduces a novel perspective by framing OOD detection as a online multiple hypothesis testing problem. The authors propose a Generalized LOND (g-LOND) algorithm with both rigorous theoretical guarantees and strong empirical performance. The g-LOND algorithm enables to control the false discovery rate (FDR). Besides, its false positive rate (FPR) converges to zero in probability. Experimental results show that the g-LOND algorithm outperforms traditional threshold-based methods across various OOD detection benchmarks.\n\n## update after rebuttal\n\nI am willing to champion this paper. I have read all reviews and rebuttals. To my best knowledge, this paper may be the first to propose a novel online hypothesis testing framework for OOD detection with a strong theoretical guarantee. Extensive experiments demonstrate the effectiveness of this framework. Besides, the authors also provide the new experiment results in the rebuttal, which further enhance the claims of this paper. So, I recommend the acceptance of the paper.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__W1__. In Algorithm 1, the proposed method utilizes a calibrated set for hypothesis testing. I think a detailed explanation for it would be helpful.\n\n__Ans-W1__. In practice, we just need to randomly sample a small number of examples from the ID training set as the calibrated set, without any special operations. In our experiments, for  CIFAR-100 as ID data, the calibrated set contains 2000 ID examples; for ImageNet-200 as ID data, the calibrated set contains 10000 ID examples.\n\n\n__w2__. The authors need to provide a simple discussion of the motivation.\n\n__Ans-W2__. Usually, the traditional threshold-based decision rule in Eq.(1) performs well on ID data but performs poorly on OOD data. Because the selection of its threshold just considers a high TPR on ID validation set. By contrast, our method considers the performance on both ID data and OOD data simultaneously. Specifically, our method can control FDR. Intuitively, to control FDR, the g-BH tends to reject more null hypotheses while maintaining small false rejections of the null hypothesis. Small false rejections means to control the number of falsely classifying the ID as the OOD (maintaining a high TPR) and more rejection of null hypotheses means to classify more testing examples as OOD (leading to a low FPR on OOD data).\n\n__Q1__. In multiple hypothesis testing, do other evaluation criteria exist similar to FDR?\n\n__Ans-Q1__. Family-wise error rate (FWER) and  marginal FDR (mFDR)  are two similar evaluation metrics. The FWER can be expressed as \n$$ FWER = P(R\\cap H _0 > 0). $$\nThe mFDR can be expressed as \n$$ mFDR = \\frac{E(R\\cap H_0)}{E(R)}. $$", "claims_and_evidence": "Yes. The claims made in the submission is supported by many theoretical results and extensive experiments.", "methods_and_evaluation_criteria": "Yes. The proposed method is based on the statistical hypothesis testing framework, and the evaluation criteria are appropriate for OOD detection task.", "theoretical_claims": "Yes. I check the proofs of Theorem 4.5, Theorem 4.6 and Theorem 5. These proofs are sound.", "experimental_designs_or_analyses": "Yes. The experimental designs and analyses are sound and well-executed, with a clear framework for evaluating the proposed method.", "relation_to_broader_scientific_literature": "This paper is based on the previous multiple hypothesis testing literature. The authors modify the traditional LOND algorithm and extend its theoretical results such that the proposed method can be applied to OOD detection task.", "essential_references_not_discussed": "No, the paper includes essential references. It adequately discusses prior works that are crucial for understanding the context of the key contributions.", "other_strengths_and_weaknesses": "Strengths \n\n- The paper introduces a novel perspective on OOD detection by framing it as an online multiple hypothesis testing problem. This departure from traditional approaches adds originality and encourages new thinking in the field.\n\n- The proposed g-LOND algorithm is innovative and theory-inspired, with good interpretability. This methodology represents a departure from conventional threshold-based methods.\n\n- The techniques of the proofs in the Appendix are sound and detailed.\n\n- The proposed method is distribution-free and easy to implement. Besides, extensive experiments are conducted to validate the proposed g-LOND procedure, demonstrating its superiority over traditional methods. \n\nWeaknesses\n\n- In Algorithm 1, the proposed method utilizes a calibrated set for hypothesis testing.  I think a detailed explanation for it would be helpful.\n\n- The authors need to provide a simple discussion of the motivation.", "other_comments_or_suggestions": "See weaknesses.", "questions_for_authors": "In multiple hypothesis testing, do other evaluation criteria exist similar to FDR?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: After review, the paper received two positive and two negative reviews. Following the author’s rebuttal, all reviewers maintained their original positions. During the AC-reviewer discussion, all reviewers actively participated—one of the negatively inclined reviewers raised concerns about the paper’s experimental setup and evaluation metrics.

Given these points, the AC conducted a thorough assessment of the paper and concluded that its technical contributions outweighed its weaknesses (including evaluation limitations and insufficient discussion of related work). Particularly, after checking the paper, the authors include reasonable baselines and evaluation metrics in their experiments. Also, related works suggested by reviewers are compared in the authors' rebuttal.

Accordingly, the AC recommends acceptance. The authors are encouraged to revise the paper carefully, addressing the reviewers’ feedbacks by providing a more thorough discussion of related work.
Decision: Accept (poster)