A Bayesian Nonparametric Perspective on Mahalanobis Distance for Out of Distribution Detection
Authors: Randolph Linderman
Noah Cowan
Yiran Chen
Scott Linderman
Venue: Accepted by TMLR ()

Abstract: Bayesian nonparametric methods are naturally suited to the problem of out-of-distribution (OOD) detection. However, these techniques have largely been eschewed in favor of simpler methods based on distances between pre-trained or learned embeddings of data points. Here we show a formal relationship between Bayesian nonparametric models and the relative Mahalanobis distance score (RMDS), a commonly used method for OOD detection. Building on this connection, we propose Bayesian nonparametric mixture models with hierarchical priors that generalize the RMDS.  We evaluate these models on the OpenOOD detection benchmark and show that Bayesian nonparametric methods can improve upon existing OOD methods, especially in regimes where training classes differ in their covariance structure and where there are relatively few data points per class.

Published in Transactions on Machine Learning Research (02/2026)
A Bayesian Nonparametric Perspective on Mahalanobis Distance for Out of Distribution Detection
Randolph W. Linderman
madobpplinndrmmnn@buk..dd
Electricnl and Computer Engineering Department
Duke University
Noah Cowan
ncwwwnnsstafoord.comm
Statistics Department
Stanford University
Yiran Chen
yiron. wtron.chen@duuke.. edu
Electrical and Computer Engineering Department
Duke University
Scott W. Linderman
scott. wet..iindermonnStaaooo....
Statistics Deportment and Wu. Tsaš Neurosciences Institate
Stanford University
Reviewed on OpenReview: https: // /ptp//veeeeeeeeee/ net/ forum? id= #3bMMPNND
Abstract
Bayesian nonparametric methods naturally suited to the problem of out-of-distribution (OOD) detection. However, these techniques have largely been eschewed in favor of simpler methods based on distances between pre-trained or learned embeddings of data points. Here we show a formal relationship between Bayesian nonparametric models and the relative Mahalanobis distance score (RMDS), a commonly used method for OOD detection. Building on this connection, we propose Baycsian nonparametric mixture models with hicrarchical priors that generalize the RMDS. We evaluate these models on the OpenOOD detection benchmark and show that Bayesian nonparametric methods can improve upon existing OOD mothods, especially in regimes where training classes differ in their covariance structure and where there are relatively few data points per class.
1 Introduction
Machine learning systems inevitably face data that deviate from their training distributions. Generally, this data is either sparsely labeled or wholly unsupervised. Faced with such dynamic enviromment, intelligent system must accurately detect outliers and respond appropriately. This capability is the subject of modern research on out-of-distribution (OOD) detection (Hendrycks & Gimpel, 2017), anomaly detection Chandola et al. (2009), and open-set recognition (Scheirer et al., 2012).
Outlier detection one of the oldest problems statistics (Anscombe, 1960), and there are well-sstablished methods for lackling this canonical problem. For example, Bayesian nonparametrie methods offer coherent, probabilistic framework for estimating the probability that data point belongs to new cluster (Ferguson, 1973; Antoniak, 1974; Lo, 1984; Sethuraman, 1994; MacEachern, 1994; Neal, 2000). Here, we use Dirichlet Process Mixture Models (DPMMs) to fit a generative model to the training data. Under this model, OOD detection reduces to a straightforward computation of the probability that data point belongs to a novel class.
Care must be taken with Bayesian nonparametric methods, however. Modern machine learning systems often built. on foundation models that have been trained massive datasets (Dosovitskiy et al., 2020:

---

Published in Transactions on Machine Learning Research (02/2026)
Caron et al., 2021: Oquab et al., 2024; Darcet et al., 2024; Chen et al., 2020a;b). These models yield feature embeddings of data points that can be used for several downstream tasks, but the embeddings are high- dimensional. When fitting a Gaussian DPMM, for example, we must implicitly estimate the covariance of embeddings within and across classes, which presents both computational and statistical challenges. We propose a hierarchical model that adaptively shares statistical strength across classes when estimating these high-dimensional covariance matrices.
Generative classifiers like these have been largely eschewed in favor of simpler distance metrics, like the relative Mahalanobis distance score (RMDS; Ren et al., 2021). Here, we show both theoretically and em- pirically that RMDS is a close approximation to the outlier probability under a Gaussian DPMM with tied covariance matrices, connecting this widely-used approach to inference in a Bayesian nonparametric model. From this perspective, we propose hierarchical models that generalize RMDS by relaxing the assumption of equal covariance matrices across classes.
We find that hierarchical Gaussian DPMMs offer a well-grounded and practically competitive approach to OOD detection. Section 2 reviews related work, and Section 3 covers important background on DPMMs. We make theoretical connection between RMDS and DPMMs in Section 4. This connection motivates our use of hierarchical models for estimating the high-dimensional covariance matrices in DPMMs - including a novel "coupled diagonal covariance" model - which we describe in Section 5 and evaluate in Section 6. We compare these models to existing baselines on synthetic datasets as well as the OpenOOD benchmark to characterize the regimes where Bayesian nonparameteric yield improved OOD performance !
2 Related Work
The OOD detection task for image classification is the binary decision that predicts if an image belongs to a class that is outside of the set of known classes that have been observed during training. This task has been widely studied and many solutions have been proposed. For example, some approaches alter the architecture or objective of classifier (Tack et al., 2020; Huang & Li, 2021; Wei et al., 2022; Linderman et al., 2023), and others exploit auxiliary outlier datasets (Hendrycks et al., 2019; Zhang et al., 2023). While training and outlier exposure based methods are important research directions for OOD detection the focus of this work is post-hoc methods. We refer readers to the available high quality OOD detection surveys to learn more about these methods (Yang et al., 2022; Zhang et al., 2024). Our approach is related to a class of post-hoc methods including max sofumax probability (MSP; Hendrycks & Gimpel, 2017), temperature-scalel MSP (Guo et al., 2017), ODIN (Liang al., 2018), energy-bases OOD Liu et al. (2020), the Mahalanobis distance score (MDS; Lee et al., 2018), and the Relative MDS (Ren et al., 2021), which derive OOD scores from embeddings or activations of pre-trained network.
Recently, Zhang et al. (2024) proposed a set of Near and Far OOD benchmarks, as well as a leaderboard named OpenOOD to facilitate comparison across methods. The OpenOOD benchmarks found (1) that post-hoc methods are more scalable to large datasets, (2) there is no method that is best on all datasets, and (3) methods are sensitive to which model was used for embedding. The best performing OpenOOD methods for vision transformer (ViT) feature embeddings are the MDS and RMDS. The relative Mahalanobis distance score was inspired by earlier work by Ren et al. (2019) that addressed the poor performance of the OOD performance with density estimation methods. Sun et al. (2022) propase to relax some of the assumptions of Mahalanobis distance methods by using the nogative k-th nearest neighbor distances instead. We will show that the relative Mahalanobis distance score (RMDS) is similar to scores derived from Bayesian nonparametric mixture models Section 4.
Bayesian nonparametric methods have previously been proposed for outlier detection and used in several applications. Shotwell & Slate (2011) proposed to detect outliers within datasets by partitioning data via DPMM and identifying clusters containing a small number of samples as outliers. Varadarajan et al. (2017) developed a method for detecting anomalous activity in video snippets by modeling object motion with DPMMs. Another line work explored Dirichlet prior networks (DPN; Malinin & Gales, 2018; 2019) that explicitly model distributional uncertainty arising from dataset shift as a Dirichlet distribution over the
The implementation for all models and experiments available at https://github. http:////gtttbb..ccco/////////// http://cwwww.//////////eo/o
2

---

Published in Transactions on Machine Learning Research (02/2026)
categorical class probabilities. More recently, Kim et al. (2024) performed unsupervised anomaly detection through an ensemble of Gaussian DPMMs fit to random projections of subset of datapoints. Our work focuses on connecting DPMMs to post-hoc confidence scores and developing hierarchical Gaussian DPMMs that share statistical strongth across classes in order to estimate their high-dimensional covariance matrices.
3 Background
We start with background on Dirichlet process mixture models (DPMMs) and the special case Gaussian DPMM with tied covariance.
3.1 Dirichlet process mixture models
Dirichlet process mixture models (Lo, 1984) are Bayesian nonparametric models for clustering and density estimation that allow for countably infinite number of clusters. There is always some probability that data point could come from cluster that has never been scen before i..., that the new point is an oullier.
Let D= = {En, yn Jn denote set of training data points In E IRD and labels Un E [K]. Likewise, let DR = (xn yn = k} denote the subset of points assigned to cluster k, and let N₂ = |Dal denote the number of such points. Now consider a new, unlabeled data point x. Under DPMM, corresponding label has probability,
Nep(2 Dk) app())
if k E [K] k = K +
ply D)
(1)
where the hyperparameter E IR, specifies the concentration of the Dirichlet process prior.
The first case captures the probability that the new point belongs to one of the training clusters (that it is an inlier). That probability depends on two factors: 1) the number of training data points in that cluster
since, intuitively, larger clusters are more likely; 2) the posterior predictive probability, which is obtained by integrating over the postcrior distribution of cluster parameters, 0x:
/ p(r 0x) P&) d0,
pl D&)
(2)
f
plx 0k)
II S,,DD
p(an 0x) p(0s) d0x.
The second case of eq. (1) captures the probability that the new point an outlier. depends on the concentration a and the prior predictive probability obtained by integrating over the prior distribution of cluster parameters,
p(x) p(x 0x) p(0x) d0,.
(3)
For many models of interest, the posterior and prior predictive distributions have closed forms.
3.2 Gaussian DPMM with Tied Covariance
For example, consider Gaussian DPMM in which each cluster is parameterized by a mean and covari- ance, 0₂- = (Mx, Σg). Assume a conjugate prior for the mean, 44 ~ ((p,, Eo). For now, assume that all clusters share the same covariance matrix, which we express through an atomic prior, E. dz. The hyperparameters of the prior are 7 (Ho. Eg, E).
Under this Gaussian DPMM, the conditional distribution of new data point's label is,
if k E [K] aN po, En + + k = K +1
No 14e E +)
ply xx D)
(4)
3

---

Published in Transactions on Machine Learning Research (02/2026)
where
14 = Eg (Zg 1o + N-E-- Σg = 4 +
(5)
and 8g Nv
1. Ex,ED, Zn is the mean of the data points assigned to cluster k.
The relative probability of theses cases is an intuitive measure of how likely a point is to be an outlier. Indeed, the next section shows that the outlier probabilities from this Bayesian nonparametric model are closely related to another common outlier detection score.
4 Theory: Connecting Relative Mahalanobis Distance and DPMMs
Here we show that widely used outlier detection method called the relative Mahalanobis distance score (RMDS; Ren et al., 2021) is closely related to the outlier probabilities obtained using Gaussian DPMM with tied covariances. RMDS outputs a score, C(z), where smaller values indicate that data point æ is more likely to be an outlier. The RMDS score of new point is defined as follows,
MDo(x) = (x - (o) S (s PP)
MD&(F)) (x- --)x)) T px) RMDx(I) MDD(0)) MD+(z)
C(x) max (M)
(6)
(r) are squared Mahalanobis distances, Pāo and En are the sample mean and covari- ance of the data, As is the sample mean of cluster k, and E E>> Py. (In Pyy.) is the sample within-class covariance.
where MDo(æ) and MDa(2)
Ren et al. (2021) motivated this score terms of log density ratios between a Gaussian distribution for each cluster and a Gaussian "background" model. Specifically,
N( pr.. Σ)
RMD (x) = 2og
+ d,
M(x
foo εn)
where d = log [$} log |Éol does not depend on x or k. Larger values of RMDx(x) indicate that x is more likely under cluster & than under the background model.
The procedure for mapping (MDx(x) values to the score C(x) is inherited from the Mahalanobis distance score (MDS; Lee et al., 2018). If the log density ratio is negative for all k then the background model is more likely than all of the existing clusters, and hence is likely to be outlier. Propositions 4.1 and 4.2 show that similar computation is at work in the outlier probabilities for DPMMs.
First, we show that the inkier probabilities under a general DPMM (not necessarily Gaussian) can be expressed terms of quantity analogous C(x). Proposition 4.1. The inlier probohilily of general DPMM with concentration can be expressed as follows, p(u [K] D) - o(((z) - log o/N) where a(u) (1 + -- the logistic function,
(8)
= Σ N₂ the average cluster size, and
K
Ö(x) = log
e Aog Mu/N
(9)
1=1
p( D2)
A = log
(10)
p(x)
Here, d the log density ratio of the posterior and prior predictive distributions from eq. (1).
We flip the sign of (x) compared Ren al. (2021), but account for in the definition of C(x) that the resulting score unchanged. Our presentation in keeping with the definition of the MDS score (Lee al., 2018).
4

---

Published in Transactions on Machine Learning Research (02/2026)
Proof. The inlier probability is one minus the outlier probability. Normalizing the outlier probability in eq. (1) and rearranging, we can write the inlier probability
ap(x)
ply E [K] 2, D) =1- ap(x) + Ek=1 NEP(E Dz)
No N//p(a Da) g/Np(xx)
1+
1 (1 ((z)
= =0(Ō(x) - n/N)
(11)
where Ö(x) is defined in eq. (9) and the last line follows from the fact that 1 -
This proposition says that the log-odds of data point belonging to existing cluster is the difference of DPMM score, C(x), which is analogous to the relative Mahalanobis score, and a threshold, log 9/N, which is tuned by the hyperparameter a.
Next, we show that in certain regimes, the DPMM score from a Gaussian DPMM with tied covariance is al- most perfectly correlated with the RMDS. Below, we define the relative covariance matrix R E-1/2 1/2 and note that as its operator norm [Rllo goes 0, then we intuitively have that Eo is growing larger with respect to E.
Proposition 4.2. Consider each cluster k has mean μı and common covariance E. If the cluster means are drawn from Gaussian prior (( En), then for any c, 5 > O there cxist Ko, No such that if n. S Ko and N 2 No then with probability at least 1 -
Suub-Gaussian data in IR generated from K clusters with equal size N, where
(Õ(x) - (C(X) - dlI < 8 log K
where C(x) is the RMDS, Ö(z) is the DPMM score from Gaussian DPMM with tied covariance E and hyperparometers (Do, Eo, Σ), and d is an additive constant, defined in eq. (7).
Proof. We give sketch here and refer the reader to the full proof in Appendix E. For each k, we decompose the difference, A 1 [RMD+(() - d], and show that it is small. The differences can be separated into terms that get smaller as и decreases,
|log P20 + 8 - log Eoll: (x (x- + 51)-11 11 - (o),
plus two more terms that decrease as N increases,
loog E + У - log |E]| (x- (u)) (xx) - (z 2/) (Ex + -x- m4).
We collect these four terms and show that cach is small with high probability. Finally, since the DPMM score is given by a log-sum-exp, which is 1-Lipschitz in the foo norm, it follows that if each d is within of 1 (R) - d], then C(x) is within 5 + log K of max; [RMD& - d = [C(x)-d)
This proposition establishes the close correspondence between the relative Mahalanobis distance score and the log-odds that point is an inlier under Gaussian DPMM with tied covariance. Note that the log K factor in Proposition 4.2 is irreducible due to the difference between the max used in RMDS and the smooth approximation used in DPMMs. We view this as feature not bug of the DPMMs: Since the log K factor only appears when the scores for different clusters are close to identical, the gap arises when the DPMM aggregates evidence across multiple equally plausible components rather than arbitrarily selecting a single cluster with RMDS. In practice, we find that a close correspondence holds in the experiments below and in Appendix I. This correspondence provides further support for using RMDS for outlier detection, beyond the original motivation in terms of log likelihood ratios. However, from this perspective, we also recognize several natural generalizations of RMDS that could improve outlier detection through richer DPMMs. We present three such generalizations below.
5

---

Published in Transactions on Machine Learning Research (02/2026)
1.0
Data
0.8
Null
Density
0.6
0.4
0.2
0.0
10
20
30
Förstner-Moonen Distance
Figure 1: Pörstuer-Moonen distance between covariance matrices of the 1000 classes in the Imagenet-1k VIT-B-16 feature space (Data) versus 1000 samples of covariance matrices from the Wishart mull distribution, W(N, E/N). See Appendix B for complete details.
5 Hierarchical Gaussian DPMMs
RMDS has proven to be highly effective outlier detection method, but it assumes that all clusters share the same covariance. This assumption helps avoid overfitting the covariance matrices for each class (Ren et al., 2021), but it is not always warranted. Figure shows a histogram of differences between empirical covariance matrices Eg and Σg for all pairs of classes (k, K²) in the Imagenet-1K dataset, as mcasured by the Porstner-Mooner distance (Förstner & Moonen, 2003).
These pairwise distances are systematically larger than what we would expect under null distribution where the true covariance matrices are the same for all classes, and the empirical estimates differ solely due to sampling variability. Complete details of this analysis are provided in Appendix B. This analysis suggests that the covariance matrices are significantly different across classes and motivates a more flexible approach.
The connection between RMDS and Gaussian DPMMs established above suggests a natural way of relax- ing the tied-covariance assumption without sacrificing statistical power: Instead of estimating covariance matrices independently, we could infer them jointly under hicrarchical Bayesian model (Gelman et al., 1995). With such a model, we can estimate separate covariance matrices for each cluster, while sharing information via prior. By tuning the strength of the prior, we obtain the tied covariance model in one limit and fully independent model in the other. Finally, we estimate these hierarchical prior parameters using a simple wwwpeetttttiimmmiiiiiaeaa algorithm that runs in matter of minutes, even with large, high-dimensional datasets.
5.1 Full Covariance Model
First, propose hierarchical Gaussian DPMM with full covariance matrices and conjugate prior. The cluster parameters, 0g = (bk; ) are drawn from a conjugate, normal-inverse Wishart (NIW) prior,
p(0)) = (Ex Po. vo. (vo - 1)1oo)) N N(x μo, Ky Ex);
(12)
where IW denotes the inverse Wishart density. Under this parameterization, E[En] Eo for vo D+1. The hyperparameters of the prior are 77 (Vo, Kin No: En).
The most important hyperparameters are and Eo, as they specify the prior on covariance matrices. As Mo + 00, the prior concentrates around its mean and we recover a tied covariance model. For small values of Vo. the hierarchical model shares little strength across clusters, and the covariance estimates are effectively independent.
We propose a simple approach estimate these hyperparameters in Appendix F. Briefly, we use empirical Bayes estimates for the prior mean and covariance, setting #40 Ωó and En Σ. We derive expectation-

---

Published in Transactions on Machine Learning Research (02/2026)
A.
Correlation of diag(Ix Σ)
4
(
diag(En1s)
3
diag($428)
200
Dimension d'
Variance
diag(E452)
diag(E158)
2
400
diag(
1
600
0
-1
0
200 400
600
200
400
600
Dimension d
Dimension
Figure A: Diagonal of empirical covariance matrices, diag(Ex) for five randomly chosen clusters (colored lines) over dimensions. Compared to the diagonal of the average covariance matrix, diag(E)), individual clusters tend to have systematically larger smaller variances than average. B: The correlation between dimensions of the deviation from the mean, Σ. Σ, of the diagonal components. The strong positive correlations between all but the first few dimensions indicates that the relationship observed
consistent across all clusters.
maximization (EM) algorithm optimize vn and Ko. Thanks the conjugacy of the model, the E-step and the M-step for Ko be computed in closed form. We leverage generalized Newton method (Minka, 2000) to update the concentration hyperparameter, Vo, effectively learning the strength of the prior to maximize the marginal likelihood of the data.
Finally, the prior and posterior predictive distributions are multivariate Student's distributions with closed- form densities. The log density ratios derived from these predictive distributions form the basis of the DPMM scores, (z))
5.2 Diagonal Covariance Model
Even with the hierarchical prior, we find that the full covariance model can still overfit to high-dimensional embeddings. Thus, we also consider simplified version of the hierarchical Gaussian DPMM with diagonal covariance matrices. Here, the cluser parameters O2 {uk,a, HL and the conjugate prior is,
D p(0)) = II x Poiaal uo,d; a..a) x N(uukd Po,di ko
(13)
1=
where -2 is the scaled inverse chi-squared density.
In addition to having fewer parameters per cluster, another advantage of this model is that it allows for different concentration hyperparameters for each dimension, Vo,d- We estimate the hyperparameters using a procedure that closely parallels the full covariance model. Likewise, the prior and posterior predictive den- sities reduce products of scalar Student's densities, which are even more efficient to compute. Complete details are in Appendix G.
5.3 Coupled Diagonal Covariance Model
The diagonal covariance model dramatically reduces the number of parameters per cluster, but it also makes strong assumption about the per-class covariance matrices. Specifically, it assumes the variances, o2

---

Published in Transactions on Machine Learning Research (02/2026)
Full
Diagonal
Coupled Diag.
Tied
A.
=
4
C.
Vo 4.0
Vo 8.0
Vo 16.0
20
1.0
10
RMDS AUC
0.9
X2
0.8
-10
22
0.7
-0
20
0.7
0.8
0.9
10.7
0.8
0.9
10.7
0.8
0.9
B.
Vo = 16
N₂ =
N₂ = 20
N₂ = 100
20
1.0
Indep RMDS AUC
10
0.9
0
0.8
-10-
20
0.7
-20
10
10
20
0.7
0.8
0.9
10.7
0.8
0.9
10.7
0.9 DPMM AUC
0.8
1
DPMM AUC
DPMM AUC
Figure 3: Synthetic experiments panel. Example sampled 2D dataset from DPMM with params Py = 4 (A) and (B). Each data set performance classifying outliers (gray dots) drawn from the prior predictive distribution. C: Performance of DPMM models vs. RMDS when sweeping over ue with N₂ = 20 shows that DPMMs outperform when 10 is small and there is greater variation in the Ex's. D: Independent RMDS performance DPMMs as function of N₂ with v = 4. Independent RMDS only performs well when there are adequate numbers samples per class.
clusters with N₂ 20 training data points each (colored dots). We evaluate
conditionally independent across dimensions. Figure suggests that this is not the case: the diagonals of the empirical covariance matricos, Σ,, tend to be systematically larger smaller than those of the average covariance matrix, E. This analysis suggests that of are not independent; rather, if of is larger than average, then okd is likely to be larger as well.
We propose novel, coupled diagonal covariance model to capture these effects. Specifically, introduce scale factor 7% E I2 that scales the variances for class l compared to the average. In this model, the cluster parameters are 0₂ = (7k, [1e,ds of ( and the prior is,
D p(0x)= = x2( oo) II d=1
x "oh.a vo.d, moo x N(uuaa Mo,d, hidd
(14)
where 7k scales the means of oha for all dimensions d to capture the correlations seen above.
Our procedure for hyperparametor cstimation and computing DPMM scores is very similar to those described above. The only complication that with the 7h the posterior distribution no longer has a simple closed form. However, for any fixed value 7k, the coupled model a straightforward generalization of the diagonal model above. Since 7 a one-dimensional variable, we can use numerical quadrature integrate over its possible values. Likewise, we can estimate the hyperparameter og using generalized Newton method, just like for the concentration parameter un- See Appendix H for complete details of this model.
8

---

Published in Transactions on Machine Learning Research (02/2026)
6 Experiments
We experimentally tested these hierarchical Gaussian DPMMs on real and synthetic datasets. First we used simulated datascts to build intuition for where hierarchical models improve performance. Then we compared hierarchical Gaussian DPMMs to other widely-used OOD metrics on the OpenOOD Benchmark, and we studied performance versus dimensionality of the embeddings.
Table 1: Performance of Hierarchical Gaussian DPMM and baseline methods the OpenOOD benchmark datasets Yang et al. (2022); Zhang et al. (2024), including 3 ID datasets (Imagenet-1K (Russakovsky et al., 2015) and CIFAR-10/100 (Krizhevsky et al., 2009)), and Near and Far OOD datasets. Accuracy of the classifiers predicting the label [K] for in-distribution test data is reported for each benchmark. Other columns report AUROC scores for OOD detection
OpenOOD benchmark datasets. "We found the tied DPMM performance on Imagenet- 1K improved when the prior parameter Eg is set to the covariance of the data. For the CTFAR experiments we set Eo to the covariance of the cluster means.
CIFAR-10
CIFAR-100
Imagenet- 1K
Method
Accuracy
Near
Far
Accuracy
Near
Far
Accuracy
Near
Far
MSP
94.93
88.36
91.80
76.19
80.33
78.88
80.90
75.80
86.30
Temp.
Scale
94.93
88.43
91.84
76.19
80.57
79.25
80.90
77.29
88.62
MDS
95.04
85.41
90.15
76,10
58.86
69. 69.29
80.41
78.97
92.57
RMDS
95.04
89.83
92.42
76.10
80.17
82.97
80.41
80.03
92.59
Hierarchical Gaussian DPMMs
Tied
95.04
89.83
92.42
76.10
80.17
82.98
80.41
79.28*
92.70*
Full
94.95
90.63
93.50
76.64
79.22
81.20
76.78
70.66
86.31
Diagonal
94.76
89.14
90.86
76.07
79.22
82.88
76.54
80.60
90.85
Coupled Diag.
94.76
88.30
90.70
76.04
78.05
79.29
76.52
80.98
90.72
6.1 Synthetically generated dataset experiments
To understand the regimes in which hierarchical models yield benefits, we simulated D = dimensional data from full covariance models with varying Yy and Ng- When My is small, covariances differ considcrably across clusters, and the assumptions of the tied model (and of RMDS) are not well met. Conversely, as ih - oo, the prior concentrates on Eg, and the covariances are effectively tied. Figure 3A and 3B show simulated datasets from these two regimes. As expected, Figure 3C shows that hicrarchical Gaussian DPMMs yielded considerable improvements when My was small, and the largest improvements came from the full covariance model, which matched the data generating process. Notably, the tied model matched the RMDS performance, as predicted in Section 4.
We then asked if the hierarchical model was strictly necessary or whether a simpler model would suffice. For example, we considered an "Independent RMDS" based on Mahalanobis distances to the per-class covariance estimates, Σg, instead of the average covariance Σ. Intuitively, we expected the hierarchical models to perform best when there were few data points per cluster relative to the dimensionality of the embeddings; i.e., when Ng/D D is small. Indeed, Figure 3D shows that DPMMs offered substantial improvements in this regime, with diminishing gains N increased.
These analyses suggest that hierarchical Gaussian DPMMs should yield benefits in regimes where covariances differ across classes and the number of data points per class is relatively small.
6.2 OpenOOD Benchmark
Next, we compared hierarchical Gaussian DPMMs to other widely-used OOD detection methods on the OpenOOD benchmark datasets Yang al. (2022); Zhang al. (2024). The benchmark consists of 3 in- distribution (ID) datasets, CIFAR-10, CIFAR-100, and Imagenet-IK For each ID dataset, several OOD datasets are grouped into Near and Far, where the Near OOD datasets
similar to ID. For the

---

Published in Transactions on Machine Learning Research (02/2026)
Full
Diagonal
Coupled Diag.
Tied
Near OOD
Far OOD
ID Accuracy
0.8
0.9
0.9
0.7
Accuracy
AUROC
0.8
AUROC
0.8
0.6
0.7
0.7
0.5
0.4
24
25
2
27
28
2
20
24
2
26
2
2
2
200
2
2
2
27
2
2
2
Dimension
Dimension
Dimension
Figure 4: Performance on "near OOD", "far OOD", and in-distribution classification as a function of the feature dimension the Imagenet-IK task. We projected the 768-dimensional ViT-B-16 features into lower dimensions using PCA, then projected into the eigenspace of the average within-class covariance. We compared the tied model (with full covariance) the hierarchical model with full, diagonal, and coupled diagonal covariance and measured performance by area under the receiver operator curve (AUROC).
Imagenct-1K experiment, we used D = 768 dimensional embeddings from the ViT-B-16 model trained ac- cording to the DeIT method (Touvron et al., 2021), which are available in the Pytorch torchvision (TorchVi- sion maintainers and contributors, 2016) package. The CIFAR experiments use OpenOOD's pretrained ResNet18 (He et al., 2016) features, D 512. We preprocessed the embeddings as described in Appendix A. As baselines, we considered both MDS (Lee et al., 2018) and RMDS (Ren al., 2021). We also compare to maximum softmax probability (MSP) (Hendrycks & Gimpel, 2017) and temperature scaled MSP with T 1000 (Temp. Scale) (Guo et al., 2017) We trained single lincar layer with gradient descent and supervised cross-entropy loss for the MSP methods. For all models, measured the accuracy of classifying which class an in-distribution test image came from, as well as the AUROC score for outlier detection across the OpenOOD datasets. For DPMMs, we computed AUROC scores using C(x).
Table shows that hierarchical models do offer improved performance, but the results are nuanced. We found the complexity and scale of the ID dataset determines which modeling assumptions are most appropriate, The full covariance model yields the highest performance 10 task. Whereas the coupled diagonal model is the best performing model in Near OOD settings for the large-scale Imagenet- task. The full covariance model performed surprisingly poorly the Tmagenet- task, and we suspected it was due to the high-dimensional embeddings. We provide expanded results tables for each ID dataset that show performance on cach OOD dataset in Appendicos J, K and M.
Near and Far OOD the small scale CIFAR-
6.3 Performance vs. Dimensionality
Finally, we asked how these methods compare as we vary the dimensionality of the embeddings. We suspected that the full covariance model would perform better in lower dimensions for two reasons. First, has O(KD2) parameters compared to only O(KD) in the diagonal models and O(D2) in the tied model, even with hierarchical prior, the full model could still overfit. This problem is exacerbated for classes that have fewer data points than the feature dimension, in which case the prior has strong effect on the conditional distribution of the per-class covariance matrix and the posterior predictive distributions. Second, we suspected that the inverse Wishart prior distribution, which has only a scalar concentration vo, may be a poor prior for high-dimensional covariance matrices.
To test this hypothesis, we swept the number of principal components retained in preprocessing (see Ap- pendix A). We found that out-of-distributtion detection of the full-covariance hierarchical model plateaued
10

---

Published in Transactions on Machine Learning Research (02/2026)
for D 2 128 dimensional embeddings (Figure 4). By contrast, the diagonal and coupled diagonal models performed considerably better, especially on Near OOD benchmarks. The diagonal covariance model out- performs the tied model across all dimensions on Near OOD detection, as well as in lower dimensions for Far OOD. However, in-distribution classification accuracy platcaus around 256 dimensions.
Altogether, these analyses of synthetic and real datasets show that hierarchical Gaussian DPMMs are ad- vantageous for OOD detection, especially in regimes where: i) covariance matrices differ across clusters; di) the number of data points per cluster is small compared to the dimension; and iii) detection relies on fine- fine-grained distinctions between training data and Near OOD test points.
7 Discussion
We developed a theoretical connection between the relative Mahalanobis distance score for outlier detection and the outlier probability under Gaussian DPMM with tied covariance. This Bayesian nonparametric perpsective led us to propose hierarchical Gaussian DPMMs that allow each cluster different covariance matrix, while still sharing statistical strength across classos via the prior. We developed efficient EM al- gorithms to estimate the hyperparameters of the hierarchical models, and we studied their performance on synthetic data as well as the OpenOOD benchmarks. We found that these models - especially the coupled diagonal covariance model - yielded improved performance on some benchmarks, especially the Near OOD benchmarks.
Limitations and Future Work Despite the hierarchical prior, we found that the full covariance Gaussian DPMMs were prone to overfitting. Further work could explore low-rank plus diagonal covariance matrices, which would interpolate between the diagonal and full covariance models. More generally, like RMDS and MDS, we assume that features are Gaussian distributed within each class. The competitive performance on the OpenOOD Imagenet benchmark using VIT features suggests that this assumption is reasonable (Yang et al., 2022; Zhang et al., 2024), but there no guarantee. Future work could consider fine-tuning the features or learning a nonlinear transformation to address this potential source model misspecification, as in prototype networks (Snell et al., 2017).
Bayesian nonparametric approaches naturally extend other closely related problems, like generalized category discovery Vaze et al. (2022a) and continual learning Van de Ven et al. (2022). For example, given collection of data points, DPMM may have sufficient evidence to allocate new classes for the out-of- distribution data. More generally, casting OOD as inference in a generative model brings modeling choices to the fore. Here, we focused on the challenge of modeling high-dimensional covariance matrices that may vary across classes, but there are several other ways in which the simple Gaussian DPMM could be improved. For example, we could attempt to capture the nonstationarity inherent in the OOD setting by allowing the prior predietive distribution to drift from what was inferred based on the training data. Such model could afford greater robustness OOD detection tasks.
Conclusion In summary, we find that Bayesian nonparametric methods with hierarchical priors are promising approach for OOD detection. If the features extracted from foundation models are reasonably well approximated as realizations of Gaussian DPMMs, the posterior inferences under such models can provide accurate estimates of outlier probability. This probabilistic perspective not only casts widely used methods in a new light, also leads to practical model improvements and enables several lines of future inquiry.
Acknowledgments
We thank Jingyang Zhang for his feedback and detailed knowledge of the OpenOOD codebase. RWL was supported in part. by the National Science Foundation (NSF) under Grant No. 2112562 to YC, the U.S. Army Research Laboratory and the U.S. Army Research Office (ARL/ARO) under grant number ARO- W911NF-23-2-0222 YC, and the Department of Defense (DoD) through the National Defense Science & Engineering Graduate (NDSEG) Fellowship Program. SWL was supported in part by fellowships from the Simons Collaboration on the Global Brain, the Alfred P. Sloan Foundation, and the McKnight Foundation.
11

---

Published in Transactions on Machine Learning Research (02/2026)
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF, the ARI,/ARO, or the DoD.
References
Frank J Anscombe. Rejection of outliers. Technometrics, 2(2):123 146, 1960.
Charles E Antoniak. Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. The Annals of Statistics, 2(6):1152-1174, 1974.
Julian Bitterwolf, Maximilian Müller, and Matthias Hein. In or out? Fixing ImageNet out-of-distribution detection evaluation. In Proceedings of the 40th International Conference on Machine Learving (ICML), volume 202, pp. 2471 2506. PMLR, 2023.
Eric Carlen. Trace inequalities and quantum entropy: An introductory course. In Robert Sims and Daniel Ueltschi (eds.), Contemporryy Mathemalics, volume 529, pp. 73 140. American Mathematical Society, Providence, Rhode Island, 2010. ISBN 978-0-8218-5247-7 978-0-8218-8208-5. doi: 10. 10.10900com/ coom//22mm 10428.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference Computer Vision (ICCV), 2021.
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM computing surveys (CSUR), 4133:11558, 2009.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119, pp. 1597-1607. PMLR, 2020a.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self- supervised models are strong semi-supervised learners. In Advances in Newral Information Processing Systems (NeurIPS), volume 33, pp. 22225-22555. Curran Associates, Inc., 2020b.
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In The Twelfth International Conference on Learning Represenlations (ICLR), 2024.
Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distributiin detection. In The Eleventh International Conference on Learning Representations, 2023. URL attpss//openreviite .nw.esn.ee/eeee
Alexcy Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. In The Ninth International Conference Learning Representations (ICLR), 2020.
Thomas S Ferguson. Bayesian analysis of some nonparamotric problems. The Annals of Statistics, 1(2): 209-230,100 1973.
Wolfgang Förstner and Boudewiju Moonen. A metric for covariance matrices. In Geodesy- The Challenge of the 3rd Millennium, pp.
Springer Berlin Heidelberg, Berlin, Heidelberg, 2003.
Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Boyesion data analysis. Chapman and Hall/CRC, 1995.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the 84th International Conference on Machine Learning (ICML), volume 70, pp. 1321-1330. PMLR, 2017.
12

---

Published in Transactions on Machine Learning Research (02/2026)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CV Conference Computer Vision and Paltern Recognition (CVPR), pp. 770 778, 2016.
Dan Hendrycks and Kevin Gimpel. bascline for detocting misclassified and out-of-distribution examples in neural networks. In The Fifth International Conference on Learning Representations (ICLR), 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In The Seventh International Conference on Learning Representations (ICLR), 2019.
Rui Huang and Yixuan Li. MOS: Towards scaling out-of-distribation detection for large semantic space. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 8710 8710-8719, 8719, 2021.
Dongwook Kim, Juyeon Park, Hee Cheol Chung, and Seonghyun Jeong. Unsupervised outlier detection using random subspace and subsampling ensembles dirichlet process mixtures. Pattern Recognition, 156:110846, 2024.
Alex Krizhevsky, GeofTrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Gustaf Kylberg. The Kylberg Texture Dataset v. 1.0. Centre for Image Analysis, Swedish University of Agricultural Sciences and Uppsala University, External report (Blue series) No. 35., 2011.
Kimin Lee, Kibok Lec, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems (NeurIPS), volume 31. Curran Associates, Inc., 2018.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In The Sizth International Conference on Learning Representations (ICLR), 2018.
Randolph Linderman, Jingyang Zhang, Nathan Inkawhich, Hai Li, and Yiran Chen. Fine-grain inference on distr--distribuutinn data with hierarchical classification. In Proceedings of The 2nd Conference on Lifelong Learning Agents (CoLLAs); volume 232, pp. 162-183. PMLR, 2023.
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Advonces in Newral Information Processing Systems (NewrIPS), volume 33, pp. 21464 21475. Curran Associates, Inc., 2020.
Albert Y Lo. On a class of Bayesian nonparametric estimates: T. Density estimates. The Annals of Statistics, 12(1):331-933 1984.
Karl Löwner. Über monotone Matrixfunktionen. Mathematische Zeitschrift, 38(1):177 216, 1934. ISSN 1432-1823. doi: 10.1007/BF0117000
Steven N MacEachern. Estimating normal means with conjugate style Dirichlet process prior. Communi- cations in "tatistissSSimalation and Computation, (3(3):7277741, 1994.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via. prior networks. In Advances in Newral Information Processing Systems (NeurIPS), volume 31. Curran Associates, Inc., 2018.
Andrey Malinin and Mark Gales. Reverse KL-divergence training of prior networks: Improved uncertainty and adversarial robustness. Advances in Neural Information Processing Systems (NeurIPS), volume 32. Curran Associates, Inc., 2019.
Tom Minka. Beyond Newton's method, April 2000.
Radford M Neal. Markov chain sampling methods for Dirichlet process mixture models. Journal of Compu- lational and Graphical Statistics, 9(2):249 2252499-65,66 2000.
13

---

Published in Transactions on Machine Learning Research (02/2026)
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Woj- ciech Galuba, Russell Howes, Po-Ye Huang, Shang- Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnacve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bo- janowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Leara- ing Research (TMLR), 2024. Featured Certification.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advonces in Newral Information Processing Systems (NeurIPS), volume 32, pp. 8024 8035. Curran Associates, Inc., 2019.
Sudarshan Regmi, Bibek Panthi, Yifci Ming, Prashnna K Gyawali, Danail Stoyanov, and Binod Bhattarai. Reweightood: Loss reweighting for Ristance-based ood detection. In Proceedings of the IEEE/CV E/V//VV Con- ference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 131 141, June 2024.
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribation detection. In Advances in Neural Infor Information Processing Systems (NeurIPS), volume 32. Curran Associates, Inc., 2019.
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan. A simple fix to Mahalanobis distance for improving near-OOD detection. In International Conference on Machine Learning Workshops, Uncertainty & Robustness in Deep Learning, 2021.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision (IJCV), 115(3):211 252, 2015.
Walter J Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance Boult. Toward open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 35()):1757-1772, 2012.
Jayaram Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinicn, 4(2):639 650, 1994.
Matthew S. Shotwell and Elizabeth H. Slate. Bayesian outlier detection with Dirichlet process mixtures. Bayesian Analysis, 6(4):665 690, 2011.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical werwsrks for few-shot learning. In Advances in Newral Information Processing Systems (NeurLPS), volume 30. Curran Associates, Inc., 2017.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-listribution detection with rectified activations. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (cds.), Advances in Newral Information Processing Systems, volume 34, pp. 144 157. Curran Associates, Inc., 2021.
Yiyou Sun, Yifci Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribatiio detection with deop nearest neigh- bors. In Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162, pp. 20827-20840. PMLR, 2022.
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. CSI: Novelty detection via contrastive learning on distributionally shifted instances. In Advances in Newral Information Processing Systems (NewrIPS), Curran Associates, Inc., 2020.
volume 33, pp. 11839-11888
TorchVision maintainers and contributors. Torchvision: Pytorch's computer vision library, 2016.
Hugo Touvron, Matthicu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In Proceedings of the 38th International Conference on Machine Learning (ICML), pp. 10347- PMT.R, 2021.
14

---

Published in Transactions on Machine Learning Research (02/2026)
Gido M Van de Ven, Tinne Tuytelaars, and Andreas S Tolias. Three types of incremental learning. Nature Machine Intelligence, 4(12): 11111185-19977 2022.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The iNaturalist species classification and detection dataset. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition (CVPR), pp. 8778, 2018.
Jagannadan Varadarajan, Ramanathan Subramanian, Narendra Ahuja, Pierre Moulin, and Jean- Marc Odobez. Active online anomaly detection using Dirichlet process mixture model and Gaussian process classification. In IEEE/CVF Winler Conference on Applicalions of Compuler Vision, (WACV), pp. 615- 623, 2017.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7492-7501, 2022a.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recoguition: A good closed-set classifier is all you need. In The Tenth International Conference on Learning Representations (ICLR), 2022b.
Roman Vershynin. High- nigh-dimensional probability: An introduction with applications in data science, vol- ume 47. Cambridge university press, 2018
Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. VIM: Out-of-distribution with virtual-logit matching. In Proceedings of the IEEE/CVF Conference Computer Vision and Pottern Recognition (CVPR), pp. 4911-4920, 2022.
Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. In Proceedings of the 39th International Conference Machine Learning (ICML), pp. 23631 23644. PMLR, 2022.
Kai Xu, Rongyu Chen, Gianni Franchi, and Angela Yao. Scaling for training time and post-hoc out-of- distribution detection enhancement. In The Twelfth International Conference on Learning Representations, 2024.
Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, WenXuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, Xueleng Du, Kaiyang Zhou, Wayne Zhang, Dan Hendrycks, Yixuan Li, and Ziwei Liu. OpenOOD: Benchmarking generalized out-of-distribution detection. In Advances in Newral Information Processing Systems (NeurIPS) Dalasets and Benchmarks Track. Curran Associates, Inc., 2022.
Jingyang Zhang, Nathan Inkawhich, Randolph Linderman, Yiran Chen, and Hai Li. Mixture outlier exposure: Towards out-of-distribution detection in fine-grained environments. In IEEE/CVF Winter Conference on Applications of Computer Vision, (WACV), pp. 5520 5529. IEEE, 2023.
Jingyang Zhang, Jingkang Yang, Pengyun Wang, Haoqi Wang, Yueqian Lin, Haoran Zhang, Yiyou Sun, Xuefeng Du, Yixuan Li, Ziwei Liu, Yiran Chen, and Hai Li. Openood v1.5: Enhanced benchmark for out- of-distribution detection. Journal of Dala-centrie Machine Learning Research (DMLR), 2, 2024. Datasel Certification.
15

---

Published in Transactions on Machine Learning Research (02/2026)
A Preprocessing
Preprocessor
Deep Network f(::6)
Rotatation into I's eigenbasis
Feature
Image
Whiten
Figure 5: W&R preprocessing flowchart.
Before computing OOD scores, we first preprocessed the embeddings using PCA whiten and sort. the dimensions in order of decreasing variance. We discarded dimensions with near variance to ensure the empirical covariance matrices were full rank. We scaled each dimension by the inverse square root of the eigenvalues so that the transformed embeddings had identity covariance. Finally, we rotated the embeddings using the eigenvectors of the average covariance matrix, so that the average within-class covariance matrix is diagonal.
More precisely, the preprocessing steps are as follows:
1. Let {x:);- denote the mean-centered embeddings.
Let Eo = U T denote the covariance of the centered embeddings and its eigendecomposition. Discard any dimensions with cigenvalues less than a threshold of approximately 10-7, and then we project and scale the embeddings by,
- A on
(15)
so that the empirical covariance of { is the identity matrix.
Compute the average within-class covariance E + DM (at ) - Byy where üu
N₂ E4
1
4. Compute the eigendeccoposition S VSVT, with eigenvalues = diag(of..... .., o sorted in in- creasing order of magnitude that the first dimension has the smallest within-class covariance. The embeddings z have unit variance in all dimensions, but along the dimension of the first cigenvector in V, the average within-class covariance is smallest.
5. Project the embeddings into this cigenbasis,
V T/
(16)
After these preprocessing steps, the resulting embeddings { are zero mean (Āo = 0), their empirical co- variance is the identity (Eo I), and the average within-class covariance is diagonal (E diag(o?, ... o))). The empirical within-class covariance matrix Σg for class will not generally be diagonal, but this sequence of preprocessing steps is intended to make them closer to diagonal on average.
Note that the relative Mahalanobis distance score is invariant to these linear transformations. They simply render the embeddings more amenable to our hierarchical models with diagonal covariance. We further test the effect of these preprocessing steps via the ablation experiment described Appendix J.
Further Details of Exploratory Analyses
First, we investigated the degree to which the sample covariance matrices differ between the 1000 classes the Imagenet- 1K dataset. In accordance with the OpenOOD benchmark, we used embeddings from the ViT- B-16 model trained according to the DelT method (Touvron et al., 2021), which are D 768 dimensional.
16

---

Published in Transactions on Machine Learning Research (02/2026)
The embeddings are available in the Pytorch torchvision (TorchVision maintainers and contributors, 2016)
package. We preprocessed the embeddings as described in Appendix A. For this analysis, we only kept the top 128 PCs to speed computation.
To measure the distance between covariance matrices for two clusters, we used the Cörstner-Moonel (FM) metric (Förstner Moonen, 2003),
Σ (log d (E) EET)"=))) 1
d(E,, En) =
(17)
where E2) is the i-th eigenvalue of ΣT'Eg Σz. We computed the FM metric for all pair of empirical covariance matrices (Ex. Σgg). We compared the distribution of FM distances under the real data to distances between covariance matrices sampled from the mull model, in which Σg truly equals E. for all k, and the differences in the estimates Σn arise solely from sampling error. The corresponding null distribution is a Wishart distribution, En W(N, Σ/N), where N is the average nunber of data points per class.
C Compute Resources
The OOD experiments were performed on a cluster consisting of compute nodes with 8 NVIDIA RTX A5000 GPUs. The OpenOOD (Yang et al., 2022; Zhang et al., 2024) experiments on the Imagenet- 1K dataset (Rus- sakovsky al., 2015) utilized weights available from Pytorch's (Paszke et al., 2019) torchvision (TorchVision maintainers and contributors, 2016) models. The compute across experiments was reduced by storing sum- mary statistics of the embeddings for the Gaussian models. The DPMM fitting and prediction was performed the CPU except for caleulating the posterior and prior predictive distributions for each sample.
D Computational Complexity
The complexity of computing outlier scores is not that much greater than computing the relative Mahalanobis distance score (RMDS). Our preprocessing procodure (Appendix A) involves whitening and rotating the embeddings, which requires O(D2) memory and O(DS) time, just like RMDS.
In order account for different covariances across classes, we incur additional costs. The first costs are those of estimating the hyperparameters via poppettationmmxxiiizzttii (EM). In the EM algorithm, need to store expected covariance matrices (or rather, their inverses) for each class. This costs O(KD²) space for the full covariance model and O(KD) space for the (coupled) diagonal model. Each M-step of the EM algorithm also involves inverting covariance matrices, so for the full covariance model the time complexity is O(TKD*), and for the diagonal and coupled-diagonae covariance model it is only O(TKD), where is the number of EM iterations. Typically, we find that = 10 iterations is enough converge.
Finally, there is the cost of computing the final outlier scores, which involves computing Mahalanobis dis- tances for RMDS and marginal likelihood ratio the DPMM. For both RMDS and full covariance DPMM scores, these can be computed in O(ND²) time using the Cholesky factors of the covariance matrices from the preprocessing or EM algorithm; for the (coupled) diagonal covariance model, this cost is only O(ND).
In sum, the additional expressivity of the Bayesian nonparametric mixture models incurs some additional cost. For the full covariance model, these costs can become burdensome (especially in memory), but for the diagonal and coupled diagonal model they are quite reasonable and, as we show in our experiments, they can lend significant. boost.
E Proof of Proposition 4.2
Using the definitions, we can write d exactly as a difference of log Gaussian densities:
\ log N(r p/. Eg + +
[oggN(>> phe Ex E) log.N(x no. Eo + +
N(x Åo: En + Σ)
17

---

where d = log Σ -log ESol- For clarity, define the ideal log-ratio using true cluster parameters as Published in Transactions on Machine Learning Research (02/2026)
Similarly, the RMDS for cluster can be written log-density ratio between cluster Gaussian and the "background" Gaussian model:
RMD 2 log M( Puk Σ) N(x Po. En)
d,
N(x Pk: Σ) N(x Ho. Ee)
px(x) = log
Notice that 1/ (RMD
(1)) in place of (nR, Σ), and px (x())) uses the true cluster We now can use the triangle inequality and decompose this into four separate terms:
log Ni= Ar Σ)
=: (x) is the same ratio but with empirical estimates (fis, E)
Nia Ao. Eoo)
and covariance. We wish to bound 2/ pxl-
2/1x- Pul - log Eo + ES - -log /201|
(A)
+ (x PTo)T [1E0 + E -3-1-10011 (x mo)
(B)
+ log 2لا + Σン - log
-log 1
(C)
+ ( fax) (z= - Pxx) - (x- (12=)))7(20 (Ex + -1)-1(x m22
(D)
To bound A, we first note that
log Eo + E log
Eoo log 7++ 1 = llog| 7 $.j 1/"5//0000-1111
1/21
log +++ R
using the fact that the determinant behaves multiplicatively with respect to the matrix product. Then given that the determinant the product of the cigenvalues we finish our bound on A
D
A S
Σ
log(1 (1 + ( - Doog(1 + x) I Dic
i=1
thus this term is less than dn when <
41 D
For term B, recall that
(A B) A 1 BBA B) B)
(which can be verified by right multiplying by A + B) combined with the fact that
12/2 = و, 1/1 + - $2/2 EE + - 0
(1 -
1/1/2 - 20
1/1/25(1
ES 1/121110 1/2(111111 + /2
R(1 + R7
Means that can say
SS0 + 1)
K S
1+
18

---

Published in Transactions on Machine Learning Research (02/2026)
By the standard fact that for B positive definite, A A 3 al if and only if B 3 a.. So combining all of these identities we have
K (x Po) - (x (4a) 1
B
Which then bound with the standard Hanson Wright inequality Vershynin (2018) to get that with probability 1 -1
1
2 log B RD / Dlog
61
E1
Which is less than 82 when < K2
/ + 22 =
82
Now to deal with term C, we again note the multiplicativity of the determinant with respect to matrix products and see that
log | + Σ log أ = log + S
But recall that Σg -
(5 -NN--11-- which immodiately gives a useful result when combined with the fact
that 5- = -
5-1122 12
log + 1 Ex = =
1ogl/+(NV+ + 2/ 122
1og15//220 2 + (NE 1/2
+ +00 1/1
1/2 RED R --1
20
log 15//////200000000 + <D (NT + R) - 1/2
$1/2
log 10g / (NI + R) 11
lo + (NT R)-11
Now we are functionally done, since we can bound this easily using known bounds the logarithm (easy to check by concavity) to say
D log I + (NI 11-
Σ
log
1 N
1=1
D
S Dlog
1
So when 2 N1 > 6g then C 83-
Finally, we turn to D. We note that we can re-write it as
(x - ple ph - (a) - u'e + phe (ax) (x - ( + 5)- - p%)
which decomposes into three parts. Trying to ease notation, let v = I - ugg and u = 1// - fax. So then is just.
T EE - T 1 + 2n E プーー
(41)
(ais)
For the first piece, we again use the identity
(A - 13
to get that (i) is equal to
1 T -1
19

---

Published in Transactions on Machine Learning Research (02/2026)
And recall the Loewner-Heing inequality Löwner (1934); Carlen (2010) tells us that when A B > A, then (A + B)-1 x A-1 Then note that we can show with not too much difficulty (using the fact that since A and B are positive definite then B B- < = A B) that this implies
ΣJ ( + Σ Σー
But since Eg 3 E we get that (i) is bounded by
T N
now here we again use Hanson Wright to say with probability at least. - <2. as long as N > N2 5a oz (D+ DDog + 21og + it bounded. For term (Mi), we have rapid convergence of 1 to eü and note that D-1 is bounded above by $ 11pl While the difference between the means is bounded by (2] (En)) if N 2 N3 = T5 appiinn (Eo)): Amn))
then have convergence. Finally, we can just use Sauwhy-Sehwarz lo argue that N is set as above, then the bounds for both of the above hold for the cross term. Combining all terms together, we have that x < min(k1, K2) and N > max(N1, N2,, Ns), then the difference between A and Pk less than = 61 - 82 + 83 + 84 + d5 with probability e as long as G + 62 S.
Finally,
((x) = log Σ exp()x(x)).
The map (A1,-..,, + log |Ex exe) is 1-Lipschitz continuous the norm in the sense that if [Ag. - Xe1 S 8 for all k then [xp LogSumExp(XXH uniformly in k, then ((x) is within 5 of LogSumExp RM)x d). Noting that
S c. Hence if Au (z) is within 8 of 2 (R) d]
LoggSumExp(As (Aa max((C(=))] < LogSumExp(AA) [
+ LogSumExp(Ca(x)) max Ck(x))
s 6 log K
Thus conclude
[(x) 1/3 C(x) d]| S 6 + log
with probability at least 1 - e. This completes the proof.
20

---

Published in Transactions on Machine Learning Research (02/2026)
F EM Algorithm for the Full Covariance Model
Here we describe an expectation-mxxiiiatioo (EM) algorithm for estimating the hyperparameters of the hierarchical covariance model. Recall that under this model,
Eg [W (vo D 1)Eo)
Ak ~ N(Ho, K0 Ex)
that the prior hyperparameter Eg specifics the mean of the per-class covariances, E[Ex] En- This hyperparameter should not be confused with Eo defined in the main text, which denoted the empirical estimate of the marginal covariance. Also note that this prior formulation requires > D +1 1.
First, we set the hyperparameters bg and Eg using empirical Bayes estimates,
1
110 p00
Tn
(18)
N
1
Eg E
Σ
(In
Py (Dn - Pyn )
(19)
N
R=1
where Pun No
1 E Em In.
To set the remaining hyperparameters, Ro and Vo, use EM. The expected log likelihood is separable over these two hyperparameters,
((vo,Ko) = C(vo) + L(Ko)
[(vo) E
Σ
log IW Vo. (vo D - 1)2u)
((K0) E
log N(us pos Ko Ex)
where the expectations taken with respect to the posterior distribution over {pu, En KK 1==1
F.1
M-step for Vo
Expanding the first objective yields,
K
(up D - 1) vo
Mo Trr D++
Eo 10-1
C(vo)
E
log
[
2
2 2 TD( W
b=1
P log
20 log Σgl
log (2)))
D+1
EE. log [x
2o D Tr<E
k=1
K
1 log
Mo D-11
+ [log Eoo Eloog [+]] - Tr |>] - log (#0).
No 2
We can maximize this objective using generalized Newton's method (Minka, 2000). We need the first and second derivatives of the objective,
K
C'(vo) =
Σ
log
(a-DD11 D-1 +
Mo D-1
+ [log Eu
Elog Σx[] Tr TrE 'D] - (
2
2
km1
("(v)) =
D
D+1
7 (
2
D-I
D-1
1=1
21

---

Published in Transactions on Machine Learning Research (02/2026)
The idea is to lower bound the objective with a concave function of the form,
g(vo) k + log vo buo
which has derivatives g' (v6) a on
and (V(o) = -// Matching derivatives implies,
= - (vo)
b = (voo
k [(vo) a aloo Vo bvn-
For and < O, the maximizer of the lower bound obtained at
(vo) v/ - b ('(vo) + (vD)
(20)
F.2 M-step for Ko
Expanding the second objective,
K
((Ko)=
log Ko
1 [(Ux - nO) T (#x nol] c.
2
The maximum obtained at,
K
(Ak - mo) TEx (us - No)l E
KD
F.3 Computing the posterior expectations
Under the conjugate prior, those posteriors are normal inverse Wishart distributions,
24 | {In Un = k} ~ NIW(1/s M%, mek mk)
1/2 1/ + N₁.
1/2 No + Ne
14 1
Σ In
KoJl0 +
Kik
1i: =k
Σg (vo
D 1) +
Kopooo
Σ
To evaluate the objectives above, we need the following expected sufficient statistics of the normal inverse Wishart distribution,
EMS '] = 11.2%/-1
Ellog Ex = log |Eye - 1/ 1/ D Dlog 2
E(( po) T Ex (Mu - pn)] Rig
+ (rk mo) "EE - 14)
Since the tied covariance approach in RMDS already works quite well, we recommend initializing the EM iterations by setting V/ Ng and Ky 0. That way, the covariances are strongly coupled across clusters and the means have uninformative prior.
22

---

Published in Transactions on Machine Learning Research (02/2026)
F.4 Marginal Likelihood
This EM algorithm maximizes the marginal likelihood,
K log (rn : yn = =k3)
log (( Un
K
Σ log / (((x Ma k3 pa, Ex) p(un, Ex) duu dEn
Jmm
K
log
II mi k
N(an Mas Ex) NIW(RR En vo, Eo, No. No) du dE
k=1
K
log Z(14, 2%, R 146) log Z(vo, (vo - - 1)Vo,,
No. No. +
k=
where
log E,, = log log ( + DD 2 log 2 log |
is the log normalizer of the normal inverse Wishart distribution, and is constant with respect the hyperparameters being optimized (but it data dependent).
G EM Algorithm for the Diagonal Covariance Model
Here we describe an expectation-maxiiizaiin (EM) algorithm for estimating the hyperparameters of the hierarchical diagonal covariance model. Recall that under this model,
Fn,d yn k~ N(uk,a, (( o..))
where
of x (vo,d, σG.a)
Ptdd N(Jo,d: R.d)
for each dimension I.....D independently. Under the prior Elo oba for large degrees of freedom Vu,d-
Vo.4
which is approximately
First, we the hyperparameters Ma,d and oia using empirical Bayes estimates,
N
1
N0.4 Po.
Fn,d
(21)
R=1
N
0
02
(In,d
() d)
(22)
N
n=l
where Py... = N Envne Fn.d.
To set the remaining hyperparameters, Ko,d and Po,d, we use EM, The expected log likelihood is separable over these two hyperparameters,
C(Po.a: Ko.a) = C(vo,)) + L(Kg.a)
K
((vo.a) = E
log ?oh vo oo.a)
((so,a) E
log N po,di No ooad
where the expectations taken with respect to the posterior distribution over (ukaaorka
23

---

Published in Transactions on Machine Learning Research (02/2026)
G.1 M-step for Vo,d
Expanding the first objective yields,
Ho.d Vo.d 0 2 T( ((2d))
K
[(vo,d) =
log
(o
20,4
e
1=
K
Vo.d log (222)) + Ho.d log of. - log I( (
aa.a+2
Ellog of
2odd 02 Blo k,d
2
km 1
Ha.d log
Ma.d
Hold log o.. Ellog o. 20 (o log [ Madd
2
2
We can maximize this objective using generalized Newton's method (Minka, 2000). We need the first and second derivatives of the objective,
K
('(v,,))
log (M2d) + 1] log oō.a - Ellog o. - o B 1 (0 (2a)
=
R= 1
L (vo,a)
t
yadd
200,
The idea is to lower bound the objective with a concave function of the form,
g(vo.d) = k + a alg Vo,d + buodd
which has derivatives g' (Vo,a) 2o.a
b and (vVvo -- 3. Matching derivatives implies,
v8 (vo,a)
C'vvo.)) Mo.a
k [(vo,a) log Vo.a bug
For a and < O, the maximizer of the lower bound obtained at
Vo, "2 ('(vo.)) + Vo,aC"(Mood
(23)
G.2 M-step for Ko.d
Expanding the second objective,
(He,d 2o,d)2
((Ko.a)
log Ko,d
No.d E
c.
2
oh.
The maximum obtained at,
K
(Wked - Ho.a)2 1)
-1
K
oha
k=
24

---

Published in Transactions on Machine Learning Research (02/2026)
G.3 Computing the posterior expectations
Under the conjugate prior, those posteriors are normal inverse chi-squared distributions,
Mk,d; k.d f Un = k) NIX kids Ki a))
U/2. Vu,a + Nss
K/g, = Ku,a + Ns
Medd = mkdd
Kofload +
Σ In
niga =k
Vo,ao + ko, +
and-
ni mk
To evaluate the objectives above, we need the following expected sufficient statistics of the normal inverse chi-squared distribution,
Elo k.d
0'-²2
E Eloog o.ul = log oho
1/( Kad (Wxd Mo,a) 2
(pidd M0,a)²
1
E
+
k,d
Kidd
Since the tied covariance approach in RMDS already works quite well, recommend initializing the EM iterations by setting Vo,d N₂ and Ko,a 0. That way, the covariances strongly coupled across clusters and the means have an uninformative prior.
G.4 Marginal Likelihood
This EM algorithm maximizes the marginal likelihood,
K log ( = =
logp((In, Un 1 =
log
pl p((xndd : Un k3 pedd, oi pluu.do dumed dok
d=1 k=1
log
N(xnn Me,d; ord) nynn=
II
NTX oh vo,ds l.. Ko,ds (a,a) disk do? k.a
Σ
(log Z ((k R.di ds He log (VV,dd a. Ko,ds Mo,d)) +
where
2 k,, - log x + log 1(5) S/B log uga log Z(v, 22
is the log normalizer of the normal inverse chi-squared distribution, and log((27) respect to the hyperparameters being optimized.
-N/2 N/2 is constant with
25

---

Published in Transactions on Machine Learning Research (02/2026)
G.5 Predictive Distributions
Under this model, the predictive distribution of data point given its cluster assigument is,
D
/ plaal uaads R plun,as ond Xtr, Ytr) du klok
pl y = k, Xtr: vtr)
II
d=1
D
/ N(xa Pk.d: R.))) NLX(HK,d ond Midd kids Rk Hk duk,d do
II
d=1
D
II
St(ra a 1k
Eeatl Red b.d))
d=1
where (1r | U, denotes the univariate Student's t distribution with degrees of freedom, location and scale a.
Under this model, the prior predictive distribution is,
p(r y K X,r, Mtr) II
St(ra
Vo,d. Ho,ds No.t Ka.a
oo.)):
d=1
which is approximately Gaussian, N(xa Mo aa,)) when Vo.d: Ko,d > 1.
H EM Algorithm for the Coupled Diagonal Covariance Model
This model introduces scale factor E R4 that is shared by all dimensions. The model is,
7 x (0o)
k,d ~ X (Vo,d, T6.a,)) Mkdd N(Jo, oo k.d)
for 1....D for d 1....D D
Since E/x]] 1, under the prior Elo 2o.d 20 2 which is approxinately ob.a for large Vo.d-
The hyperparameters of the model are 17 (av, (Vo,d. of No.d: (o.a)). We set the hyperparameters Po,d and of using empirical Bayes estimates,
N
1
P0.d Paodd
Fn,d
(24)
N
N
1
T 02
(on,d d)
(25)
N
where tA 1
No Eng. In and N₂ = E. My, k).
To set the remaining hyperparameters, use EM.
H.1 E-step
Note that the posterior distribution factors as, 12
p(ikk (pk,d, de
Xx) = p(7k
1D Xx) ph)p((pkad o nk: Xx)
= =p(n Xx) Il
p(u of o.d Nx).
d=1
26

---

Published in Transactions on Machine Learning Research (02/2026)
The posterior distribution over " doesn't have simple closed form, but since it's only one-dimensional, we can approximate it on dense grid of points, {7(0) Conditioned and of is normal inverse chi-squared. For each point,
7 = =((p), the distribution of paa,
p(ua,d R.d 7h = 710), (((P),n)) Xx) - NIX(UXx,, Rid V/ V 12 kp (k.d:Hk)))
undd = Vo,d + NE
Kk,d = Ko,d + NE
1
Σ In
KoMOod +
kidd
12gg/ =k
12 k.p.d
1
vo
0b.
ko +
Σ n:y==
Ind - a
Vkkd
Note that this is practically the same as above, but with 7h scaling the prior for of For any value of 7e the posterior probability is,
p(76 = 7(0) Xx)) x p(7n = 760) p(Xx
7h =0(1))
D
= p(akk = ,0)
II
p(Xx,d g(1))
d=1
D
Z(1k K.p.a Kk Hk.a)
p(7 (1))
II
Z (Vo,a 7 (1)) Ko,ds (o.a)
P Wh
where we reused the marginal likelihood calculation from the hicrarchical diagaonal DPMM above. Finally, denote the normalized posterior probabilities as,
wh k.p Σ, 1
wh.p
H.2 M-Step
To set the hyperparameters, Ko,d; Mo,ds and og, we use EM. The expected log likelihood separable over these two hyperparameters,
L(vo,d, Ki,dd on)
L(vo,a) + L(so,a) + L(ao)
((vo.a)
log 20h Mo,a: mmo
K
[(Ko,d) = E
log N(ued Po.d. ko.d
A=1
C(ao)
EE [log Ga(?x. ap)]
where the expectations taken with respect to the posterior distribution over (7k, (Mn,d, d
1K
27

---

Published in Transactions on Machine Learning Research (02/2026)
H.3 M-step for Po,d
Expanding the first objective yields,
((vo,))
E log
( 7.
1 2b.d
Pa.a
2
(o
a 2
6
a0 20
k=1
vo.d log
2 + Vo.a TE [lg +
log ng log [ (1)
vo.u Elloo o.
2 " a o
k=1
Kodd log
Ma.d +
Eloo 74] + log of - Ellog ollogok.d R - B(m log [ ((=22) + c.
2
2
We can maximize this objective using generalized Newton's method (Minka, 2000). We need the first and second derivatives of the objective,
K
('Vvvd)) Σ
log
Mo.d 2
1] + Elog 7a) + log ob. BBloog a - Blxo
1 20 20,4
2
k=1
L"
(
The idea is to lower bound the objective with a concave function of the form,
g(vo,a) = k + a aog Vo,d + buood
which has derivatives g' (Vo.a) = 2o.a
b and g"(vo,a) 2.
Matching derivatives implies, vo."
- (vo.a)
b L'Vvo,)) No.
k = C(vo.a) log Vo.d buo,a.
For and b < O, the maximizer of the lower bound is obtained at
Vo.d a (2.aC"(vo.d)
(26)
('(vod)) + Vo,aL" (Vo,a)
H.4 M-step for Ro.d Expanding the second objective,
K
(Hidd - Ho,a)
((s0,a)
log Ko.d
Ko.d E
+ c.
2
2
o kid
The maximum is obtained at,
K
(mked mooaa) ))
Ki,d
K
oRd
k
H.5 M-step for ag
Expanding the final objective,
L(ao)
ag log ag log I'(()) + agl [log yu]
aoE [Tx]] +
1=1
28

---

Published in Transactions on Machine Learning Research (02/2026)
Its derivatives are,
K
('(o)) = Klog og + Kib(oo) +
> E
log 7k
k=1
K KV((oo)
(oo)
0g
We can optimize og using the generalized Newton's method described in the M-step for vo-
H.6 Computing the posterior expectations
To evaluate the objectives above, we need the following expected sufficient statistics of the normal inverse chi-squared distribution,
E[rue]
Σ (
P
E Eloog nxl = Σ
u//pp logn (p)
0 = Eny [og dhe ln = Σ (
1-2 k.p.d
Ellog Rhal
Eng [B0g [log hall Σ
wkkpllogg
" ai
P
(uk.d 2 o
(Hind m),
(Pbd - mu,a) o
Buyg Bunaroiel
WKe Σ P
+
Rnd
Rik.d
k.p.a
where y(p) are the centers of discretized posterior 7 and Wk.p are the corresponding weights.
Since the tied covariance approach in RMDS already works quite well, we recommend initializing the EM iterations by setting Vo,d 2 N₂ and Ko.d s 0. That way, the covariances are strongly coupled across clusters and the means have an uninformative prior.
H.7 Computing the predictive distributions
The prior predictive is,
p(a" ;
D
II
N( Ha. N2) of Vu,a, Ko,d, Po,d; yo {Zn, yn N= dua do Galy au, ax) dry
where No = oo, (Mo,a; oz. Ko,d, ) are the model hyperparamters.
The y integral can be estimated by numerical integration over dense grid of points,
p
D
p(=":T0) ~
w0.
II
N( pa, a N o3 Do.d, No,d; po.d, (p) ob. {I W dua do
Ld=1
P
Vo,d Ho,ds II (t
Ko.d (0) (0))) aa.a)
wo.p
Ka,a
p=1
d=1
where,
Ga(7(P)
Ogg Orl) (n)) (p)
Wo.p = Σ, Ga(z(1)
ao (0)a0)))) (r(r)
29

---

Published in Transactions on Machine Learning Research (02/2026)
We renormalize the weights to ensure that the numerical integration satisfies that E,[1] 1. In practice, we evaluate the prior log predictive probability using a log-sum-exp,
logp(x"; no) ~ logsumexp, log Wo.p +
Σ d=1
Mo.ds Ho,d, Ka.at 7 (00 oo,a) log St
By the same logic, the posterior log predictive is, log p(>" y² = ki (Ini Un 1. no) s logsumexpp
log wk +
Σ d=
log St(aa
1 Me, Keatt (oott o Roppaa) Khin
H.8
Marginal Likelihood
This EM algorithm maximizes the marginal likelihood,
K
log (1 Vn 1.
log p( {zn Un = k})
log
p((Tn Uh kf I {ux,a, okal) pl pl mw p(TR) dune do drk
II p(2n,d pk.d: ok.a)p(uk obd 7h) duped io ] ni
Σ log
p()x) dry
d=
Z Z k. " Hedd
Σ log II
p((n)) drk +
Z(vo,a Tho Ko,ds Ho,a)
d=1
D
Z(% h.p.o Ke Medd) +
log
Wo.p II
7 (vo,d, (()) o No,d, (o,a)
d=1
logsumexp, [l +
where
tn A log Wor
Σ
log Z(/ o2 hepaa Mlea)) log Z(Vo,d, (00) 0z.a: Ko,ds Ho.a)
d=1
and c
N.D. 2
log 2x.
30

---

Published in Transactions on Machine Learning Research (02/2026)
Score Correlation between RMDS and the Tied Covariance Model
We observe that the DPMM model with shared covariance and the RMDS Ren et al. (2021) are highly correlated, as illustrated in Figure 6. Here, we plot the RMDS vs the Tied Covariance Gaussian DPMM for all the real datasets (differentiated by color) in the Imagenet-1K OpenOOD task and note the tight agreement between the two. This empirical result supports the theoretical relationship derived in Proposition 4.2.
500
-500
-1000
-1500
-2000
1000
-750
-500
-25
0
250
500
Č
Figure 6: Tied DPMM OOD score, č, correlation to RMDS Ren et al. (2021) score, C, the Imagenet- 1K dataset. The colors represent different or OOD datasets.
31

---

Published in Transactions on Machine Learning Research (02/2026)
J Imagenet-1K Experiment
Table 2: OpenOOD performance different preprocessing methods for expectation maximization trained hier- archical DPMM models. Thee preprocessing methods are the raw VII features (VTT), marginal covariance whitening followed by rotation the average class-covariance eigenspace (W&R), and PCA. Baselines: Mahalanobis dis- tance score to the closely related Mahalanobis distance score methods, MDS (Lee al., 2018) and RMDS (Ren et al., 2021). We also compare maximum softmax probability (MSP) (Hendrycks scaled MSP with 7 1000 (Temp. Scale) (Guo et al., 2017) which is the ODIN (Liang et al., 2018) method without input preprocessing. the MSP and ODIN methods. Accuracy values marked with asterisk* are the VIT classification layer's perfor- mance not the method itself. Additional benchmark performance metrics were obtained from the OpenOODv1.5 leaderboard (Zhang al., 2024) including ASH (Djurisic al., 2023) and ReAct (Sun et al., 2021).
Gimpel, 2017) and temperature
single linear layer was trained with gradient descent and supervised cross-entropy loss for
Near
Far
Model
Pre
Accuracy
SSB Hard
NINCO
Avg.
iNaturalist
OpenImage-O
Textures
Avg.
MSP
VII
80.94
73.80
82.72
78.26
92.08
88,67
88.39
89.71
W&R
80.90
71.74
79.87
75.80
88.65
85.62
84.64
86.30
PCA
80.91
73.67
82.94
78.31
92.08
88.60
88.34
89.67
Temp.
VIT
80.94
75.60
84.36
79.98
94.22
90.82
90.82
91.95
MSP
W&R
80.90
73.29
81.28
77.29
91.21
87.82
86.81
88.62
T=1000
PCA
80.91
75.24
84.77
80.00
94.25
90.78
90.75
91.93
MDS
VIT
80.22
71.45
86.44
78.94
95.96
92.33
89.37
92.55
W&R
80.41
71.45
86.48
78.97
96.00
92.34
89.38
92.57
PCA
80.41
71.45
86.48
78.97
96.00
92.34
89.38
92.57
RMDS
VIT
80.22
72.78
87.18
79.98
96.00
92.23
89.28
92.50
W&R
80.41
72.79
87.28
80.03
96.09
92.29
89.38
92.59
PCA
80.41
72.79
87.28
80.03
96.09
92.29
89.38
92.59
ASH
VIT
81.14*
53.21
51.56
ReAct
VIT
81.14*
69.26
85.69
Hierarchical Gaussian DPMMs
Tied
VIT
80.40
71.79
86.75
79.27
95.99
92.40
89.71
92.70
W&R
80.41
71.80
86.76
79.28
96.00
92.40
89.72
92.70
PCA
80.40
71.79
86.75
79.27
96.00
92.40
89.70
92.70
Full
VIT
76.82
62.64
78.32
70.48
85.76
84.95
88.03
86.24
W&R
76.78
62.84
78.48
70.66
85.88
85.03
88.02
86.31
PCA
76.82
62.64
78.33
70.49
85.76
84.95
88.03
86.25
Diag.
VIT
75.96
72.38
85.96
79.17
94.14
90.18
87.20
90.51
W&R
76.54
73.89
87.32
80.60
95.36
90.78
86.42
90.85
PCA
75.76
71.99
85.52
78.75
93.91
90.18
87.39
90.49
Coupled
VIT
75.93
72.80
86.15
79.48
94.08
90.20
87.19
90.49
Diag.
W&R
76.52
74.47
87.48
80.98
95.51
90.63
86.02
90.72
PCA
75.76
72.40
85.97
79.19
95.02
90.92
88.09
91.34
32

---

Published in Transactions on Machine Learning Research (02/2026)
Table 3: Performance of Hierarchical Ganssian DPMM and baseline methods on the OpenOOD benchmark datasets Yang et al. (2022); Zhang al. (2021), including both Near (SSB Hard (Vaze al., 2022b) and NINCO (Bit- terwolf et al., 2023)) and Far (iNaturalist (Van Horn et al., 2018), Openlmage-O O (Wang et al., 2022), and Tex- tures (Kylberg, 2011)) OOD datasets. The first column reports the accuracy of the classifiers predicting the label E [K] for in-distribution test data. Other columns report AUROC scores for OOD detection OpenOOD benchmark datasets.
Near
Far
Method
Accuracy
SSB Hard
NINCO
Avg.
iNaturalist
OpenImage
Textures
Avg.
MSP
80.90
71.74
79.87
75.80
88.65
85.62
84.64
86.30
Temp. Scale
80.90
73.29
81.28
77.29
91.24
87.82
86.81
88.62
MDS
80.41
71.45
86.48
78.97
96.00
92.34
89.38
92.57
RMDS
80.41
72.79
87.28
80.03
96.09
92.29
89.38
92.59
Hierarchical Gaussian DPMMs
Tied
80.41
71.80
86.76
79.28
96.00
92.40
89.72
92.70
Full
76.78
62.84
78.48
70.66
85.88
85.03
88.02
86.31
Diagonal
76.54
73.89
87.32
80.60
95.36
90.78
86.42
90.85
Coupled Diag.
76.52
74.47
87.48
80.98
95.51
90.63
86.02
90.72
33

---

Published in Transactions on Machine Learning Research (02/2026)
K CIFAR-10 Experiment
Table OpenOOD CIFAR 10 performance trained hierarchical DPMM models, 'he preprocessing methods are the raw ResNet18 features and marginal covari- ance whitening followed by a rotation into the average class-covariance eigenspace (W&R). Baselines: Mahalanobis distance score to the closely related Mahalanobis distance score methods, MDS (Lee et al., 2018) and RMDS (Ren et al., 2021). We also compare maximum softanax probability (MSP) (Hendrycks Cimpel, 2017) and temperature scaled MSP with = 1000 (Temp. Scale) (Guo et al., 2017) which is the ODIN (Liang et al., 2018) method without input preprocessing.
different preprocessing methods for expectation maximization
single linear layer was trained with gradient descent and supervised cross-entropy loss for the MSP and ODIN methods. Standard deviation across 3 ResNet18 training runs with different seeds is provided in parentheses and smaller lont. Additional benchmark performance metrics were obtained from the OpenOODv1.5 leaderboard (Zhang al., 2024) including ASH (Djurisic et al., 2023), ReAct (Sun et al., 2021), SCALE (Xu et al., 2024), and Reweight.OD (Regmi et al., 2024). f: Alternative training OOD method, *: Accuracy value inherited from ResNet classifier.
Near
Par
Aocuracy
CIFAR
Tiny Inaggonet
Avg.
Placan365
Toxtures
Avg.
MSP
ResNet18
00.01(0-29;
87.24
88. (1-44)))
88. 08tc as)
92.088c1 12)
89. 89.42:0.1 71)
91.46 083
80. 7017 953
90. 10.831e as)
Waztt
94.93(o.27)
87.38(0.55)
89. 60.38(o.30)
88.36co 45)
94.21g1 04)
88.49:1 200
98.4911.22
91.00ic.53
91,80cc.
Thmp--
R8
90:
a1)
04)
19)
Warft
94.93(0.27
73).033)
db)
94.18;1
93.51(1.95)
83)
91.84(1.30
MTS
18
95-01(0.00
50ro a773
84. (2.)9)
84. 28 ao)
90. 10:110cs.0 a1)
84.90 te)
91.17 ca)
92.60r 05)
80. (1 as)
WAR
84.63/9.11 17)
86.19(2 40)
66.41
91.45c1 18)
86.97( 10)
90.19/o.50)
92.00 07)
RMDS
ResNet18
80
00.01co.20
84.0.35)
83(0-26)
80
93,
01,96
92.
Whstt
83)033)
(0.88(a.38)
89.83co
98.67c0
91.07:0.00
92.25/0.ca)
92.20/0.0
92.458 02.442c0.m
ASII
ResNett8
56.00 00.00*
75.27
40
ReAct
ResNet18
95.06*
87.11
90. 92.42
ResNect8
ReweightO0!
ResNet18
91.56
98.05
Herarchical Gauertan DPMMs
Tied
R8 W&R
65 28:
53t 1093
90. d0)
89.4Tic 45)
98 01.0 20)
90.8300 51)
93.5 393
04.522p 144
03. 19 0n)
95.04 ae)
83 139
000-83(0790 (9)
89-33ie 95)
98.07/0 90
91.57 10)
02:25 bal
201 m6)
022.42pe 30)
Pull
ResNet18
95 25.00ca.25)
89. 350.3000.
91. (3.46)0.07))
90.41cc
B4.71 (0.70)
91.06:0.3
93.36 203
92. 30.339c... 27)
922 88.8888c..
W&R
$4.955 $4.95co
89.89)
91.57 (
90.683(039)
94 944.3220 ov)
01.95
93.37
94.35
9.3.50
(7/0.18)
(3(0-20)
00.000c 211
42)
91.34/o
WAR
94.76(aass
00.27(0-38)
89.14cc 20)
93.50cc 21g)
91.70:o on)
88.07 (..))
ox)
90.96 a4)
Coupled Diag.
ResNer18
D4 D4..8dco 2d:o
89. 1300.15)
50. (0.08((-27))
90.06ic 22)
9/ 82:0 sm)
91.47/o
91.3 91.3110.57)
02. 16to oay
01. 91.94k6 42)
WALER
94.766c. 34..66co
87.17/0.199
89. (0.48(o.20)
88.30lc 23)
91.11;0 07)
90.84;0.8
88.07
92, 02.778c0. 17)
80.700c..o 201
Table 5: Performance of Hierarchical Gaussian DPMM and baseline methods on the OpenOOD CIFAR-10 benchmark.
Near
Far
Model
Accuracy
CIFAR 100
Tiny Imagenet
Avg.
MNIST
Places365
SVHN
Textures
Avg.
MSP
94.93
87.38
89.33
88.36
94.21
88.49
93.49
91.00
91.80
Temp. MSP
94.93
87.42
89.43
88.43
94.18
88.59
93.51
91.08
91.84
MDS
95.04
84.63
86. 19
85.41
91.45
86.97
90.19 19
92.00
90.15
RMDS
95.04
88.83
90.83
89.83
93.67
91.57
92.25
92.20
92.42
Hierarchical Gaussian DPMMs
Tiod
95.04
88.83
90.83
89.83
93.67
91.57
92.25
92.20
92.42
Full
94.95
89.69
91.57
90.63
94.32
91.95
93.37
94.35
93.50
Diag.
94.76
88.01
90.27
89.14
91.50
91.70
88.07
92.16
90.86
Coupled Diag.
94.76
87.17
89.43
88.30
91.11
90.84
88.07
92.78
90.70
34

---

Published in Transactions on Machine Learning Research (02/2026)
L CIFAR-100 Experiment
Table 6: OpenOOD CIFAR 100 performance trained hierarchical DPMM models. 'hee preprocessing methods are the raw ResNet18 features and marginal covari- ance whitening followed by a rotation into the average class-covariance eigenspace (W&R). Baselines: Mahalanobis distance score to the closely related Mahalanobis distance score methods, MDS (Lee et al., 2018) and RMDS (Ren el al., 2021). We also compare maximum softanax probability (MSP) (Hendrycks Cimpel, 2017) and temperature scaled MSP with 7 1000 (Temp. Scale) (Guo et al., 2017) which is the ODIN (Liang et al., 2018) method without input preprocessing.
different preprocessing methods for expectation maximization
supered single linear layer was trained with gradient descent and supervised cross-entropy loss for the MSP and ODIN methods. Additional benchmark performance metrics leaderboard (Zhang et al., 2024) including ASH (Djurisic et al., 2023), ReAct (Sun et al., 2021), SCALE (Xu et al., 2024), and ReweightOOD (Regmi et al., 2024). f: Alternative training OOD method, *; Accuracy value inherited from Res.et classifier.
obtained from the OpenOODv1.5
Near
Far
Model
Poo
Ascurncc
CIFAR 10
Tiny Imaagenee
Avg.
MNNST
Planed66
8VHN
Toxlaune
Avg.
BenNet18
36
80.32,o 15)
75.76/2.813
(2)6)))))
(7.94(o-0n)
W&R
T6.19ro.04)
78.48:0.00)
82.19)0.18)
80.33;c.10)
77.01(2.30)
79.72/0.00
80.60;1.213
78.88(o.a5)
Temp-
RewNet18
76.91:0.30)
79.16
82.25 013
80. 80.71cc
77.044:1000
78 60ce 19)
81. 81.10:1 433
78. (1.0770-777
18 05(0 49)
MSP
W&R
76.19)).04)
78.87
(2.27(0.16)
90,37(c.07)
77.41(1.04)
80.06ro.0g 05)
81.72/0.50)
77. (7.44)
79.25(0-32)
18
ong
30.87(0.22 223
61.84:0.19)
58.86/c.0s
67.47
70.24/6.513
70.28(33.0)
2912
RMDS
R8sNwi18
77.76:0.m0 103
82.58 0o3
80. 80.17(c.00)
79.74:2.40
83.40re 46)
85.10 .003
83. (8.655)..
(7 45)
W&R
76. 10cou00
77.70 77.7000.39)
82. 82.58...02
80.17 (0.0M)
79.74 79.74:2.49
63.40co
85. (0.11011.00
63. (..6650)00)
#2 (7
ASH
ResNot18
77.25*
20
80.08
ReAct
ReaNet18
77.26
80.77
80.39
SCALE
RewNet18
77.20"
60.99
81.42
HeweightOODt
Heraarhhcaa Gauasian DPMMs
Tied
ResNet18
82.56/o.02)
80.11cc.0p
79.82(2.47)
10.38ro.47
85.17:1.13)
83.377c044)
W&R
00
77.75/o
80.17 om)
79.75
63.65(9.89)
Pull
ResNet18
76.79to.23)
76.81(0.15)
(2.97(0.15)
79.89(c.05)
82.10(1.95)
79. 79.16rr.113 13)
81.22:2.54)
82 82.07(0.611
81.11 21.144o.19)
WAR
76 76.64ça.14 143
76.04
82.40 16)
79.22 00
82.10(1.0 (0)
78 82. 10)
81.20:0.19)
82 (2.66
81. (1.20((.a0)
ReaNN1188
75.54(a..e)
81.50/o.a1
77.91c0.02
70.077o.11
81.46
84.06:1.88
63.31(3.04)
Coupled
ReaNet18
76.54(a.05)
75.80
(2.77
79.28rc.24 24)
79.26;1.70)
20.29kco.m ao)
(3.29:1.60)
84.77:0-07)
(1.990(0-43)
Diag.
W&R
78. 78.04(9-90) 983
76. 18c0
79.93 533
78.05ee 14)
75.97(9.00)
79. 80ce
80.58
80. (0.81(
79. 20.20000
Table 7: Performance of the Hierarchical Gaussian DPMM and baseline methods on the OpenOOD CIFAR-100 benchmark.
Near
Far
Model
Accuracy
CIFAR
Tiny Imagenet
Avg.
MNIST
Places365
SVHN
Avg.
MSP
76.19
78.48
82
80.33
77.01
79.72
80.60
78.18
78. 88
Temp. MSP
76.19
78.87
82.27
80.57
77.41
80.06
81.72
77.81
79.25
MDS
76.10
55.87
61.84
58.86
67.47
63.18
70.24
76.26
69.29
RMDS
76.10
77.75
82.58
80.17
79.74
83.40
85.10
83.65
82.97
Hicrarchical Gaussian DPMMs
Tied
76.10
77.75
82.59
80.17
79.75
83.41
85.11
83.65
82.98
Pull
76.64
76.04
82.40
79.22
82.10
78.82
81.20
82.66
81.20
Diag.
76.07
76.30
82, 13
79.22
81.46
81.79
84.98
83.55
82.88
Coupled Ding.
76.18
79.50
60.58
80.81
35

---

Published in Transactions on Machine Learning Research (02/2026)
M
Feature Dimension Study with Independent RMDS
Full
Diagonal
Coupled Diag.
Tied
Independent RMDS
Near OOD
Far OOD
ID Accuracy
0.9
0.9
0.8
0.8
0.6
Accuracy
AUROC
0.7
AUROC
0.7
0.4
0.6
0.6
0.5
0.5
0.2
0.4
0.0
24
2
26
2
28
2
20
24
2
26
2
28
2
2
2
2
2
27
2
2
2
Dimension
Dimension
Dimension
Figure 7: Performance on "near OOD", "far OOD", and in-distribution classification as a function of the feature dimension the Imagenet-1K task. We projected the 768-dimensional VIT-B-16 features into lower dimensions using PCA, then projected into the eigenspace of the average within-class covariance. We compared the tied model (with full covariance) the hierarchical model with full, diagonal, and coupled diagonal covariance and measured performance by area under the receiver operator curve (AUROC).
36


Reviews:
[{"review_id": "Rh4FphE2QO", "reviewer": "Reviewer_xdPP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Theoretically, this paper establishes a bridge between the Relative Mahalanobis Distance Score (RMDS) and the Dirichlet Process Mixture Model (DPMM). Moreover, the authors show that the DPMM score from a Gaussian DPMM with tied covariance may correlate perfectly with the RMDS. In addition, three hierarchical Gaussian DPMMs (Full, Diagonal, and Coupled Diagonal) are proposed, which generalize the RMDS. \n\nFrom the experimental perspective, the authors analyze the results and differences of these Gaussian DPMMs (Full, Diagonal, and Coupled Diagonal) on two-dimensional synthetic data. Furthermore, they conduct experiments on the OpenOOD benchmark. By evaluating the performance of different models on embeddings from CIFAR-10 and CIFAR-100 using ResNet-18, as well as embeddings from ImageNet-1K using ViT-B/16, they draw the conclusion that Bayesian nonparametric methods achieve superior performance when covariance structures differ, or training data is scarce.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors provide theoretical derivations, along with visualizations of the synthesized data and numerical results on CIFAR10/100 and ImageNet-1k.", "audience_yn": "Yes", "audience_explanation": "This paper addresses the problem of out-of-distribution (OOD) detection, which is a popular and rapidly growing topic in the field of machine learning.", "requested_changes": "- I would recommend that the authors include a concrete **algorithm flowchart or table** after presenting the formulas for the three models in the Methods section. This would help to clearly illustrate processes such as PCA and EM, improving readability. Currently, most of the PCA and EM operations are described in the appendix; it is suggested to briefly summarize these procedures in the main text.\n\n- The authors may include an analysis of the **computational complexity** of the baseline method RMDS, and the three Hierarchical Gaussian DPMM methods proposed in this paper, or to report the actual runtime of experiments. This would demonstrate that the Hierarchical Gaussian DPMMs are not significantly more complex than RMDS, thereby supporting their practical applicability.\n\n- In Section 6.1, the paper conducts experiments using two-dimensional **synthetic data**. This raises several questions regarding experiments on real data: Are the results on real data similar to those on synthetic data? Admittedly, reducing high-dimensional complex image data (e.g., CIFAR, ImageNet) to two dimensions results in significant information loss, but what happens with simpler image datasets, such as MNIST? If the results on real data are similar to those on synthetic data, what exactly are those results? Conversely, if the results on real data differ substantially from those on synthetic data, what justifies using synthetic data for experiments? What causes these differences, and how might they affect the conclusions drawn in this paper?\n\n- How does the choice of **feature extractor** (i.e., different foundation models) affect the experimental results? For instance, this paper observes relatively poor performance on ImageNet (using ViT-B/16), which may be due to the high-dimensional embeddings. How would the results change if embeddings of lower dimensionality were obtained using other models, such as ResNet?\n\n- Since the differences in classification results are generally small, including the standard deviation (std) of the results could better reflect the stability of the models, making the findings more convincing—at least in the main figures and tables, such as Table 1.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ht7A1T0Iwu", "reviewer": "Reviewer_xay4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper establishes a theoretical link between the Relative Mahalanobis Distance Score (RMDS) and Gaussian Dirichlet Process Mixture Models (DPMMs) with tied covariance. Building on this, it proposes hierarchical Gaussian DPMMs (full, diagonal, coupled diagonal) to model class-specific covariance structures.", "claims_and_evidence_yn": "No", "claims_explanation": "The authors pay a lot of effects in describe their algorithm, but realisation details and heuristics in their design choices are missing.", "audience_yn": "Yes", "audience_explanation": "OOD detection is an important area of machine learning, researchers in safety and trustworthiness may be interested in it.", "requested_changes": "The clarity and the concreteness of this paper should be improved. \n\nMore experiments, baselines, and base models should be considered. \n\nPlease see Additional Comments for more comments.", "broader_impact_concerns": "Bayesian framework is of broad interest in the literature of OOD detection, the authors may contribute to this sub area.", "additional_comments": "Although post-hoc methods are the main focus of this paper, I think the authors still need to mention fine-tuning based methods in the related works. I listed below for some works that may of the practical interests. \n[1]  Deep Anomaly Detection with Outlier Exposure \n[2] POEM: Out-of-Distribution Detection with Posterior Sampling \n[3] Learning to Augment Distributions for Out-of-Distribution Detection \n\nIt might be hard for the beginner to understand the basic problem setup of ood detection, considering add more detailed backgrounds about OOD detection in Sec 3. Also, the definition of \\theta_k in Eq 2 seems to be missing, seemingly to be the joint of \\mu and \\simga. Similar issues also appear for the IW in Eq 12. I am also confused about why DPMM is of the particular interests of this paper, any particular benefits of it in the context of OOD detection, especially for the use of N_k and alpha in Eq. 1? \n\nI am not sure if EM is indispensable for estimating v and kappa, is it possible to set them as hyperparameters? Because EM also cannot guarantee the convergence to the optimal solution. \n\nIf the proposed method still overfits to high dimensional data, I am quite concerned about its effectiveness. Two baseline methods may be required for comparisons, both in heuristics and experiments. \n1. Estimating \\mu and \\sigma for each class directly by empirical expectation, further assuming that \\sigma is diagonal. \n2.  Conducting dimension reductions for to low dimensional space, maybe three to ten dimensions, then using empirical expectation to estimate \\mu and \\sigma\n \nThe paper is not easy to read, a lot of Bayesian designs are presented without detailed discussions. \n\nMore baselines and experimental setups are required for the experimental part.", "extra_scores": {}, "extra_text": {}}, {"review_id": "CsFydX6xsw", "reviewer": "Reviewer_pzYS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Strengths\n\n1.Rigorous proofs (Propositions 4.1–4.2) formalize the RMDS-DPMM connection, filling a gap between heuristic distance-based methods and BNP generative modeling.\n\n2.Hierarchical models address real-world limitations (e.g., heterogeneous covariances in ImageNet-1K, verified via Förstner-Moonen distance analysis) and are computationally feasible (EM runs in minutes, even for high-dimensional embeddings).\n\n3.Tests on synthetic data (to isolate covariance/sample size effects) and OpenOOD (to validate real-world performance) with ablation studies (e.g., embedding dimensionality, preprocessing) ensure results are robust.\n\nWeaknesses\n\n1.The coupled diagonal model exhibits significant performance variation across datasets. On ImageNet-1K, it performs optimally in Near OOD detection, achieving an AUROC of 80.98—outperforming the tied model (79.28) and full covariance model (70.66) . However, on CIFAR-10 and CIFAR-100, its performance deteriorates markedly. For CIFAR-10, the coupled diagonal model’s Near OOD AUROC is 88.30, which is lower than the full covariance model’s 90.63 and the tied/RMDS models’ 89.83 . For CIFAR-100, its Near OOD AUROC drops to 78.05, trailing the tied model (80.17) and RMDS (80.17) . The paper fails to provide a clear explanation for this dataset-dependent behavior: it does not analyze how differences between datasets (e.g., ImageNet-1K’s 1000 classes with heterogeneous covariance vs. CIFAR-10’s 10 classes with more homogeneous covariance, or ViT vs. ResNet embeddings) interact with the coupled diagonal model’s design (e.g., its $\\gamma_k$ scale factor for cross-dimensional variance correlation) to cause performance gaps.\n\n2. The paper only conducts ViT-B for ImageNet-1k and ResNet-18 for CIFAR. How about try one or two more different models on these dataset.\n\n3.The paper does not compare with state-of-the-art OOD methods from 2024 or non-Gaussian BNP models, which may limit context for its advantages. The baselines are too limited and old, please compare with more latest methods [1-5].\n\n[1] Djurisic, Andrija, et al. \"Extremely simple activation shaping for out-of-distribution detection.\" arXiv preprint arXiv:2209.09858 (2022).\n\n[2] Jiang, Xue, et al. \"Detecting out-of-distribution data through in-distribution class prior.\" International conference on machine learning. PMLR, 2023.\n\n[3] Sun, Yiyou, Chuan Guo, and Yixuan Li. \"React: Out-of-distribution detection with rectified activations.\" Advances in neural information processing systems 34 (2021): 144-157.\n\n[4]  Xu, Kai, et al. \"Scaling for training time and post-hoc out-of-distribution detection enhancement.\" arXiv preprint arXiv:2310.00227 (2023).\n\n[5] Regmi, Sudarshan, et al. \"Reweightood: Loss reweighting for distance-based ood detection.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1.Propositions 4.1 and 4.2 are rigorously proven (with full details in Appendix D) to link RMDS to Gaussian DPMM inlier probabilities. For example, Proposition 4.2 bounds the difference between DPMM scores and RMDS under sub-Gaussian data, using tools like Hanson-Wright inequality and Loewner-Heinz inequality to ensure correctness.\n\n2.Synthetic experiments (Figure 3) isolate key variables (covariance heterogeneity via \\(\\nu_0\\), per-class sample size \\(N_k\\)) and show: (a) hierarchical DPMMs outperform RMDS when \\(\\nu_0\\) is small (heterogeneous covariances), (b) DPMMs dominate \"Independent RMDS\" when \\(N_k/D\\) is small (small samples vs. dimensions). These results directly validate the model’s design goals.\n\n3.On OpenOOD (Table 1, Appendices I–K), hierarchical models outperform baselines in targeted regimes: (a) full covariance model achieves highest AUROC (93.50) on CIFAR-10 Far OOD, (b) coupled diagonal model leads on ImageNet-1K Near OOD (80.98 AUROC). Preprocessing ablation (e.g., W&R vs. PCA) and dimensionality analysis (Figure 4) confirm results are not artifacts of data preparation.", "audience_yn": "Yes", "audience_explanation": "The paper revitalizes BNP methods for OOD detection, a task where BNP’s strength in handling infinite clusters (novel classes) has been underutilized. The DPMM-RMDS connection bridges BNP theory to practical, widely used distance-based methods, opening new directions for BNP applications in anomaly detection.", "requested_changes": "Please refer to weakness.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)