Out-of-Distribution Detection with a Single Unconditional Diffusion Model

Authors: alvin  heng
alexandre h thiery
harold  soh
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-09 10:48:21

Abstract: Out-of-distribution (OOD) detection is a critical task in machine learning that seeks to identify abnormal samples. Traditionally, unsupervised methods utilize a deep generative model for OOD detection. However, such approaches require a new model to be trained for each inlier dataset. This paper explores whether a single model can  perform OOD detection across diverse tasks. To that end, we introduce  Diffusion Paths (DiffPath), which  uses a single diffusion model originally trained to perform unconditional generation for OOD detection. We introduce a novel technique of measuring the rate-of-change and curvature of the diffusion paths connecting samples to the standard normal. Extensive experiments show that with a single model, DiffPath is competitive with prior work using individual models on a variety of OOD tasks involving different distributions. Our code is publicly available at https://github.com/clear-nus/diffpath.

Main paper: 
 introduction: 
 Out-of-distribution (OOD) detection, also known as anomaly or outlier detection, seeks to detect abnormal samples that are far from a given distribution. This is a vital problem as deep neural networks are known to be overconfident when making incorrect predictions on outlier samples [1, 2] , leading to potential issues in safety-critical applications such as robotics, healthcare, finance, and criminal justice [3] . Traditionally, OOD detection using only unlabeled data relies on training a generative model on in-distribution data. Thereafter, measures such as model likelihood or its variants are used as an OOD detection score [4] [5] [6] . An alternative approach is to utilize the excellent sampling capabilities of diffusion models (DMs) to reconstruct corrupted samples, and use the reconstruction loss as an OOD measure [7] [8] [9] .However, these conventional methods require separate generative models tailored to specific inlier distributions and require retraining if the inlier data changes, such as in continual learning setups. This prompts the question: can OOD detection be performed using a single generative model? We answer in the affirmative and present Diffusion Paths (DiffPath) in this paper. While the use of a single model for OOD detection has been proposed in the discriminative setting [10] , to the best of our knowledge, we are the first to explore this for generative models. We believe that the generative setting is particularly salient in light of recent trends where single generative foundation models are utilized across various tasks [11, 12] .Our method utilizes a single pretrained DM. In a departure from prior works that utilize variants of likelihoods [6, 5, 4] or reconstruction losses [7] [8] [9] , we propose to perform OOD detection by measuring characteristics of the forward diffusion trajectory, specifically its rate-of-change and curvature, which can be computed from the score predicted by the diffusion model. We provide theoretical and empirical analyses that motivate these quantities as useful OOD detectors; their magnitudes are similar for samples from the same distribution and different otherwise. We summarize our contributions as follows: 1. We introduce a novel approach to OOD detection by examining the rate-ofchange and curvature along the diffusion path connecting different distributions to standard normal.DDIM Inversion2. Through comprehensive experiments with various datasets, we show that a single generative model is competitive with baselines that necessitates separate models for each distribution.3. We offer a theoretical framework demonstrating that our method characterizes properties of the optimal transport (OT) path between the data distribution and the standard normal. 
background: 
 Score-based Diffusion Models. Let p 0 (x) denote the data distribution. We define a stochastic differential equation (SDE), also known as the forward process, to diffuse p 0 (x) to a noise distribution p T (x):EQUATIONwhere f (•, t) : R D → R D is the drift coefficient, g(t) ∈ R is the diffusion coefficient and w t ∈ R D is the standard Wiener process (Brownian motion). We denote p t as the marginal distribution of Eq. 1 at time t. By starting from noise samples x T ∼ p T , new samples x 0 ∼ p 0 (x) can be sampled by simulating the reverse SDE dx t = [f (x t , t) -g(t) 2 ∇ x log p t (x t )]dt + g(t)d wt , x T ∼ p T (x)where wt is the standard Wiener process when time flows backwards from T to 0, and dt is an infinitesimal negative timestep. The diffusion process described by Eq. 1 also has an equivalent ODE formulation, termed the probability flow (PF) ODE [13] , given byEQUATIONThe ODE and SDE formulations are equivalent in the sense that trajectories under both processes share the same marginal distribution p t (x t ). Hence, given an estimate of the score function s θ (x t , t) ≈ ∇ x log p t (x t ), which can be obtained using score-matching approaches [14, 13] , one can sample from the diffusion model by solving the reverse SDE or integrating the PF ODE backwards in time.In this work, we focus on the variance-preserving formulation used in DDPM [15] , which is given by an Ornstein-Uhlenbeck forward processEQUATIONwhere β t are time-dependent constants. Under Eq. 4, diffused samples x t can be sampled analytically via p t (x t |x 0 ) = N (x t ; √ ᾱt x 0 , σ 2 t I), where β t = -d dt log ᾱt and σ 2 t = 1 -ᾱt . The score estimator, ϵ θ (x t , t) ≈ -σ t ∇ x log p t (x t ), can be trained via the following objectiveEQUATIONwhere Unsupervised OOD Detection. Given a distribution of interest p(x), the goal of OOD detection is to construct a scoring function which outputs a quantity S θ (x) ∈ R that identifies if a given test point x test is from p(x). In this work, a higher value of S θ (x test ) indicates that the sample is more likely to be drawn from p(x). We will use the notation "A vs B" to denote the task of distinguishing samples between A and B, where A is the inlier distribution and B is the outlier distribution. In unsupervised OOD detection, one must construct the function S θ using only knowledge of A.ϵ = -σ t ∇ x log p t (x t |x 0 ) = (x t - √ ᾱt x 0 )/σ t . 
diffusion models for ood detection: 
 An overview of our method, DiffPath, is illustrated in Fig. 1 . DiffPath is based on the insight that the rate-of-change and curvature of the diffusion path connecting samples to standard normal differ between distributions, making them effective indicators for OOD detection. This section outlines the methodology behind DiffPath. We begin in Sec. 3.1, where we provide evidence that likelihoods from a diffusion model are insufficient for OOD detection. Next, Sec. 3.2 shows that the score function is a measure of the rate-of-change and motivates the use of a single generative model. We then motivate the curvature as the derivative of the score in Sec. 3.3. We consider the curvature statistic as one variation of our method and abbreviate it as DiffPath-1D. In Sec. 3.4, we contextualize our method in terms of the optimal transport path between samples and standard normal, and finally propose a higher-order, hybrid variation called DiffPath-6D in Sec. 3.5, which incorporates both the rate-of-change and curvature quantities.When leveraging a likelihood-based generative model for OOD detection, the most natural statistic to consider is the likelihood itself. As DMs are trained to maximize the evidence lower bound (ELBO) of the data, one would expect that in-distribution samples have higher ELBO under the model compared to out-of-distribution samples. However, prior works [2, 16] have shown that the opposite behavior was observed in deep generative models, such as normalizing flows, where the model assigned higher likelihoods to OOD samples.In Fig. 2 , we plot the distributions of the negative ELBO (denoted NLL) of the CIFAR10 and SVHN training sets for a DM trained on CIFAR10. Our results corroborate earlier findings that likelihoods are not good OOD detectors; the NLL of CIFAR10 samples are higher than SVHN samples, meaning in-distribution samples have lower likelihoods than out-of-distribution samples. The poor AUROC score in Table 1 quantitatively demonstrates the inability of likelihoods to distinguish between inlier and outlier samples. This motivates us to search for better statistics that we can extract from DMs for OOD detection.Scores as KL Divergence Proxy. We start by rewriting the PF ODE, Eq. 3, in the following form: where we have parameterized the score as ϵ p (x t , t) = -σ t ∇ x log p t (x). Theorem 1. Denote ϕ t and ψ t as the marginals from evolving two distinct distributions ϕ 0 and ψ 0 via their respective probability flow ODEs (Eq. 6) forward in time. We consider the case with the same forward process, i.e., the two PF ODEs have the same f (x t , t), g(t) and σ t . Under some regularity conditions stated in Appendix A.1,EQUATIOND KL (ϕ 0 ∥ψ 0 ) = 1 2 T 0 E x∼ϕt g(t) 2 σ t ∥ϵ ϕ (x t , t) -ϵ ψ (x t , t)∥ 2 2 dt + D KL (ϕ T ∥ψ T ).The term D KL (ϕ T ∥ψ T ) vanishes as ϕ T = ψ T = N (0, I) by construction, assuming the true scores are available. In practice, we rely on a score estimator ϵ θ obtained via score matching approaches. Theorem 1 suggests that the scores of the marginal distributions along the ODE path serve as a proxy for the KL divergence: as D KL (ϕ 0 ∥ψ 0 ) increases, so should the difference in the norms of their scores. Another interpretation is that this difference,E[∥ϵ ϕ (x t , t) -ϵ ψ (x t , t)∥ 2 2 ], is a measure of the Fisher divergence between the two distributions, which forms the foundation for score matching [17] . This motivates using the norm of the scores as a statistic for distinguishing two distributions.However, Theorem 1 is not immediately useful as it requires a priori knowledge of both distributions, whereas in unsupervised OOD detection, only knowledge of the inlier distribution is available. Interestingly, we empirically observe that it is possible to approximate the forward probability flow ODE for different distributions using a single diffusion model. Recall that as the PF ODE has the same marginal as the forward SDE, if the score estimate ϵ θ has converged to the true score, then forward integration of a sample x 0 using Eq. 6 should bring the sample to approximately standard normal, x T ∼ N (0, I). Specifically, we consider the following parameterization [18] of the PF ODEEQUATIONwhereγ t = 1-ᾱ2 t ᾱt and xt = x t 1 + γ 2 t . Let ϵ θ (x t , t) = -σ t ∇ x log p t (x)be a score model trained on p 0 (x). It is known that the DDIM sampler [19] is Euler's method applied to Eq. 7. In Fig. 3 , we integrate Eq. 7 forward in time using DDIM for samples from various distributions, most of which are unseen by the model during training. Qualitatively, we observe the surprising fact that both the ImageNet and CelebA models are able to bring the samples approximately to the standard normal. We ablate the choice of p 0 in Sec. 5.2. This motivates ϵ θ as a replacement for arbitrary ϵ ϕ when integrating Eq. 7 forward with samples from ϕ 0 . In Fig. 2 , we see that the distributions of t ∥ϵ θ (x t , t)∥ 2 2 , the square root of the sum of L 2 norms of scores over time, applied to the two datasets using a single model trained on ImageNet are better separated than the likelihoods. Note that Theorem 1 tells us only that the score norms of inlier and outlier samples are different, not whether one is higher or lower than the other. Thus, we propose the following OOD detection scheme: fit a Kernel Density Estimator (KDE) to t ∥ϵ θ (x t , t)∥ S θ . This way, the likelihoods of outlier samples under the KDE are lower than for inlier samples, allowing us to compute the AUROC. We provide pseudocode in Algorithm 1. With this scheme, the AUROC scores in Table 1 show a large improvement over likelihoods; however, we would like to pursue even better OOD statistics, which we discuss in Sec. 3.3.Score as First-Order Taylor Expansion. We provide a second interpretation of the score, and subsequently motivate a new statistic that can be used for OOD detection. Recall that the numerical DDIM solver is the first-order Euler's method applied to Eq. 7. In general, we can expand the ODE to higher-order terms using the truncated Taylor method [20, 18] :EQUATIONwhere h n = γ tn+1 -γ tn . The norm of the first-order score, ∥ϵ θ ∥ 2 , therefore measures the rate-ofchange of the ODE integration path. Intuitively, the ODE integration path necessary to bring different distributions to the standard normal in finite time differs (c.f. the PF ODE path is also the optimal transport path, see Sec. 3.4) , hence the rate-of-change differs as well.Based on the preceding discussion, it is natural to consider if higher-order terms in the ODE Taylor expansion can also serve as an effective OOD statistic. We answer in the affirmative by considering the second order term, dϵ θ dγt . Intuitively, the second-order term measures the curvature of the ODE integration path. We expand the derivative as follows [18] :EQUATIONWe see that the derivative contains two Jacobian-Vector Products (JVP) and a simple time derivative term. In principle, all three terms can be computed using automatic differentiation. However, this makes inference twice as costly due to the need for an additional backward pass after every forward pass of the network, and significantly more memory-intensive due to storage of the full computation graph. Fortunately, the time derivative term can be computed using simple finite difference:EQUATION)where the pairs (x t , x t+∆t ) are obtained during standard DDIM integration. Thus, no additional costs associated with gradient computation are incurred.Surprisingly, we observe that for high-dimensional distributions such as images that we consider in this work, the time derivative in Eq. 10 alone provides an accurate enough estimate for OOD detection. Using the same ImageNet model as in Sec. 3.2, we observe an improvement in AUROC scores in CIFAR10 vs SVHN in Table 1 when using the second-order statistic. Qualitatively, the distributions oft ∥∂ t ϵ θ (x t , t)∥ 2 2are more spread out in Fig. 2 as compared to the first-order scores, although the distinction is subtle; the quantitative results provide a more reliable confirmation of the benefit of using the second-order statistic.We thus consider the second-order statistic alone,t ∥∂ t ϵ θ (x t , t)∥ 2 2, as a possible statistic for OOD detection. As it is a one-dimensional quantity, we abbreviate it as DiffPath-1D. We evaluate DiffPath-1D in Sec. 5.2.Recent works have viewed DDIM integration as an encoder that maps the data distribution to standard normal [21, 22] . They prove that this map is the optimal transport (OT) path if the data distribution is Gaussian, while providing numerical results suggesting likewise for high-dimensional data like images. As a result, we can view the OOD statistics proposed in Sec. 3.2 and Sec. 3.3 as characterizing different derivatives of the OT path: the score ∥ϵ θ ∥ 2 represents the rate-of-change of the path, while the time derivative ∥∂ t ϵ θ ∥ 2 represents its curvature.To justify our proposition that derivatives of OT paths serve as meaningful OOD statistics, we consider the following toy example [21] of distinguishing two multivariate Gaussians (detailed derivation in Appendix A.2). Let the distributions be p i 0 (x) ∼ N (a i , I), i ∈ {0, 1}, where a i ∈ R d and I ∈ R d×d . The marginal densities along the forward diffusion can be computed exactly using the transition formulas for SDEs [23] and is given by p i t (x) ∼ N (a i e -t , I), with PF ODE dxi dt = -a i e -t . This path corresponds exactly to the OT map between p i 0 and N (0, I). In this case, the corresponding first and second-order OOD statistics are equal and given by dxidt 2 = d 2 xi dt 2 2= ∥a i e -t ∥ 2 . Crucially, they are proportional to ∥a i ∥ 2 , meaning that as the two distributions move farther apart (i.e., as ∥a 0 -a 1 ∥ 2 increases), so should the L 2 norms of the OOD statistics, thereby increasing their ability to distinguish samples between the two. Owing to its simplicity, the one-dimensional statistic proposed in Sec. 3.3 may suffer from edge cases or perform suboptimally as information is condensed to a single scalar. For instance, given an image x 0 with pixels normalized to the range [-1, 1], one such edge case is distinguishing x 0 from itself with the sign of the pixels flipped, -x 0 . The two samples will produce symmetric OT paths differing only by a negative sign, resulting in identical statistics after taking the L 2 norm. We can see this from Table 2 where DiffPath-1D fails to distinguish CIFAR10 samples from itself with signs flipped, which we call negative CIFAR10. As such, we propose a higher-dimensional statistic that does not utilize the standard form of the L p norm: ∥x∥ p = i |x i | p . We define a new scalar quantity, ⟨x⟩ p = i (x i ) p , which retains the sign information, and propose a new six-dimensional statistic we dub DiffPath-6D:t ⟨ϵ θ (x t , t)⟩ 1 , t ⟨ϵ θ (x t , t)⟩ 2 , t ⟨ϵ θ (x t , t)⟩ 3 , t ⟨∂ t ϵ θ (x t , t)⟩ 1 , t ⟨∂ t ϵ θ (x t , t)⟩ 2 , t ⟨∂ t ϵ θ (x t , t)⟩ 3 ⊤which concatenates scalars based on the first, second and third powers of ϵ θ and ∂ t ϵ θ into a vector. From Table 2 , DiffPath-6D is able to distinguish CIFAR10 from neg. CIFAR10 near perfectly. We validate both DiffPath-1D and DiffPath-6D on a wider suite of experiments in Sec. 5. 
related works: 
 Modern OOD detection can be divided roughly into three categories: feature-based, likelihood-based, reconstruction-based. Feature-based methods extract features from inlier samples and fit a likelihood or distance function as an OOD detector. For instance, one can obtain the latent representations of a test sample using an autoencoder and measure its Mahalanobis distance to the representations of inlier samples [24] . Distances between features derived from self-supervised learning models are also utilized in similar contexts [25, 26] . Similar to our work, Xiao et al. [10] showed that one can perform OOD detection using features from a single discriminative model.Likelihood-based approaches leverage a generative model trained on inlier samples. These methods typically employ variants of the log-likelihood of a test sample under the model as the OOD detection score. Nalisnick et al. [2] first pointed out that deep generative models might erroneously assign higher likelihoods to outlier samples. Several explanations have been proposed, such as the input complexity [16] and typicality [27] of samples. As a result, just as we show in Sec. 3.1, vanilla likelihoods are rarely used. Instead, variants derived from the log-likelihood have been proposed, such as likelihood ratios [4] , ensembles of the likelihood [5] , density of states [6] , energy-based models [28] and typicality tests [27] . Diffusion Time Estimation [29] estimates the distribution over the diffusion time of a noisy test sample and uses the mean or mode as the OOD score. MSMA [30] uses the score function over discrete noise levels for OOD detection. One can view MSMA as a specific case of DiffPath which only utilizes the first-order statistic, while we generalize to higherorder terms. MSMA proposes to measure the score at various noise levels, while our method sums over the entire diffusion path. It is worth emphasizing that MSMA requires different models for different inlier distributions, unlike our single model setup.Reconstruction-based approaches evaluate how well a generative model, trained on in-distribution data, can reconstruct a test sample. Earlier works utilize autoencoders [31] and GANs [32] for reconstruction. Recent works have adapted unconditional DMs to this approach due to its impressive sample quality. A test sample is first artificially corrupted before being reconstructed using the DM's sampling process. DDPM-OOD [7] noises a sample using the forward process and evaluates the perceptual quality [33] of the reconstructed sample. Projection Regret [9] adopts a similar approach, but uses a Consistency Model [34] and introduces an additional projection regret score. LMD [8] corrupts the image by masking and reconstructs the sample via inpainting. Evidently, DiffPath differs from these diffusion approaches as we do not utilize reconstructions. We also stress again that these baselines require different models for different inlier tasks.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: Full experimental details are provided in main paper and appendix. Opensource code is provided. 
experiments: 
 Based on our earlier analysis, we hypothesize that DiffPath can be utilized for OOD detection across diverse tasks using a single model. In this section, we validate our hypothesis with comprehensive experiments across numerous pairwise OOD detection tasks and compare DiffPath's performance against state-of-the-art baselines.Datasets. All experiments are conducted as of pairwise OOD detection tasks using CIFAR10 (C10), SVHN, and CelebA as inlier datasets, and CIFAR100 (C100) and Textures as additional outlier datasets. Unconditional diffusion models are employed at resolutions of 32 × 32 and 64 × 64. The model utilizing ImageNet as the base distribution is trained at 64 × 64 resolution, while all other models are trained at 32 × 32.Methodology and Baselines. Our methodology features two variants of our model, DiffPath-1D using KDE and DiffPath-6D using a Gaussian Mixture Model for OOD scoring, as outlined in Sec. 3. We compare against a variety of generative baselines, including Energy-based Model (EBM) such as IGEBM [28] , VAEBM [35] and Improved CD [36] , as well as Input Complexity (IC) [16] , Density of States (DOS) [6] , Watanabe-Akaike Information Criterion (WAIC) [5] , Typicality Test (TT) [27] and Likelihood Ratio (LR) [4] applied to the Glow [37] model. Additionally, we compare against diffusion baselines such as vanilla NLL and IC based on the DM's likelihoods and re-implementations of DDPM-OOD [7] , LMD [8] , and MSMA [30] based on open-source code for full comparisons.Table 3 summarizes our main results. Here, we report outcomes for DiffPath-6D using ImageNet and CelebA as base distributions. DiffPath-6D-CelebA achieves an average AUROC of 0.931, comparable to the leading diffusion-based approach MSMA and outperforming all other baselines, while utilizing only a single model. Similar to MSMA, we attain this result using 10 NFEs, significantly surpassing other diffusion baselines that require an order of magnitude or more NFEs. When using ImageNet as the base distribution, the average AUROC of 0.850 is competitive with LMD, which requires several The empirical results indicate that the CelebA-based model outperforms the ImageNet-based model primarily due to its superior performance on tasks involving CelebA samples, whether they are in-distribution or out-of-distribution. For instance, DiffPath-6D-CelebA achieves near-perfect performance on all tasks where CelebA is in-distribution (rightmost columns), and in the CIFAR10 vs CelebA task. On tasks that do not involve CelebA samples, the two models exhibit roughly comparable performance. This suggests that distinguishing CelebA from other samples is particularly challenging, and that DiffPath benefits from exposure to inlier samples from the respective distributions during training. Next, we discuss the effect of the base datasets and other design considerations via ablations. Choice of Diffusion Training Set. We investigate the impact of the base dataset on the performance of DiffPath-6D. In Table 4 , we compare four different base distributions alongside a randomly initialized model. As a baseline check, we observe that the average AUROC of the randomly initialized model is 0.501, which is consistent with random guessing. This indicates that training on a base distribution is essential for the model to learn features for effective OOD detection.Our ablations include CIFAR10 and SVHN as base distributions, in addition to CelebA and ImageNet shown in Table 3 . Among these four base distributions, CelebA yields the best performance overall.Notably, the models trained on SVHN and CelebA demonstrate superior results when the inlier data aligns with their respective training distributions. This supports the established principle of training models on in-distribution samples, and we show that DiffPath-6D similarly benefits from such training. However, we underscore the key finding of our work: while in-distribution training enhances performance, it is not strictly necessary. DiffPath-6D exhibits strong performance even on tasks involving samples from distributions that the model has not encountered during training.DiffPath 1D vs 6D. Here we ablate on the choice of DiffPath-1D and DiffPath-6D. Table 6 shows that DiffPath-6D performs better than its 1D counterpart for both choices of base distributions. We attribute this to the increased robustness of aggregating multiple statistics, c.f. Sec. 3.5, and recommend practitioners to use DiffPath-6D in general. However, DiffPath-1D outperforms DiffPath-6D in certain instances. For instance, for the ImageNet model, DiffPath-1D achieves the best performance on CIFAR10 vs SVHN and in three out of four tasks where SVHN is the inlier distribution.Number of DDIM Steps. We investigate how the performance of our method varies with the number of NFEs (DDIM steps) in Table 5 . Overall, the changes in average AUROC are relatively minor as the NFEs are varied, suggesting that DiffPath is robust to the number of integration steps. The best result is obtained at 10 NFEs. While the finite difference approximation of the derivative (Eq. 10) should become more accurate as the number of NFEs increases, the aggregation of multiple statistics involving scores and its derivatives makes this effect less pronounced. We leave the investigation of this phenomena in greater detail to future work.Using a single DM with a fixed input resolution necessitates resizing all images to match the model's resolution during evaluation. However, when datasets have differing original resolutions, naive resizing-upsampling lower-resolution images and downsampling higher-resolution ones-can lead to evaluation inaccuracies. Specifically, upsampling introduces blurriness due to pixel interpolation, while downsampling does not. This discrepancy allows the model to differentiate samples based on image blur rather than semantic content, resulting in overly optimistic performance metrics.For instance, when evaluating DiffPath-6D trained at 64 × 64 pixels on the CelebA vs CIFAR10 task, CIFAR10 images are upsampled (introducing blur) while CelebA images are downsampled. This imbalance enables trivial distinction between the samples. As illustrated in the first row of Table 7 , tasks where only one distribution undergoes upsampling yield artificially high AUROC scores.To mitigate this issue, we propose equalizing the resizing process by first downsampling higherresolution images to the lower resolution of the other distribution, then upsampling both to the model's required resolution. In the CelebA vs CIFAR10 example, CelebA images are downsampled to 32 × 32 pixels before both samples are upsampled to 64 × 64 pixels. This method ensures consistent blurring effects across all samples, reducing confounding factors. The second row of Table 7 demonstrates more accurate evaluations using this approach. We adopt this resizing procedure in all relevant experiments to ensure fair comparisons. In short, we highlight the importance of consistent image resizing for fair evaluation in OOD detection, which we hope will guide future research.Near-OOD tasks refer to setups where the inlier and outlier samples are semantically similar, making them challenging for most methods. From Table 3 , DiffPath, like most baselines, does not perform strongly on near-OOD tasks like CIFAR10 vs CIFAR100. This motivated us to conduct further near-OOD experiments, the results of which are presented in Table 8 of Sec. C of the appendix. Note that near-OOD tasks are not a standard evaluation on generative methods. We defer detailed analysis of the results to the appendix, and leave further investigations on near-OOD tasks to future work.Question: Does the paper provide open to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: Open-source code is provided.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion: 
 In this work, we proposed Diffusion Paths (DiffPath), a method of OOD detection using a single diffusion model by characterizing properties of the forward diffusion path. In light of the growing popularity of generative foundation models, our work demonstrates that a single diffusion model can also be applied to OOD detection. There are several interesting future directions that arise from our work; for instance, applying DiffPath to other modalities such as video, audio, language, time series and tabular data, as well as investigating if higher-order terms of the Taylor expansion, or leveraging different instantiations of the PF ODE might lead to better performance.Limitations and Future Work. We only calculate the simple time derivative and found that it works well experimentally, although one might compute the full derivative to quantify the curvature completely. We leave this for future work. Also, we consider DMs trained on natural images like CelebA and ImageNet, which may not be appropriate for specialized applications such as medical images. For such purposes, one could consider including domain-specific data during training.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Provided in the appendix and code.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Error bars are not conventionally reported for AUROC performance. Furthermore, our method is deterministic (ODE-based), so repeated evaluations produce the same result (up to initialization of the EM algorithm when fitting the GMM, which is negligible in our experiments).Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: While we utilize a diffusion model, we are not leveraging its generation abilities, so societal impacts of generative models like bias/disinformation/deepfakes etc are not relevant to our work. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The diffusion models utilized in this work are trained on standardized, publicly available image datasets (e.g., CelebA, ImageNet) that to our knowledge, do not contain risks of misuse or harmful generation.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: Citations to relevant models and works are given.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside assets?Answer: [Yes]Justification: Documentation is provided in the appendix and open-source code.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: No human subjects are required in this work.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: No human subjects or crowdsourcing are required in this work.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A.1 Theorem 1 Theorem 1. Denote ϕ t and ψ t as the marginals from evolving two distinct distributions ϕ 0 and ψ 0 via their respective probability flow ODEs (Eq. 6) forward in time. We consider the case with the same forward process, i.e., the two PF ODEs have the same f (x t , t), g(t) and σ t . Under some regularity conditions stated in Appendix A.1,D KL (ϕ 0 ∥ψ 0 ) = 1 2 T 0 E x∼ϕt g(t) 2 σ t ∥ϵ ϕ (x t , t) -ϵ ψ (x t , t)∥ 2 2 dt + D KL (ϕ T ∥ψ T ).Proof. The proof is a modification from Song et al. [38] . Let us first state the PF ODEs of the two distributions explicitly:EQUATION)EQUATION)We make the following assumption about ϕ t and ψ t :EQUATIONWe start by rewriting the KL divergence between ϕ 0 and ψ 0 in integral form:EQUATIONAs we can treat the PF ODE as a special case of an SDE with zero diffusion term, we can obtain the Fokker-Planck of the marginal density of the PF ODEs, also known as the continuity equation, as follows:EQUATIONwhere we define h ϕ := -f (x t , t) -g(t) 2 2σt ϵ ϕ (x t , t) for simplicity. Similarly, ∂ψt ∂t = ∇ x • (h ψ ψ t (x)). Let us now rewrite the time-derivative ∂DKL(ϕt∥ψt) ∂t in Eq. 14 as follows:∂D KL (ϕ t ∥ψ t ) ∂t = ∂ ∂t ϕ t (x) log ϕ t (x) ψ t (x) dx = ∂ϕ t ∂t log ϕ t (x) ψ t (x) dx + ∂ϕ t (x) ∂t dx =0 - ϕ t (x) ψ t (x) ∂ψ t (x) ∂t dx = ∇ x • (h ϕ (x, t)ϕ t (x)) log ϕ t (x) ψ t (x) dx - ϕ t (x) ψ t (x) ∇ x • (h ψ (x, t)ψ t (x))dx (i) = ϕ t (x)[h ⊤ ϕ (x, t) -h ⊤ ψ (x, t)][∇ x log ϕ t (x) -∇ x log ψ t (x)]dx = - 1 2 ϕ t (x) g(t) 2 σ t ∥ϵ ϕ (x, t) -ϵ ψ (x, t)∥ 2 2dx, where (i) is due to integration by parts and the fact that lim x→∞ h ϕ (x, t)ϕ t (x) = 0 and lim x→∞ h ψ (x, t)ψ t (x) = 0 due to assumption 13. Combining with Eq. 14 gives us the desired result:EQUATIONWe derive in detail the toy example discussed in Sec. 3.4, where we will prove the optimal transport map between a multivariate Gaussian and standard normal is identical to the diffusion PF ODE path. We consider our source distribution as p 0 (x) ∼ N (a, I), a ∈ R d and I ∈ R d×d . We choose our forward SDE to be parameterized as:EQUATIONwhich is the same Ornstein-Uhlenbeck process as the DDPM forward SDE Eq. 4 with a constant noise schedule β(t) = 2. This is also commonly referred to as the Langevin equation.As Eq. 17 has affine drift coefficients and a starting distribution which is normal, we know that the marginal distributions at intermediate times are also normal, p t (x) ∼ N (µ(t), Σ(t)). Furthermore, we can calculate the means and variances analytically using Eq. 5.50 and Eq. 5.51 of Särkkä and Solin [23] :EQUATIONwith solutionsEQUATION)Thus, the marginal density has the form p t (x) = N (ae -t , I), from which we compute the score as ∇ x log p t (x) = -x + ae -t . We can substitute this into the corresponding PF ODE to obtain:EQUATIONThe optimal transport map denoted E p0 (x) is obtained by solving Eq. 20 to get x t = x 0 + a(e -t -I) and taking the limit t → ∞. This gives us E p0 (x) = xa, which is precisely the OT map between p 0 (x) ∼ N (a, I) and N (0, I) (cf. Eq. 2.40 in Peyré and Cuturi [39] ).DiffPath. As mentioned in Sec. 5, we utilize a single unconditional diffusion model trained on CelebA and ImageNet at 32 × 32 and 64 × 64 resolution respectively. We train our own CelebA model and utilize the ImageNet checkpoint trained using Improved DDPM's L hybrid objective (Eq. 16 of Nichol and Dhariwal [40] ) from the official repository 2 . Both models use a cosine noise schedule with a total of 4000 diffusion steps. For DiffPath-1D, we fit a KDE using a Gaussian kernel with a bandwith of 5. For DiffPath-6D, we fit a GMM with hyperparameters obtained by sweeping over a predefined number of mixture components (e.g., 50, 100) and covariance type (e.g., diagonal, full, tied). Both are implemented using the sklearn library.On a single Nvidia A5000 GPU, DiffPath takes approximately 0.25s and 0.94s per integration step on 32 × 32 and 64 × 64 images respectively with a batch size of 256.Diffusion Baselines. For all diffusion baselines, we rely on the official GitHub repositories and open-source checkpoints where possible. The repositories are listed as follows: MSMA 3 , DDPM-OOD 4 , LMD 5 . For NLL and IC, we use the pre-trained CIFAR10 checkpoint from Improved DDPM and train our own model for SVHN using the same hyperparameters at 32 × 32 resolution. We calculate the NLL using the default implementation in Improved DDPM, while we compute IC using code from the LR repository 6 due to lack of official code from the IC authors. We train all baselines using 1-3 A5000 GPUs.8: Average AUROC results for near-OOD tasks as proposed in OpenOOD [41] . We use DiffPath-6D with ImageNet as the base distribution with 10 DDIM steps. Bold and underline denotes the best and second best result respectively. We also show the number of function evaluations (NFE) for diffusion methods, where lower is better. To further investigate the performance of DiffPath on near-OOD tasks, we ran experiments on two tasks proposed in OpenOOD [41] . The first task involves CIFAR10 as the in-distribution data with CIFAR100 and TinyImageNet (also known as ImageNet200) as out-of-distribution datasets. The second involves TinyImageNet as in-distribution data and SSB [42] (hard split) and NINCO [43] as out-of-distribution data (see the official repository 7 for full details).We compare our results against the four latest discriminative baselines reported in OpenOOD [41] under the "w/o Extra Data, w/o Training" category, which are KLM [44] , VIM [45] , KNN [46] and DICE [47] . We report the average AUROC of each task in Table 8 . The results are mixed: DiffPath performs the best for the TinyImageNet task, but obtains the poorest result for the CIFAR10 task.As noted in the main text, to our knowledge near-OOD tasks are not a standard evaluation setup for generative methods. We hypothesize that such tasks are better suited to discriminative methods as gradient-based classification training enables the model to learn fine-grained features specific to each in-distribution class, which we believe is crucial in this context. In contrast, generative models focus on maximizing the likelihood of the overall data distribution and are not explicitly trained to identify subtle discriminative features. This could potentially lead to weaker performance in tasks where the distributions exhibit a high degree of overlap. It should be noted that discriminative methods typically require class labels, while unconditional generative methods do not, thus constraining the use of the former to cases with labelled in-distribution data.Complete proofs for Theorem 1 and the optimal transport example are provided in the appendix.• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.38th Conference on Neural Information Processing Systems (NeurIPS 2024).Results obtained from Morningstar et al.[6].https://github.com/openai/improved-diffusionhttps://github.com/ahsanMah/msmahttps://github.com/marksgraham/ddpm-oodhttps://github.com/zhenzhel/lift_map_detecthttps://github.com/XavierXiao/Likelihood-Regrethttps://github.com/Jingkang50/OpenOOD 


Decision: The paper proposes a novel approach to OOD detection using a single diffusion model trained on a diverse dataset, leveraging the rate-of-change and curvature of diffusion paths. The reviewers found the core idea innovative and the experiments generally compelling, demonstrating competitive results across various OOD tasks. Reviewer 5CB8 appreciated the theoretical connections and clear presentation but expressed concerns about resolution mismatches and the benefits of using a single model, which the authors addressed through additional experiments, showing that “DiffPath performs competitively with existing baselines” even when accounting for image resolution discrepancies. Reviewer 3gUh noted the method’s novelty and solid results but pointed out limitations related to model generalizability, which the authors acknowledged and positioned as future work. Reviewers eKbL and k7Q2 commended the clear presentation and the significance of extending OOD detection to large pre-trained generative models, with eKbL stating that “the method seems to work better than competitors” and k7Q2 highlighting the value of a solution that reduces the need for task-specific models. The authors’ comprehensive rebuttals effectively addressed the reviewers’ concerns, supporting the recommendation for acceptance due to the paper’s strong methodological contributions and potential impact on the field.

Reviews: review: paper_summary: paper_summary: The paper proposes DiffPath, an OOD Detection method with foundation diffusion models (i.e., diffusion models over diverse data) that can be applied to any in-distribution dataset. By measuring properties of the diffusion trajectory mapping images to noise, the paper demonstrates some improvements on OOD detection. Notably, the method is faster than most other diffusion-based techniques for OOD detection.
strength_weakness: strength_weakness: - Achieves lower performance for near-ood detection tasks. 
- The evaluations do not sufficiently demonstrate benefits of a single unconditional diffusion model.  Since two sets of examples are said to be OOD if there is no class-overlap between them and the diffusion model is trained on 1k Imagenet classes, I believe that some of the experiments should be focused on demonstrating ease of constructing OOD detectors for arbitrary sets of imagenet classes as in-distribution or perhaps the entire imagenet-data as in-distribution. 
- As a follow-up to the above, it seems like this technique can also be used for learning one-class classification networks. In fact, I feel that the paper should discuss the performance on this task first before considering the natural extension to OOD detection where several classes are in-distribution. It also helps understand the generality of the statistics used to identify OOD examples. 
- The use of Theorem 1 (fisher-divergence) to motivate L2-norm of score-function as an indicator for OOD detection is not entirely clear. For example, the fisher-divergence uses difference between score-function outputs estimated over a batch of examples. 
- The ablation study with resizing is useful. However, it seems like the diffusion model is trained on 64x64 images and hence, the CIFAR10, CIFAR100 and SVHN images are scaled-up to 64x64 while celeba and textures are scaled down to 64x64. For a fair comparison with methods which directly work at 32x32 resolution,  celeba and textures should be first downsampled to 32x32 and then upsampled to 64x64. Otherwise, ood detection between pixelated CIFAR10 images and comparatively high-res celeba/textures images would not be so meaningful.
questions: questions: See weaknesses. 
1. What hyperparameters do you use for KDE/GMM algorithms? How to select these hyperparameters? 
2. Intuitively, why do the trajectory statistics such as norm and rate-of-change vary from class to class?
3. Using an SVHN/CIFAR10/CELEB-A model to directly perform OOD detection was not as successful as using a diffusion model trained on imagenet. However, considering a SVHN diffusion model, if we select one of the SVHN classes as in-distribution, can we achieve effective OOD detection?
limitations: limitations: Yes.
paper_summary: paper_summary: This paper presents a diffusion model trained on a single dataset that can also perform well in OOD detection across diverse tasks. The core concept is Diffusion Paths (DiffPath), which characterizes the properties of the forward diffusion path. Specifically, they measure the rate-of-change and curvature of the diffusion paths, using these as the derivatives of the score and contextualize regarding the optimal transport concept. The method is tested on various benchmarks and generally shows improvement over compared baselines.
strength_weakness: strength_weakness: - In Figure 3, how extensively does the diffusion model need to be trained on a large, diverse dataset?
     - The method is limited by the coverage of the ImageNet-trained diffusion model.
     - Ultimately, to achieve broad coverage, a large-scale pretrained diffusion model is required (e.g. more varisous OOD dataset, medical, manufacturing). Is the contribution of guaranteed generalizability compared to foundational generative models still valid?
     - Add a single trained case that can cover datasets from different domains, not just ImageNet.
- In Table 4, performance drops significantly compared to the baseline in hard-settings like CIFAR10 vs. CIFAR100.
     - The proposed method struggles particularly in distinguishing between ID/OOD datasets with similar semantics.
     - Include experiments and improvement strategies for hard-settings in OpenOOD [1].
- Table 4, add and compare with the latest research baseline, Projection Regret [2].

[1] Zhang, Jingyang, et al. "Openood v1. 5: Enhanced benchmark for out-of-distribution detection." arXiv preprint arXiv:2306.09301 (2023).

[2] Choi, Sungik, et al. "Projection regret: Reducing background bias for novelty detection via diffusion models." Advances in Neural Information Processing Systems 36 (2023): 19230-19245.
questions: questions: N/A
limitations: limitations: see weakness
paper_summary: paper_summary: This paper looks at statistics calculated from the path through the diffusion model based on $\int_0^T \| \epsilon_{\phi}(x_t, t) - \epsilon_{\psi}(x_t, t)\|dt$ and use it to discriminate between two distributions $\psi$ and $\phi$. A KDE of the distance of ID scores is used to compute the likelihood of the score of a new test image, and then perform OOD detection.
strength_weakness: strength_weakness: 1. The training distribution matters (as stated by the authors themselves), so saying that the DM is an universal OOD detector is a bit of an overclaim.
2. In 3.1, It is stated that "likelihood does not work", but the content does not demonstrate such a strong statement. It only shows that *likelihood obtained directly from a diffusion model does not work as is for OOD detection*. It seems from prior work that likelihood coming from the training of generative models is not suitable for OOD detection -- which is a weaker claim than "likelihood does not work" -- but even this weaker statement is not demonstrated properly since showing it for diffusion models is not sufficient evidence. In addition, the authors end up using a likelihood (that of the KDE) to build their score which makes this wording confusing. This part is nothing more than a motivating experiment (which is, by the way, interesting and fits well in the flow of the paper), so I would not provide such an overinterpretation.
questions: questions: 1. Why use the KDE and not directly the score $\sqrt{\sum_t  \| \epsilon_{\theta} (x_t, t) \|^2_2}$ 
2. 3.5 why use the first-order term, which disconnects the statistics from OT theory? Why powers are chosen up to three? 
3. The method requires 50 FE, but what is the cost of one individual FE? OOD detection applications are often real-time, online or embedded so it is important to detail this point.
limitations: limitations: The authors have adequately addressed the limitations.
paper_summary: paper_summary: This paper proposes an unsupervised anomaly detection method based on diffusion models. The core idea is to leverage the properties of the score function of a pre-trained diffusion model to distinguish samples from different distributions, rather than relying on log-likelihoods or reconstruction error. To this end, the authors first demonstrate that log-likelihoods are not a good metric for differentiating samples from different datasets.  The authors then motivate using the score function, specifically the L2 norm of the score function summed across different time steps as a statistic for differentiating OOD samples. This statistic can be interpreted as the rate of change of diffusion trajectories from the original distribution to the standard Normal distribution. Extending this observation, the authors also incorporate the curvature of the trajectory by considering the derivative of the score function. One key contribution is showing that a single diffusion model trained on a large, diverse dataset, such as ImageNet, can be used for OOD detection across multiple datasets. Experiments are performed on CIFAR-10, CIFAR-100, CelebA, and SVHN datasets, demonstrating higher average AUCROC scores compared to existing methods. Ablation experiments show the effect of various design choices.
strength_weakness: strength_weakness: There are certain concerns regarding the applicability of the approach to a broader range of problems, and some clarifying questions for the authors.

- The method relies on the availability of a reasonably large and diverse dataset for the domain of interest. For images, ImageNet is an obvious choice, but this raises the question of the applicability of this method to tabular domains, which are especially relevant for scientific and industrial applications.
- The connection to OT in Section 3.4 is valid only if the data distribution is Gaussian, as per my understanding. However, the data distribution, including the image datasets analyzed in this paper, is typically highly multi-modal. This invalidates the connection, and an explanation from the authors would be beneficial.
- It is not clear how the 6D statistic introduced in Section 3.5 is used to distinguish samples since it is not a scalar value that can directly be compared.
- Since a single diffusion model trained on ImageNet is used for the experiment, it is not clear what distinguishes an ID and OOD dataset. What makes a dataset ID in this scenario and what is the difference when evaluating C10 vs SVHN and SVHN vs C10 in Table 3?
- There is at least one other diffusion-based anomaly detection method that leverages properties of the diffusion schedule rather than relying on reconstruction error [1]. A brief discussion on the differences between this approach and the one discussed in the paper would be beneficial.

[1] Livernoche, V., Jain, V., Hezaveh, Y. and Ravanbakhsh, S., On Diffusion Modeling for Anomaly Detection. In *The Twelfth International Conference on Learning Representations*.
questions: questions: - The results in Table 4 are a bit surprising. Why is a model trained on ImageNet better than a model trained on the ID dataset? For example, when treating C10 as the ID dataset, shouldn’t a diffusion model trained on C10 perform better?
- In this paper, the authors motivate the use of ImageNet due to it being a large and diverse dataset. Do the authors have some thoughts on how to pick such a ‘base’ dataset when applying this method to other domains? Is the idea that this dataset should provide coverage over the ID and OOD datasets?
- The authors provide an explanation for why a higher number of DDIM steps slightly hurt performance, but shouldn’t a smaller time difference make the finite approximation method more accurate?
limitations: limitations: The authors discuss the limitations of their approach in sufficient detail. Some of my points in the weaknesses section echo this discussion.

Comments: title: Thank you to the reviewer
Thank you to the reviewer
Thank you to the reviewer
Response to reviewer
Thank you!
Thank you to the reviewer
Response to reviewer 2/2
Response to reviewer 1/2
Thank you for your response
comment: As the discussion period comes to a close, we sincerely thank the reviewer for their helpful comments. We hope that we have addressed the reviewer's concerns in our rebuttal. If the reviewer still has unaddressed concerns, please let us know and we will be happy to discuss further.
As the discussion period comes to a close, we sincerely thank the reviewer for their helpful comments and for engaging with us. We hope that we have addressed the reviewer's concerns in our rebuttals and if so, we kindly request that the reviewer consider revising their score. If the reviewer still has unaddressed concerns, please let us know and we will be happy to discuss further.
We thank the reviewer for their positive remarks and for acknowledging the contributions of our work!
Thank you to the reviewer for their prompt response and for providing further clarifications. We better understand the concerns raised.

From our interpretation, the reviewer uses the term 'pixelated' to describe images that have been upsampled from a lower resolution (e.g., 32x32 CIFAR10 to 64x64), and 'non-pixelated' to refer to images that have been downsampled from a higher resolution (e.g., CelebA, Textures). We agree with the reviewer’s observation that when comparing 'pixelated' versus 'non-pixelated' images, the blur introduced during upsampling adds additional information—absent in downsampling—that could potentially simplify the OOD task.

To address this concern, we have re-evaluated the results presented in Table 3 of the paper, focusing exclusively on OOD tasks where comparisons are made only between 'pixelated vs pixelated' and 'non-pixelated vs non-pixelated' images. In other words, we have omitted cases where an upsampled image is compared to a non-upsampled image, thereby eliminating any potential advantage due to the blur of upsampling. The revised results are presented below:

| ID           | CIFAR10 | CIFAR10 | SVHN   |  SVHN   | CelebA | Average |
|--------------|---------|---------|--------|---------|--------|---------|
| OOD          | SVHN    | CIFAR100| CIFAR10| CIFAR100| Textures|         |
| Diffusion NLL| 0.091   | 0.521   | 0.99   | 0.992   | 0.809  | 0.681   |
| Diffusion IC | 0.921   | 0.519   | 0.08   | 0.1     | 0.559  | 0.436   |
| MSMA         | 0.957   | 0.615   | 0.976  | 0.98    | 0.967  | 0.899   |
| DDPM-OOD     | 0.39    | 0.536   | 0.951  | 0.945   | 0.773  | 0.719   |
| LMD          | 0.992   | 0.604   | 0.919  | 0.881   | 0.972  | 0.874   |
| DiffPath     | 0.92    | 0.593   | 0.924  | 0.936   | 0.946  | 0.864   |

From this revised evaluation, DiffPath remains highly competitive with existing baselines. Specifically, it outperforms DDPM-OOD, NLL, and IC, while closely matching the performance of MSMA and LMD. Again, this is achieved using a single model that is not trained on in-distribution data. We will revise the results to only include comparisons between datasets of the same resolution, e.g., these new results and the new 64x64 results in the rebuttal, as well as moderate our claims accordingly. In light of these findings, along with our methodological contributions, we believe that our work remains novel and represents a significant advancement in the generative OOD literature. 

Regarding the reviewer's remarks on the OpenOOD benchmarks, we would like to highlight that in OpenOOD, the near-OOD tasks involve datasets where images vary in size *even within the same dataset* (e.g., NINCO and SSB-hard) [1]. Consequently, there is no standardized approach to ensuring comparisons are exclusively between 'pixelated vs pixelated' and 'non-pixelated vs non-pixelated' images, *even in the original settings tested in OpenOOD*. Therefore, we believe that the focus on resolution within the OpenOOD context may be less critical than anticipated. 

[1] https://github.com/Jingkang50/OpenOOD
I thank the authors for their reply and new insights from additional experiments. I also appreciate the clear explanation of their perspectives. While I strongly subscribe to the claim-based evaluation, I feel that the unconventional experimental settings using both pixelated and non-pixelated images gives the wrong message. More specifically, there are 3 types of comparisons in your experimental setup:
1. **Pixelated vs Pixelated**: 
- Examples: C10 vs C100, C10 vs SVHN, SVHN vs C10, SVHN vs C100. 
- Finding: In Table 3, we see that baselines outperform DiffPath in all these cases while DiffPath still offers comparable performance in some cases. 
2. **Pixelated vs Non-pixelated**: 
- Examples: C10 vs Celeb-A, C10 vs Textures, SVHN vs Celeb-A, SVHN vs Textures, Celeb-A vs C10, Celeb-A vs C100, Celeb-A vs SVHN
- Finding: In each of these cases, DiffPath achieves close to 100% detection rate. However, this is also very different from the conventional setting and distinguishing between pixelated and non-pixelated images may almost be trivial. In Table 3, DiffPath outperforms all baselines but under fair settings, it seems like all baselines achieve close to 100% detection rate (at least for Celeb-A vs C10/C100/SVHN as in your rebuttal response). 
3. **Non-pixelated vs Non-pixelated**: 
- Examples: Celeb-A vs Textures
- Finding: All baselines outperform DiffPath both in Table 3 and in the new Celeb-A results in your rebuttal response.

As stated above, my concern is that these are non-conventional settings with over-optimistic results in some cases and cannot be compared directly with other works. A standardized benchmark such as OpenOOD allows us to standardize comparison between methods. It is impressive that DiffPath achieves improvements on Imagenet-200 on the near-OOD tasks in OpenOOD; but the baseline methods in that table operate using 224x224 images while DiffPath operates at 64x64 resolution and hence, it is not clear if it is meaningful to compare between them directly. 

I would appreciate a standardized comparison between baselines and your method that does not involve pixelated images (i.e., resizing standard 32x32 datasets to 64x64 resolution). This would allow for a fair comparison between DiffPath results and other results in the literature. Finally, while it is not necessary to improve over the baselines in every case, it would be good to demonstrate _some_ improvements over the baseline in the standardized evaluation settings. 

I request the authors to kindly correct me in case I am misinterpreting the results. Thank you!
Thank you for the clarifications which clear most of my questions. 
After reading the other reviews and the authors' responses, I think the paper is a useful contribution and should be accepted. I maintain my positive score.
We thank the reviewer for their positive response and for raising their score!
We thank the reviewer for suggesting the above experiments that have suggested where improvements can be made to DiffPath. We hope that the reviewer will evaluate our work in the context of our main claims:

- We propose the use of diffusion path statistics, namely the rate-of-change and curvature, for OOD detection.
- We show that these statistics are obtained from a Taylor expansion about the diffusion process, and can be estimated simply via finite difference of the DDIM sampler.
- We further show that one can obtain these statistics for different data distributions using a single model, even when the model is not trained on task-specific data.
- We connect these statistics to score-matching and optimal transport theory, providing a theoretical motivation for DiffPath.
DiffPath is competitive with generative baselines requiring individually-trained models.

Our experiments were geared towards evaluating these claims. We do **not** claim that DiffPath is the best possible OOD detector in all settings.  

[1] Ahmed, Faruk, and Aaron Courville. "Detecting semantic anomalies." 

[2] Zhang, Richard, et al. "The unreasonable effectiveness of deep features as a perceptual metric."

[3] Song, Yang, and Stefano Ermon. "Generative modeling by estimating gradients of the data distribution." 

[4] Lu, Cheng, et al. "Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps."
We thank the reviewer for their timely reply. Below is our response to the reviewer’s comments.

**OpenOOD transformation pipeline**

We use standard transformations in diffusion image modeling: we resize the images to 64x64 using bilinear interpolation, followed by normalization of pixels to the range [-1,1].

**One class learning**

We thank the reviewer for this suggestion. To our knowledge, generative and discriminative OOD methods are evaluated differently in the literature and we are unaware of any generative baselines that are evaluated via the one-class experiments suggested. We agree that such tests would be very challenging for unconditional generative models as they may not learn the "right" features for discriminating the classes. 

Due to time constraints, we could only run preliminary trials with select ImageNet dog breeds (Ibizan hound vs bluetick and beagle). Without further tuning, we found that DiffPath did not distinguish the dog breeds. We also tested LMD on the same task and found similar performance. We hypothesize that methods that perform well on this task are likely to be discriminative models which utilize class labels during training, such as those evaluated in [1].

Gradient-based classification training enables the model to learn fine-grained features specific to each dog class, which we believe is crucial in this context. In contrast, generative models focus on maximizing the likelihood of the overall data distribution and are not explicitly trained to identify subtle discriminative features. This could potentially lead to weaker performance in tasks where the distributions exhibit a high degree of overlap.

To enhance the performance of generative methods in this challenging context, one potential improvement could involve augmenting the OOD score calculation with discriminative features [2]. This approach could leverage the strengths of both methodologies: the generative model (e.g., DiffPath) would effectively manage most tasks, while the discriminative features would address more complex cases characterized by high distribution overlap. We leave this to future studies and will include this discussion as a limitation of DiffPath in the revision. 

**"DiffPath also has access to a much stronger diffusion model since its trained on much more data. So, matching the results with the baseline methods is not very surprising"**

We respectfully disagree with the reviewer on this statement. We find it surprising that one can perform OOD detection using a generative model that has *not* been trained on ID data or labels, *regardless of the quantity of other data the model is trained on*. 

While one could argue that ImageNet provides broad coverage over images like those in CIFAR10 and faces in CelebA, exact data from these distributions are **not** included in ImageNet. Digits of SVHN are even more sparsely covered by ImageNet. Thus, we believe this is a surprising discovery, one which connects OOD detection with recent findings in foundation generative models, where one model excels at multiple tasks. Our work shows that  the curvature statistics of the Imagenet diffusion model captures general properties of images that are useful for OOD detection. 

**Fair comparison**

For a more complete comparison with diffusion baselines at 64x64 resolution, we were able to train MSMA and DDPM-OOD for all ID setups. We will include complete results, including LMD, in the final revision (reconstruction for LMD takes $>10^3$ steps per sample, thus we are unable to obtain results at present). We present the averaged AUROC result over all 12 tasks in Table 3 of our paper for simplicity

| Method             | Average |
|--------------------|---------|
| MSMA               | 0.951   |
| DDPM-OOD           | 0.765   |
| DiffPath           | 0.942   |


Upsizing leads to more optimistic results for MSMA (and LMD in earlier experiments), and only marginally better results for DDPM-OOD. In this setting, MSMA and DiffPath are competitive with each other.

In terms of function evaluations (FEs), MSMA is cheaper than DiffPath (10 vs 50) but this isn't a fundamental limitation of DiffPath: MSMA is a discrete noise level NCSN [3], while DiffPath utilizes the continuous-time diffusion formulation with DDIM sampling. DiffPath can be made more efficient by using better diffusion samplers like DPM-Solver [4]. 

Taken as a whole, the experimental results show strong performance for DiffPath, as it is competitive with baselines using roughly the same order-of-magnitude in FEs (or one/two magnitudes better FE versus DDPM-OOD/LMD), *while using a single model and without significant hyperparameter tuning*. We believe the last point to be most significant, as this work is the first to show this for generative OOD methods.
I thank the authors for their response. I increase my rating accordingly.
Thank you for your response! I still have a few outstanding concerns/questions:

1. Open-OOD results: Could you please describe the transformation pipeline (e.g., the sequence of image transformation you applied in order to get the 64x64 images) you used for ImageNet-200? 
2. One-class learning results: these results seem encouraging! Instead of arbitrary pairs of in-distribution and out-of-distribution classes, it will be more effective to consider semantically related categories for ID/OOD pairs. For example, see Table 8 of [1]. Here, if you consider the dogs subset, you may consider one of them as ID (e.g., beagle) and the remaining 11 dog breeds as OOD. This is a challenging setting while also having important practical applications -- so, demonstrating improvements in this case is going to be significant. I understand that this experiment requires time and compute resources and may be more suitable for a later revision.
3. Fair Comparison: Thank you for the additional experiments on this task. Comparing these numbers to the original table, it seems to confirm my speculation that comparing between pixelated images (i.e., upsampling C10/SVHN/C100) and higher-resolution images (e.g., downsampling Celeb-A) leads to over-optimistic results. For e.g., in Table 3, MSMA is shown to yield an AUROC of 0.871 for CelebA vs C10 when using a diffusion model trained at 32x32 resolution (is this correct?) and comparing between original C10 images and downsampled CelebA images. Next, in this new experiment using a diffusion model trained at 64x64 resolution, we notice that it achieves an AUROC of 1.0 and in my understanding, this can attributed to upsampling C10 images to 64x64.  While I understand that DiffPath uses a single model while baseline methods rely upon a separate model in each case, DiffPath also has access to a much stronger diffusion model since its trained on much more data. So, matching the results with the baseline methods is not very surprising; in fact, it also seems to me that MSMA achieves better results and requires fewer function evaluations as compared to DiffPath.

***
EDIT: in Q3, I was referring to an experimental setting where a diffusion model is trained on SVHN images and one of the SVHN classes (e.g., 0) is considered as in-distribution and others as OOD. It is similar to the one-class detection results for ImageNet already provided in the above response.  This experiment can evaluate if DiffPath can generalize to a diffusion model trained on much smaller data. Again, this may be more suitable for consideration in a later revision. 

[1] Ahmed and Courville. Detecting Semantic Anomalies.