Diffusion-based  Layer-wise Semantic Reconstruction for Unsupervised Out-of-Distribution Detection
Authors: Ying Yang
De Cheng
Chaowei Fang
Yubiao Wang
Changzhe Jiao
Lechao Cheng
Nannan Wang
Xinbo Gao
Venue: NeurIPS 2024 poster (2024)
Keywords: Out-of-Distribution detection
diffusion model
feature reconstruction
Primary area: safety_in_machine_learning

Abstract: Unsupervised out-of-distribution (OOD) detection aims to identify out-of-domain data by learning only from unlabeled In-Distribution (ID) training samples, which is crucial for developing a safe real-world machine learning system. Current reconstruction-based method provides a good alternative approach, by measuring the reconstruction error between the input and its corresponding generative counterpart in the pixel/feature space. However, such generative methods face the key dilemma, $i.e.$, improving the reconstruction power of the generative model, while keeping compact representation of the ID data. To address this issue, we propose the diffusion-based layer-wise semantic reconstruction approach for unsupervised OOD detection. The innovation of our approach is that we leverage the diffusion model's intrinsic data reconstruction ability to distinguish ID samples from OOD samples in the latent feature space. Moreover, to set up a comprehensive and discriminative feature representation, we devise a multi-layer semantic feature extraction strategy. Through distorting the extracted features with Gaussian noises and applying the diffusion model for feature reconstruction, the separation of ID and OOD samples is implemented according to the reconstruction errors. Extensive experimental results on multiple benchmarks built upon various datasets demonstrate that our method achieves state-of-the-art performance in terms of detection accuracy and speed.

Diffusion-based Layer-wise Semantic Reconstruction for Unsupervised Out-of-Distribution Detection
Ying Yang', De Cheng Chaowei Fang! Yubiao Wang',
Changzhe Jiao', Lechao Cheng", Nannan Wang', Xinbo Gao
Xidian University
Hefei University of Technology
CChongiing University of Posts and Telecommunications Cyycfq, ybwang_3) 3] xidian edu. chenglc@hfut. cerffut.edu.ccm en, gaoxbêcqupt edu. {dcheng, cwfang, cjiao, nnwang) @xidian edu, cn
edu. en
Abstract
Unsupervised out-of out-of-distribution (OOD) detection aims to identify out-of-domain data by learning only from unlabeled In-Distribution (ID) training samples, which is crucial for developing a safe real-world machine learning system. Current reconstruction-base method provides good alternative approach, by measuring the reconstruction error between the input and its corresponding generative coun- terpart the pixel/feature space. However, such generative methods face the key dilemma, i.e., improving the reconstruction power of the generative model, while keeping compact representation of the ID data. To address this issue, we propose the diffusion-based layer-wise semantic reconstruction approach for unsupervised OOD detection. The innovation our approach that we leverage the diffusion model's intrinsic data reconstruction ability to distinguish ID) samples from OOD samples in the latent feature space. Moreover, to set up a comprehensive and discriminative feature representation, extraction strategy. Through distorting the extracted features with Gaussian noises and applying the diffusion model for feature reconstruction, the separation of ID and OOD samples is implemented according to the reconstruction errors. Extensive experimental results multiple benchmarks built upon various datasets demon- strate that our method achieves state-of-the- performance in terms of detection accuracy and speed. Code available at httpss/ggihubb.om/b/y///.....
devise multi-layer semantic feature
Introduction
Unsupervised Out-of-Distribun (OOD) detection aims to identify whether data point belongs the In-Distribution (ID) or OOD dataset, by learning only from unlabeled in-distribution training samples. OOD detection plays vital role in developing safe real-world machine learning system, which ensures that the model only performed on data drawn from the same distribution as its training data. If the test data does not follow the training distribution, the model could unintentionally produce nonsensical predictions, resulting some misleading conclusions. Naturally, OOD detection is one of the key techniques for ensuring the model's robustness and safety.
Existing research studies the OOD detection mainly under two settings, i.e., supervised and unsu- pervised. The supervised OOD detection methods usually deem this task as binary classification problem, which relies on training with data labeled as OOD from disjoint categories or adversaries
Equation Contribution. 1 Corresponding authors: De Cheng and Chaowei Fang.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

[Hendrycks et al. 2018), Ming et al. 2022]. However, in many practical applications. it is almost impossible to access representative OOD samples, as the OOD data usually can be highly diverse and unpredictable. Therefore, we prefer to study the more challenging while practical unsupervised OOD detection problem. We will build an OOD detector trained solely on unlabeled ID data, as large amounts of unlabeled data are readily available and widely utilized due to their ease of acquisition.
Current reconstruction-bart based methods provide a good alternative approach for OOD detection, by measuring the reconstruction error between the input and its corresponding generative counterpart in the pixel/feature space. Obviously, the generative models and metric learning evaluation strategies are the main research directions. However, such methods of the generative models always face the following key dilemma: The projected in-distribution latent feature space should be compressed sufficiently to capture the exclusive characteristics of ID images, while it should also provide sufficient reconstruction power for the large-scale ID images of various categories. Existing generative-baser methods (e-g., auto-encoder (AB), variational AE [Kingma and Welling 2013) and Generative Adversarial Network(GAAN) [Goodfellow et al.) 2014), can not always fulfill these two requirements
simultaneously, and a good balance between them is required. Besides, recent OOD detection
methods based on diffusion models such as [Graham et al. 2023], [Gao et al. 2023 and (Liu
et al. 2023] often involve the pixel-level reconstruction of distorted images, which consume much traiing/inference time and computation resources.
To address the above sooee-menionee issues, and inspired by the latent space noise addition mechanism in Latent Diffusion Models (LDM) Rombach al. [2022, we propose the diffusion-based layer- wise semantic reconstruction approach for unsupervised OOD detection. Specifically, the proposed method makes full use of the diffusion model's intrinsic data reconstruction ability, to distinguish in-distribution samples from OOD samples in the latent feature space. In the diffusion denoising probabilistic models (DDPM) [Ho et al. 2020], the model is trained to incrementally remove noise from the noised inputs of different levels. Clearly, we can see that, instead of faithfully reconstructing inputs from the distribution it was trained on as previous VAE Kingma and Welling (2013) or GAN Goodfellow et al. [2014) the diffusion-based model shows more powerful reconstruction capabilities. Practically. our model involves reconstructing an input image feature from multiple values of the time step, this allows a single trained model to handle large amount of noise applied to the input, obviating the need for any dataset-specifie fine-tuning.
Moreover, to set up comprehensive and discriminative feature representation, we devise multi- layer semantic feature extraction strategy. Performing feature reconstruction on top of the multi-layer semantic features, encourages to restrict the in-distribution latent features distributed more compactly within a certain space, so as to better rebuild in-distribution samples while not reconstructing OOD comparatively. Overall, by distorting the extracted multi-layer features with Gaussian noises and applying the diffusion model for feature reconstruction, the separation of ID and OOD samples is implemented according to the reconstruction errors. Note that, the proposed Latent Feature Diffusion Network (LFDN) is built on top of the feature level instead of the traditional pixel level, which could significantly improve the computation efficiency and achieve effective OOD detection. The other potential strength of such a strategy is that it avoids the reconstruction of minor characteristics unrelated image understanding. In summary, the contributions of this work are as follows:
We propose diffusion-based layer-wise semantic reconstruction framework to tackle OOD detection, based on multi-layer semantic feature distortion and reconstruction. Meanwhile, We are the first to successfully incorporate generative modeling of features within the framework of OOD detection in image classification tasks.
The layer-wise semantic feature reconstruction encourages restricting the in-distribution latent features to be more compactly distributed within a certain space, enabling better reconstruction of ID samples while limiting the reconstruction of OOD samples.
Extensive experiments on multiple benchmarks across various datasets show that our method achieves state-of-the-art detection accuracy and speed.
2 Related Work
Existing researches study the OOD detection mainly under two settings: supervised and unsupervised. The Supervised method is generally based on classification. The method usually uses the maximum softmax probability [Hendrycks and Gimpel 2016) from the final fully connected (FC) layer as the score to judge the ID sample. But the classification-bet based OOD detection methods often encounter

---

70
7t
21
2p
p
Avg- Pe--Polling Z-wore Norm
Latent Feature
ft
Pixel space
8(:
F2
Diffusion Network
OOD Detection
Res Block
Book
Res Block
f
Guide
pM-1
M-1
iM-1
PM
Figure 1: Overview of proposed diffusion- based layer-wise semantic reconstruction framework for unsupervised OOD detection. It includes multi-layer semantic feature extraction, Diffusion-baid Feature Distortion and Reconstruction, and OOD detection head modules.
issues with assigning high softmax probability to OOD samples. Recent works Liu et al., 2020].
[Sun and Li 2022), [Djurisic et al., 2022], [Zhao et al., 2024]. attempt to alleviate this issue. The unsupervised OOD detection can be roughly categorized as the distance-basee metric evaluation and the generative- beasee reconstruction methods.
Distance- bestance-based methods assume that OOD data lies far from ID class centroids. [Ren et al. 2021
improved OOD detection by separating image foregrounds from backgrounds and computing the Mahalanobis distance for each, then combining them. [Sun et al. [022] used a non-parametric nearest neighbor distance for OOD detection. [Techapanurak
al. 20200 and [Chen et al., 2020) used cosine
similarity to measure distances between test data features of in in-distribution data to identify OOD data. (Huang et 2020] applied Euclidean distance, while (Gomes al.
2022] used Geodesic distance
for OOD detection. These methods often fail to capture sample distribution accurately.
Among the generative-barte based methods, the Likelihood-basee methods be traced back as early as [Bishop] 1994]. This method assumes that the generative model assigns high likelihood to ID data, while the likelihood for OOD data tends to be lower. Recently, several deep generative models have supported the computation of likelihood, such as VAE [Kingma and Welling, 2013). PixeICNN+ [Salimans
2017], and Glow [Kingma and Dhariwal] 2018). However, some studies ([aliiniik)) et al. 2018:: [Choi et al. 2018); Kirichenko et al. 2020 have found that probabilistic generative models might also assign high likelihood OOD data.
series of studics have attempted to mitigate this issue. (Serrà et al., 2019 explored the relationship between image complexity and likelihood values, which adjusted likelihoods based the size of image compression. (Ren et al., 2019) enhanced OOD detection by comparing likelihood values derived from different models. Another closely related approach highlights that these indicators not well suited for VAEs. [Xiao et al. 2020) proposed a specialized metric known as likelihood regret for OOD detection in VAEs. [Cai and Li 2023 suggested to leverage the high-frequency information of images to improve the model's 's ability to recognize OOD data. Additionally, a range of studies [Nalisnick et al.) 2019). [Wang al. 2020),
BBergamin et al. 2022). [Osada et al., [002]], have proposed typicality tests, estimating layer activation distributions and other statistical measures on training data, which are then evaluated through hypothesis testing or density estimation.
Another type of OOD detection methods leverage the idea that generative networks produce different reconstruction errors for ID and OOD data. Some methods such as (Sakurada and Yairi 2014). [Zong 2018). and [Zhou and Paffenroth] (017)), used auto-encoders to analyze reconstruction errors. GAN based methods [Schlegl et al.) 2017], [Zenati et al. 2018). and [Madzia- Maddzia-Madzou and Kuijf 2022 utilized reconstruction errors and discriminator confidence to detect anomalies. Recent works
et al.
[Gaham et al. 2023), [Gao al.! 2023]. and [Liu et al. 2023] applied diffusion models to model the pixel-level distribution of images, using errors from multiple reconstructions for OOD detection. Different from previous methods, we propose leverage diffusion models perform multi-layer semantic reconstruction in the latent feature space, not only for their stability in generation but also for significantly reducing training and inference time costs.
3 Method
Unsupervised OOD detection leverages intrinsic information from an unlabeled ID dataset DD train detector. Suppose D contains N images, namely ID {x, where denotes the i-th image.
3

---

The target is to learn an OOD detector denoted as S(-). which can effectively evaluate an OOD score for each input image. The judgment of whether the input image belongs ID or OOD is implemented by thresholding the OOD score. For example, given a testing image x, it is recognized as an IDD sample if the OOD score S(x) lower than the pre-defined threshold X; otherwise, recognized as an OOD sample.
In this paper, we propose a diffusion-basee layer-wise semantic reconstruction framework accom- plish the OOD detection task. Specifically, as illustrated in Figure the proposed framework consists of the following three components: the multi-laye -laee semantic feature extraction module, the latent feature diffusion stage, and the OOD detection head.
3.1 Multi-layer Semantic Feature Extraction
The proposed semantic reconstruction--asee method achieves OOD detection by measuring the reconstruction error between the input and its generative counterpart in the feature space. Specifically, devise multi-layer semantic feature extraction strategy. to set up comprehensive and discrimi- native feature representation for each input image. Such multi-layer features could better rebuild the samples and encourage the ID semantic features distributed more compactly within a certain space from different semantic layers.
Specifically, given image R3xaxh with and h being the width and height of the input image, passing through an image encoder E()). (e.g., EfficientNet [Tan and Le 2019). D. we can extract its feature maps from different layers (i.e., low-level high-level semantic blocks). The multi-layer intermediate feature map from the m-th block can be defined as F E IR Km xhm m E (1.... 3). where Cm. Wm and htm are the number of channels, width and height of the feature map FT, and M is the total number of intermediate feature maps. Then, each feature map Fm undergoes the global average pooling, obtaining the one-dimensional feature vector f E Rem Afterward, Z-- score normalization [Al Shalabi et al.) 2006) is applied to each feature vector fm resulting in pmm for the m-th intermediate feature vector fm of the input image where Var(f")
F"
the variance of f along the channel elements, and ò is small constant value. Finally, obtain the overall multi-layer feature vector for the input image x as: Zo H(() = [ ....,ff .'....... F E R by concatenating all the intermediate feature vectors, where Σ m=1 Cms and (xx denotes the whole feature extraction process.
3.2 Diffusion-based Feature Distortion and Reconstruction
Fitting the semantic feature distribution of ID samples crucial for identifying whether the input is an ID or OOD sample. However, it is difficult explicitly model the semantic feature space which has moderate complexity. Existing cenerative-based models [Zhou 2022], [Cai and Li 2023) address the modeling of complex data/feature space by transferring the orig- inal data/features into an implicit bottleneck space and learning a generator capable of recovering ID samples from the bottleneck space. Since the gen- erator can not generalize well in recovering unseen OOD samples, it can be used as the OOD detector. Inspired by this, we set up a fiffusion-basee feature distortion and reconstruction framework, considering the strength of diffusion models in data reconstruction. Our framework is innovative in the intro- duction of diffusion models in modeling semantic features, while previous works [Graham et al.
Time Embed
Linear
SiLU
Linear
Groupnorm
Grropporm
SiLUU
Linear
SiLU
Linear
Res Block
Figure 2: Residual Block Structure in LFDN.
2023). Liu et al. 2023], [Gao et al. 2023) focus on applying diffusion models for straightforward pixel-level distortion and reconstruction.
Semantic Feature Distortion.
The semantic feature distortion process can be conceptualized as transforming the semantic features into distorted counterparts with different levels of noise. For each step t belonging to [I...., ......,J. the

---

generation of data point Z₁ follows the formula:
2 ennoise(Za, t) = va, x 2o -1 x e, c~ C~N(00 rexe)
(1)
where R represents Gaussian noise vector; (:---) denotes the Gaussian distribution; 0 and rexe denote the c-dimensional zero vector and the identity matrix, respectively. oz is predefined noise level that controls the amount of noise added at each step.
Semantic Feature Reconstruction. For reconstructing the semantic features from their distorted counterparts, we build up Latent Feature Diffusion Network (LFDN) constituted by 16 residual blocks (ResBlock), as shown in Fig.
The structure of ResBlock is illustrated in Fig. Its residual branch is formed with two groups of Groupnorm [wu and He 2018). SiLU, and linear layers, as well as a MLP used for absorbing in the time embedding.
Following the calculation process of the denoising diffusion implicit model [Song et al. 2020), we employ LFDN to remove the noises injected into the semantic features with skipping step stride denoted as s. The detailed noise-removing process for 2₁ is described follows. 8 set to a value randomly selected from {1,2, ,t).
1) We first input ,, and the time embedding of t into LFDN, generating an initial reconstruction state denoted as ž. The calculation formulation can be summarized as: z, LFDN(Z,,)) t), where LF) denotes the feed-forward process of LFDN.
2) Afterwards, we estimate the noise correction vector for 2, denoted as é, as follows,
Z,
04 x 2
E
=
C2
where Ti, is the predefined noise level of the 1-th feature distortion step.
3) Then, sample the input (āv- for implementing the t'-th step's feature reconstruction where = max(t s,0) 0) as:
Zt -1-VI-ai E:) + V V1-ar σe - -o x é + +0/6,
Ze = =ar
(3)
Vaa
where of represents the variance of the additional noise at step t. Regarding 2 and time embedding of t' as inputs, LFDN predicts reconstruction results of the t'-th step as Z (Zr (Z),
4) Repeating steps 2 and 3 until L' = 0, yields the final reconstructed semantic features Zn-
We summarize the above process as 2μ = denoise t). This framework ensures that žu is not solely derived from the LFDN output but is continuously refined by DDIM, integrating detailed corrections to achieve high accuracy in reconstructing the original data from its noisy observations.
Objective Function. For optimizing the network parameters of LFDN, the mean square error is used as the loss function for pulling close the outputs of LFDN with the original semantic features. The calculation formulation is as follows:
Izo
LFDN(2,, t)ll
xED
During training, is randomly selected from (1,2, ,T). The detail is illustrated in Algorithm I
3.3 OOD Detection Head
Our approach can be integrated with three metrics to detect OOD data. Firstly, we utilize the Mean Squared Error (MSE) to measure the feature reconstruction eтror. Secondly, we use the Likelihood Regret metric (LR = MSEmaial MSE [Xiao et al. 2020), which quantifies the change in MSE from the initial epoch to the final epoch. This metric reflects the model's evolving certainty during training. Generally, the reconstruction errors for samples decrease as the model becomes more familiar with these samples, whereas the errors for OOD samples remain relatively stable.
5

---

Lastly, we employ the Multi-layer Semantic Feature Similarity (MFsim), t.e., the cosine similarity.
We assesses the cosine similarity between the original features Zo = [[ L" reconstructed features uo [ [ fm ..... fM] at various layers: Sim(f" Im)) 11 F
and the
The
OOD detection score MFsim, then computed as the negative average of these similarities: MFsim TT Σ mmm1 Sim(f", ) where M is the number of feature maps, A higher Mi score indicates greater likelihood of the data being OOD, Algorithm details the MFsim calculation. The flows for MSE and LR calculations are provided in Appendix A
Algorithm Training Algorithm
Algorithm 2 Testing Algorithm
1: Input: Train image x R3xhxx
1: Input: An image 3x
2: Zo
H(x)= = [ I"
ERR
2: Output: OOD score
3: repeat
3: 2o H(x) [ .....
p E ER
4: Draw t ~ Uniform (......T T}
4: Ze & ennoise 2o.
5: Draw ~ 10,
5: 2n denoise Zt, 1)
6:
Compute 2, and
[f] ..... fm
PM
Zu
7:
Z:
7 + /1 - The
7: for m 1 to M do
8:
L L= ExED I LFDN(Z,, 1)
Sm + Sim F fm)
9:
Update the parameters via the A optimizer.
9: end for
10: MFsim < Em==l Sm) /M
10: until convergence
11: return MFsim
Experiments
4.1 Datasets and Evaluation Metrics
Datasets: We train the OOD detection model on three in-distribution (ID) datasets: CIFAR-10
Krizhevsky et al. 2009, CIFAR-100 and CelebA (Liu et al., 2013). When testing models learned
on specific ID dataset, we select several datasets from SVHN [Netzer et al. 2011]. SUN [Xiao
al. 2010,, LSUN-c (Yu al 2013), LSUN-r, iSUN [u et al. 20155, iNaturalist Van Hom et
2018 Textures [Cimpoi et al. 2014.. Places365 (hhou et al. 2017], 7), MNIST (Degg 2012), FMNIST,
KMNIST [Clanuwat et al. 2018). Omniglot [L.kee et al. 2015], and NotMNiST as OOD data.
We employed the area under the receiver operating characteristic (AUROC) and the false positive rate at 95% true positive rate (FPR95) as evaluation metrics, Results in FPR95 metric are provided in Appendix C.I
Evaluation Metrics:
4.2 Implementation Details
We utilize EfficientNet-b4 [Tan and Le 2019 ResNet50 [He al. 2016) pre-trained ImageNet [Deng et al. 2009] as our encoder. The main text presents results using EfficientNet-b44 while results using ResNet50 are detailed in Appendix C.2 For EfficientNet-b4, we select feature maps from the first to fifth stages (M = 5) to construct the multi-layer semantic features, resulting in a feature dimension (c) of 720. The LFDN is consisting of 16 residual blocks. Inside each residual block, the number of groups in Groupnorm and the intermediate feature dimension of the residual branch are set to 1 and 1440, respectively. We employ the AdamW optimizer with weight decay of 10-4, Our method is trained on NVIDIA Geforce 4090 GPU for 150 epochs, with batch size of 128 and constant learning rate of 10-4 throughout the training phase.
4.3 Comparison with State-of-the-art Methods
Compared Generative-basel Methods: In Table regarding CIFAR-10 as the ID dataset, we
compare our method against pixel-level generative- methods including GLOW (Serrà et al. 2019]. PixeICNN++ [Serrà et al.] 2019). VAE [Xiao et al., 2020). and DDPM |Graaam et al. 2023 To validate the effectiveness of LFDN, we implement variant our method through replacing L.DN with AutoEncoder in which MFsim is used for estimating the OOD score. In comparison with the best

---

pixel-level method, VAE, our method achieves 9.1% improvement in average AUROC when using MFsim for OOD score estimation. Compared to DDPM, our method variants show a significantly improvement in average AUROC. For example, when integrated with MSE, our method achieves 20.4% higher AUROC than DDPM. This indirectly indicates that performing OOD detection the pixel level is much worse than performing OOD detection at the feature level. Generating pixels may reconstruct more content unrelated to the image's 's semantics, which may interfere the identification of OOD samples. Making the model focus on the reconstruction of compactly distributed semantic features benefits in separating ID and OOD samples. In terms of testing speed, our method is nearly 100 times faster than DDPM, significantly enhancing performance while reducing detection costs. Moreover, the final version of our method built upon LFDN improves average AUROC by 18.5% % compared to the variant basd on AutoEncoder, the diffusion model captures data distribution more effectively.
In Table we compare our method with VAE, DDPM and AutoEncoder, using CelebA as the II) dataset. Our method integrated with MFsim achieves state-of-the-an performances, with an AUROC improvement of 19.89% compared to DDPM, and the performance of the remaining two metries also far exceeds the baseline, demonstrating the generalizability of our approach.
Compared Classification-based and Distance-based Methods: In Table 3 we compare our method with classification- based methods including MSP [enndrycks and Gimpel 2016), EBO (Liu et al. 2020], DICE [Sun and Li, [202]] and ASH-S (Djurisic et al.) [022]. as well as sesuuce-bbsed methods including "SimCLR+Maaalann et al. 2022]. All methods are evaluated using EfficientNet-br b4 as the backbone. Compared to classification-bass and distance-based methods, our approach consistently shows a clear advantage. Specifically, for CIFAR- 100 as the in-distribution dataset, our method integrated with MFsim achieves an average AUROC of 13.84% higher than the classification-basee method DICE. Moreover, unlike classification-based methods, our approach does not require labeled data.
Distance' [iiao et al. 2021] and 'SimCLR+KNN' [Sun
The inference speed of our method based MSE or MFsim is faster than that of distance-based methods SimCLR+Mahh and SimCLR+KNN, because the computation of covariance matrix or K nearest neighbors occupies part of time. Our method is also comparable to classifier- based methods including MSP. EBO, DICE and ASH-S. This demonstrates the effectiveness of leveraging the strong ability of diffusion models to reconstruct original distributions from different noise levels for reconstructing low-dimensional features and performing OOD detection.
Table 1:
The AUROC values for OOD detection, where CIFAR-10 is used as the in-distribution dataset. The results are compared with generative-based methods. Higher AUROC values indicate better performance, with the best results highlighted in bold for clarity.
Dataset
Pixet-Generative-Bas
Teature-Generativ--Bass
ID
OOD
GLOW
PixeICNN++
VAE
DDPM
AutoFncoder
our(*MSF)
ours(+1.R)
ours(+
SVHN
73.7
95.9
97.3
97.3440.0
98.240.00
98.9-0.J 0.11
LSUN
21.3
64.0
40.3
68.2
81.5
97. 6..60
97.8+0..1
99
MNIST
85.8
96.7
99.9
83.2
95.8
99.4+0.0
98.910.11
99.9=0.00
CIFRA10
FMNIST
71.2
90.7
99.1
84.3
79.6
99.0:10.0
98.8.10.0
99.9-10.0
KMNIST
38.0
82.6
99.9
89.7
90.5
99.5=10.0
99.11:000
99,9=0.00
Onniglot
95.5
98.9
99.6
35,9
81.5
99.1:10.1
97.110011
99.9-10.0
NoIMNIST
53.9
82.6
99.4
88.7
81.6
99.8.10.1
99.5:10.0
99.9 0.0
average
84.2
90.6
78.2
98.8.10.1
98.5.10.1
99.7.10.1
Time
Num img/s (1)
38.6
19.3
0.7
11.4
1224.2
999.3
273.6
999.3
4.4 Ablation Study
Illustration of the generation ability of the diff'ssinn model on OOD detection. To demonstrate the evolution of the generative model's reconstruction capability for both ID and OOD samples before and after training, we compare the distributions of the MFsim scores at the first epoch and the final epoch in Figure(3 CIFAR-10 serves as the ID dataset, while the other six datasets listed in Table 3 are employed as OOD data. Our observations reveal that the diffusion model's reconstruction ability enhances across most datasets, with a notably more pronounced improvement for the in-distribution samples. This indicates that ID samples are reconstructed more effectively, thereby validating the efficacy of our method.
7

---

Table The AUROC values for OOD detection, where CelebA used the in-distribution dataset. The results are compared with generative- methods. Higher AUROC values indicate better performance, with the best results highlighted bold for clarity.
Dataset
Pixel-Gemerative--ars
Feature-Generaatie-BBssd
ID
OOD
VAE
DDPM
AutoEncoder
ours(+MSE)
ours(+LR)
ours(+MFsim)
SUN
95.89
83.41
32.90
99.98_10.01
97.15.10.02
99.9810.01
INaturalist
95.52
82.38
41.56
100+0.00
99.96:10.01
99.99=10000
CelebA
Textures
91.73
78.33
56.33
99.93:t0.02
98.51++0.02
9.09
Places365
97.58
76.25
35.90
99,96=00.01
97.47=0.033
99.98==0.00
average
95.18
80.09
41.67
99.97
98.27:0.02
99.98-10.01
Time
Num img/s (1)
18.7
10.2
1357.6
1033.8
290.4
1033.8
Table The AUROC values for OOD detection, where CIFAR-10/1001 is used as the in-distribution dataset. The results are compared with Classification-base and Distance-basee methods using EfficientNet-bà as the backbone. Higher AUROC values indicate better performance, with the best results highlighted in bold for clarity.
Melad
Nun ings
SVHN
LSUUN=
[SUNN=
ISUN
Testrnes
Placza 365
average
MSP
10903
56.55
96.17
91.00
92.23
55.95
977 90
94. 14.714
EBO
1060.5
96.79
97.34
94.42
91.64
96.30
98.34
96.31
Clasidier-used
DICE
1060-3
98.53
99:03
94.49
95.25
97.68
99.03
97.
94.13
CIRARIO
Distance-eeeed
SinCLR+XKN
oun(cMSE)
9606
97.31:0.02
07. 07.50.10.01
93.83
92 02.7810.01
100:0.000
90.96 00.6600000
96 06.010001
oun(eL.)
360.
98.22 02:2220002
07.84:20.2
05.37
94.3110.02
109:00.00
50.9120001
97 07:002
min--Mhimii
969.6
98.09 0.01
98.83 0..0
08.53 IU01
98.82/0.02
100:0.000
100 11,00
98.34 0.01
MSP
1000
37.55
84.00
72.09
71.51
54102
39.00
W0/70
1000
76.51
78.92
76
75.51
Clasifez--eccl
DICE
CIPARIO0
Distunce besed
SimC1 JRKKNN
SinC MMnCLR-MMahssmmeh
919.3
54.97
$1.49
$3.80
77.21
75.33
53.31
63.44
61.21
nun(-MSE)
060.6
$3:03:0.00 0100
56 08.8640.00
75:38:0.01
11.0930.002
90:09-0.00
59.97:00001
56 35:00.0
Genenative Based
oun(et.R)
360.2
$8.94:00.01 01
87
37.00:40.02
80.96:0.01
77.71::0:00
09:08:0001
50.02::000
89 17:10.01
(unt-MMhim)
900.5
93.90-0.01
99.14:0.00
0.01
94.48:10001
100:00.00
97.30
Performance variations across different sampling time steps: Figure 4 iillsstrates the variations average AUROC and FPR95 values different evaluation metries at various sampling time steps, using CIFAR-10 the ID data, with the final time step T = 100. It is observed that all metrics perform poorly at = 1 primarily due to minimal noise added, making z, too similar to Zo and thus, limiting the denoising capability of LFDN; both ID and OOD data are well reconstructed. As t increases to about 3-10 steps, the appropriate amount noise allows MSE, LR, and MFsim to reach optimal performances. However, Zu enlarges, with 2, gradually approaching random noise, thereby worsening the reconstruction differences between 2n and zo for both ID and OOD samples.
continues to increase, the difference between 2; and the original
CIFARIO test
CIFAR10 test
ISUN
40
ISUN
SVHN
SVHN
LSUN-r
LSUN-r
Density
30-
LSUN-c
Density
LSUN-c
Textures
Textures
20
Places365
20
Places365
10
0
-.995
85
-0
-0.65
-1.00
-0.90
-0.85
-0.80 -0.75
-0.65 -0.60 -0
The Milsim Score
The Mfsim Score
Figure The MFsim score distributions of the first epoch (left) and the last epoch (right)

---

- Dum(+MSF)
0.8
0.7
Ous(-1R)
0.6
Ouss--IFain))
0.7
Ohr(++MEE)
Ouns(+LR)
Our! Mi'im)
0.5
0.4
1-10
1-25
e-d0
10100
1-2
1:3
1:10
1:50
10100
Sumpling Timse
Sangling
Figure 4: CIFAR-10 dataset is the ID data, the six datasets listed in Table 3 used OOD data. The average AUROC and FPR95 for the three metrics are evaluated at different sampling time steps.
Table 4: Changes in Average AUROC Across Six Datasets listed in Table 3 for CIFARIO0 ID.
Metries
MSE
LR
MFsim
Linear
Lineur=720
Linear-1440
Linear=720
Linear=1440
Linear=720
Linear= 1440
Average
83.35
86.35
84.05
89.17
96.43
97.20
Number of Blocks
Number=8
1umber=16
Number=8
Number=16
Number=8
Number=16
Average
85.26
86.35
87.32
89.17
97.13
97.20
Comparison of MFsim across different feature scales. Figure displays performance compar- isons of MFsim when reconstructing the last block (i.e., f4. 448) versus multi-layer semantic features under an EfficientNet-be encoder. The results demonstrate that multi-layer semantic features generally outperform single-layer ones, indicating that multi-layer semantic features contain richer semantic information and are more representative of samples across different in-distribution datasets. Furthermore, considering the diverse semantic information represented by different layers, combing various layers of semantic features helps to boost the OOD performances of LFDN.
Ablation study LFDN network parameters. We conducted ablation experiments on groups of parameters within the LFDN network: the dimension of the linear layers and the number of ResBlocks. For each experiment, we reduced one of these parameters to half of its original size while keeping all other parameters unchanged. Table presents the results of these experiments, showing how these modifications affect the performance. It is observed that the performance of our MFsim metric remains relatively stable, indicating that it continues to provide effective OOD detection capabilities even under conditions of reduced network size.
CIFARIO
CIFAR100
CelebA
19190
15.00
71.00
40.90
AEROCCN)
66
32.00
65
73.00
75.00
93.51
10
94.00
01:00
7V
30.00
19000
X100
Sagle-ayer
LX
Mirire
NB0
1-K
Raye
Mhirte
Maln
Mhvins
Figure 5: Variation of Average AUROC Values across Different Scales
5 Conclusion and Limitation
In this paper, we propose diffusion-based layer-wise semantic reconstruction framework for unsu- pervised OOD detection. We leverage the diffusion model's intrinsic data reconstruction ability to distinguish in-distribution and OOD samples the latent feature space. Specially, the feature generation is built top of the devised multi-layer semantic feature extraction strategy, which

---

sets up a comprehensive and discriminative feature representation benefiting the generative OOD detection methods. Finally, we hope our proposed OOD detection method could make contributions to develop a safe real-world machine learning system. Additionally, it needs to point out that the performance of our method also relies on the quality of features extracted by the encoder. There- fore, selecting encoder with strong feature extraction capabilities is crucial for achieving good performances.
Acknowledgement
This work supported in part by the National Key R&D Program of China under Grant No. No.2023YFA 1008600, in part by NSFC under Grant NO.62376206, 62176198 and U22A2096, in part by the Key R&D Program of Shaanxi Province under Grant 2024GX- YBXM-1355, in part by the Key Laboratory of Big Data Intelligent Computing under Grant BDIC-2023-A-004,
Broader Impacts
Positive Societal Impacts: The proposed diffusion- layer-wise semantic reconstruction method for unsupervised out-of-distribution (OOD) detection can significantly enhance the security and safety of machine learning systems. By effectively identifying OOD data, the system can prevent incorrect or potentially harmful decisions, making Al applications more reliable in critical areas such healthcare, autonomous driving, and financial systems. This method increases the robustness of AI systems by ensuring they can handle unexpected inputs gracefully. This contributes to the overall stability and trustworthiness of AI deployments various industries, thereby promoting wider acceptance and integration of AI technologies. Negative Societal Impacts: As with any advanced detection method, there a risk that the technology could be misused. For instance, surveillance applications, it could be employed to monitor individuals without their consent, leading to privacy violations and ethical concern..
References
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 088...04666 2018.
Yifei Ming, Ying Fan, and Yixuan Li. Poem: Out-of-distribution detection with posterior sampling.
In International Conference on Machine Learning, pages 15650-15665. PMLR, 2022.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. orXiv:1312.000 14, 2013.
arXiv preprint
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL TRLpattss///pooceeiiii mww.eeididinnnnnn.nn..ee.e.e newrips...... cc/ Sapor_files/eee
Paper. .pfp
Mark S Graham, Walter HL Pinaya, Petru-Daniel Tudosiu, Parashkev Nachev, Sebastien Ourselin, and Jorge Cardoso. Denoising diffusion models for -iitt--iistiiuutin detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2947-2956, 2023.
Ruiyuan Gao, Chenchen Zhao, Lanqing Hong, and Qiang Xu. Diffguard: Semantic mismatch- gugidd dutt---distiibution detection using pre-trained diffusion models. In Proceedings of the IEFE//VV International Conference on Computer Vision, pages 1579-1589 2023.
Zhenzhen Liu, Jin Peng Zhou, Yufan Wang, and Kilian Q Weinberger. distribution detection with diffusion inpainting. In International Conference on Machine Learning, pages 22528 22538. PMLR, 2023.
Unsupervised out-of-
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pages 10684-10695, 2022.
10

---

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 15:6840-6851, 2020.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint 1w1.:::610.0211111 2016.
Weitang Liu, Xiaoyun Wang. John Owens, and Yixuan Li. Energy-based out-of-distribution detection.
Advances in neural information processing systems, 33:21464-21475, 2020.
Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision, pages 691-708. Springer, 2022.
Andrija Djurisic. Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. arXiv preprint arXiv:2209.09858 2022.
Qinyu Zhao, Ming Xu, Kartik Gupta, Akshay Asthana, Liang Zheng, and Stephen Gould. Towards op-
timal feature-shaping methods for out- off-isttriuuton detection. arXiv preprint arXiv:2402.00865 2024.
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshmi- narayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint 00X0022006.0902.. 2021.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning, pages 20827-20840. PMLR, 2022.
Engkarat Techapanurak, Masanori Suganuma, and Takayuki Okatani. Hyperparameter-ffee out-of- distribution detection using cosine similarity. In Proceedings of the Asian conference on computer vision, 2020.
Xingyu Chen, Xuguang Lan, Fuchun Sun, and Nanning Zheng. boundary based out-of-distribution classifier for generalized zero-shot learning. In European conference computer vision, pages 572-588. Springer, 2020.
Haiwen Huang, Zhihan Li, Lulu Wang, Sishuo Chen, Bin Dong, and Xinyu Zhou. Feature space singularity for diitt--distriuution detection. arXiv preprint arXiv:201 14654, 2020.
Eduardo Dadalto Camara Gomes, Florence Alberge, Pierre Duhamel, and Pablo Piantanida. Igcood: An information geometry approach to out-of-distribution detection. arXiv preprint arXiv:2203.07798, 2022.
Christopher M Bishop.
Novelty detection and neural network validation. IEE Proceedings- Vision, Image and Signal processing, 141(4):17-222, 1994.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelenn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv: 1701.055777 2017.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions.
Advances in neural information processing systems, 31, 2018
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? arXiv preprint arXiv: 180.0038, 2018, 2018.
Hyunsun Choi, Eric Jang, and Alexander A Alemi Waic, but why? generative ensembles for robust anomaly detection. arXiv preprint arXiv:1810.01392. 2018
Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-of-distribution data, Advances in neural information processing systems, 33:20578-20589, 2020.
Joan Serrà, David Álvarez, Vicenę Gómez, Gregory Slabaugh, and Isabel Diez. Input complexity and out-of-distribution detection with likelihood-basee generative models. Proceedings of the International Conference Learning Representations, 2019,
11

---

Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark DePristo, Joshua Dillon, and Balaji Lakshminarayanain Likelihood ratios for -uttrf--dittibbution detection. Advances Neural Information Processing Systems, 32, 2019.
Zhisheng Xiao, Qing Yan, and Yali Amit. Likelihood regret: An out-of-distribution detection score for variational auto-encoder. Advances neural information processing systems, 33:20685-20696, 2020.
Mu Cai and Yixuan Li. Out-of-distribution detection via frequency-rregularized generative models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5521 3521-5530, 2023.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-of- distribution inputs to deep generative models using typicality. arXiv preprint 2019.
Ziyu Wang, Bin Dai, David Wipf. and Jun Zhu. Further analysis of outlier detection with deep generative models. Advances in Neural Information Processing Systems, 33:8982-8992, 2020.
Federico Bergamin, Pierre-Ale Alexandle Mattei, Jakob Drachmann Havtorn, Hugo Senetaire, Hugo Schmutz, Lars Maaløe. Soren Hauberg. and Jes Frellsen. Model-agnostic out-of-distribution detection using combined statistical tests. In International Conference on Artificial Intelligence and Statistics, pages 10753-10776. PMLR, 2022.
Genki Osada, Tsubasa Takahashi, Budrul Ahsan, and Takashi Nishide. Out-of-distribution detection with reconstruction error and typicality-basel penalty. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pages 5551-5563, 2023.
Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with nonlinear dimen- sionality reduction. In Proceedings of the MLSDA 2014 2nd workshop machine learning for sensory data analysis, pages 4 4-1 2014.
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International conference on learning representations, 2018.
Chong Zhou and Randy C Paffenroth. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 665 674, 2017.
Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Schmidt-Erfurth, and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International conference on information processing in medical imaging, pages 146-157. Springer, 2017.
Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, and Vijay Ramaseshan Chan- drasekhar. Efficient gan- gan-based anomaly detection. arXiv preprint arXiv:1802.06222 2018.
Djennifer K Madzia-Madzou and Hugo J Kuijf. Progressive ganomaly: anomaly detection with progressively growing gans. In Medical Imaging 2022: Image Processing, volume 12032, pages 527-540. SPIE, 2022.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-6114 14. PMLR, 2019.
Luai Al Shalabi, Zyad Shaaban, and Basel Kasasbeh. Data mining: A preprocessing engine, Journal of Computer Science, 299)7735-73, 2006.
Yibo Zhou. Rethinking reconstruction autoencoder- based distribtiitiiiiionnnn detection. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 73877 2022.
Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European conference on computer vision (ECCV), pages 3-19, 2018.
12

---

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Proceedings of the IEEE international conference computer vision, pages 3730-3738, 2015.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng. al. Reading digits natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, page Granada, Spain, 2011.
Jianxiong Xiao, James Hays, Krista Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 3485-3499 -492. IEFE, 2010.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of scage image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 0ee.i15060000333 2015.
Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755, 2015.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8769-8778, 2018.
Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describ- ing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606-3613, 2014.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions pattern analysis and machine intelligence, 40(66) 2017.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6):141-142 2012.
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv: 18/200/1/ 2018.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266): 13322666:1332-33383 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248 255. leee, 2009.
Zhisheng Xiao, Qing Yan, and Yali Amit. Do we really need to learn representations from in-domain data for outlier detection? arXiv preprint 205.0:210500920 2021.
13

---

Appendix
A
Supplementary Algorithm
This section provides two key algorithms used for evaluating our approach: the MSE (Mean Squared Error) calculation and the Likelihood Regret (LR) calculation.
The MSE calculation, as shown in Algorithm B computes the mean squared error between the original and reconstructed latent features. It serves basic measure of reconstruction error for detecting OOD samples.
The LR calculation, detailed in Algorithm measures the reduction in reconstruction error by comparing the MSE values at the initial and final epochs of training. This metric reflects how well the model has adapted to the ID data over time, with higher reduction indicating better adaptation.
Algorithm 3 Testing Algorithm for MSE Calculation
1: Input: An image x
2: Output: MSE score
3: Zo H(x)
4: Z, ennoise(Zo)
5: Zo = denoise Zg: 1)
6: MSE +- 1/ > [zoli] 2old))²
i indexes the elements of zo and Zo
7: return MSE
Algorithm 4 Testing Algorithm for LR Calculation
1: Input: An image MSE at initial and final epochs
2: Output: LR score
3: zinitill H(x) the beginning of training
4: z mnitial <- ennoise initiall
5: aigitial denoise initial
6: MSEnnitiil f- >i=
20
initial [ initiall [i))
20
7: 2
final = H(x) at the end of training
8: zinal ennoise(2 (zfinnl) (2ffal))) Fnna denoise 2 final
hnal
9: 20
10: MSEffal <- N Di (z. hual
stgaal (1))
LR MSF ana
12: return LR
B More Experimental Details
B.1 Dataset Details and Testing Speeds
Table CIFAR-10 Dataset The CIFAR-10 test set consisted of 10,000 images. The SVHN dataset contained 26,032 images, LSUN-r had 10,000 images, and Fashion-MNIST. MNIST, and KMNIST each comprised 10,000 images. Omniglot included 13, 180 images, and notMNIST had 18,724 images, totaling 97,936 OOD samples. The testing of the MFsim metric took total of 98 seconds, with an average speed of 999.3 images per second.
CelebA Dataset The CelebA test set comprised 60,780 images, SUN included 10,000 images, iNaturalist had 100,000 images, Textures consisted of 1 1,678 images, and Places365 had 1,002 images, making up a total 112,680 OOD samples. Testing the MFsim metric took total of 109 seconds, processing average of 1033.8 images per second.
Table 2:
14

---

B.2
Training Details
Both CIFAR-10 and CelebA datasets were trained for 200 epochs using the VAE model. The GLOW model was trained for 150 epochs with a learning rate of x 10-4, and PixeICNN+ was trained for 150 epochs at the same learning rate. Under the DDPM model, both datasets 350 epochs, following the experimental setups and code provided in the original papers. We used LFDN without time-step embeddings our autoencoder, used MF metrics, and kept all remaining training details consistent with our approach.
trained for
C Supplementary Experiments
C.1 Experimental Results for FPR95 Values Using EfficientNet-h4 Backbone
We conducted tests to evaluate the FPR95 (False Positive Rate at 95% True Positive Rate) values using CIFARIO 10 and CIFARI 100 datasets in-distribution data while treating the remaining six datasets as out-of-distribution datasets. The specific FPR95 values are summarized in Table 5
Table 5: The FPR95 values for OOD detection, where CIFAR-10/100 is used as the in-distribution dataset. The results are compared with glassification-based and Distance-based methods using EfficientNer-b4 as the backbone. Higher AUROC values indicate better performance, with the best results highlighted in bold for clarity.
OOD
ID
Bused
Method
Num ingh (1)
LSSUNec
Touturce
Placca565
averuge
ERO
1050.5
11.52
23.01
17.42
Clesifizsteeall
11.55
AS33-5
1047.6
4.15
1.32
12.53
CIFARIO
Dissmnstraa
SinCI Mabaaamuuh
SmCLR-KKN
674.8
919.5
49.15
9:34
54,89
67.73
75.43
76.97
64.32
73.45
56.22
15.27
39.39
72.15
51.53
$7.52
(un(+MSF)) ou))++L
990.5
21.15-000
10 00.00+0.00
30.670000
7600000
0+1009
0.42-003
20:75-0000
Gescrativs-beall
300.2
14.26:00,02
18,67.10.00 10.00
31.220002
37 76100.022
0,06:0.001
01335:0022
17,20:10.02
Musss+Missmm)
950.5
4.34:9002
0.04::0.01
4.43:0002
6.26::0032
01:000
0::0,00
2.51.:9.01
Clasifter based
78.67
45.77
66.30
AS35-5
34.50
CIFARI00
Dissmnstraal
SmnC Ma-aaaaauusi
BeCCR-KNN
674.8
919.8
94.05
95.32
96, 35
97.11
85.05
78.45
84.38
86.29
95.28
$0.37
94.82
95.50
90.89
89.75
aun(+MFF)
990.5
89.05-0002
69 14+0000
69.61-0000
89 70+00.01
0.12--0.00
0-00.00
53.95-10.00
Gescratino-beall
umn(+1.)
360.2
60.06-0000
73. 10:40.04
85.5750002
86. 17++0.02
0,42:00.00
2:81:0002
51.72-0008
ours-MMisim)
990.5
37.49:00.02
1.99::9.01
23.05:9.002
26. 26.00::0.02
01:0.00
0:10,00
14.78:00.22
As shown in 5 our method demonstrates significant advantage in terms of FPR95 values compared to other classification-base and distance-based approaches. Specifically, when using CIFARION 100 as in-distribution data, our method achieves an average reduction of 36.93% in FPR95 values compared to the state-of-the-ae classification based approach, ASH-S.
C.2 Experimental Results with ResNet50 as Encoder
Besides using EfficientNer-b4 as the encoder, we also employed the commonly used ResNet50 to extract multi-layer semantic features. For ResNet50, feature maps from stages to 3 were selected, with channel counts of 256, 512, and 1024, respectively. These feature maps were concatenated to form 792-diimnsioonal vector, which was then used as input for the LFDN. The results for three OOD detection metrics are presented in Table 6 and Table 7 Both tables compare our method with classification-basee and distance-based methods.
As shown in Table and Table when using ResNet50 as the backbone, our method still achieves the best performance. Specifically, with CIFAR-I0 the in-distribution dataset, the average AUROC and MFsim values are 98.30% and 8.89% respectively, outperforming the classification based SOTA method DICE by 6.% in AUROC and reducing the FPR95 by 24.43%.
Figures and illustrate the differences the MFsim score distributions for various datasets, with ResNet50 as the encoder and CIFARIO 10 as the in-distribution dataset, across the first and last epochs.
15

---

Table 6: The AUROC values for OOD detection, where CIFAR-10/100 is used as the in-distribution dataset. The results are compared with Classification-be based and Distance- methods using ResNet50 as the backbone. Higher AUROC values indicate better performance, with the best results highlighted in bold for clarity.
OOI
Based
Method
Num (1)
SVHI
LSUN C
LSUN
ISUN
Testures
Places366
MSP
1321.5
80.64
95.05
91.65
$9.05
87.76
00:45
80.
EBO DICE
1321.5
81.90
95.21
94.43
93.09
85.84
92-45
91.15
Classiter
1369.3
91.92
99.18
91.13
50.51
$7.39
92.65
92.13
ASHI-S
1307.4
98.16
98.76
95.00
94.40
$7.88
91.63
91.97
CIFARIO
Distance based
SinCTRR-Muhalazoee
XinNCLK-XNN
1179
92.36
97.75
91.82
79.41
$9.52
81.40
81.11
$1.36
100.00
100.00
Generative based
ouns+1RR)
90.95
92.55
100.00
100.00
92.44
Puns->Mi'sng
95.9810.02
99,86.0.00
97.06620.02
96.39_00.0
100:00,00
100.10,00
96.30:00.01
MSP
13215
76.35
$4.38
75.95
78.09
76.54
72.00
78,03
EBO DICE
1321.5
83.13
$9.35
83.83
83.35
75.85
71.42
$1.66
1369.3
87.93
93.12
92.41
$2.20
79.29
09
$2.47
ASHL-S
1307.4
91.66
93.24
60.93
72.78
87.75
71.00
$1 10.06
91.93
57.18
87.47
38.00
91.95
71.85
Distance
SinCT
87.78
82.51
Generutive hased
89.17
100,00
100.00
95.72
39.68:00022
03.6420002
92.94:0002
90.91
Table 7: The FPR95 values for OOD detection, where CIFAR-10/16 is used as the in in-distribution dataset. The results are compared with Classification- based and Distance-based methods using ResNet50 as the backbone. Higher AUROC values indicate better performance, with the best results highlighted in bold for clarity.
QOD
Based
Melhnd
Num ingl (T)
SVHN
LSSUN:
ISSUN+
ISUN
Testure
Plasect65
MSP
13215
60.56
16.08
28.21
35.63
45.4
32.93
K.07
Claseifer-bueed
17077
ASELS
1307.4
53.03
5.57
22.31
64.36
34.36
CIPARIO
Bistne--hsed
SimC3l R-Maaahaaahis
SinCTI
24.53
(in(+MSF)
654.
94.99
45,0
91.63
0.00
0,00
$4.72
Guneneiss-haee
mundal R))
296
14.99
25,73
66.54
$2.05
0.00
0.00
76 16
ouss-+MEsino
654.9
21.10::005
0.121:011
15.75:0.03
16.48:0.044
0.0910.00
0.00:0..00
$,89 1.8920..2
MSP
13015
$3.98
43.50
56.23
57 09
61.63
77.53
58.66
FRO
13215
47004
$4.15
$1.14
51. 96
61.05
$0.95
$4.78
DICE
1369
38.7
25.73
56.21
56.74
65:21
$2.63
$4.71
ASE S
1307.4
2933
25.25
$9.48
85.22
51.3
$1,48
51.09
CIBAR100
Battnnc-hased
SincCLR-KNN
74.99
57.15
Gunennine-hsee
ouns(1LR)
65.93
1.57
38.78
0.00
0.00
23,67
ouns-+Misino
64.721:005
1.0810002
39
39.06:10.04
39.79:60.03
0.08:6.00
0.00.00.00
24.21
40
CIFAR10
CIRAR1O
35
ISUN
ISUN
LSUN-r
100
LSUNTT
25
LSUN-c
80
LSUNic
Textures
Textures
60
Places365
10
40-
20-
-1.00
-0.95
-090
-0.85
-030
-0.75
-0.70
-085
-0.60
-1.00
-0.95
-0's0
-0.85
-0.0
-0.75
-0.70
-0'es
-0.50
The MFzim Store
The MFsim Store
Figure 6: The MFsim score distributions of the First Epoch with ResNet50 Encoder
Figure 7: The MFsim score distributions of the Last Epoch with ResNet50 as Encoder
16

---

C.3 CIFAR-10 as ID and CIFAR-100 OOD
As shown in Table g when CIFAR-I0 used as ID dataset and CIFAR-100 as the OOD dataset, method consistently achieves the best performance across different evaluation metrics, including AUROC and FPR95. Compared to classification based methods, the improvement is not significantly large, but approach still shows consistent edge, particularly in feature-generative generative-based models, demonstrating the robustness of our method.
Table 8: The FPR95 and AUROC Values for CIFAR-10 ID Samples and CIFARIO0 as OOD Samples.
OOD
Based
Method
FPR95
AUROCT
MSP
52.04
86.14
Classification Based
EBO
51.32
86.19
ASH-S
51.29
87.13
GLOW
73.60
CIFARIO
CIFARI00
Pixel-Generraiive Based
VAE
90.41
55.95
DDPM
67.38
82.43
ours(+MSE)
48.87
87.54
Feature-Generraiie--Bsss
ours(+LR)
49.48
87.24
ours(+MMFsim)
53.70
85.60
C.4 Comparisons with recent generative methods
The comparison between our method and Table Our method outperforms DDPM consistently on benchmarks using CIFAR10 or CelebA as ID data.
et al.
2023 can be referred to Table and
The comparison between method and Diffuard Gao et al. of Diffuard are taken from its original paper. Here, CIFARIO is regarded as ID data, while CIFAR100 Tinylmagenet regarded OOD data. Our method based on MFsim achieves overall better performance than 'Diffuard+Deep Ens', with 1.55 higher AUROC and 21.77 lower FPR95.
2023 is provided in Table 9 Results
The comparison between our method and LMD et al. 2023 is shown in Table 10 The evaluation metric AUROC. The average AUROC of our method based on MFsim is 6.94 higher than that of LMD.
Table 9: The AUROC and FPR95 values compared to DiffGuard [ao et al. as the dataset and CIFAR-100/Ting Imageee as the OOD datasets.
2023
using CIFAR-10
CIFAR-100
TINYIMAGENET
average
Method
AUROC1
FPR95
AUROCT
FPR95.
AUROCT
FPR95
Diffuard Diffuard+EBO
89.88
52.67
91.88
45.48
90.88
49.08
89.93
50.77
91.95
43.58
90.94
47.18
Diffuaed+Deep Ens
90.40
52.51
91.98
45.04
91.19
48.78
ours(+MSB)
87.54
48.87
97.68
13.42
92.61
31.15
ours(+LR)
87.24
49.48
97.11
15.04
92.18
32.26
ours(+MFsim)
85.60
53.70
99.88
0.39
92.74
27.01
C.5
Comparison against other methods using the multi-scale feature encodings as the input.
In Table II we have made comparison of our method against AE and VAE using the multi-layer feature encodings as inputs. For AB (AutoEncoder), we use the LFDN network without the timestep embedding, i.c., 16-layer linear network. For VAE,
5-layer linear network the encoder
17

---

Table 10: The AUROC values compared LMD Liu the ID dataset and
2023 using
as
as the OOD datasets.
ID
OOD
LMD
ours(+MSE)
ours(+LR)
ours(+MFsim)
CIFARI00
60.70
87.54
87.24
85.6
CIFARIO
SVHN
99.20
97.31
98.22
98.89
CIFAR10
56.80
70.52
72.86
64.58
CIFARIO0
SVHN
98.50
83.93
88.84
93.9
AVERAGE
78.80
84.83
86.79
85.74
and 8-layer linear network the decoder. Compared AE and VAE, the diffusion model has significant advantages when modeling complex multidimensional distributions.
C.6 Comparisons with pixel-level denoising approaches.
We provide the distribution differences of the MSE score and MFsim score at two levels after training, with CIFAR-10 10 as ID dataset and other datasets as OOD; The results are shown in Figures 8 and
It can be observed that at the pixel Ievel the reconstruction error distributions of ID and OOD samples are very similar. The mixed MSE scores make very hard distinguish ID samples from OOD samples. However, at the feature level, the reconstruction score distribution of ID samples shows clear distinction from that of OOD samples. The reason that, our feature-level diffusion- based generative model makes the projected in-distribution latent space not only be compressed sufficiently to capture the exclusive characteristics for the large-scale ID images of various categories. In other words, the pretrained encoder has inherent generalization capabilities, and the multi-layer features it extracts are more discriminative than the high-dimensional pixels of the images themselves.
ID images, but also provide sufficient reconstruction power
Table 11: The AUROC values compared the other generative models using CIFAR-IO as the ID dataset.
Dataset
Method
ID
OOD
AE(+MFsim)
VAE(+MFFsim)
Diffusion(+MFsii)
SVHN
57.68
83.96
98.89
LSUN
81.47
97.69
99.83
MNIST
95.85
99.98
99.99
CIFRA10
FMNIST
79.61
98.69
99.99
KMNIST
90.51
99.96
99.99
Omniglot
81.50
97.69
99.99
NotMNIST
81.61
99.88
99.99
average
81.18
96.84
99.81
Time
Num img/s (1)
1224.2
1179.4
999.3
C.7 ImageNet100 as ID Dataset
In Table 12 our method using MSE outperforms the classification-bar based SOTA method DICE, achieving an improvement of 3.91% in AUROC when ImageNet100 is used the ID dataset and various datasets such as SUN, iNaturalist, Textures, and Places365 are used as OOD datasets. The significant improvements performance metries demonstrate that our generative-bee can effectively model the in-distribution characteristics, leading to better OOD detection capabilities. This indicates that our proposed method is particularly suitable for more complex datasets like ImageNet100, where capturing detailed features is crucial for accurate OOD detection.
approach
18

---

Table 12: The AUROC and FPR95 Values for Different Methods with ImageNet100 as ID Dataset and Www.Naauralss/Txxtuus//PPPeeeee as OOD Datasets.
(1)
INotoralist
Testures
Average
AURUCT
FPR05
AUROC?
FFRMS
AUROCT
FFR955
FPR95
AURUCT
FPRUS]
956
956.2
90.83
87.83
DICE:
950.
93.78
91.56
1m0
ASHLS
977.3
90.12
04.34
80.94
31.45
$3.38
45.23
88.76
49.70
53.55
47.73
OaSE--MSE)
954.9
98.81
5.91
86.59
71.85
97.08
10.76
98.79
5.73
95.47
23,57
mars(-1RR)
313.4
93.98
16.51
51.30
87. 09.99
76.02
47.54 54
94.05
16.25
79.09
42.07
nars(-MPssnn)
954-9
98.49
8.41
84.63
77.86
90L0X
9.98
98.37
8.75
94.89
26.25
SVHN
40-
ISUN
LSUN-r
Dersity
LSUN-c
Textures
20
Places365
10-
0.85
-0.90
-075
The MS2 Store
The
Figure 8: Reconstruction Error Distribution of ID and OOD Samples for Pixel-level
Figure 9: Reconstruction Error Distribution of ID and OOD Samples for Feature-level
D Qualitative results.
The first type, shown in Figure 10 represents ID samples misclassified as OOD. can be observed that these misclassified samples often have significant shadows and lack semantic information, resulting in high reconstruction errors and being incorrectly classified as OOD samples. The second type, shown in Figure [1] represents OOD samples misclassified as ID. It can be observed that these OOD samples have categories very similar those the ID samples (CIFAR-10), such as cars ships, which are categories present in CIFAR-10. 10. The third type, shown in Figure [2 represents OOD samples with colors very similar to the ID samples, leading to their misclassification ID)
We have included three types of failure cases Figures 10 I1 and 12
Figure 10: Examples of ID Samples Misclassified OOD (Lacking Semantic Information).
Figure 11: Examples of OOD Samples Misclassified as (Similar to Sample Categories).
Figure 12: Examples of OOD Samples Misclassified as ID (Similar to ID Sample Colors),
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer:
[Yes] Yes
Justification: The main claims in the abstract and introduction accurately reflect our con- tributions. We propose a diffusion-based layer-wise semantic reconstruction method for unsupervised out-of-distribution (OOD) detection. Our method demonstrates superior performance in detecting OOD samples, as detailed in Section and Section 4pf our paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes] Yes
Justification: The limitations of our work are discussed in detail in Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper. but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
20

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA] N/A
Justification: Our paper focuses on an experimental approach to out-of-distribun detection and does not include theoretical results. Therefore, this question applicable.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes] Yes
Justification: Our paper fully discloses all necessary information to reproduce the main experimental results, including detailed descriptions of the experimental setup, datasets used, and evaluation metrics. This information is provided in Sections 4 of the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
21

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes] Yes
Justification: We have provided open access to our data and code, along with detailed instructions for reproducing the main experimental results. These resources are described in the supplemental material and can be accessed via the provided links.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
public//uiides/CoeSttnii www.sooabsssiioooooooooooo.o. for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips https///ccccllicgggeeeeeeeeeeeee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes] Yes
Justification: Our paper specifies all necessary training and test details, including data splits, hyperparameters, and optimizer settings. These details are provided in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes] Yes
22

---

Justification: We have reported error bars for our main experimental results, calculated as the mean and standard deviation over three runs. Details on the calculation of error bars and the factors of variabilityy considered (such train/test split and random initialization) are provided Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes] Yes
The paper provides detailed information on the compute resources used for the experiments, including the type of compute workers (GPU), memory, and execution time. These details are specified in the experimental setup section 4
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:pppps...poap.....ooopp/teee
Answer: [Yes] Yes
Justification: We have thoroughly reviewed the NeurIPS Code of Ethics and confirm that our research conforms to these guidelines in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
23

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes] Yes
Justification: We discuss the potential positive and negative societal impacts of our work in Section[] Specifically, we address how our method could improve unsupervised out-of- distribution detection, well the potential risks associated with misuse surveillance applications.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation. generating fake profiles, surveillance), fairness considerations (e.g.. deploymentt of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA] N/A
Justification: Our paper does not involve the release of data or models that have high risk for misuse. Therefore, this question is not applicable.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
24

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes) Yes
Justification: We have properly credited the creators and original owners of the datasets and models used in our work. The licenses and terms of use are explicitly mentioned in Section of our paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes] Yes
Justification: We have introduced new assets in the form of original code, and they are well documented. Detailed documentation is provided alongside the assets to ensure reproducibil- ity and ease of use.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots. if applicable, as well as details about compensation (if any)?
Answer: [NA] N/A
Justification: Our paper does not involve crowdsourcing nor research with human subjects. Therefore, this question is not applicable.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
25

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA] N/A
Justification: Our paper does not involve crowdsourcing nor research with human subjects. Therefore, this question is not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
26


Reviews:
[{"review_id": "oXMW3URDku", "reviewer": "Reviewer_yKR8", "summary": "This paper proposes a diffusion-based layer-wise semantic reconstruction strategy for unsupervised Out of Domain (OOD) detection. Specifically, they leverage the intrinsic data reconstruction ability of the diffusion model to differentiate between the In-distribution (ID) and OOD samples in the feature space. The features/data reconstruction at the multi-layer feature spaces helps the generative OOD detection.  The experiments suggest the superiority of the proposed approach compared to existing approaches.", "questions": "The questions are there in the weaknesses section, specifically, experimental evidence of points 2-6 in weaknesses section will significantly improve the completeness.", "limitations": "The authors mentioned about the limitations related to backbone architecture. However, the large models pre-trained on large datasets are more generalised. So, how this generalisation will effect the models performance?", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper proposes a novel scheme for unsupervised OOD detection based on semantic features  reconstruction at multiple layers.\n\n- Building the diffusion network on top of multi-level features instead of pixel-level outputs, helps in better preserving the ID information. \n\n- State-of-the-art performance on benchmark datasets.", "weaknesses": "1- The paper needs a writeup revision, for example the contributions and idea of the paper is repeated multiple times unnecessarily.\n\n2- The third step of the proposed approach, I.e., OOD detection step, is not clear. The authors defines three metrics for OOD detection based on some threshold, however, the paper never mentioned those values nor provided any experimental  study, that how were they selected.\n\n3- The comparison with recent generative methods based OOD is not provided. Instead only VAE is results are compared, despite there are some approaches available ([Graham et al., 2023], [Gao et al., 2023] and [Liu  et al., 20]). \n\n4- A comparative analysis between the pixel level and feature level denoising is not performed, which is the base of the proposed approach. \n\n5-The authors needs to clearly indicate that how the proposed approach is better than the existing pixel level approaches, both theoretically and experimentally. \n\n6- The proposed approach, and most of the methods cited in this paper experimented with classification problems. However, they build  their motivation and need based on the natural scenarios. In natural scenarios, the input images may be more complex and the task may be more difficult than classification only. A study is required to evaluate the proposed approach on complex scenes, object detection and semantic segmentation datasets, having more information instead of having a single primary object.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1. Writeup revision.   \n\nA: Thank you for your valuable feedback. We appreciate your observation regarding the repetition of the contributions and the main idea of the paper. We will address these issues to ensure clarity and conciseness.  \n\nQ2. Clarification of OOD detection step.  \n\nA: During inference, samples having higher reconstruction errors measured by MSE, MFsim, or LR  are more likely to be OOD samples.  FPR95 indicates the false positive rate when the true positive rate reaches 95\\%. When calculating FPR95, the threshold is determined by the measurement value where the true positive rate (TPR) reaches 95%. AUROC measures the area under the ROC curve. When calculating this metric, we vary the true positive rate from xxx to xxx in step of xxx to choose the threshold. In practical usage, we can select the threshold value with the OTSU algorithm. We report the F-scores of different methods using OTSU to determine the threshold value as follows:  \n\n|ID|Method|SVHN|LSUN-c|LSUN-r|iSUN|Textures|Places365|**average**|\n|-|-|-|-|-|-|-|-|-|\n|CIFAR10|MSP|48.20|80.88|75.74|75.85|81.75|75.90|73.05|\n||EBO|46.76|92.72|81.93|81.30|82.53|81.17|77.74|\n||ASH-S|67.44|93.55|90.99|91.33|83.12|82.39|84.80|\n||ours(+MSE)|79.21|91.04|83.16|82.93|97.43|97.43|88.53|\n||ours(+LR)|86.18|91.85|85.74|85.03|97.43|97.43|90.61|\n||ours(+MFsim)|91.91|97.42|95.20|94.71|97.43|97.43|**95.68**|\n|CIFAR100|MSP|47.92|72.47|70.29|72.08|78.51|68.02|68.22|\n||EBO|46.81|75.34|70.91|72.39|78.48|67.84|68.63|\n||ASH-S|44.35|74.83|67.94|69.86|78.20|68.79|67.33|\n||ours(+MSE)|49.80|73.75|67.56|70.11|97.43|97.43|76.01|\n||ours(+LR)|52.39|74.05|68.75|71.13|97.43|97.42|76.86|\n||ours(+MFsim)|65.48|96.52|87.39|87.20|97.43|97.43|**88.58**|\n\nQ3. Comparisons with recent generative methods based OOD  \n\nA:  The comparison between our method and DDPM [Graham et al., 2023] can be referred to Table 1 and 2 of our paper.  Our method outperforms DDPM consistently on benchmarks using CIFAR10 or CelebA as ID data.   \n\nThe comparison between our method and Diffuard [Gao et al., 2023] is provided in the table below. Results of Diffuard are taken from its original paper. Here, CIFAR10 is regarded as ID data, while CIFAR100 or TinyImagenet is regarded as OOD data. Our method based on MFsim achieves overall better performance than ‘Diffuard+Deep Ens’, with 1.55 higher AUROC and 21.77 lower FPR95. \n\n|Method|CIFAR-100 AUROC↑|CIFAR-100 FPR95↓|TINYIMAGENET AUROC↑|TINYIMAGENET FPR95↓|average AUROC↑|average FPR95↓|\n|-|-|-|-|-|-|-|\n|Diffuard|89.88|52.67|91.88|45.48|90.88|49.08|\n|Diffuard+EBO|89.93|50.77|91.95|43.58|90.94|47.18|\n|Diffuaed+Deep Ens|**90.40**|52.51|91.98|45.04|91.19|48.78|\n|ours(+MSE)|87.54|**48.87**|97.68|13.42|92.61|31.15|\n|ours(+LR)|87.24|49.48|97.11|15.04|92.18|32.26|\n|ours(+MFsim)|85.6|53.7|**99.88**|**0.39**|**92.74**|**27.01**|\n\nThe comparison between our method and LMD [Liu et al., 2023] is shown in the following table. The evaluation metric is AUROC. The average AUROC of our method based on MFsim is 6.94 higher than that of LMD. \n\n|ID|OOD|LMD|ours(+MSE)|ours(+LR)|ours(+MFsim)|\n|-|-|-|-|-|-|\n|CIFAR10|CIFAR100|60.70|87.54|87.24|85.6|\n||SVHN|99.20|97.31|98.22|98.89|\n|CIFAR100|CIFAR10|56.80|70.52|72.86|64.58|\n||SVHN|98.50|83.93|88.84|93.9|\n||**AVERAGE**|78.80|84.83|**86.79**|85.74|\n\nQ4. Comparisons with pixel-level denoising approaches. \n\nA: The quantitative comparisons between our method and pixel-level approaches are as follows:  \n\nWe provide the distribution differences of the MSE scores at two levels after training, with CIFAR-10 as the ID dataset and other datasets as OOD; The results are shown in Figure A of the PDF attached in the global rebuttal.\n\nIt can be observed that at the pixel level, the reconstruction error distributions of ID and OOD samples are very similar. The mixed MSE scores make it very hard to distinguish ID samples from OOD samples. However, at the feature level, the reconstruction score distribution of ID samples shows a clear distinction from that of OOD samples. The reason is that, our feature-level diffusion-based generative model makes the projected in-distribution latent space not only be compressed sufficiently to capture the exclusive characteristics of ID images, but also provide sufficient reconstruction power for the large-scale ID images of various categories.\n\nQ5. Focusing on Classification Problems\n\nA: Thanks for the valuable comment. Our experiments, along with most of the cited works, focus on classification problems. This focus is largely because the field of OOD detection has traditionally concentrated on classification tasks. However, we believe our method has the potential to extend beyond classification. Our approach can serve as an effective data pre-processing step for more complex tasks like semantic segmentation and object detection. By identifying and filtering out OOD inputs, our method can enhance data security and improve the performance of these models. Additionally, our method can be integrated into semantic segmentation and object detection pipelines to judge whether detected or segmented object proposals are out of distribution.\n\nQ6. Effect of generalisation on model performance. \n\nA generalised backbone model is crucial for ensuring the extracted features are comprehensive and representative of the samples. When modeling in multi-scale feature spaces, it's essential that the features extracted are thorough and meaningful. Small-scale models may have shallower layers, which might not fully capture the complexity of a sample. The generalization ability of large models allows them to extract a more complete and representative set of features, which is essential for the effective performance of our OOD detection method. This ensures that the model can adequately generalize across various inputs, leading to a more robust and reliable detection system.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l1n9mi0B8A", "reviewer": "Reviewer_ch15", "summary": "The paper addresses the problem identification of out of distribution models in an unsupervised manner. The idea is that OOD samples have the largest reconstruction error. The method is evaluated on a plethora of setups including SVHN [Netzer et al., 2011], LSUN(and variants) iNaturalist, Textures, Places365.", "questions": "None. Questions I had (e.g. other models as encoder) have been answered in extended material.", "limitations": "not the case", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The theme is interesting. Identification of the OOD samples has many applications. Limiting factor is that the paper tests in general and synthetic setups for OOD and it does identifies a particular scenarios with OOD samples. Yet compared to other papers which focuses on synthetic evaluation, the setup in this paper is realistic.\n- The paper contains innovation, although is somehow limited. To my best knowledge, which is somehow similar with prior work as presented in the paper, the idea to use reconstruction error as a measure of how OOD is a sample,  is indeed novel. Limiting factors lies for instance in triplet loss, where the network is forced to do opposite: adjust weights such that samples which initially were OOD, are pushed towards the middle of the distribution. Yet turning a weak point of a training algorithm in a strong point of a different problems is, in my view, interesting and novel.  \n- Evaluation is strong: comparison is with strong previous works and in many setups. The evaluation is carried benchmark derivied from 10 datasets and compared against 7 strong solutions. In my view, it is far beyond minimum evaluation required to prove that a method is working.\n- ablation is informative. I particularly appreciate the experiment \"before and after training\"", "weaknesses": "- at the core, the idea is that reconstruction error points to OOD. This is new, but is in a broader family (including AE, contrastive leraning), thus some limited\n- the limitation sections points only to the strength of the encoder. However the assumption that OOD samples lead to large reconstruction error is also a prior assumption (although a strong one given the evaluation), but one may identify cases where is not true.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1.at the core, the idea is that reconstruction error points to OOD. This is new, but is in a broader family (including AE, contrastive leraning), thus some limited. \n\nA: Thanks for the insightful comment. We agree that the core idea of using reconstruction error to indicate OOD is indeed related to broader methodologies, including Autoencoders (AEs) and contrastive learning. However, we believe our approach introduces a novel perspective and significant innovations in this field. Our overall framework is innovative in that it applies diffusion models to multi-scale feature layers, establishing reconstruction error from this unique vantage point. This new approach attempts to leverage the strengths of diffusion models in capturing and reconstructing complex data distributions, setting it apart from traditional AE and contrastive learning methods. We appreciate the recognition of the novelty within the broader context and are confident that our contribution offers valuable advancements to the OOD detection domain.  \n\n  Additionally, we compared the performance of AE and Diffusion for multi-scale feature modeling to observe their performance differences. For AE (AutoEncoder), we use the LFDN network without the timestep embedding, i.e., a 16-layer linear network. The results are as follows: \n\n|ID|OOD|AE(+MFsim)|Diffusion(+MFsim)|\n|-|-|-|-|\n|CIFRA10|SVHN|57.68|98.89|\n||LSUN|81.47|99.83|\n||MNIST|95.85|99.99|\n||FMNIST|79.61|99.99|\n||KMNIST|90.51|99.99|\n||Omniglot|81.50|99.99|\n||NotMNIST|81.61|99.99|\n||average|81.18|**99.81**|\n|Time|Num img/s (↑)|1224.2|999.3||\n\nThis further demonstrates the advantage of using diffusion for feature modeling. \n\nQ2.the limitation sections points only to the strength of the encoder. However the assumption that OOD samples lead to large reconstruction error is also a prior assumption (although a strong one given the evaluation), but one may identify cases where is not true.\n\nA: Thanks for the valuable feedback. We acknowledge the point regarding the prior assumption that OOD samples lead to large reconstruction errors. While this is generally a prior  assumption supported by our evaluation, there are indeed scenarios where it may not hold true. Some difficult in-distribution (ID) samples may exhibit large reconstruction errors, and certain OOD samples similar to ID samples may have smaller reconstruction errors. The reconstruction errors for ID and OOD samples after training, as shown in Figure 3, also indicate this point. There is a small overlap between the reconstruction errors of some ID and OOD samples. \n\nWe also considered the detection performance of our method when the reconstruction errors are quite close. For example, the ID dataset is from CIFAR10, and the OOD dataset is CIFAR100 which shares similar distributions with CIFAR10. Below are our results: \n\nCIFAR-10 as ID and CIFAR-100 as OOD: \n\n|ID|OOD|Method|FPR95↓|AUROC↑|\n|-|-|-|-|-|\n|CIFAR10|CIFAR100|MSP|52.04|86.14|\n|||EBO|51.32|86.19|\n|||ASH-S|51.29|87.13|\n|||GLOW|➖|73.60|\n|||VAE|90.41|55.95|\n|||DDPM|93.21|54.00|\n|||ours(+MSE)|**48.87**|**87.54**|\n|||ours(+LR)|49.48|87.24|\n|||ours(+MFsim)|53.70|85.60|\n\nCIFAR-10 as ID and noisy CIFAR-10 as OOD: \n\n|ID|OOD|Method|FPR95↓|AUROC↑|\n|-|-|-|-|-|\n|CIFAR10|CIFAR10(add noise)|MSP|90.23|61.63|\n|||EBO|88.01|62.49|\n|||ASH-S|86.95|62.16|\n|||VAE|89.20|51.76|\n|||DDPM|95.80|45.00|\n|||ours(+MSE)|7.84|98.43|\n|||ours(+LR)|16.27|96.76|\n|||ours(+MFsim)|**0.14**|**99.97**|\n\nIt can be observed that our method still achieves the best performance in these two specific scenarios. This validates the reasonableness of this prior assumption.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w0PJyipyiQ", "reviewer": "Reviewer_aPqn", "summary": "The authors introduce a method for unsupervised out-of-distribution (OoD) detection for image classification tasks. Their approach is based on the semantic reconstruction of latent features in multiple layers of an image encoder using a diffusion model and does not require any OoD data for training. Unlike existing methods, this approach operates on the feature level instead of the pixel level, leading to more efficient OoD detection due to the lower dimensionality of feature encodings. Experiments on multiple established benchmarks, such as CIFAR10 vs. SVHN, LSUN, FMNIST, Omniglot, and CelebA vs. SUN, Textures, and Places365, demonstrate the effectiveness of the OoD detection approach with respect to the evaluation metrics AUROC and FPR95.", "questions": "- In line 142ff: “Such multi-layer features could better rebuild the samples and encourage the ID semantic features distributed more compactly within a certain space from different semantic layers.” To me it is unclear what is meant by semantic features being encouraged to be distributed more compactly. As far as I understand, the features of the encoders are only extracted and not modified during the training of the OoD detection model.\n- Another insightful metric to measure OoD detection performance could be the AUPRC, in particular if there are significant less OoD examples than ID examples. The numbers for AUROC seem saturated for couple of datasets.\n- The OoD detection task with the given datasets seems rather simple. It would be interesting to see the proposed method in other settings, when the original task is more challenging. Are there limitations in terms scalability?", "limitations": "The authors have discussed limitations. Potential negative social impact have not been mentioned.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Modeling the distribution of feature encodings for OoD detection is an intuitive yet established approach. However, the authors are the first to successfully incorporate generative modeling of features within the framework of OoD detection in image classification tasks.\n- The paper is generally well-written and easy to follow. The modules of their OoD detection method—namely, the multi-layer feature extraction, diffusion-based feature reconstruction, and the OoD detection score—are clearly explained. In the ablation studies, the authors demonstrate the effectiveness of each of these modules.\n- The proposed method outperforms the state-of-the-art across multiple datasets. In addition to the clear performance gain, the method is also fast. Particularly when compared to other types of generative models, the speedup is significant and relevant in safety-critical applications.", "weaknesses": "- The comparisons in the experiments could be extended to ensure fairness. The proposed diffusion and feature-based approach is compared against other types of generative models, such as GLOW and VAE. However, these generative models are only applied at pixel level and rely on image reconstruction error as the OoD score. In my opinion, a fair comparison would involve applying all generative models to the same input, i.e., the multi-scale feature encodings. Different generative models come with their own advantages and drawbacks, and a fair comparison could better show why the diffusion model is the best choice.\n- The speed comparison is incomplete. As mentioned, the other generative models are only applied at pixel level, so the time comparison is based on different inputs. Additionally, it would be interesting to see the speed gap between the proposed method and classification or distance-based methods.\n- The feature-based approach relies on the feature encodings of the image encoder. From the comparison of EfficientNet-b4 and ResNet50 as encoders, it appears that OoD detection performance benefits from stronger encoders (although this is not explicitly validated). From this perspective, it is unclear what the underlying model is for the classification or distance methods. For a fair comparison, these approaches should also use EfficientNet-b4 and ResNet50 as backbones.\n- There are no images to show some qualitative results. Particularly, examples of failure cases would be interesting to see.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1. Comparison against other methods using the multi-scale feature encodings as the input.   \n\nA: We have made comparison of our method against AE and VAE using the multi-layer feature encodings as inputs.  \n\nFor AE (AutoEncoder), we use the LFDN network without the timestep embedding, i.e., a 16-layer linear network. For VAE, we use a 5-layer linear network as the encoder and an 8-layer linear network as the decoder.   \n\n|ID||OOD|AE(+MFsim)|VAE(+MFsim)|Diffusion(+MFsim)|\n|-|-|-|-|-|-|\n|CIFRA10||SVHN|57.68|83.96|98.89|\n|||LSUN|81.47|97.69|99.83|\n|||MNIST|95.85|99.98|99.99|\n|||FMNIST|79.61|98.69|99.99|\n|||KMNIST|90.51|99.96|99.99|\n|||Omniglot|81.50|97.69|99.99|\n|||NotMNIST|81.61|99.88|99.99|\n|average|||81.18|96.84|**99.81**|\n|Time|Num img/s (↑)||1224.2|1179.4|999.3|\n\nCompared to AE and VAE, the diffusion model has significant advantages when modeling complex multidimensional distributions.  \n\nQ2. Speed comparison with classification or distance-based methods. \n\nA: The speed comparison between classification-based and distance-based methods is presented below. All experiments were conducted on an NVIDIA Geforce 4090 GPU. \n\n|Method|MSP|EBO|DICE|ASH-S|SimCLR+Mahalanobis|SimCLR+KNN|ours(+MSE)|ours(+LR)|ours(+MFsim)|\n|-|-|-|-|-|-|-|-|-|-|\n|Type|Classifier-based|Classifier-based|Classifier-based|Classifier-based|Distance-based|Distance-based|Genetive-based|Genetive-based|Genetive-based|\n|img/s (↑)|1060.5|1060.5|1066.3|1047.6|674.8|919.8|960.6|360.2|960.6|\n\nThe inference speed of our method based on MSE or MFsim is faster than that of distance-based methods SimCLR+Maha and SimCLR+KNN, because the computation of covariance matrix or K nearest neighbors occupies part of time. Our method is also comparable to classifier-based methods including MSP, EBO, DICE and ASH-S. \n\nQ3. Comparison using the same image encoder as backbone. \n\nA: The experimental results based on classification and distance methods are taken from the best results in the original papers using their optimal backbones.\n\nWe also conducted experiments to compare our method with other methods using the same image encoder. The FPR95  and AUROC metrics for methods using EfficientNet-b4 as the backbone are as follows. \n\nWe provide the average FPR95 and AUROC values across six OOD datasets: SVHN, LSUN-c, LSUN-r, iSUN, Textures, and Places365. \n\n|ID|Type|Method|**average FPR95**|**average AUROC**|\n|-|-|-|-|-|\n|CIFAR10|Classifier-based|MSP|34.70|94.74|\n|||EBO|17.42|96.31|\n|||DICE|11.55|97.44|\n|||ASH-S|12.53|96.51|\n||Distance-based|SimCLR+Mahalanobis|57.52|78.87|\n|||SimCLR+KNN|51.53|92.67|\n||Genetive-based|ours(+MSE)|20.75±0.02|96.93±0.01|\n|||ours(+LR)|17.20±0.02|97.61±0.02|\n|||ours(+MFsim)|**2.51±0.01**|**99.34±0.01**|\n|CIFAR100|Classifier-based|MSP|69.02|80.70|\n|||EBO|84.36|79.31|\n|||DICE|60.39|83.72|\n|||ASH-S|51.71|83.13|\n||Distance-based|SimCLR+Mahalanobis|89.75|62.71|\n|||SimCLR+KNN|91.56|61.94|\n||Genetive-based|ours(+MSE)|52.95±0.02|86.35±0.01|\n|||ours(+LR)|51.72±0.03|89.17±0.01|\n|||ours(+MFsim)|**14.78±0.02**|**97.20±0.01**|\n\nThe average FPR95 and AUROC metrics of methods using ResNet-50 as the backbone are provided in Table II of the PDF file attached in the global rebuttal.\n\nQ4. Qualitative results. \n\nA: We have included three types of failure cases in the PDF. \n\nThe first type, shown in Figure B, represents ID samples misclassified as OOD. It can be observed that these misclassified samples often have significant shadows and lack semantic information, resulting in high reconstruction errors and being incorrectly classified as OOD samples. \n\nThe second type, shown in Figure C, represents OOD samples misclassified as ID. It can be observed that these OOD samples have categories very similar to those of the ID samples (CIFAR-10), such as cars and ships, which are categories present in CIFAR-10. \n\nThe third type, shown in Figure D, represents OOD samples with colors very similar to the ID samples, leading to their misclassification as ID. \n\nQ5. Explanation to more compact semantic features.\n\nA: Due to the higher dimensionality and complexity of the feature distribution at the pixel level, our proposed multi-scale feature fusion and compression strategy significantly reduces the feature dimensions and makes the distribution more compact. This allows the sample reconstruction process to focus more on the primary features with inter-class discriminative power, rather than secondary details, thereby achieving the goal of distinguishing between ID and OOD samples.\n\nQ6. AUPRC Measurement \n\nA:  We use the AUPRC metric to evaluate OOD detection methods based on the EfficientNet-b4 backbone model. We provide the average AUPRC values across six different datasets in Table I of the PDF. Our method achieves better performance than existing methods as well. \n\nQ7. Experiments on more challenging datasets. \n\nWe provide experiments on more challenging datasets using EfficientNet-b4 as the encoder.\n\n(1) We use CIFAR-10 as in-distribution data and CIFAR-100 as OOD data, which has relatively high similarity with CIFAR-10. The results are shown in Table III of the PDF.\n\n(2) We also test OOD detection methods in experiments where ImageNet100 is regarded as the ID dataset and SUN, iNaturalist, Textures, and Places365 are used as OOD datasets. The results are shown in Table IV of the PDF.\n\nOur method still shows significant advantages over the baselines, which demonstrates the scalability of our approach. \n\nQ8. Potential negative social impact have not been mentioned.  \n\nA: We have discussed potential negative social impact in the appendix of our paper: As with any advanced detection method, there is a risk that the technology could be misused. For instance, surveillance applications, it could be employed to monitor individuals without their consent, leading to privacy violations and ethical concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LekEnCup0n", "reviewer": "Reviewer_Jgzd", "summary": "This paper focuses on out of distribution detection. The detection is based on the reconstruction error. The authors use a diffusion model as their generative model and focus on the feature space instead of the original images. The authors test their proposed methods on several different datasets.", "questions": "Basically, my major concerns include: 1) why it works? 2) the experiments. Please see [Weaknesses] for more details.", "limitations": "yes.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The overall presentation of this paper is good. It is very easy to follow.\n2. Have conducted experiments on different datasets.", "weaknesses": "There are several concerns for this paper.\n\n1) The novelty of this paper is very limited. It looks like it simply replaces a generative model with the diffusion model.\n\n2) The current version only presents the \"what\"---for example, the model includes three components (multi-layer semantic feature extraction, diffusion model for feature reconstruction, and OOD detection head)---but not the \"why\". More explanations and theoretical analysis may need.\n\n3) The evaluation metrics may be problematic. Due to the natural of the OOD problem, the dataset would be highly imbalanced. In addition to the AUROC and FPR95, other metrics that could capture the imbalance characteristics should be used.\n\n4) The current ID and OOD are very different (from different datasets), what would happen if the difference between ID and OOD is not so significant? For example, both ID and OOD come from the same dataset (e.g., CIFAR10) but OOD is curated by adding some distortions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Novelty of this paper\n\nThanks for the valuable comments. I believe the reviewer may misunderstand our contributions on OOD detection by stating that “It looks like it simply replaces a generative model with diffusion model”. Our main novelty lies in the following three aspects:  \n\nFirstly, we are the first to successfully incorporate generative modeling of features within the framework of OOD detection in image classification tasks. This demonstrates that when using diffusion models for OOD detection in such downstream tasks, it is not necessarily required to operate in the pixel space. \n\nSecondly, we devise a multi-layer semantic feature reconstruction mechanism.  Performing feature reconstruction on top of the multi-layer semantic features, encourages to restrict the in-distribution latent features distributed more compactly within a certain space, so as to better rebuild in-distribution samples while not reconstructing OOD comparatively. As a result, the projected in-distribution latent feature space should be compressed sufficiently to capture the exclusive characteristics of ID images, while it also provides sufficient reconstruction power for the large-scale ID images of various categories with the high-level semantic features. \n\nThirdly, the proposed Latent Feature Diffusion Network (LFDN) is built on top of the feature level instead of the traditional pixel level, which could significantly improve the computation efficiency and achieve effective OOD detection. \n\n2. More explanations and theoretical analysis may need.\n\nThanks for the valuable comments. We will improve these descriptions in the camera-ready version. \nAs illustrated in Figure 1, the proposed framework consists of the following three components: \n\n(1)The multi-layer semantic feature extraction module, which sets up a comprehensive and discriminative feature representation for each image, and could help to better rebuild the samples and encourage the in-distribution features distributed more compactly within a certain space from different semantic layers. \n\n(2) The latent feature diffusion stage, which introduces the DDPM to model the multi-layer semantic features. It builds a latent feature diffusion network to reconstruct the semantic features from their distorted counterparts.\n\n (3) The OOD detection head, which utilizes different evaluation metrics (i.e., MSE, MFsim and LR metric), to measure the reconstruction error. Finally, we can use the reconstruction error to justify whether the input sample belongs to ID or OOD. \n\nThe success of our method lies in that, the projected in-distribution latent feature space can be compressed sufficiently to capture the exclusive characteristics of ID images, while it also provides sufficient reconstruction power for the large-scale ID images of various categories. \n\nBesides, to further illustrate the effectiveness of the proposed method, **we also compare the MFsim score distributions between the initial model and the final model after training. We can clearly see that the reconstruction errors of the ID data tend to decreasing as the model training, and the final scores can be distinguished from the OOD data.**\n\n3. In addition to the AUROC and FPR95, other metrics that could capture the imbalance characteristics should be used. \n\nBesides the commonly used AUROC and FPR95 evaluation metrics, we also adopt the F1-score and AUPRC evaluation metrics which could capture the imbalance characteristics of the dataset, for comprehensive experimental comparison. As shown in the table below, we compare the proposed method with some of the latest representative Classification-based methods under the F1-score and AUPRC evaluation metrics, respectively. \n\nComparison of F1-Score \n\n|ID|Method|SVHN|LSUN-c|LSUN-r|iSUN|Textures|Places365|**average**|\n|-|-|-|-|-|-|-|-|-|\n|CIFAR10|MSP|48.20|80.88|75.74|75.85|81.75|75.90|73.05|\n||EBO|46.76|92.72|81.93|81.30|82.53|81.17|77.74|\n||ASH-S|67.44|93.55|90.99|91.33|83.12|82.39|84.80|\n||ours(+MSE)|79.21|91.04|83.16|82.93|97.43|97.43|88.53|\n||ours(+LR)|86.18|91.85|85.74|85.03|97.43|97.43|90.61|\n||ours(+MFsim)|91.91|97.42|95.20|94.71|97.43|97.43|**95.68**|\n|CIFAR100|MSP|47.92|72.47|70.29|72.08|78.51|68.02|68.22|\n||EBO|46.81|75.34|70.91|72.39|78.48|67.84|68.63|\n||ASH-S|44.35|74.83|67.94|69.86|78.20|68.79|67.33|\n||ours(+MSE)|49.80|73.75|67.56|70.11|97.43|97.43|76.01|\n||ours(+LR)|52.39|74.05|68.75|71.13|97.43|97.42|76.86|\n||ours(+MFsim)|65.48|96.52|87.39|87.20|97.43|97.43|**88.58**|\n\nWe provide the average AUPRC values across six different datasets in Table I of the PDF attached  in the global rebuttal.\n\nWe can clearly see that our method also obtains the best performances under all the settings.\n\nBesides, to analyze the effect of the imbalanced dataset, we design different ratios of ID and OOD data samples for evaluations. As shown in the table below, the detection accuracy tends to decreasing as the sample ratio (ID: OOD) gets larger. \n\n|ID|ID:OOD Ratios|**average F1-Score**|\n|-|-|-|\n|CIFAR10|1:0.1|97.39|\n||1:0.5|97.29|\n||1:1|97.23|\n||1:5.7|95.68|\n|CIFAR100|1:0.1|97.25|\n||1:0.5|96.47|\n||1:1|95.49|\n||1:5.7|88.58|\n\n\n4. What would happen if the difference between ID and OOD is not so significant? \n\nTo analyze the effect of the difference between ID and OOD on the model performance, we also artificially design the following two experiments:  \n\n1) The ID dataset is from CIFAR10, and the OOD dataset is CIFAR100 which shares similar distributions with CIFAR10. \n\nThe results are in Table III of the PDF.\n\n2) Both the ID and OOD datasets are CIFAR10, while the OOD dataset is CIFAR10 by adding  random Gaussian noise.  \n\nThe results are as follows: \n\n|ID|OOD|Method|FPR95↓|AUROC↑|\n|-|-|-|-|-|\n|CIFAR10|CIFAR10(add noise)|MSP|90.23|61.63|\n|||EBO|88.01|62.49|\n|||ASH-S|86.95|62.16|\n|||VAE|89.20|51.76|\n|||DDPM|95.80|45.00|\n|||ours(+MSE)|7.84|98.43|\n|||ours(+LR)|16.27|96.76|\n|||ours(+MFsim)|**0.14**|**99.97**|\n\nWe can clearly see that our method is superior to other methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their valuable feedback, with three reviewers (yKR8, ch15 and aPqn) supporting our work. We are encouraged that reviews think our paper:
- **A novel and interesting unsupervised OOD detection scheme.** (by Reviewer yKR8, ch15, aPqn)
- **State-of-the-art performances on benchmark datasets.** (by Reviewer yKR8, ch15, aPqn)
- **Strong evaluation and informative ablation.** (by Reviewer ch15)
- **Good presentation.** (by Reviewer aPqn, Jgzd)


Concerns of reviewers are addressed in the rebuttal to each reviewer with extra tables and figures in the attached pdf document.

1.In response to Reviewer aPqn’s question on evaluation metric, we provide AUPRC to measure OOD detection performance in Table I.

2.In response to Reviewer aPqn’s concern on using same backbone, we provide the average FPR95 and AUROC values for different methods using ResNet-50 as the backbone in Table II. 

3.In response to Reviewer aPqn’s concern on scalability to more challenging settings, we conduct experiments using CIFAR-10 as the ID dataset and CIFAR-100 as the OOD dataset. The results are presented in Table III. We also conducted experiments using ImageNet100 as ID dataset. The results are presented in Table IV. 

4.In response to Reviewer yKR8’s question on comparative analysis between the pixel-level and feature-level denoising, we visualize the reconstruction error distribution for both pixel-level and feature-level reconstructions in Figure A. It can be observed that at the pixel level, the reconstruction error distributions of ID and OOD samples are very similar. However, at the feature level, the reconstruction score distribution of ID samples shows a clear distinction from that of OOD samples.

5.In response to Reviewer aPqn’s request on qualitative results of failure cases, we provide three groups of samples representing three main types of failure cases. 

We have tried our best to answer all the reviewers’ questions about our paper. We sincerely hope that our responses can address all the concerns.

Meta-review: Dear authors,

This draft has received 1 Accept, 2 Weak Accept, and 1 borderline. After carefully reviewing the comments, the draft is being accepted for publication. Authors are encouraged to update the draft according to the suggestions and concerns mentioned by the reviewers.

Congratulations.

regards
AC
Decision: Accept (poster)