BOOD: Boundary-based Out-Of-Distribution Data Generation
Authors: Qilin Liao
Shuo Yang
Bo Zhao
Ping Luo
Hengshuang Zhao
Venue: ICML 2025 poster (2025)
Keywords: OOD detection
Diffusion models
Training data generation
Primary area: general_machine_learning->everything_else
TL;DR: This paper proposes a novel framework called Boundary-based Out-Of-Distribution data generation (BOOD), which synthesizes high-quality OOD features and generates human-compatible outlier images using diffusion models.

Abstract: Harnessing the power of diffusion models to synthesize auxiliary training data based on latent space features has proven effective in enhancing out-of-distribution (OOD) detection performance. However, extracting effective features outside the in-distribution (ID) boundary in latent space remains challenging due to the difficulty of identifying decision boundaries between classes. This paper proposes a novel framework called Boundary-based Out-Of-Distribution data generation (BOOD), which synthesizes high-quality OOD features and generates human-compatible outlier images using diffusion models. BOOD first learns a text-conditioned latent feature space from the ID dataset, selects ID features closest to the decision boundary, and perturbs them to cross the decision boundary to form OOD features. These synthetic OOD features are then decoded into images in pixel space by a diffusion model. Compared to previous works, BOOD provides a more training efficient strategy for synthesizing informative OOD features, facilitating clearer distinctions between ID and OOD data. Extensive experimental results on common benchmarks demonstrate that BOOD surpasses the state-of-the-art method significantly, achieving a 29.64\% decrease in average FPR95 (40.31\% vs. 10.67\%) and a 7.27\% improvement in average AUROC (90.15\% vs. 97.42\%) on the Cifar-100 dataset.

BOOD: Boundary-based Out-Of-Distribution Data Generation
Qilin Liao' Shuo Yang?" 2* Bo Zhao
Ping Luo'
Hengshuang Zhao
Abstract
Distribution (OOD) detection, which flags outliers during training, is non-trivial solution for helping models form boundary around the ID (in-distribution) data (Du et al., 2023). Recent works have shown that training neural net- works with auxiliary outlier datasets is promising for help- ing the model to form a decision boundary between ID and OOD data (Hendrycks et al., 2019; Liu et al., 2020; Katz-Samuels al., 2022; Ming et al., 2022). However, the process of manually preparing OOD data for model training incurs substantial costs, both in terms of human resources investment and time consumption. Additionally, it's impos- sible to collect data distributed outside the data distribution boundary, which can not be captured in the real world as shown in Figure 1.
Harnessing the power of diffusion models to syn- thesize auxiliary training data based on latent space features has proven effective in enhanc- ing out-of-distribttion (OOD) detection perfor- mance. However, extracting effective features outside the in-distribution (ID) boundary in latent space remains challenging due to the difficulty of identifying decision boundaries between classes. This paper proposes a novel framework called Boundary-based Oft-tr-Distiibuttion data gener- ation (BOOD), which synthesizes high-quality OOD features and generates coman-coeeatible outlier images using diffusion models. BOOD first learns a text-conditioned latent feature space from the ID dataset, selects ID features closest to the decision boundary, and perturbs them to cross the decision boundary to form OOD features. These synthetic OOD features are then decoded into images in pixel space by a diffusion model. Compared to previous works, BOOD provides more training efficient strategy for synthesiz- ing informative OOD features, facilitating clearer distinctions between ID and OOD data. Exten- sive experimental results on common benchmarks demonstrate that BOOD surpasses the state-of- the-art method significantly, achieving 29.64% decrease in average FPR95 (40.31 % vs. 10.67%) and 7.27% improvement in average AUROC (90.15% vs. 97. 97.42%) on the CIFAR- 100 dataset.
To address this problem, recent works have demonstrated pipelines regarding automating OOD data generation, which significantly decreases the labor intensity during creating auxiliary datasets (Du et al., 2022; Tao et al., 2023a; Du et al., 2023; Chen et al., 2024). As a representative of them, DreamOOD (Du et al., 2023) models the training data distri- bution and samples visual embeddings from low-likelihood regions OOD auxiliary data in text-conditioned latent space, then decoding them into images through diffusion model. However, due to the lack of an explicit relationship between the low-likelihood regions and the decision bound- aries between classes, the DreamOOD (Du et al., 2023) can not guarantee the generated images always lie on the decision boundaries, which have demonstrated efficacy in enhancing the robustness of the ID classifier and refining its decision boundaries (Ming al., 2022). Chen al. (2020) proposes OOD detection frameworks targeting the OOD boundary by searching for a similarity threshold be- tween class center features and encoded input features. Pei ct al. (2022) harnesses GAN (Goodfellow et al., 2014) to train a boundary-aware discriminator and an OOD gener- ator for distinguishing ID and OOD data. However, these two methods still lack the ability find an explicit distri- bution boundary between ID and OOD area, and generate image-level OOD dataset.
100
1. Introduction
In the field of open-world learning, machine learning mod- els will encounter various inputs from unseen classes, thus be confused and make untrustworthy predictions. Out-Of-
'The University of Hong Kong. Hong Kong, China 2 DDpartmeett of Computer Science, Harbin Institute of Technol- ogy (Shenzhen), Shenzhen, China "Scoool of AI, Shanghai Jiao Tong University, Shanghai. China. Correspondence to: Shuo Yang <shuoyang @ hit.cdu.cnn. Hengshuang Zhao <hszhao cs cs..ku.hkk.
In this paper, we introduce a new framework, BOOD (Boundary-based Out-Of-Distribttion data generation), which explicitly enables us to generate images located around decision boundaries between classes, thus providing high-quality and informative features for OOD detection.
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

BOOD: Boundary-based Out-Of-Distribation Data Generation
Bear
Tiger
ID
OOD
Figure 1. Top: images generated from features. Bottom: images generated from OOD features. Compared to preparing ID image datasets, preparing OOD image datasets incurs substantial costs in terms of resource allocation, particularly with respect to labor and time investment. Moreover, certain OOD images, as illustrated in the above figure, are impossible to aequire through real-world data collection methods. Consequently, there exists pressing need for the development of automated pipelines capable of generating OOD datasets.
The challenging part lies in the following: (1) Identifying the data distribution boundary accurately, and (2) Synthesiz- ing the informative outlier features based on the identified data distribution boundaries. Our innovative framework addresses the aforementioned challenges by: (1) an adver- sarial perturbation strategy, which successfully identifies the features closest to the decision boundary by calculat- ing the minimal perturbation steps imposed on the feature to change the model's prediction, and (2) an outlier fea- ture synthesis stralegy, which generates the outlier features by perturbing the identified boundary ID features along with the gradient ascent direction. The synthetic outlier features are subsequently fed into a diffusion model to gen- erate the OOD images. To guarantee the synthetic feature space is compatible with the diffusion-model. iiseddl-ippu-sppcce (class token embedding space), we employ a class embed- ding alignment strategy during the image encoder training following Du et al. (2023).
improvement across two challenging benchmarks, achieving state-of-the-arr results on CIFAR-100 and IMAGENET-100 100 datasets. For instance, on CIFAR- 100, BOOD improves the average performance on detecting 5 OOD datasets from 40.31 % to 10.67% FPR95 and from 90.15% to 97.42% AUROC. Moreover, we conducted extensive quantitative abla- tion analyses to provide a deeper insight into BOOD's efficiency mechanism.
2. Preliminaries
Latent space formation. Given an ID training dataset, Dis = {(x,, y) where H; E X and ys E y. X denotes the input space and y E {1,2,...V) denotes the label space. Let hxr) : X age feature encoder, where 1" denotes the feature space. The output of he is supposed to be an n-dimensional vec- tor representing the encoded image feature. We denote f(x) = CosSim(ha [(y)) as the cosine image classi- fier, whose output is assumed to be o-dimensional vector that performs as a discrete probability function representing prediction probability for each class. T(y) represents the class token embedding encoded by feeding class name y into CLIP (Radford et al., 2021) text encoder.
R denote the im-
Before delving into details, we summarize our contributions as below:
To our best knowledge, BOOD is the first framework that enables generating image-level OOD data lying around the decision boundaries explicitly, thus pro- viding informative features for shaping the decision boundaries between ID and OOD data.
OOD detection. In real-world applications of machine learning models, a reliable classification system must ex- hibit dual capabilities: distribution (ID) samples, and it able recognize and flag out-of- distrribuuiin (OOD) inputs that belong to unknown classes not represented in the original training set y/ y. Thus, having an OOD detector can solve this problem. OOD detection can be formulated as a binary classification prob- lem (Ming et al., 2022), and the goal is to decide whether an input is from ID or OOD. We denote the OOD detection as go go(z)) : X + (TD, OOD) mathematically.
We propose two key methodologies to address the chal- lenges in synthesizing the OOD features: (1) Identify- ing the ID boundary data by counting their minimum perturbation steps to cross the decision boundaries for all ID features. (2) Synthesizing the informative OOD features lying around the decision boundaries by per- turbing the ID boundary features towards the gradient ascent direction.
should categorize familiar in-
Our method demonstrates superior performance
2

---

BOOD: Boundary-based Out-Or-Distribatiin Data Generation
gradient ascent direction
Tiger Fish
0,904
0.788
0.254
0.072
0.172
0. 2.643
0.973
1
Streetcar
0.994
0.795
0.199
0.062
0.568
decision boundary
Figure Illustration of perturbing boundary feature process. charts under each image represent the prediction probability the perturbed features by the image classifier. After each perturbation, the prediction probability of the original class decreases. When the prediction of the image classifier switches, we consider the obtained feature crossed the decision boundary.
Diffusion-based image generation. Diffusion models demonstrate formidable prowess generating authentic and lifelike content. Their robust capabilities extend to vari- ous applications, with particular efficacy in tasks such as the creation of synthetic images. We synthesize images in specific distribution by conditioning descriptions (Ramesh et al., 2022). Stable Diffusion (Rom- bach et al., 2022) is a text-to-image model which enables generating particular images conditioned by text prompts. For a given class name y, the generating process can denoted by:
thetic images decoding from latent space features lying around the decision boundaries among ID classes. The chal- lenging part lies in identifying the ID boundary features and synthesizing outlier features located around the decision boundary, which have demonstrated efficacy in enhancing the robustness of the ID classifier and refining its decision boundaries (Ming et al., 2022).
class labels text
3.1. Building the Text-Conditions Latent Space
Aiming ensuring the image features suitable for being decoded by the diffusion model, we first create an image feature space that is aligned with the diffusion diffusion-model-iinee model-input space. To achieve this, we train the image encoder he by aligning the extracted image features ha))) with their corre- sponding class token embeddings N(y), which match the in- put space with the diffusion model. The resulting generated features form text-conditioned latent space. Following DreamOOD (Du et al., 2023), we train the image encoder hg with the following loss function:
Zy)
where Z, = r(Y) denotes a specific textual representation of class label with prompting, and we denote the whole prompting as For instance, Y "A picture of [y]". denotes the CLIP (Radford et al., 2021) model's text en- coder.
3. BOOD: Boundary-based Out-Of-Distribution data generation
exp(T(y)) T 2/t) L. = (err) ΣC e (Tτ,) 22t))
(2)
Images situated near the decision boundary offer informative OOD insights, which significantly enhance the ability of OOD detection models to establish accurate boundaries between ID and OOD data, thereby improving overall de- tection performance. In this paper, we propose a framework BOOD (Boundary-batt based CnttO--DDitribttion data genera- tion), which enables us to generate human-come compatible syn-
where = ho(x)/ is the Iz-normalized image fea- ture embedding, t is the temperature, he denotes the text- conditioned image feature encoder, and r(y) denotes the class token embedding encoded by feeding class name into CLIP (Radford al., 2021) text encoder. After training
3

---

BOOD: Boundary-based Out-Of-Distribation Data Generation
Table OOD detection results for CIFAR- 100 as the in-distribution data. We report standard deviations estimated across 3 runs. Bold numbers are superior results, and the last row is the improvement of our method over previous satt--of-the-aee DreamOOD (Du et al., 2023).
Methods
SVHN
PLACES365
LSUN
ISUN
TEXTURES
Average
ID ACC
FPR95
AUROC1
FPR95
AUROCI
FPRO5I
AUROCT
FPR95L
AUROC
FPR95L
AUROC
FPR95,
AUROCT
MSP (Hendrycks & Gimpel, 2017)
87.35
69.08
81.65
76.71
76.40
80.12
76.00
78,90
79.35
77.45
80.15
76.45
79.04
ODIN (1 (Liang et al., 2018)
90.95
64.36
79.30
74.87
75,64)
78.04
53.10
87.40
72.60
79.82
74.31
76.90
79.04
Mahulanobis (Loz
2018bb
77.90
86.46
62.45
84.43
81.92
Energy (1.in et al., 2020)
84,90
70.90
82.05
76.00
81.75
78.36
73.55
81.20
78.70
78.87
80.19
77.07
79.04
GODIN (Hbu al., 2020)
63.95
88, $5.96
80.65
77.19
60,65
88.36
51.60
92.07
71.75
85.02
65.72
86,32
76.34
KNN
2022)
81.12
79.77
79.51
VIM (Wang et al., 2002)
81.20
77.24
79.20
77.81
43. 10
90.43
74.55
83.02
61.85
85.57
67.98
82.81
79.04
RoAct (Sun et al., 2021)
K2.85
70.12
81.75
76.25
80,70
83,03
67.40
83.28
74 74,60
81.61
77,46
T8
79.04
DICE &
83.55
85.05
75.92
80.90
79.80
77.83
83.53
76.15
79.04
Sym/hgatt-bused mechoudy
GAN (Lee et al., 20155)
89.45
66.95
88.75
66.76
82.35
75.87
83:45
73.49
92.80
62.99
87.36
69.21
70.12
VOS (Do et 2022)
78.50
84.55
75.85
59:05
75.35
73.98
NPO8 (Tio et al., 2025a)
11.14
97.884 84
79.08
71.30
56.27
82.43
51.72
M5.48
35.20
92. 44
46.68
85,90
78.23
DreamOOD (Du et 2023)
58.75
$7.01
70.85
79.94
24.25
95.23
1.10
99.73
46.60
88.82
40.31
90.15
78.94
BOOD
$.42 :0.5
98.43:10.1
40.55=1
90.76:00.5
2.06:00.8
99.25:001
0.22::0.15
99.91:0.00 0.00
5.1:11
98.74:0.2
10.67:0.95
97,42:0.11
78.03 10.1
A (Improvements)
+53,33
+11.42
+30.3
+10.82
+22.19
+4,02
40.18
+9.92
+29.64
+7.27
the image encoder hgg the image classifier f can be simply formulated as a cosine classifier between the encoded image features hz) and the class token embeddings I'(y).
Sec. 3.2.2. We finally decode the synthesized OOD features through the diffusion model and generate OOD images in Sec. 3.2.3, Figure 4 is a visual representation of Sec. 3.2.1 and Sec. 3.2.2.
3.2. Synthesizing OOD features and Generating images
3.2.1. BOUNDARY FEATURE IDENTIFICATION
Algorithm 1 BOOD: Boundary-based Oft-tt-DDitrrbuttion
We believe that the ID features distributed near the decision boundary are more sensitive to perturbation, as slight pertur- bation can push them across the decision boundary, making them ideal candidates for synthesizing OOD features in Sec. 3.2.2. Thus, the target at this stage to select fea- tures that are closest the decision boundary. Introduced by Chakraborty et al. (2018) and Kurakin et al. (2017), adversarial attack endeavors to perturb data point to the smallest possible extent to cross the model's decision bound- ary. Inspired by Yang et al. (2024b), Mickisch et al. (2020) and Yousefzadeh & O'Leary (2020), our objective to de- termine the minimal distance required for an in-distribution (ID) feature traverse the decision boundary. This is ac- complished by quantifying the number of steps, denoted as k; necessary to perturb the ID feature along the gradient ascent direction until it changes the model's prediction.
data generation
In-distribution training data Di {(x,, ys) y,) ) initial model parameters 0 for learning the text-conditioned latent space, diffusion model.
Input:
Output: Synthetic images Food-
Section. 1:1: Building the text-conditioned latent space Extract token embeddings (yy) of the ID label y E V. Learn the text-conditioned latent representation space by Equation22 2.
// Section. 3.2: Synthesizing OOD features and generating images
1. Calculate the distances for each feature and select the ID boundary features with Equation 3.
2. Perturb the selected ID boundary features to cross the
decision boundary with Equation 4 and Equation 5.
3. Decode the outlier embeddings into the pixel-space OOD
Below is the working principle for a given feature (2, ()
images via diffusion model by Equation 6.
After obtaining a well-established text- cext-conditioned image feature latent space, our framework proposes the generation of outlier images through a three-step process. Firstly, we estimate each feature's 's distance the decision boundary by counting their perturbation steps to cross the decision boundaries, and select the ID boundary features by choosing those features with minimal distances in Sec. 3.2.1. Sub- sequently, we push the identified ID boundary features to the location around the decision boundary to synthesize OOD features by perturbing them along with the gradient ascent direction until the model's prediction switches in
(k+1) (k) Fddu Radv + a sign( ( (R) Sade
[[fo v)), E [0, K]
where a denotes the step size of a single perturbation, ((k) denotes adversarial feature at step k. / is the loss function, fe denotes the image classifier and K denotes the maximum iteration. The process keeps iterating until fo # y or k K, indicating that the adversarial feature 2*() has crossed the decision boundary or & exceeds the maximum allowed
add
iteration number K. We provide visualization of this process

---

BOOD: Boundary-based Out-Or-Distribation Data Generation
Table OOD detection results for IMAGENET- 100 the in-distribution data. We report standard deviations estimated across runs,
Bold numbers are superior results, and the last row is the improvement of our method over previous state-of 2023).
ofte-of-the-ar the DreamOOD (Du et al.,
OOD Datasets
Methods
INATURALIST
PLACES
SUN
TEXTURES
Average
ID ACC
FPR95]
AUROCT
FPR95]
AUROCT
FPR95]
AUROCT
FPR95]
AUROCT
FPR951
AUROCT
MSP (Hendrycks & Gimpel. 2017)
31.80
94.98
47.10
90.84
47.60
90.86
65.80
83.34
48.08
90.01
87.64
ODIN (Liang et ol., 2018)
24.40
95.92
50.30
90.20
44,90
91.55
61.00
81.37
45.15
89.76
87.64
Mabalanobis (Lee
201 8b)
91.60
75.16
96.70
60.87
97.40
62.23
36.50
91.43
80.55
72.42
Energy (Liu et al., 2020)
32.50
94.82
50,80
90.76
47.60
91.71
63.80
80.54
48.68
89.46
87.64
GODIN (Hsu ct al., 2020)
39.90
93.94
59.70
89,20
58.70
90.65
39,90
92.71
49.55
91.62
87.38
KNN (Sun al.,
28.67
95.57
65.83
88.72
58.08
90.17
41.38
91.20
87.64
VIM (Wang el al., 2022)
75,50
87.18
83,30
81.25
88,70
81.37
15.60
96.63
67.03
86.61
87.64
ReAct (Sun et al., 2021)
22.40
96.05
45.10
92.28
37,90
93.04
59.30
85.19
41.17
91.64
87.64
DICE (Sun & Li, 2022)
37.30
92.51
53,80
87.75
45,60
89.21
50,00
83.27
46.67
88.19
87.64
Syathesis-based methods GAN (Lee et 2018a)
83.10
71.35
83.20
69.85
84,40
67.56
91,00
59.16
85.42
66.98
79.52
VOS
2022)
43.00
93.77
47.60
91.77
39.40
93.17
81.42
90.03
NPOS (Tao al., 2023a)
53.84
86.52
59.66
83,50
53.54
87.99
8,98
98.13
44.00
89.04
85.37
DreamOOD (Du et al., 2023)
24.10
96.10
39.87
93.11
36.88
93.31
53.99
85.56
38.76
92.02
87.54
BOOD
18.33=0.3
96.74=0.2
33.33=0.5
94.08=0.4
37.92+0.2
93.52:0.1
51.8820.5
85.41=00.5
35.37:0.3
92.44:0.1
87,92+0.05
3 (Improvements)
+5.77
+0.64
+6.54
+0.97
-1.04
+0.21
+2.11
-0.15
+3.39
+0.42
in Figure 2.
Consider selected ID boundary feature we perturb it following the direction of gradient ascent until the pre- diction the image classifier fo switches (f(zia) * y). We continue perturbing it for e steps to guarantee ade- quately distant from the ID boundary. We provide ablation studies and in Sec. 4.3.2.
During each iteration, our method perturbs the adversar- ial feature in a direction that maximizes the change in the model's prediction. The minimum number of iterations necessary to create an adversarial example Zede from given feature that crosses the decision boundary, can be employed as proxy for the shortest distance between that data point and the decision boundary. This relationship is expressed as d(z, y) = k. where k is bounded by [0, K]. Thus, we can obtain the distance set for all ID features to the decision boundaries ID and select the boundary features that have minimal distances the decision boundary, de- } } where D,* denotes the smallest 5% of distance set D and denotes the selection ratio of ID boundary selection.
3.2.3. OOD IMAGE GENERATION
To generate the outlier images, we finally decode the syn- thetic OOD feature embeddings Food through a diffusion model. Following Du et al. (2023), we replace the origin token embedding I'(y) in the textual representation Z, with our synthetic OOD embedding 2ood- The generation process can be formulated
noted as 2id E {z|d(2, y) E D,%
Food P(x ((oo))
(6)
3.2.2. OOD FEATURE SYNTHESIZING
The features distributed around the decision boundary can provide high-quality OOD information to facilitate the OOD detection model form precise ID-OOD boundaries. So aim to perturb the selected ID boundary features zıa
where Food denotes the synthetic OOD images and Zood denotes the textual representation Zy with [(y) replaced by
Food. We summarize our methodology in Algorithm 1.
the location around the decision boundary, where we might synthesize informative features. These OOD features, de- noted as Rood, will be decoded into outlier images that are distributed around the OOD detection boundary. We sum- marize the perturbation process in the following module:
3.3. Regularizing OOD detection model
After synthesizing the OOD images, we regularize the OOD classification model with the following loss function:
log (E(g)((
LOOD
E ExpgonDid
While (f(aa) do Fi Aid + a sign (VinnVa,,, ([,,d() vi)
I (raaa)
(4)
llog
end
1 (Ep(o
Road
Zid
for
Fool aood sign ((
(5)
where Φ denotes 3-layer MLP function of the same struc- ture as VOS (Du al., 2022), E. denotes the energy function
end
5

---

BOOD: Boundary-based Out-Of-Distribation Data Generation
steps crossing boundary
step size
0
0.045
0.09
0.135
NV
decision boundary
Figure Left: the effect of perturbing steps c after crossing the boundary, Right: the effect of step size &.
2018) and TEXTURES (Cimpoi et al., 2014), following MOS (Huang & Li, 2021).
Training details. The ResNet-34 (He et al., 2016) archi- tecture was employed the training network for both the CIFAR-100 and IMAGENET- 100 datasets. The initial leamn ing rate set to 0.1, with a cosine learning rate decay schedule implemented. A batch size of 160 was utilized. In the construction of the latent space, the temperature param- eter was assigned a value of 1. In the boundary feature selection process, the initial pruning rate r was established at 5, with an initial total step K of 100. The step size a was configured 0.015. The hyper parameters for the OOD feature synthesis step were maintained consistent with those of the boundary feature identification process. A total of 1000 images per class were generated using Stable Diffusion v1.4, yielding a comprehensive set of 100,000 OOD images. For the regularization of the OOD detection model, the parameter was set to 1.0 for IMAGENET- 100 and 2.5 for CIFAR-100.
festuses
Decision boundary
TD boundary features
OOD features
Figure 4. Illustration of the identified ID boundary features and perturbing them to cross the decision boundary.
and go denotes the output of OOD classification model. The final training objective function combines cross-entropy loss and OOD regularization loss, which can be reflected by Lee + 3 LOOD, where 3 denotes the weight of the OOD regularization.
Evaluation metrics. We evaluate the performance using three key metrics: (1) the false positive rate at 95% true positive rate (FPR95) for OOD samples, (2) the area under the receiver operating characteristic curve (AUROC), and (3) in-distribution classification accuracy (ID ACC). These metrics collectively assess the model's discriminative capa- bility, overall performance, and retention of in in-distribution task proficiency.
4. Experiments and Analysis
4.1. Experimental setup and Implementation details
Datasets. Following DreamOOD (Du et al., 2023), we se- lect 11FAR-100 and IMAGENET- 100 (Deng al., 2009) as ID image datasets. As the OOD datasets should not overlap with ID datasets,
choose SVHN (Netzer al.,
4.2. Comparison with State-of-the-art
2011), PLACES365 (Zhou et al., 2018), TEXTURES(Cinpoi et al., 2014), LSUN (Yu et al., 2015), ISUN (Xu et al., 2015) as OOD testing image datasets for CIFAR-100 100. For IMAGENET- 100, we choose INATURALIST (Hom et al., 2018), SUN (Xiao et al., 2010), PLACES (Zhou et al.,
BOOD shows outstanding performance improvement com- pared to previous state-of-the-art methods. As shown in Table 1 and 2, we compare BOOD with other methods,
6

---

BOOD: Boundary-based Out-Of-Distribution Data Generation
The effect a
FPRES
AUROC
The effect
97.5
14.0
22
97.4
9/21
97.0
20
97.3
18773
19:53
13.0
30:32
18
97.2
17P295
AUROC
AUROC
16
96.0
12.0
11 M6
95.5
11.5
11.33
57.
02
97.0
26.36
10.67
95.0
11.0
96.9
05
10.5
0.001
0.005
0.015
0.025
0.05
0.1
0
4
Figure 5. Leftt: The effect of step size Right: The effect of perturbation steps after crossing the boundary.
for achieving the best result. ID boundary features are more sensitive to perturbation, which makes them optimal candi- dates for perturbation. The generated features lying around the decision boundary can provide high-quality OOD in- formation to help the OOD detection model regularize the ID-OOD decision boundary.
Table Ablation OOD feature synthesizing methodologies
Methods
Criteria (Avg.)
boundary identification
perturbation
TPR95
AUROCT 1
ID ACC
76.51
89.79
10.67
97.42
77.64
4.3.2. HYPER PARAMETERS SENSITIVE ANALYSIS
The effect of step size a. We show the effect of step size in Figure 5 (left). Employing a smaller step size allows for minor perturbations of the instance in each iteration and facilitates a more nuanced differentiation between samples across different distances. It also guarantees that the per- turbed features are more accurate direction towards the decision boundary. We choose the step size 0.015 our experiments. Figure 3 (left) illustrates the effect of as when increases, the discrepancy between iteration increases.
including Maximum Softmax Probability (Hendrycks & Gimpel, 2017), ODIN score (Liang et al., 2018), Maha- lanobis score (Lee et al., 2018b), Energy score(Liu et al., 2020), Generalized ODIN (Hsu et al., 2020), KNN dis- lance (Sun et al., 2022), VIM score (Wang et al., 2022), ReAct (Sun et al., 2021) and DICE (Sun & Li, 2022). Ad- ditionally, we compare BOOD with another four synthesis- based methods, including GAN-based synthesis (Lee al., 2018b), VOS(Du al., 2022), NPOS (Tao et al., 2023a) and DreamOOD (Du et al., 2023) as they have closer relationship with us. BOOD surpasses the state-of-the-art method significantly, achieving 29.64% decrease in aver- age FPR95 (40.31% vs. 10.67%) and a 7.27 7.27% improvement average AUROC 150.15% vs. 97.42%% on the CIFAR-100 dataset. BOOD's performance also surpasses the state-of. of- the-art methodologies on the IMAGENET-100 dataset.
The effect of perturbation steps after crossing the boundary. We analyze the effect of perturbation steps c after crossing the boundary in Figure 5 (right) to explore whether will extract more efficient features. We vary steps {0. 1, 2, 3, 4, 5) and observe that when c 2, BOOD shows the best performance. Employing a large may force the feature to step into the ID region, and choosing a small may not guarantee the perturbed feature is adequately dis- tant from the ID boundaries. Figure 3 (right) shows the effect of e: as the number of steps crossing the boundary augment, the generated images gradually transform into an- other classes or distribute outside the distribution boundary.
4.3. Ablation Studies and Hyper-parameter Analysis
1. ABLATION ON OOD FEATURE SYNTHESIZING METHODOLOGIEES
We ablate the effect of boundary identification and feature perturbation. As shown in Table 3, conduct experi- ments: (1) directly decode the ID features selected by Sec- tion 3,2.1. (2) randomly choose ID features and perturb them to the boundary (Section 3.2.2), (3) full BOOD. The results demonstrate that both the boundary feature identifi- cation and OOD feature perturbation modules are essential
The effect of r. We show the effect of pruning rate in Figure 6 (left). We vary rate E {2.5, 5, 10, 20} and ob- serve that BOOD shows best performance when we employ moderate pruning rate. Insufficient pruming (small r) may limit the diversity of generated OOD images (not enough features), while excessive pruning (large r) risks selecting

---

BOOD: Boundary-based Out-Of-Distribution Data Generation
Effect of
Effect of
F7R95
Effect of
16.0
15.5
97.30
97.0
2:10
97.25
95.5
96.75
97,30
16
19988
14.5
AUROC
EPR85
ATTROC
96..0
97.15
15
12
12.7
407 1.0
95.5
13.5
2348
97.05
50.1
12.6
95.0
13.0
11
mmhh
-10
05.75
12.5
2.5
5.0
10.0
20.0
1.5
2.0
2.5
3.0
50
100
200
400
8
Figure 6. Left:The effect of pruning rate r. Middle: The effect of (3. Right: The effect of maximum perturbation steps K
Table Computational cost comparison
Computational
Building latent space
OOD
OOD
OOD detection model regularization
Total
Cost
features synthesizing
image generation
BOOD
0.62h
0..1
77.5
8..5h
16..2h
DreamOOD
~061h
0005h
~7.5h
~855h
16.666
(1) the building of latent space, (2) OOD features synthesiz- ing, (3) the OOD image generation and (4) regularization
Table Memory requirements comparison
Memory
00D
OOD
Total
of OOD detection model. To provide quantitative evidence, present a detailed comparison of computational require- ments between BOOD and DreamOOD (Du et al., 2023) in table 4. We also summarize the storage memory re- quirements of BOOD and DreamOOD (Du et al., 2023) on CIFAR-100 table 5. Our empirical evaluation reveals that the differences between these approaches are not statisti- cally significant. Thus, our proposed framework is not time consuming or has strict memory requirements.
requirements
features
images
BOOD
~7,32MB
11.7G
1117G
DreamOOD
22.9
11.67G
144.57
ID features proximally distributed to the anchor.
The effect of 3. From Figure 6 (middle), we can conclude that empirical evidence suggests optimal performance is achieved with moderate regularization weighting 2.5
5. Related Work
excessive OOD regularization can compromise OOD
detection efficiency.
OOD detection. OOD detection has experienced a no- table increase research attention, as evidenced by numer- ous studies (Tajwar et al., 2021; Fort et al., 2021; Elflein et al., 2021; Fang et al., 2022: Yang et al., 2022: 2024a). A branch of research approach to addressing the OOD detec- tion problem through designing scoring mechanisms, such as Bayesian approach (Gal & Ghahramani, 2016; Laksh- minarayanan et al., 2017; Malinin & Gales, 2018; Osawa et al., 2019), energy-based approach (Liu et al., 2020; Lin et al., 2021; Choi et al., 2023) and distance based meth- ods (Abati al., 2019; Ren et al., 2021; Zaeemzadeh et al., 2021; Ming al., 2023). Katz-Samuels et al. (2022) and Du et al. (2024) propose methods harnessing unlabeled data to enhance OOD detection, illustrating possible directions
The effect of K. We analyze the effect of maximum it- eration number K in Figure 6 (right). We vary K E (5. 50, 100, 200, 400} and found that relatively large max iteration number K to ensure comprehensive boundary crossing for most features. While increased iterations do affect computational overhead in boundary identification, the impact remains manageable.
4.3.3. COMPUTATIONAL COST AND MEMORY REQUIREMENTS
In this section, we conducted a comparative study of com- putational efficiency between BOOD and DreamOOD (Du al., 2023). We specifically focus on four key processes:
8

---

BOOD: Boundary-based Out-Of-Distribation Data Generation
to leverage wild data which is considered abundant. Most of these works need auxiliary datasets for regularization.
benchmarks.
VOS (Du et al., 2022) and NPOS (Tao et al., 2023a) pro- pose methodologies for generating outlier data in the fea- ture space, DreamOOD (Du et al., 2023) and SONA (Yoon et al., 2025) synthesizes OOD images in the pixel space. Compared to DreamOOD (Du et al., 2023), NPOS (Tao al., 2023a) and SONA (Yoon al., 2025) which samples features with Gaussian-based strategies or in the nuisance region, BOOD synthesizes features located around the de- cision boundaries, providing high-quality information to the OOD detection model. Additionally, compared to Chen et al. (2020) and Pei et al. (2022) who train the OOD dis- criminator harnessing the OOD samples at the ID-OOD boundary at feature level, our framework locate the distribu- tion boundary more explicitly and can generate image-level OOD images.
7. Limitations
Although BOOD achieves excellent performance on com- mon benchmarks, still has some shortcomings. For some datasets that has large domain discrepancy from the diffusion model's training distribution, BOOD's perfor- mance might be affected by the diffusion model's capability. The classification error for unseen outlier features in Sec- tion 3.2.2 might result in deviations in determining whether perturbed feature has crossed the decision boundary, lead- ing to generating low-quality OOD features. Besides, might difficult for tuning the hyper parameters. the fu- ture study, designing a automatic adaptive method for tuning might improve the performance and reduce the training time.
Diffusion- Diffusion-model-baseee data augmentation. The field of data augmentation with diffusion models attracts various attention (Tao et al., 2023b; Zhu et al., 2024; Ding et al., 2024; Yeo al., 2024). One line of work performed image generation with semantic guidance. Dunlap et al. (2023) proposes caption the images of the given dataset and leverage the large language model (LLM) to summarize the captions, thus generating augmented images with the text-to- image model. Li et al. (2024) generated augmented images with the guidance of captions and textual labels, which are generated from the image decoder and image labels. A branch of research proposed perturbation-baan based approaches to synthesize augmented images (Shivashankar & Miller, 2023; Fu et al., 2024). Zhang et al. (2023) perturbed the CLIP (Radford et al., 2enooedd feature embeddings, guided the perturbed features by class name token features, and finally decoded it with diffusion model. Our framework BOOD creates an image feature space aligning with the class token embeddings encoded by CLIP (Radford al., 2021). proposes perturbation strategy to generate informative OOD features that are located around the decision boundary.
A Acknowledgment
This research is supported by the National Natural Science Foundation of China (No. 62306046, 62422606, 62201484). We would also like to thank ICML anonymous reviewers for helpful feedback.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning, specifically the OOD data genera- tion and detection field. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
References
Abati, D., Porrello, A., Calderara, S., and Cucchiara, R. Latent space autoregression for novelty detection. In CVPR, 2019.
Bitterwolf, J., Mueller, M., and Hein, M. In or out? fix- ing imagenet nut-of-distribution detection evaluation. In ICML, 2023.
6. Conclusion
In this paper, we propose an innovative methodology BOOD that generates effective decision boundary-band based OOD im- ages via diffusion models. BOOD provides two key method- ologies in identifying the ID boundary data and synthesizing OOD features. BOOD proves that generating OOD images located around the decision boundaries is effective in help- ing the detection model to form precise ID-OOD decision boundaries, thus delineating a novel trajectory for synthesiz- ing OOD features within this domain of study. The empiri- hunndry--bssed
Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., and Mukhopadhyay, D. Adversarial attacks and defences: A survey. arXiv: 18 180000069 2018.
Chen, J., Zhang, T., Zheng, W., and Wang, R. Tagfog: Textual anchor guidance and fake outlier generation for visual out-of-distribution detection. In AAAI, 2024.
cal result demonstrates that the generated outlier images are high-quality and informative, resulting in remarkable performance on popular OOD detection
Chen, X., Lan, X., Sun, F., and Zheng, N. A boundary based out diitt--iistiittiinn classifier for generalized zero- shot learning. In ECCV, 2020.

---

BOOD: Boundary-based Out-Or-Distributiin Data Generation
Choi, H., Jeong, H., and Choi, Y. Balanced energy regu- larization loss for out-of-distribution detection. In CVPR, 2023.
Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In CVPR, 2014.
Hendrycks, D., Mazeika, M., and Dietterich, T. G. Deep anomaly detection with outlier exposure. In ICLR, 2019.
Horn, G. V., Aodha, O. M., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S. J. The inaturalist species classification and detection dataset. In CVPR, 2018.
Deng, J., Dong, W., Socher, R., Li, L.., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Hsu, Y., Shen, Y., Jin, H., and Kira, Z. Generalized ODIN: detecting aut-of-distribution image without learning from out-of-distribution data. In CVPR, 2020.
Ding, B., Qin, C., Zhao, R., Luo, T., Li, X., Chen, G., Xia, W., Hu, J., Luu, A. T., and Joty, S. Data augmentation using llms: Data perspectives, learning paradigms and challenges. In ACL, 2024.
Huang, R. and Li, Y. MOS: towards scaling out-of- distribution detection for large semantic space. In CVPR, 2021.
Du, X., Wang, Z., Cai, M., and Li, Y. VOS: learning what you don't t know by virtual outlier synthesis. In ICLR, 2022.
Katz-Samuels, J., Nakhleh, J. B., Nowak, R., and Li, Y. Training ood detectors in their natural habitats. In ICMI., 2022.
Du, X., Sun, Y., Zhu, J., and Li, Y. Dream the impossible: Outlier imagination with diffusion models. In NeurlPS, 2023.
Katz-Samuels, J., Nakhlch, B., Nowak, R. D., and Li, Y. Training OOD detectors in their natural habitats. In ICML, 2022.
Du, X., Fang. Z., Diakonikolas, I., and Li, Y. How does un- labeled data provably help out-of-distribution detection? In ICLR, 2024.
Kurakin, A., Goodfellow, I. J., and Bengio, S. Adversarial examples in the physical world. In ICLR, 2017.
Dunlap, L., Umino, A., Zhang, H., Yang, J., Gonzalez, J. E., and Darrell, T. Diversify your vision datasets with automatic diffusion-based augmentation. In NeurlPS, 2023.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. In NewrlPS, 2017.
Elflein, S., Charpentier, B., Zügner, D., and Günnemann, S. On out-of-distribution detection with energy-based models. arXiv:2107.08785 2021.
Lee, K., Lee, H., Lee, K., and Shin, J. Training confidence- calibrated classifiers for detecting -fitt--disttibuttin sam- ples. In ICML, 2018a
Lee, K., Lee, K., Lee, H., and Shin, J. simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, 2018b.
Fang, Z., Li, Y., Lu, J., Dong, J., Han, B., and Liu, F. Is out- of-distribution detection learnable? In NeurlPS, 2022.
Fort, S., Ren, J., and Lakshminarayanan, B. Exploring the limits of out-o--distribution detection. NeurIPS, 2021.
Li, B., Xu, X., Wang, X., Hou, Y., Feng, Y., Wang, F., Zhang, X., Zhu, Q., and Che, W. Semantic-guided generative image augmentation method with diffusion models for image classification. In AAAI, 2024.
Fu, Y., Chen, C., Qiao, Y., and Yu, Y. Dreamda: Generative data augmentation with diffusion models. 1rXiv:2403.12803, 2024.
Liang, S., Li, Y., and Srikant, R. Enhancing the reliability of outto--disti image detection in neural networks. In ICLR, 2018.
Gal, Y. and Ghahramani, Z. Dropout as a bayesian approxi- mation: Representing model uncertainty in deep learning. In ICML,, 2016.
Lin, Z., Roy, S. D., and Li, Y. Mood: Multi-level out-of- distribution detection. In CVPR, 2021.
Goodfellow, I., Pbudet-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In NeurlPS, 2014.
Liu, W., Wang, X., Owens, J., and Li, Y. Energy-based distr--iistribution detection. In NeurIPS. 2020.
Hc, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, 2016.
Malinin, A. and Gales, M. Predictive uncertainty estimation via prior networks. In NeurlPS, 2018
10

---

BOOD: Boundary-based Out-Of-Distribatiin Data Generation
Mickisch, D., Assion, F., Greßner, F., Günther, W., and Motta, M. Understanding the decision boundary of deep neural networks: An empirical study. arXiv:2002.01810, 2020.
Tao, L., Du, X., Zhu, J., and Li, Y. Non-parametric outlier synthesis. In ICLR, 2023a.
Tao, M., Bao, B., Tang, H., and Xu, C. GALIP: generative adversarial clips for lext-to-image synthesis. In CVPR, 2023b.
Ming, Y., Fan, Y., and Li, Y. Poem: Out-of-distribution detection with posterior sampling. In ICMI,, 2022.
Vaze, S., Han, K., Vedaldi, A., and Zisserman, A. Open-set recognition: A good closed-set classifier all you need, In ICLR, 2022.
Ming, Y., Sun, Y., Dia, O., and Li, Y. How exploit hyper- spherical embeddings for -ut-of-distribution detection? In ICLR, 2023.
Wang. H., Li, Z., Feng, L., and Zhang, W. Vim: Out-of- distribution with virtual-logit matching. CVPR, 2022.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A. Y., et al. Reading digits in natural images with unsupervised feature learning. In NeurlPS Workshop, 2011.
Xiao, J., Hays, J.. Ehinger, K. A., Oliva, A., and Torrala,, A. SUN database: Large-scale scene recognition from abbey to 2oo. In CVPR, 2010.
Osawa, K., Swaroop, S., Khan, M. E., Jain, A., Eschen- hagen, R., Turner, R. E., and Yokota, R. Practical deep learning with bayesian principles. In NeurlPS, 2019.
Xu, P., Ehinger, K. A., Zhang, Y., Finkelstein, A., Kulkarni, S. R., and Xiao, J. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. vrXiv:150.00655. 2015.
Pei, S., Zhang, X., Fan, B., and Meng. G. Out-of- distribution detection with boundary aware learning. In ECCV, 2022.
Yang, J., Wang, P., Zou, D., Zhou, Z., Ding, K., Peng, W., Wang, H., Chen, G., Li, B., Sun, Y., Du, X., Zhou, K., Zhang, W., Hendrycks, D., Li, Y., and Liu, Z. Openood: Benchmarking generalized out-of-distribution detection. In NeurIPS, 2022.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, Learning transferable visual models from natural language supervision. In ICML, 2021.
Yang. J., Zhou, K., Li, Y., and Liu, Z. Generalized out-of- distribution detection: survey. IJV,, 2024a.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv:2204.06/25, 2022.
Yang, S., Cao, Z., Guo, S., Zhang, R., Luo, P., Zhang, S., and Nie, L. Mind the boundary: Coreset selection via reconstructing the decision boundary. In ICML, 2024b.
Ren, J., Fort, S., Liu, J., Roy, A. G., Padhy, S., and Lakshmi- narayanan, B. A simple fix to mahalanobis distance for improving near-ood detection. arXiv:2106.0900 2021.
Yeo, T. Atanov, A., Benoit, H., Alekseev, A., Ray, R., Akhoondi, P. E., and Zamir, A. Controlled training data generation with diffusion models. arXiv:2403. 15309, 2024.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
Yoon, S., Yoon, S., Sim, Y. S., Choi, S., Lee, K., Cho, H., Lee, H., and Lim, W. Diffusion- semantic outlier generation via nuisance awareness for disttiiiittiitoinn detection. In AAAI, 2025.
Shivashankar, C. and Miller, S. Semantic data augmentation with generative models. In CVPR, 2023.
Yousefzadeh, R. and O' O'Larry D. P. Deep learning interpreta- tion: Flip points and homotopy methods. In Mathematical and Scientific Machine Leurning, 2020.
Sun, Y. and Li, Y. DICE: leveraging sparsification for out- of-distribution detection. In ECCV, 2022.
Sun, Y., Guo, C., and Li, React: Dutto--dist detection with rectified activations. In NeurIPS, 2021.
Yu, F., Zhang, Y., Song, S., Seff, A., and Xiao, J. LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv:1506.0335 03360335.000.. 2015.
Sun, Y., Ming, Y., Zhu, X., and Li, Y. Out-of-distribution detection with deep nearest neighbors. In ICML, 2022.
Zaeemzadeh, A., Bisagno, N., Sambugaro. Z., Conci, N., Rahnavard, N., and Shah, M. Out-of-distribution detec- tion using union of I-dimensional subspaces. In CVPR, 2021.
Tajwar, F., Kumar, A., Xie, S. M., and Liang, P. No true sfate-of-the-arts ood detection methods are inconsistent across datasets, arXiv:2 100:2/0000000 095005554, 2021.

---

BOOD: Boundary-based Out-Or-Distribation Data Generation
Zhang, Y. Zhou, D., Hooi, B., Wang, K., and Feng. Expanding small-scale datasets with guided imagination. In NeurIPS, 2023.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Tor- ralba, A. Places: A 10 million image database for scene recognition. TPAMI, 2018.
Zhu, H., Yang, L., Yong. J., Yin, H., Jiang. J., Xiao, M., Zhang, W., and Wang. B. Distribution-awat data expan- sion with diffusion models. In NeurIPS, 2024.
12

---

BOOD: Boundary-based Out-Or-Distribation Data Generation
A. Datasets details
ImageNet-100 For IMAGENET-ION 100, we choose the following 100 classes from IMAGENET- 1K following DreamOOD's (Du
al., 2023) setting: m01498041, m01514859, nU1582220, nb1608432, n01616318, mb1687978, n01776313, ab1806567, ab1833805, ab1882714, #01910747, 601944990
a01985128, a02007558, n02071294,, n02085620, m02114855, n02123045, n02128385, u02129165, d02129604, e02165456, a02190166, n02219486, n02226429, n02279972.
n02317335, H02326432, m02342885, a02363005, m02391049, m02395406,
nf12422699, b02442845, n024448199 a024800855 m02510455, #02640242,
n02687172, n02701002, n02730930, n02769748, n02782093, n02787622, n02793495, n02799071, n02802426, n028 n02840245, m02906734, n02948072, n02980441,
w02994100 a03014705, n03028079, nU3032252. n03125729, n03160309, n03179701, 103220513, 103249569, 003291819, a05384352, a03388043, n03450230, n03481172.
n03594734, n08594945, n03627232, m03644206, n03649909, n03661043, n03676483, 003724870, 00573328 003759954, n05761084, n08773504, n03804744, n03910031,
m03938244, m04000767, nD4026417, n04090263, H04135789, n04153751, nd4296562, m04330267, DIM371774, 41M404412, n04465501, a0M485082, a04507155, n04536866.
n04579432, m04606251, n97714990, n07745940.
B. Additional visualization of the generated images
In this section, we provide additional visualizations of generated images.
Crab
Panda
MORT
Mouse
Zebra
Peur
Tractor
Scal
Violin
CIFAR-100
ImageNet- 100
Figure 7. Left: OOD images generated for CIFAR- 100. Right: OOD images generated for IMAGENET-100.
KruSuns
uay
HCT
lcopaad
Cooprd
doydii
dondey
DreamOOD
BOOD
Figure Left: OOD images generated with DreamOOD (Du et al., 2023). Right: OOD images generated with BOOD.
13

---

BOOD: Boundary-based Out-Of-Distribation Data Generation
C. Comparison between perturbation methods
To gain a deeper insight of the effectiveness of our strategy, we provide additional ablation studies (see table 6) on different perturbation strategies this section, including (1) adding Gaussian noises the latent features, (2) displacing features away from class centroids and (3) BOOD's perturbation strategy. The results illustrates that our perturbation strategies solid.
Table 6. Comparison of BOOD with different perturbation methods
Method
FPR95 4
AUROC f
(I)
18.99
95.04
40.51
91.63
BOOD
10.67
97.42
D. BOOD's performance on NearOOD datasets
We conduct the experiment selected NearOOD datasets: NINCO (Bitterwolf et al., 2023) and SSB-IIARD (Vaze al., 2022). Below performance results NearOOD using IMAGENET- 100 the ID dataset:
Table 7. Performance NearOOD datasets using ImageNet- 100 as the ID dataset
NINCO
SSB-HARD
Average
Method
FPR95 |
AUROC 1
FPR95
AUROC1
FPR95
AUROC
DreamOOD (Du et al., 2023)
57.08
89.51
77.29
74.77
67.19
82. 14
BOOD
54.37
89.82
75.48
80.82
64.93
85.32
From the table above we can see that in the NearOOD dataset, BOOD shows better performance than DreamOOD (Du et al., 2023), thus indicating that BOOD can handle more complex OOD datasets.
E. Architectures of model
For code reproducibility, (Sec 3.3) here: we choose standard ResNet-34 (He et al., 2016) for both of them, with the final linear transformation layer changed to 512 768 for image encoder (aligns with class token embeddings).
introduce model selection for image encoder(Sec 3.1) and OOD regularization model
We also provide comparison between BOOD's performance on ResNet-18 (He et al., 2016), ResNet-34 (He et al., 2016) and ResNet-50 (He et al., 2016) using CIFAR- as ID dataset:
Table Performance comparison across different ResNet backbones
Backbone
FPR95
AUROC
ID ACC
ResNet-18 (He et al., 2016)
10.83
97.74
78.11
ResNet-34 (He et al., 2016)
10.67
97.42
78.03
ResNet-50 (He et al., 2016)
11.23
96.89
79.68
From the table above we can see that the selection backbone architecture will not significantly affect the performance of BOOD.
14


Reviews:
[{"review_id": "ueFkd1vemR", "reviewer": "Reviewer_wxMm", "summary": "This paper introduces a framework for generating synthetic out-of-distribution (OOD) data by explicitly targeting decision boundaries in latent feature space. The proposed method, BOOD, employs an adversarial perturbation strategy to identify in-distribution (ID) features closest to the decision boundary and perturbs them along the gradient ascent direction to synthesize informative OOD features. These features are then decoded into human-compatible OOD images using a diffusion model. Experiments and ablation studies have presented to validate the efficiency of the framework.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to reviewer wxMm\n\nWe thank the reviewer for the comments.\n> Weakness 1: The compared methods are 2023 and before, how about the performance regarding to the most recent works?\n\nThank you for your concern regarding baseline methods. Please note that our work is based on using diffusion models to generate image-level OOD datasets. However, there are few work in this specific area. We provide comparison between BOOD and **a SOTA methods** FodFom[1] from ACMMM 2024, which also harnesses diffusion model to generate OOD images for enhancing OOD detection model. The results are summarized in the table below:\n\n|       | SVHN |  | LSUN-R |  | LSUN-C | | iSUN | | Textures |   | Places365 | |  Average |  |\n|:-:|:-:|:--:|:-:|:-:|:--:|:---:|:--:|:-:|:--:|:-:|:-:|:-:|:-:|:-:|\n| Method  | FPR95 &#8595; |  AUROC &#8593;  | FPR95 &#8595; | AUROC &#8593; | FPR95 &#8595;  |  AUROC &#8593; |FPR95 &#8595; |AUROC &#8593; |FPR95 &#8595; | AUROC &#8593; | FPR95 &#8595; | AUROC &#8593;   |  FPR95 &#8595; | AUROC &#8593; |\n|  FodFoM   | 33.19 | 94.02 | 28.24 | 95.09 | 26.79 | 95.04 | 33.06 | 94.45 |  35.44 | 93.38 | 42.30 | 90.68 | 33.17 | 93.78 | \n| BOOD | 5.42 | 98.43 | 0.10 | 99.94 | 2.06 | 99.25 | 0.22 | 99.91 | 5.1 | 98.74 | 40.55 | 90.76 | 8.91 | 97.84 |\n\nCompare to FodFom[1], **BOOD demonstrates superior performance**, illustrating its competitiveness.\n\n> Weakness 2: The discussion of OOD methods misses most works on 2024.\n\nThanks for your question related to the discussion on recent OOD methods. We will include them into our camera-ready version.\n\n> Weakness 3: The complexity analysis in appendix should be included in the main text, as the timing of diffusion-based methods is always an interested point for researchers in the community. Besides, the details about the complexity analysis are unclear. For instance, what the dataset used for Table 5, how about the timing when generating samples one by one? Does the memory requirements in Table 6 are for training? How about the inference?\n\nThanks for pointing out the this. We will include the detailed complexity analysis in the main text in our revised manuscript.\n\nRegarding the dataset used in Table 5, we use **CIFAR-100** as the ID dataset. It takes around **4** second to generate a single image in our framework on **Stable Diffusion v1.4** with **one NIVDIA L40S GPU**. And for Table 6, the memory requirements indicates the space needed for **storing the generated OOD images** used for training the final OOD detection model. There's no extra memory requirements during the inference time.\n\n> Weakness 4: There are many hyper-parameters. Although the paper provided curves of using different values, setting them for a new dataset is still not an easy job. This could be a disadvantage of this method for practical applications.\n\nWe appreciate the reviewer for the meaningful concerning. While fine tuning the parameter $c$ and $\\alpha$ may be somehow time-consuming, we would like to emphasize that $r$, $\\beta$ and $K$'s effect are **not remarkable** (Figure 6): the difference in AUROC are within 0.5% when $r < 10$ and within 0.4% for $\\beta \\in \\{1.5, 2, 2.5, 3\\}$, the performance will not change if $K > 50$. Thus, our suggestions for tuning the hyperparameters can make the process of tuning less difficult. We are working on a possible automatic adaptive method to adjust the additional pertuabation steps $c$ in the next step, aiming to reduce the tuning time.\n\n> Weakness 5: The quality of figures can be improved. The size of tables in the appendix can be adjusted.\n\nThank you for your advice, we will polish all the figures and tables in our camera-ready version of paper.\n\nWe thank the reviewer for taking the time to read our response and the positive feedback again. If you have any further concerns, we are willing to have discussions with you.\n\n[1] Jiankang Chen, Ling Deng, Zhiyong Gan, Wei-Shi Zheng and Ruixuan Wang. \"FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detector.\" ACMMM 2024.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "yes", "experimental_designs_or_analyses": "yes", "relation_to_broader_scientific_literature": "The paper could have broader impact on OOD detection, open-world pattern recognition methods, and image generation.", "essential_references_not_discussed": "no", "other_strengths_and_weaknesses": "Strengths:\n1. The paper is well written and easy to follow.\n2. The idea of exploring ood features in the boundary and using diffusion models to generate images from ood features is reasonable and interest.\n3. The method achieves better results than the compared methods on most evaluated datasets.\n4. The ability of generating image-level OOD samples would be beneficial to other domains of the community.\n\nWeaknesses:\n1. The compared methods are 2023 and before, how about the performance regarding to the most recent works?\n2. The discussion of OOD methods misses most works on 2024.\n3. The complexity analysis in appendix should be included in the main text, as the timing of diffusion-based methods is always an interested point for researchers in the community. Besides, the details about the complexity analysis are unclear. For instance, what the dataset used for Table 5, how about the timing when generating samples one by one? Does the memory requirements in Table 6 are for training? How about the inference?\n4. There are many hyper-parameters. Although the paper provided curves of using different values, setting them for a new dataset is still not an easy job. This could be a disadvantage of this method for practical applications.\n\n\n=================== post rebuttal ========================\nAfter read the rebuttal and other reviews, I would like to raise my score from weak accept to accept.", "other_comments_or_suggestions": "The quality of figures can be improved.\n\nThe size of tables in the appendix can be adjusted.", "questions_for_authors": "no", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m1EhCU75Ot", "reviewer": "Reviewer_Smhu", "summary": "This paper introduces a framework called Boundary-based Out-Of-Distribution data generation (BOOD). BOOD synthesizes high-quality OOD features and generates outlier images using diffusion models. \n\nThe BOOD framework learns a text-conditioned latent feature space from the ID dataset, selects ID features closest to the decision boundary, and perturbs them to cross the decision boundary to form OOD features.  These synthetic OOD features are then decoded into images in pixel space by a diffusion model.\n\nThe authors claim that BOOD provides a more efficient strategy for synthesizing informative OOD features, facilitating clearer distinctions between ID and OOD data. Experimental results on common benchmarks demonstrate that BOOD surpasses the state-of-the-art method significantly.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to reviewer Smhu\n\n> Essential References Not Discussed.\n\nThanks for recommendation for essential references. We will add them into our camera-ready paper.\n> Weakness 1: the process of tuning hyperparameters may be challenging.\n\nWe appreciate the reviewer for the meaningful concerning. While fine tuning the parameter $c$ and $\\alpha$ may be somehow time-consuming, we would like to emphasize that $r$, $\\beta$ and $K$'s effect are **not remarkable** (Figure 6): the difference in AUROC are within 0.5% when $r < 10$ and within 0.4% for $\\beta \\in \\{1.5, 2, 2.5, 3\\}$, the performance will not change if $K > 50$. Thus, our suggestions for tuning the hyperparameters can make the process of tuning less difficult. We are working on a possible automatic adaptive method to adjust the additional pertuabation steps $c$ in the next step, aiming to reduce the tuning time.\n> Weakness 2: BOOD's performance on ImageNet-200, CIFAR-10 and NearOOD datasets.\n\nSince our baseline method DreamOOD[1] does not include the performance on ImageNet-200 and CIFAR-10, we do not choose them as the ID dataset to guarantee comparison fairness. Due to the limited author response time, we cannot present the result of these two dataset, but we will include them in our camera-ready paper.\n\nWe summerize the experiment results on two selected NearOOD datasets **NINCO and SSB-hard** as below:\n\n||NINCO||SSB-hard||Average||\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Method|FPR95 &#8595;|AUROC &#8593;|FPR95 &#8595;|AUROC &#8593;|FPR95 &#8595;| AUROC &#8593;|\n|DreamOOD[1]|57.08|89.51|77.29|74.77|67.19| 82.14|\n|BOOD|54.37|89.82|75.48|80.82|64.93|85.32|\n\nFrom the table above we can see that in the NearOOD dataset, BOOD shows **better performance** than DreamOOD[1], thus indicating that BOOD can handle more complex OOD datasets.\n\n> Question 1: importance of class embedding alignment strategy, capability between feature space and the diffusion model.\n\nWe appreciate the reviewer for the reasonable concern. We aim to create a latent space that is compatible for the input space of diffusion model, so the alignment between the image features and their corresponding class token embeddings is necessary. By training an image encoder with **Formula (2)**, we ensure that the image features can be decoded by the diffusion models. Please check **Section 3.1** in our paper for detailed explanation.\n\n> Question 2: hyperparameter $\\alpha$ and $K$'s sensitivity on accuracy of identifying features closest to the decision boundary.\n\nWhile it's difficult to directly measure the accuracy, from our experiment on ablation of OOD feature synthesizing methodologies (Sec 4.3.1 and Table 3), we can find that **perturbing random ID features to the boundaries will significantly decrease the OOD detection performance** compared to using the ID features closest to the decision boundary. Therefore, we can measure the accuracy of identifying features closest to the decision boundary by the final OOD detection performance. \n\nEmploying **a relatively small $\\alpha$** facilitates a more nuanced differentiation between samples, and a large $\\alpha$ may lead to **large discrepancy** between each iteration in adversarial perturbation, making the counting of distance to decision boundaries not accurate. Selecting **a relatively large max iteration number $K$** can ensure **comprehensive boundary crossing for most features**. While increased $K$ do affect computational overhead in boundary identification, **the impact remains manageable**. We provide detailed discussion about the sensitivity of the hyperparameters $\\alpha$ and $K$ in Section 4.3.2, Figure 5 (left) and Figure 6 (right).\n\n> Question 3: backbone architecture and diffusion model's impact on BOOD's performance.\n\nWe provide comparison between BOOD's performance on Resnet-18, Resnet-34 and Resnet-50 using CIFAR-100 as ID dataset:\n\n|Backbone|FPR95 &#8595;|AUROC &#8593;|ID ACC &#8593;|\n|:-:|:-:|:-:|:-:|\n|ResNet-18|10.83|97.74|78.11|\n|ResNet-34|10.67|97.42|78.03|\n|ResNet-50|11.23|96.89|79.68|\n\nFrom the table above we can see that the selection of backbone architecture will **not significantly affect the performance of BOOD.**\n\nFor some datasets that has **large domain discrepancy** from the diffusion model's training distribution (Textures), BOOD's performance might be affected by the **diffusion model's capability**. But this **constraint is inherent to all methodologies utilizing diffusion-based generative data augmentation**. While future developments in generative modeling may address these limitations, **we emphasize that our primary goal is to leverage diffusion models to generate informative OOD images** thus increasing the OOD detection model's performance.\n\nWe believe we have responded to all your concerns. If anything remains unclear, we would be pleased to have further discussions with you.\n\n[1] Xuefeng Du, Yiyou Sun, Xiaojin Zhu and Yixuan Li. Dream the impossible-Outlier imagination with diffusion models. NeurIPS, 2023", "claims_and_evidence": "The claims made in the submission are generally well-supported by clear and convincing evidence. The authors have provided a thorough evaluation of their proposed method, BOOD, and compared it with several state-of-the-art approaches. The experimental results on common benchmarks demonstrate that BOOD surpasses the state-of-the-art method significantly.\n\nSpecifically, the main claims are supported by the following evidence:\n\n* **Claim:** BOOD surpasses the state-of-the-art method significantly.\n   \n    * **Evidence:** The experimental results in Table 1 and Table 2 show that BOOD outperforms other methods on both CIFAR-100 and IMAGENET-100 datasets.\n\n* **Claim:** BOOD provides a more efficient strategy for synthesizing informative OOD features, facilitating clearer distinctions between ID and OOD data.\n   \n    * **Evidence:** The ablation studies in Section 4.3.1 and the visualization of generated images support this claim.\n\n* **Claim:** The proposed framework is not time-consuming or has strict memory requirements.\n   \n    * **Evidence:** The computational cost comparison in Table 5 and the memory requirements comparison in Table 6 demonstrate that BOOD's computational and memory demands are comparable to those of DreamOOD, a state-of-the-art method. \n\nHowever, it is important to note that BOOD had many hyperparameters that affect the results significantly such as the step size and the number of perturbations steps and require some tuning.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria in this paper are generally well-suited for the problem of Out-Of-Distribution (OOD) detection.\n   \n* BOOD's approach of synthesizing OOD features by perturbing ID features near the decision boundary is a sound method for generating informative outliers.\n   \n* The use of diffusion models to generate human-compatible outlier images is a reasonable choice, given their strong generative capabilities\n   \n* The alignment of the image feature space with the diffusion-model-input space is a crucial step to ensure compatibility and effectiveness of the generated OOD data as done in DreamOOD.\n\n**Evaluation Criteria:**\n\n* The paper employs widely used benchmark datasets for OOD detection, such as CIFAR-100 and IMAGENET, which allows for comparison with other state-of-the-art methods.\n   \n* The evaluation metrics used, including False Positive Rate at 95% True Positive Rate (FPR95) and Area Under the Receiver Operating Characteristic Curve (AUROC), are standard metrics for evaluating OOD detection performance.\n   \n* The inclusion of in-distribution classification accuracy (ID ACC) as an evaluation metric is important to ensure that the OOD detection method does not compromise the model's performance on in-distribution data.\n\nOverall, the proposed methods and evaluation criteria are appropriate and well-justified for addressing the problem of OOD detection.\n\nHowever, it would be beneficial to report OOD performance on ImageNet-200 and CIFAR-10 and also to add distinction in test set between Near and Far OOD.", "theoretical_claims": "While the paper presents a novel framework and demonstrates strong empirical results, it doesn't contain significant theoretical claims that require in-depth proof verification. The core contributions are algorithmic and experimental, focusing on a new way to generate OOD data.", "experimental_designs_or_analyses": "The authors have taken reasonable steps to ensure the reliability and robustness of their findings. Here's a breakdown of the key aspects:\n\n**1. Ablation Studies:**\n\n* The authors conduct ablation studies to analyze the contribution of different components of their proposed BOOD framework.\n   \n* Specifically, they ablate the effect of boundary identification and feature perturbation.\n   \n* By comparing the performance of the full BOOD framework with variants where these components are removed or replaced, they demonstrate the importance of each component for achieving the best results.\n   \n* This ablation analysis helps to validate the design choices made in the BOOD framework.\n\n**2. Hyperparameter Analysis:**\n\n* The authors perform a hyperparameter sensitivity analysis to evaluate the impact of different hyperparameter settings on the performance of BOOD.\n   \n* They analyze the effect of step size, perturbation steps after crossing the boundary, pruning rate, OOD regularization weighting, and maximum perturbation steps. \n   \n* By varying these hyperparameters and observing the resulting changes in performance, they gain insights into the optimal settings for BOOD and the sensitivity of the method to these parameters.\n   \n* This analysis helps to ensure that the reported results are not due to a specific choice of hyperparameters and provides guidance for applying BOOD in different settings.\n\n**3. Comparison with State-of-the-Art Methods:**\n\n* The authors compare the performance of BOOD with several state-of-the-art OOD detection methods on benchmark datasets.\n   \n* This comparison allows them to demonstrate the superiority of BOOD over existing methods and provides evidence for the effectiveness of their proposed approach.\n   \n* The use of standard benchmark datasets and evaluation metrics ensures that the comparison is fair and objective.\n\n\nOverall, the experimental designs and analyses in the paper are well-structured, comprehensive, and appropriate for evaluating the proposed BOOD framework. The authors have carefully considered various factors that could affect the validity of their results and have taken steps to address them through ablation studies, hyperparameter analysis, and comparison with state-of-the-art methods.", "relation_to_broader_scientific_literature": "The key contributions of the paper are related to the broader scientific literature in the following ways:\n\n* **Out-of-Distribution (OOD) Detection:** The paper addresses the problem of OOD detection, which is a well-established area of research in machine learning.\n   \n* The goal of OOD detection is to identify inputs that come from a different distribution than the one the model was trained on.\n   \n* **Data Augmentation with Diffusion Models:** The paper leverages the power of diffusion models for data augmentation, which is a rapidly growing area of research.\n   \n* Diffusion models have shown remarkable success in generating high-quality and diverse images.\n   \n* The paper's approach is related to other works that have used diffusion models for data augmentation, including those that perform image generation with semantic guidance and those that use perturbation-based approaches to synthesize augmented images.\n   \n* **Synthesizing OOD Data:** A core contribution of the paper is the development of a novel framework, BOOD, for synthesizing OOD data.\n   \n* This contribution is closely related to prior work that has explored the use of auxiliary outlier datasets to improve OOD detection.\n   \n* **Explicitly Generating OOD Images near Decision Boundaries:** BOOD's innovation lies in its ability to generate image-level OOD data located around the decision boundaries between classes.", "essential_references_not_discussed": "The key contribution of this paper relies primarily on identifying the in-distribution (ID) features located near the decision boundary and subsequently perturbing these features away from that boundary to generate out-of-distribution (OOD) data. To provide context for this approach, it would be valuable to include references [1,2], as they propose methods for analyzing and approximating the distance to the decision boundary using adversarial attack concepts similar to those employed in this paper. These works offer complementary perspectives on understanding and characterizing decision boundaries in deep neural networks.\n\n\n### References:\n\n1- Mickisch, David, et al. Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study. arXiv:2002.01810, arXiv, 5 Feb. 2020. arXiv.org, https://doi.org/10.48550/arXiv.2002.01810.\n\n2- Yousefzadeh, Roozbeh, and Dianne P. O’Leary. “Deep Learning Interpretation: Flip Points and Homotopy Methods.” Proceedings of The First Mathematical and Scientific Machine Learning Conference, PMLR, 2020, pp. 1–26. proceedings.mlr.press, https://proceedings.mlr.press/v107/yousefzadeh20a.html.", "other_strengths_and_weaknesses": "**Strengths:**\n\n* **Originality:**\n    * The paper introduces a novel framework, BOOD, for generating OOD data. \n    * BOOD's originality lies in its explicit focus on generating OOD images located around the decision boundaries between classes. \n* **Significance:**\n    * The proposed BOOD framework offers a promising approach to improve OOD detection by generating informative OOD data.\n    * The experimental results demonstrate that BOOD outperforms state-of-the-art methods on benchmark datasets, highlighting the significance of the contribution. \n* **Clarity:**\n    * The paper is generally well-written and easy to follow.\n    * The problem is clearly defined, and the proposed approach is well-motivated. \n\n**Weaknesses:**\n* **Hyperparameter Sensitivity:**\n    * The performance of BOOD depends on several hyperparameters, such as step size, perturbation steps, pruning rate, and OOD regularization weighting.\n    * Although the authors conduct a hyperparameter sensitivity analysis, the process of tuning these parameters for new datasets or applications may be challenging. \n* **Generalization to More Complex Datasets:**\n    * The experiments are conducted on CIFAR-100 and IMAGENET-100. \n    * While these are standard benchmark datasets, it is not clear how well BOOD would generalize to more complex and datasets or to NearOOD test sets as the one in the paper are considered FarOOD.", "other_comments_or_suggestions": "NA", "questions_for_authors": "1 -  The authors mention that they \"employ a class embedding alignment strategy during the image encoder training following Du et al. (2023)\". It would be helpful to understand why this strategy is important for generating OOD images. Does this alignment ensure compatibility between the feature space and the diffusion model?\n\n2- The authors use an adversarial perturbation strategy to identify ID boundary features. How sensitive is this boundary identification process to the choice of hyperparameters, such as the step size α and the maximum iteration number K? How do these parameters influence the accuracy of identifying features closest to the decision boundary?\n\n3- The authors compare BOOD with several state-of-the-art OOD detection methods. Could they discuss how BOOD's performance might be affected by the choice of the backbone architecture or the diffusion model used for image generation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7Ex8QZrUwP", "reviewer": "Reviewer_Mr32", "summary": "This paper studies the problem of out-of-distribution (OOD) detection for image tasks. The authors leverage text-to-image latent diffusion models to synthesize OOD images that are used to train the binary OOD detector. In doing so, they follow a three-step strategy: 1) Identifying the ID samples that are closest to the decision boundary by measuring the number of steps (k) required to perturb the ID feature along the gradient ascent direction until the model's prediction changes (this number of steps serves as a proxy for the shortest distance between the InD sample and the decision boundary); 2) perturbing the identified features from step 1 to get OOD embeddings, and 3) using OOD embeddings from step 2 to generate OOD images using the diffusion model. Experiments on CIFAR-100 and ImageNet-100 as ID show improvements in performance from previous scoring-based methods as well as synthesis-based methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to reviewer Mr32\n\nWe appreciate the review for providing valuable advice. Below are our responses:\n\n> Claims and Evidence 1: there is no clear evidence showing that the method is truly efficient\n\nWe apologize for the inaccurate expression in the abstract: BOOD provides a more **training efficient** strategy for synthesizing informative OOD features. We will fix this part in the updated manuscript, thank you again for pointinng this out.\n\n> Claims and Evidence 2: need to provide a visual example showing that the feature indeed returns to the ID region.\n\nThank you for your advice. Please check **Figure 2** in our paper, which shows the perturbation process of two features from one ID class to another ID class: one from **Tiger** to **Fish**, another one from **Streetcar** to **House**. We will also include more visual examples illustrating the perturbation process.\n\n> Some essential References Not Discussed.\n\nWe extend our gratitude to the reviewer for the recommendation of essential references. We will update the corresponding part in the related works of paper in the camera-ready version.\n\n> Weakness 1: The third step of the method (the OOD image generation) is based on the approach in Du et al. (2023), while the regularization technique is derived from VOS (Du et al., 2022). These elements are largely borrowed from prior works and integrated into the BOOD framework. This raises concerns about the novelty of the method.\n\nWe thank the reviewer for the concern regarding the novelty. We would like to emphasize that **our framework mainly focus on the generation of imformative OOD features** in the latent space. **To guarantee the fairness in comparison with the previous SOTA method DreamOOD in synthesis-based OOD detection frameworks**, we follow the same regularization function as them. After generating the OOD dataset, the regularization function can be substitute by other common functions to train the OOD detection model.\n\n> Weakness 2: Code not provided.\n\nThanks for concern regarding the code. We will release our code, data and model once the paper is accepted.\n\n> Comments: In the related work section, use ‘citet’ instead of ‘citep’ for “…guidance. (Dunlap et al., 2023) proposes to caption the images…” and “…model. (Li et al., 2024) generated augmented images with the guidance…”.\n\nThanks for pointing out the minor mistakes, we will update them in the revised manuscript.\n\n> Question: Why is regularization used in the OOD detection model in Section 3.3? What is the significance of this regularization, and what is the underlying intuition? It would be helpful to mention this in this paragraph. Moreover, why does setting such a large value of $\\beta$ help; what could be a potential rationale?\n\nThe significance for regularization the OOD detection model is to keep model's ability of performing visual recognition task while regularizing the model to identify the generated OOD images. Equation (7) shapes the uncertainty surface, which predicts high probability for ID data and low probability for OOD data. Compared to other regularization method such as Energy-bounded[1], Equation (7) is **hyperparameter-free so easier to be implemented.**\n\nOur analysis suggest setting a relatively larger regularization weighting $\\beta$ to ensure better performance (Figure 6 middle). The potential reationale could be: using a relatively larger $\\beta$ will help to **force the model to push OOD samples away** thus enhance the model's ablility for distinguishing ID and OOD input. However, if the $\\beta$ is too large, the model will be **over-regularized** and the performance for OOD detection will decrease.\n\nWe sincerely thank you for reviewing our response and your constructive feedback. If any aspects still need clarification, we are happy to discuss them further.\n\n[1] Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li. Energy-based out-of-distribution detection. NeurIPS, 2020", "claims_and_evidence": "Yes, most claims are supported by evidence.\n\nHowever, some parts could be clearer. For example, in the abstract, the authors state that “…BOOD provides a more efficient strategy for synthesizing informative OOD features…”. However, there is no clear evidence showing that the method is truly efficient. Although the authors compare computation and memory usage in Tables 5 and 6, the differences are not significant, and in many cases, BOOD uses more computation than DreamOOD. I also believe the comparison should include more baselines, not just one. It might be better to either remove this claim or rephrase it more modestly.\n\nAdditionally, in Section 4.3.1, the authors claim that “Employing a large c may force the feature to step into the ID region.” But is there evidence to support this, apart from the performance degradation? Would it be possible to provide an experiment (like a visual example) showing that the feature indeed returns to the ID region? For instance, it would have been useful to see this in Figure 3 (left) with more steps, where the image might begin to resemble the ID image.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Yes, I checked the algorithm and experimental designs, and they look reasonable to me.", "relation_to_broader_scientific_literature": "The paper addresses the OOD detection problem, which is a well-established problem in the machine learning literature. Most previous works have focused on using scoring methods (e.g., MSP, DICE, Energy, etc.) to tackle the OOD detection problem. More recent works have shifted towards generating OOD samples using generative models, such as GANs and diffusion models. This work adopts the latter approach, utilizing diffusion models to generate OOD samples and then trains a binary OOD detector using both ID samples and the synthesized OOD samples.", "essential_references_not_discussed": "While the paper discusses most of the relevant works, there is a recent synthesis-based method that was not mentioned, let alone compared against, in the paper. Specifically, the work titled Diffusion-based Semantic Outlier Generation via Nuisance Awareness for Out-of-Distribution Detection by Suhee Yoon et al. (2024) (https://arxiv.org/abs/2408.14841) presents an interesting approach that leverages diffusion models for OOD detection. Including a discussion of this work would have enhanced the paper's context and provided a valuable comparison.\n\nThere are some other important studies that could have been included to provide a more comprehensive context. Recent works, for instance, have explored the use of unlabeled wild data to enhance OOD detection, achieving state-of-the-art results on datasets similar to those used in this paper. For example, Du et al. (2024) achieve an FPR95 of 0.07 on the CIFAR100-SVHN ID-OOD pair, compared to 5.42 in this paper. Similarly, on the CIFAR100-PLACES365 ID-OOD pair, Du et al. (2024) report an FPR95 of 3.53, while this paper has an FPR95 of 40.55. Some of them are:\n\n•\tXuefeng Du, Zhen Fang, Ilias Diakonikolas, and Yixuan Li. \"How does unlabeled data provably help out-of-distribution detection?\" ICLR, 2024.\n\n•\tJulian Katz-Samuels, Julia Nakhleh, Robert Nowak, and Yixuan Li. \"Training OOD detectors in their natural habitats.\" ICML, 2022.\n\nWhile these studies use unlabeled data, whereas this paper focuses on a synthesis-based method, it would have been valuable to discuss the literature on unlabeled data and its impact on OOD detection, as it provides important context and alternative approaches to the problem.", "other_strengths_and_weaknesses": "**Strengths:**\n\nThe paper is well-motivated, well-written, and easy to follow. The methodology is clearly explained, with illustrative examples provided in the figures, which helped in understanding better.\n\nGenerally, I finding the algorithm design decisions quite intuitive and systematic. The method is simple yet effective.\n\n**Weaknesses:**\n\nThe third step of the method (the OOD image generation) is based on the approach in Du et al. (2023), while the regularization technique is derived from VOS (Du et al., 2022). These elements are largely borrowed from prior works and integrated into the BOOD framework. This raises concerns about the novelty of the method.\n\nThe authors did not provide their code to reproduce the results, which raises concerns about the transparency of the findings.", "other_comments_or_suggestions": "Minor: in the related work section, use ‘citet’ instead of ‘citep’ for “…guidance. (Dunlap et al., 2023) proposes to caption the images…” and “…model. (Li et al., 2024) generated augmented images with the guidance…”.", "questions_for_authors": "Why is regularization used in the OOD detection model in Section 3.3? What is the significance of this regularization, and what is the underlying intuition? It would be helpful to mention this in this paragraph. Moreover, why does setting such a large value of $\\beta=2.5$ help; what could be a potential rationale?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OZMfV5g6JA", "reviewer": "Reviewer_sEzZ", "summary": "This paper focuses on addressing the OOD detection task by synthesizing OOD samples. To generate plausible OOD samples, samples near the OOD boundary are first selected and then perturbed along the direction of gradient ascent until their predicted labels change. Finally, a diffusion model is applied to generate OOD samples from these perturbed features, which are then used to train an OOD classifier. Experiments on various datasets demonstrate that this method outperforms existing approaches.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to reviewer sEzZ\n\nWe thank the reviewer for the feedback and constructive suggestions. Our response to the reviewer’s concerns is below:\n\n> Weakness 1: How can we ensure that during the generation from 0 to c, the original ID sample of class y is not mistakenly perturbed into another ID class? Simply controlling the step size α and number of steps c may make the model overly sensitive to hyperparameters.\n\nWe appreciate this important inquiry regarding hyperparameters. We would like to emphasize that OOD features are the features that distributed between classes, so they are **ambiguous and likely to confuse the classifier**. But if BOOD generates features that are distributed in the ID area, the final OOD image will be considered as **noisy data** and **be harmful to the model performance**. While we don't have an explicit way to measure whether image features have been mistakenly perturbed into another ID class, **by setting relatively small $\\alpha$ and $c$ values**, we can guarantee that the generated OOD features will located around the decision boundaries.  Figure 5 shows that the performance of OOD detection rises firstly and latter decrases as $c$ increases, which illustrates **our control of $c$ are effective and most of the generated OOD images are effective**. In the next step, we are working on filtering mechanism to rule out the potential mistakenly perturbed features.\n\n> Weakness 2: I would like to understand whether the performance improvement over the DreamOOD method arises from generating OOD data with lower error rates, or from a higher number of OOD samples being situated closer to the decision boundary.\n\nThank you for the interesting point. Would you clarify the meaning of \"**lower error rates**\" in the context?\n\nAssume the \"lower error rates\" indicates **the rate of mistakenly generated ID images in the generated OOD data**. Since OOD data are the samples that are not distributed in the input data distributions, thus they are hard to be classified by the classifier and difficult to decide whether they are OOD samples. Our method intend to find the boundaries of ID data and perturb ID features which are cloestest to the boundary to the area between classes and close to the decision boundaries in the latent space, thus it's diffiucult to quantify the error rate of the generated OOD data.\n\nWe also conduct the experiment comparing the feature's average distance to their closest decision boundaries between DreamOOD[1] and our method:\n\n| Average distance | ImageNet-100 | CIFAR-100 |  \n|:--:|:--:|:--:|\n|DreamOOD | 3.70 | 5.23 |\n|BOOD| 2.29 | 4.01 |\n\n\nThe results show that BOOD's generated features are **closer to the closest decision boundaries**, resulting in the improvement of performance in OOD detection task.\n\n> Weakness 3: I have noticed that 100,000 generated OOD images were used during training. Would reducing or increasing the number of images have an impact on the ID accuracy or OOD metrics?\n\nGreat point! Here we list the performance statistics of BOOD and DreamOOD[1] training on different number of OOD images (use CIFAR-100 as ID dataset):\n\n| Total OOD images | FPR95 $\\downarrow$ | AUROC $\\uparrow$ | ID ACC $\\uparrow$ |\n|:--:|:--:|:--:|:--:|\n| DreamOOD-10k | 60.23 | 81.84 | 65.39 |\n| DreamOOD-50k | 48.66 | 85.71 | 72.95 |\n| DreamOOD-100k | 40.31 | 90.15 | 78.94 |\n| BOOD-10k | 25.21 | 93.63 | 65.14 |\n| BOOD-50k | 15.83 | 96.1 | 73.18 |  \n| BOOD-100k | 12.47 | 97.34 | 78.17 |\n\nFrom the table above we can find that **as the number of OOD training images increases**, both BOOD and DreamOOD's[1] performance on OOD detection and ID classification **increases**.\n\nWe hope we have responded to all your concerns. If you have further questions, we are pleased to discuss with you. Thank you again for taking the time to read our response and your constructive feedback!\n\n[1] Xuefeng Du, Yiyou Sun, Xiaojin Zhu and Yixuan Li. Dream the impossible-Outlier imagination with diffusion models. NeurIPS, 2023", "claims_and_evidence": "The claims are clear; however, the method has inherent limitations that may push an in-distribution (ID) class toward another ID class.", "methods_and_evaluation_criteria": "The proposed methods are clear; however, there are certain limitations. The performance on the benchmark datasets is acceptable.", "theoretical_claims": "I have reviewed the theoretical claims, which primarily involve gradient ascent and related theoretical concepts. The limitations of the proposed method are already reflected in the theoretical claims.", "experimental_designs_or_analyses": "Yes, I have checked the experimental designs. The paper uses CIFAR-100 and ImageNet-100 as the ID datasets, which are consistent with the DreamOOD settings. The experimental setup seems sound and appropriately chosen for the task at hand.", "relation_to_broader_scientific_literature": "This paper, inspired by DreamOOD's use of nearest neighbor distance for OOD sample generation, employs gradient ascent to generate OOD samples, providing a more directed generation approach.", "essential_references_not_discussed": "All relevant works have been appropriately cited in the paper, and there are no crucial studies or findings that have been overlooked.", "other_strengths_and_weaknesses": "Strengths:\n1.\tThis paper proposes a novel boundary-based OOD data generation method that leverages a diffusion model to identify ID data closest to the decision boundary and applies an outlier feature synthesis strategy to generate images near the decision boundary. This approach provides high-quality and information-rich features for OOD detection.\n2.\tIn terms of performance, the gain on CIFAR-100 as the ID is significant compared to existing methods.\n3.\tThe writing and expression of the paper are clear. The images generated under boundary conditions are reasonable.\nWeaknesses:\n1.\tHow can we ensure that during the generation from 0 to c, the original ID sample of class y is not mistakenly perturbed into another ID class? Simply controlling the step size α and number of steps c may make the model overly sensitive to hyperparameters.\n2.\tI would like to understand whether the performance improvement over the DreamOOD method arises from generating OOD data with lower error rates, or from a higher number of OOD samples being situated closer to the decision boundary.\n3.\tI have noticed that 100,000 generated OOD images were used during training. Would reducing or increasing the number of images have an impact on the ID accuracy or OOD metrics?", "other_comments_or_suggestions": "I have no additional comments or suggestions.", "questions_for_authors": "I have no further questions for the authors.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces BOOD, a framework for synthesizing out-of-distribution (OOD) data by explicitly perturbing in-distribution (ID) features near decision boundaries in latent space and decoding them into images via diffusion models. The method aims to generate informative OOD samples to improve OOD detection performance. All reviewers (wxMm, Smhu, Mr32, sEzZ) acknowledged the paper’s merits, including technical soundness and empirical performance.

The authors addressed some concerns (e.g., adding FodFoM comparisons, complexity analysis), but some limitations remain relevant for future work. The camera-ready version could better contextualize these weaknesses (e.g., computational costs as inherent to diffusion models, hyperparameter tuning guidelines), if the paper is finally accepted.
Decision: Accept (poster)