VOS: Learning What You Don't Know by Virtual Outlier Synthesis
Authors: Xuefeng Du
Zhaoning Wang
Mu Cai
Yixuan Li
Venue: ICLR 2022 Poster (2022)

Abstract: Out-of-distribution (OOD) detection has received much attention lately due to its importance in the safe deployment of neural networks. One of the key challenges is that models lack supervision signals from unknown data, and as a result, can produce overconfident predictions on OOD data. Previous approaches rely on real outlier datasets for model regularization, which can be costly and sometimes infeasible to obtain in practice. In this paper, we present VOS, a novel framework for OOD detection by adaptively synthesizing virtual outliers that can meaningfully regularize the model's decision boundary during training. Specifically, VOS samples virtual outliers from the low-likelihood region of the class-conditional distribution estimated in the feature space. Alongside,  we introduce a novel unknown-aware training objective,  which contrastively shapes the uncertainty space between the ID data and synthesized outlier data. VOS achieves competitive performance on both object detection and image classification models, reducing the  FPR95 by up to 9.36% compared to the previous best method on object detectors. Code is available at https://github.com/deeplearning-wisc/vos.

Published as a conference paper at ICLR 2022
VOS: LEARNING WHAT YOU DON'T KNOW BY VIRTUAL OUTLIER SYNTHESIS
Xuefeng Du, Zhaoning Wang. Mu Cai, Yixuan Li Department of Computer Sciences University of Wisconsin Madison {xfdu, mucai, sharonli ess. wisc.
ABSTRACT
Out-of cuttof-diittribttinn (OOD) detection has received much attention lately due to its importance the safe deployment of neural networks. One of the key challenges is that models lack supervision signals from unknown data, and result, produce overconfident predictions on OOD data. Previous approaches rely on real outlier datasets for model regularization, which can be costly and sometimes in- feasible to obtain in practice. In this paper, we present VOS, novel framework for OOD detection by adaptively synthesizing virtual outliers that meaningfully regularize the model's decision boundary during training. Specifically, VOS sam- ples virtual outliers from the low-likelihood region of the class-conditional distri- bution estimated in the feature space. Alongside, we introduce novel unknown- aware training objective. which contrastively shapes the uncertainty space be- tween the ID) data and synthesized outlier data. VOS achieves competitive per- formance on both object detection and image classification models, reducing the FPR95 by to 9.36% compared to the previous best method on object detectors. Code is available at https: /ttg//ggttttttttee com wttp://eee/ee..n....e..eee wisc/ vos
INTRODUCTION
Modern deep neural networks have achieved unprecedented success in known contexts for which they are trained, yet they often struggle to handle the unknowns. In particular, neural networks have been shown to produce high posterior probability for out-of-distributiin (OOD) test inputs (Nguyen et al.] 2015), which arise from unknown categories and should not be predicted by the model. Taking self-driving car as an example, an object detection model trained to recognize in-distribution objects (e.g., cars, stop signs) can produce high-confidence prediction for unseen object of moose; see Figure [a). Such a failure case raises concerns model reliability, and worse, may lead to catastrophe when deployed in safety-critical applications.
The vulnerability to OOD inputs arises due to the lack explicit knowledge of unknowns during training time. In particular, neural networks typically optimized only the in-distribution (ID) data. The resulting decision boundary, despite being useful on ID tasks such as classification, can be ill-fated for OOD detection. We illustrate this in Figure The ID data (gray) consists of three class- conditional Gaussians, on which a three-way softmax classifier is trained. The resulting classifier overconfident for regions far away from the ID data (see the red shade Figure [bb),, causing trouble for OOD detection. Ideally, a model should learn more compact decision boundary that produces low uncertainty for the ID data, with high OOD uncertainty elsewhere (e.g., Figure [c)). However, achieving this goal is non-trivial due to the lack of supervision signal of unknowns. This motivates the question: Can we synthesize virtual outliers for effective model regularization?
In this paper, we propose a novel unknown-aware learning framework dubbed VOS (Virtual Outlier Synthesis), which optimizes the dual objectives of both ID task and OOD detection performance. In a nutshell, VOS consists of three components tackling challenges of outlier synthesis and ef- fective model regularization with synthesized outliers. To synthesize the outliers, we estimate the class-conditional distribution in the feature space, and sample outliers from the low-likelihood re- gion of ID classes (Section 3.I). Key to our method, we show that sampling in the feature space more tractable than synthesizing images in the high-dimensional pixel space (Lee et al. 201 8a).

---

Published as a conference paper at ICLR 2022
High 00
High Daors
Far 97% pedestrian 89%
Low piae
(a) Overcenfident predictions on OOD data
(b) Uncertainty measure on model trained without outliers (not idcal)
(e) Uncertainty measure model trained with virtual outliers (ours)
2015 model trained BDD- 100k dataset Yu et al. 2020 produces overconfident predictions for OOD object (e.g., moose). (b)-(c) The uncertainty measurement
Figure 1: A Faster-RChy (Ren et
and without virtual outlier training. The in-distribution data x E X = R is sampled from a Gaussian mixture model). Regularizing the model with virtual outliers (c) better captures the OOD uncertainty than without (b).
Alongside, propose novel unknown-aware training objective, which contrastively shapes the uncertainty surface between the ID data and synthesized outliers (Section 3.2). During training, vos simultaneously performs the ID task (e.g., classification or object detection) as well the OOD un- certainty regularization. During inference time, the uncertainty estimation branch produces larger probabilistic score for ID data and vice versa, which enables effective OOD detection (Section 3.3
VOS offers several compelling advantages compared to existing solutions. (1) VOS is a general leariing framework that is effective for both object detection and image classification tasks, whereas previous methods were primarily driven by image classification. Image-level detection can be lim- iting as an image could be OOD in certain regions while being in-distribution elsewhere. Our work bridges critical research gap since OOD detection for object detection is timely yet underexplored in literature. (2) VOS enables adaptive outlier synthesis, which can be flexibly and conveniently used for any ID data without manual data collection cleaning. In contrast, previous methods ing outlier exposure (Hendrycks et al. 2019) require an auxiliary image dataset that is sufficiently be arguably prohibitive to obtain. Moreover,
diverse, which cleaning to ensure the auxiliary outlier dataset does not overlap with ID data. (3) VOS synthesizes outliers that can estimate compact decision boundary between ID and OOD data. In contrast, ex- isting solutions use outliers that are either too trivial to regularize the OOD estimator, or too hard to be separated from ID data, resulting in sub-optimal performance. Our key contributions and results are summarized as follows:
needs to perform careful data
We propose a new framework VOS addressing a pressing issue-unknwwn-wwarr deep learning that optimizes for both ID and OOD performance. VOS establishes state-of-the-art results on a challenging object detection task. Compared to the best method, VOS reduces the FPR95 by up to 9.36% while preserving the accuracy the ID task.
We conduct extensive ablations and reveal important insights by contrasting different out- lier synthesis approaches. We show that VoS is more advantageous than generating outliers directly in the high-dimensional pixel space (e.g., using GAN (Lee et al. 2018a)) using noise as outliers.
We comprehensively evaluate our method on common OOD detection benchmarks, along with more challenging yet underexplored task in the context of object detection. Our effort facilitates future research evaluate OOD detection a real-world setting.
PROBLEM SETUP
We start by formulating the problem of OOD detection in the setting of object detection. Our frame- work can be easily generalized to image classification when the bounding box is the entire image (see Section 4.2) Most previous formulations of OOD detection treat entire images anomalies, which can lead to ambiguity shown in Figure I particular, natural images are composed of nu- merous objects and components. Knowing which regions of an image anomalous could allow for safer handling unfamiliar objects. This setting is more realistic in practice, yet also more challenging it requires reasoning OOD uncertainty at the fine-grained object level.
2

---

Published as a conference paper at ICLR 2022
Regression Head
Lice
Backbone Network
ID Samples h(x. b) Classification Head
b
Lineer Transformtiion
Ccls
A AnchogPruposa Generator
Uncerainty sooce E
000
Detaily
Virtual Outliess
Nonlinear MLP
Input x
Euncertainty
Negalive Enenty Soott
Generated Virual Outliers
Po b) V K
The framework of VOS. We model the feature representation of ID objects as class-conditional Gaussians, and sample virtual outliers v from the low-likelihood region. The virtual outliers, along with the ID objects, used to produce the uncertainty loss for regularization. The uncertainty estimation branch (Luncortainty) ) is jointly trained with object detection loss (Cioey Lela).
Figure 2:
Specifically, we denote the input and label space by Rd and y = (1.2..., K). respectively. Let x X' be the input image, b I be the bounding box coordinates associated with object instances in the image, and y E y be the semantic label for K K-way classification. An object detec- tion model is trained on in-distribution data D = {(x,, b,, }= drawn from an unknown joint distribution P. We use neural networks with parameters pe Pe(bix x) and the classification pa ( b).
model the bounding box regression
The OOD detection can be formulated as binary classification problem, which distinguishes be- tween the in- vs. out-of-distribution objects. Let Px denote the marginal probability distribution on X. Given a test input x* Px. as well an object instance b" predicted by the object detector, the goal is to predict po (g) x", b"). We use g = 1 to indicate detected object being in-distribution, and g = being out-of -if-iistriiution, with semantics outside the support of y
METHOD
Our novel unknown-aware learning framework is illustrated Figure 2 Our framework encom- passes three novel components and addresses the following questions: (1) how synthesize the virtual outliers (Section 3.1). (2) how to leverage the synthesized outliers for effective model regu-
larization (Section 3.2). and (3) how perform OOD detection during inference time (Section 3.3
3.1
VOS: VIRTUAL OUTLIER SYNTHESIS
Our framework VOS generates virtual outliers for model regularization, without relying on external data. While straightforward idea is to train generative models such GANs (Goodfellow et al.) 2014
Lee et al. (018a),, synthesizing images the high-dimensional pixel space can be difficult to optimize. Instead, our key idea is to synthesize virtual outliers in the feature space, which is more tractable given lower dimensionality. Moreover, our method is based on a discriminatively trained classifier in the object detector, which circumvents the difficult optimization process in training generative models.
Specifically, assume the feature representation of object instances forms class-conditional mul- tivariate Gaussian distribution (see Figure 3):
Pe (1 b)ly = k) = NN E),
where st is the Gaussian mean of class k E 21, K), E is the tied covariance matrix, and h(x, b) Rm the latent representation of an object instance (x, b). extract the latent repre- sentation, we use the penultimate layer of the neural network. The dimensionality m is significantly smaller than the input dimension
3

---

Published as a conference paper at ICLR 2022
To estimate the parameters of the class-conditional Gaussian, com- pute empirical class mean fåg and covariance Σ of training samples {(x,, b; v,)l
1
Σ
h(x,, bi)
(1)
NE
sigi=k
Σ
N Σ
Σ u
(h(x;, b,) (ixs) (h(x;; b.) m
UMAP vi- sualization of feature em- beddings of PASCAL-VOC (on subset of classes).
Figure 3:
where N₂ is the number of objects in class k; and N is the total number of objects. We use online estimation for efficient training, where maintain a class-conditional queue with Qu object instances from each class. In each iteration, we enqueue the embeddings of objects to their corresponding class-conditinal queues, and dequeue the same number of object embeddings.
Sampling from the feature representation space. We propose sampling the virtual outliers from the feature representation space, using the multivariate distributions estimated above. Ideally, these virtual outliers should help estimate a more compact decision boundary between ID and OOD data. To achieve this, we propose sampling the virtual outliers V4 from the e-likelihood region of the estimated class-conditional distribution:
Vi {vill (2/)m/21511/2 exp (- -ix) T5)
-pxi) <t}, -vb-
where VA Mi Σ) denotes the sampled virtual outliers for class k. which are in the sublevel set based on the likelihood. is sufficiently small so that the sampled outliers are near class boundary.
Classification outputs for virtual outliers. For a given sampled virtual outlier v E R", the output of the classification branch can be derived through a linear transformation:
[(v; 0) = W...
(4)
where Was E m is the weight of the last fully connected layer. We proceed with describing how to regularize the output of virtual outliers for improved OOD detection.
3.2 UNKNOWN-AWARE TRAINING OBJECTIVE
We now introduce training objective for unknown-aware learning, leveraging the virtual out- liers in Section 3.] The key idea is to perform visual recognition task while regularizing the model produce low OOD score for ID data, and a high OOD score for the synthesized outlier.
Uncertainty regularization for classification. For simplicity, we first describe the regularization in the multi-class classification setting. The regularization loss should ideally optimize for the sepa- rability between the ID vs. OOD data under some function that captures the data density. However, directly estimating log p(x) can be computationally intractable as it requires sampling from the en- tire space X. We note that the log partition function E(x; 8) log ΣK to log p(x) with some unknown factor, which can be seen from the following:
Ek=1 (xx()) is proportional
p(x, v) p(x)
ex((x:0)
p(uxx
K km
exe(x))))
where fx(x; 8) denotes the y-th element of logit output corresponding to the label y. The nega- tive log partition function also known as the free energy, which was shown to an effective uncertainty measurement for OOD detection (Liu et al. 2020a).
Our idea is to explicitly perform a sevel-set estimation based on the energy function (threshold at 0), where the ID data has negative energy values and the synthesized outlier has positive energy:
Luncentinty Ev-y 1{E(v;; 0) 0} EXED 1(E(x; 0) S 0}
4

---

Published as a conference paper at ICLR 2022
This is a simpler objective than estimating density. Since the 0/1 loss is intractable, replace it with the binary sigmoid loss, a smooth approximation of the 0/1 loss, yielding the following:
1
exp exp -(((x)))
Lunneetainty E
log + exp (E(v:)))
+ Ex
log
(5)
1 exp (osppo(FF(x))) =(
Here ((-) is nonlinear MLP function, which allows learning flexible energy surface. The learning process shapes the uncertainty surface, which predicts high probability for data and low probabil- ity for virtual outliers v. [Liu et al. (2020a) employed energy for model uncertainty regularization, however, the loss function is based on the squared hinge loss and requires tuning two margin hyper- parameters. In contrast, our uncertainty regularization loss is completely hyperparameter--Pnn and much casier to use in practice. Moreover, VOS produces probabilistic score for OOD detection, whereas Liu et al. (2020a) relies
cunppobbabilistie energy score.
Object-level energy score. In case of object detection, we can replace the image-level energy with object-level energy score. For ID object (x, b). the energy defined as:
Fl F((x b:
log
Wok exp ((x,)):))
where (xx b); WT b) is the logit output for class in the classification branch. The energy score for the virtual outlier can be defined in a similar way as above. In particular, we will show Section 4 that a learnable w is more flexible than a constant w, given the inherent class imbalance in object detection datasets. Additional analysis on wi. is in Appendix G
Overall training objective. In the case of object detection, the overall training objective combines the standard object detection loss, along with regularization loss in terms of uncertainty:
min ((x. [Lets + Live] + B Luncertainty.
where B is the weight of the uncertainty regularization. Lets and Lax are losses for classification and bounding box regression, respectively. This can be simplified to classification task without Lic- We provide ablation studies in Section 4.1 demonstrating the superiority of our loss function.
3.3 INFERENCE-TIME OOD DETECTION
During inference, we use the output of the logistic regression uncertainty branch for OOD detection. In particular, given test input x", the object detector produces bounding box prediction b* The OOD uncertainty score for the predicted object (x", b*) ) is given by:
exp ((E(x",bb
po(g x b*) 1 exp ((E(' (b-))
(8)
For OOD detection, one can exercise the thresholding mechanism to distinguish between ID and OOD objects:
if po x" b*) 7 if polg x", b²) < 7.
G(x*, b(x;;b") -
(9)
The threshold y is typically chosen so that high fraction of ID data (e.g., 95%) is correctly classi- fied. Our framework VOS summarized in Algorithm [
Algorithm 1 VOS: Virtual Outlier Synthesis for OOD detection
Input: ID data {x;, b,, y4) 1 randomly initialized detector with parameter queue size Q&l for Gaussian density estimation, weight for uncertainty regularization 3, and c.
Output: Object detector with parameter d", and OOD detector G.
while train do
Update the ID queue Qe with the training objects ((x, b, 3))-
Estimate the multivariate distributions based on ID training objects using Equation 1 and
Sample virtual outliers using Equation 3
Calculate the regularization loss using Equation 5 update the parameters 0 based on Equation
end
while eval do
Calculate the OOD uncertainty score using Equation g
Perform thresholding comparison using Equation 9
end
5

---

Published as a conference paper at ICLR 2022
In-distribution D
Method
FPR95
AUROCT
mAP (ID)
MS-COCO/
Opennlagges
MSP (enddrycks & Gimpel
2017
70.997 73.13
8344781991
48.7
ODIN Lanng al. 2018]
59.82/
82.20 82.59
48.7
Mahalanobis Lee
201
96.46/ 96.27
29/25/57.45
48.7
Energy score [LU cl
20208
56.89/ 58.69
83.69 82.98
48.7
PASCAL-VOC
Gram matrices Saaiiy Oure
77.62
Generalized ODIN (Tisu el 2020
59.57 70.28
83.12 79.23
48.
CST (Tack et al. 20:00
59.91 57.41
81.83 82.95
48.
GAN-3YRISSS LS èt al
2018a
60.93 59.97
83.67 82.67
48.5
VOS Res-RReNNe500 (nirs)
4 11.33+1..6
88. 88.70 1.2/ 85.23+ 0.6
48,9=0.2
VOS-RegX4.0 (ours)
47.77 18.
89.00=0.4
87.59
51.6=0.1
MSP (Heddrvcks & Gimpel
2017
80.94 / 79.04
75.87 / 77.38
31.2
ODIN [Luing et al. 2018
62.85 58.92
74,44/ 76.61
31.2
Mahalanobis Lce 2018b
57.66 66
8499 / 86, 88
31.2
Energy score (Lurera
2020g
60.06 $4.97
71.48/ / 79.60
31.2
Berkeley DepDriie-100k
Grum matrices Sastry Core
2020
60.93
74.93
Generalized ODIN (H50
2020
57.27 50.17
85.22 / 87.18
31.8
CSI (Tack et al. 2020)
4 37,06
84.09 87.99
30.6
GANosynliess
2018a
57.03 50.61
78.82 81.25
31.4
VOS ResNer50 (PiES)
44.274-2.0/ 1..54
86.87-22.1 88 88.52- 1.3
31.3==0.0
VOS-RegX4.0 (ours)
36.61=0.9/ 27.24
89.08=0.6 02.
32.5=0.1
Main results. Comparison with competitive out-of-distribution detection methods. All baseline methods are based on model trained on ID data only using ResNet-50 as the backbone, without using any real outlier data. t indicates larger values are better and & indicates smaller values are better. All values are percentages. Bold numbers superior results. We report standard deviations estimated across 3 runs. RegX4.0 denotes the backbone of RegNetX-4.0GF (Radosavovic al. 2020) for the object detector.
Table 1:
4 EXPERIMENTAL RESULTS
In this section, we present empirical evidence to validate the effectiveness of VOS on several real- world tasks, including both object detection (Section 4.1) and image classification (Section 4.2
4.1 EVALUATION ON OBJECT DETECTION
Experimental details. We use PASCAL VOC* (Everingham et al. 2010) and Berkeley Drive (BDD-100K2) (Yu et al. 2020) datasets as the ID training data. For both tasks, we evaluate on two OOD datasets that contain subset of images from: MS-COCO (Lin et al. (014) and OpenImages (validation set) (Kuznetsova et al. 2020). We manually examine the OOD images to ensure they do not contain ID category. We have open-sourced our benchmark data that allows the community to easily evaluate future methods on object-level OOD detection.
We use the Detectron2 library GGirshick et al. 2018) and train on two backbone architectures: ResNet-50 (He et al. 2016) and RegNetX-4.0Git (Radosavovic et al. 2020). We employ two- layer MLP with ReL.U nonlinearity for & in Equation 5 with hidden layer dimension of 512. For each in-distribution class, the threshold can be infinitesimally small, we instead choose e based on the t-th smallest likelihood in pool of 10,000 samples (per-class), generated from the class-conditional Gaussian distribution. A larger t corresponds to a larger threshold c. As shown in Table 6 a smaller yields good per- formance. We set t 1 for all our experiments. Extensive details the datasets described in Appendix A along with a comprehensive sensitivity analysis of each hyperparameter (including the queue size Qu,,, coefficient B, and threshold e) in Appendix
1,000 samples to estimate the class-conditional Gaussians. Since
Metrics. For evaluating the OOD detection performance, we report: (1) the false positive rate (FPR95) of OOD samples when the true positive rate of ID samples is at 95%; (2) the area under the receiver operating characteristic curve (AUROC). For evaluating the object detection performance on the ID task, we report the common metric of mAP.
VOS outperforms existing approaches. In Table I we compare VOS with competitive OOD de- tection methods in literature. For fair comparison, all the methods only use ID data without using auxiliary outlier dataset. Our proposed method, vos, outperforms competitive baselines, including Maximum Softmax Prohability (Hendryeks & Gimpel, 2017). ODIN (Liang et al. 2018), energy
PASCAL--VOC consists of the following ID labels: Person, Car, Bicycle, Boat, Bus, Motorhike, Train, Airplane, Chair, Bottle, Dining Table, Potted Plant, TV, Sofa, Bird, Cat, Cow, Dog, Horse, Sheep. BBDD-100K consists of ID labels: Pedestrian, Rider, Car, Truck, Bus, Train, Motorcycle, Bicycle, Traffic light, Traffic sign.

---

Published as a conference paper at ICLR 2022
Method
AUROC 1 mAP
"GAN (Lee al.. 2018a
83.67
48.5
Image synthesis
"ixxup" Zhang el al. 2018
(mixing ratio 0.4)
61.23
44.3
Mixup Zhang et al. 2018
(mixing ratio 1)
63.99
46.9
8 Additive Gaussian noise to ID features
68.02
48.7
Noise as outliers
Trainable noise added to the ID features
66.67
48.6
Gaussiian noise
85.98
48.5
All negative proposals
63.45
48.1
Negative proposals
Random negative proposals
66.03
48.5
Proposals with large background prob JJosphh al. 2021
77.26
48.5
VOS (ours)
88.70
48.9
Table 2: Ablation on outlier synthesis approaches (on backbone of ResNet-50, COCO the OOD data).
score (Liu ct al., 2020a). Mahalanobis distance (Lee et al., 2018b), Generalized ODIN (Hsu et al., 2020), CSI (Tack al. 2020) and Gram matrices (Sastry & Oore 2020). These approaches rely on a classification model trained primarily for the ID classification task, and can be naturally extended to the object detection model due to the existence of classification head. The comparison precisely highlights the benefits of incorporating synthesized outliers for model regularization.
Closest to our work is the GAN-based approach for synthesizing outliers (Lee et al. 2018al Com- pare to GAN-synthesis, vos improves the OOD detection performance (FPR95) by 12.76% on BDD-100k and 13.40% on Pascal VOC (COCO as OOD). Moreover, we show in Table | that vos achieves stronger OOD detection performance while preserving high accuracy distribution task (measured by mAP). This is in contrast with CSI, which displays degradation, with mAP decreased by 0.7% BDD- 100k. Details of reproducing baselines are in Appendix E
the original in-
Ablation on outlier synthesis approaches. We compare VOS with different synthesis approaches in Table Specifically, we consider three types of synthesis approach: (i9) synthesizing outliers in the pixel space, (iil) using noise as outliers, and (iii using negative proposals from RPN as outliers. For type
consider GAN-based (Lee et al., (0188) and mixup (Zhang et al. 2018) methods. The outputs of the classification branch for outliers are forced to be closer to a uniform distribution. For mixup, we consider two different beta distributions Beta(0.4) and Beta(1), and interpolate ID objects in the pixel space. For Type II, we use noise perturbation to create virtual outliers. We consider adding fixed Gaussian noise the ID features, adding trainable noise the ID features where the noise is trained to push the outliers away from ID features, and using fixed Gaussian noise as outliers. Lastly, for type III, we directly use the negative proposals in the ROI head as the outliers for Equation 5 similar to Joseph et al. (2021). We consider three variants: randomly sampling negative proposals (n is the number of positive proposals), sampling n negative proposals with a larger probability, and using all the negative proposals. All methods are trained under the same setup, with PASCAL-VOS as in-distribution data and ResNet-50 as the backbone. The loss function is the same as Equation for all variants, with the only difference being the synthesis method.
The results are summarized in Table 2 where VOS outperforms alternative synthesis approaches both in the feature space (&, t) the pixel space (o). Generating outliers in the pixel space (o) is either unstable (GAN) or harmful for the object detection performance (mixup). Introducing noise (t), especially using Gaussian noise as outliers is promising. However, Gaussian noise outliers are relatively simple, and may not effectively regularize the decision boundary between ID and OOD as not effective, because they are distributionally
vos does. Exploiting the negative proposals close to the ID data.
Ablation on the uncertainty loss. We perform ablation on several variants of VOS, trained with different uncertainty loss Lumcertainty- Particularly, we consider: (1) using the squared hinge loss for regularization as in Liu et al., (2) using constant weight w = [1, 1, ..... 1] for energy score in Equa- 6 and (3) classitying the virtual outliers as an additional K+1 class in the classification branch. The performance comparison is summarized in Table 3 Compared to the hinge loss, proposed logistic loss reduces the FPR95 by 10.02% on BDD-T0Ok. While the squared hinge loss Liu et al. requires tuning the hyperparameters, In addition, we find that a learnable w for energy score is more desirable than a constant w, given the inherent class imbalance in object detection datasets. Finally, classifying the virtual outliers as
uncertainty loss is completely hyperparameter free.
7

---

Published as a conference paper at ICLR 2022
Method
FPR95
AUROC
object detection mAP (ID)
VOS w/ hinge loss
49.75
87.90
46.5
VOS w/ constant
51.59
88.64
48.9
PASCAL-VOC
VOS w/ K + class
65.25
85.26
47.0
VOS (ours)
47.53
88.70
48.9
VOS w/ hinge loss
54.29
83.47
29.5
Berkeley
VOS w/ constant w
49.25
85.35
30.9
DeepDrive-
VOS w/ K + class
52.98
85.91
30.1
VOS (ours)
44.27
86.87
31.3
Table 3: Ablation study. Comparison with different regularization loss functions (on backbone of ResNeet550, COCO is the OOD data).
an additional class increases the difficulty of object classification, which does not outperform either. This ablation demonstrates the superiority of the uncertainty loss employed by VOs.
VOS is effective alternative architecture. Lastly, demonstrate that vos is effective alternative neural network architectures. particular, using RegNet (Radosavovic et al. 2020) as backbone yields both better IDD accuracy and OOD detection performance. We also explore using intermediate layers for outlier synthesis, where we show using VOS on the penultimate layer is the most effective. This is expected since the feature representations are the most discriminative at deeper layers. We provide details in Appendix F
Comparison with training on real outlier data. We also compare with Outlier Expo- sure (Hendrycks et al. 2019) (OE). OE serves as strong baseline since it relies on the real outlier data. We train the object detector on PASCAL--OOC using the same architecture ResNet-50, and use the OE objective for the classification branch. The real outliers for OE training are sampled from the OpenImages dataset [Kuznetoova et al. 2020). We perform careful deduplication to ensure there is no overlap between the outlier training data and PASCAL-VOC. Our method achieves OOD de- tection performance on COCO (AUROC: 88. 70%7 that favorably matches OE (AUROC: 90. 18%), and does not require external data.
4.2 EVALUATION ON IMAGE CLASSIFICATION
Going beyond object detection, show that VOS is also suitable and effective on common image classi- fication benchmark. We use CIFAR-10 (Krizhevsky & Hinton 2009) as the ID training data, with stan- dard train/val val splits. We train on WideResNet- 40 (Zagoruyko & Komodakis 2016) and DenseNet- 101 (Huang al.J (017). where we substitute the object detection loss in Equation 7 with the cross- entropy loss. We evaluate on six OOD datasets: Textures (Cimpoi et al.] 2014). SVHN (Netzer [Zhou et al.) 2018). (Yu
Method
FPR95
AUROC 1
WideResNer
DenseNet
MSP
$1.057 11.0748..7
90.5079244 92.46
ODIN
35.71 1671122..5 24.57 9
Mahalanobis
37.08 36.26 93.27
87.12
Encrgy
33.01 27.44
91.88/ /94.51
Gram Matrices
27.33/23.13 93.00 89.83
Generalized ODIN
39.94 26.97
29.97
92,
993.76
CST
35.66
35.66/47.88 47.83
92.45 / 85.3
GAN-synthesis
37.30/
83.71
89.60
54.14
VOS (ours)
24.87
94.06 95.33
Table 4: OOD detection results of VOS com- parison with competitive baselines on two archi- tectures: WideResNet-40 and DenseNet-1011
(et al., 2011), Places 365
LSUN-C Yuu et al. 2015). ) ASUN-Resion et al. 2015), and iSUN Xu et al. 2015). The com- parisons are shown in Table with results averaged six test datasets. VOS demonstrates com- petitive OOD detection results on both architectures without sacrificing the ID test classification accuracy (94.84c 99.844 on pre-trained WideResNet 94.68% using VOS).
4.3 QUALITATIVE ANALYSIS
In Figure 4 we visualize the prediction on several OOD images, using object detection models trained without virtual outliers (top) and with vos (bottom), respectively. The in-distribution data is BDD-100K. VOS performs better in identifying OOD objects (in green) than a vanilla object detector, and reduces false positives among detected objects. Moreover, the confidence score of the false-positive objects of VOS is lower than that of the vanilla model (see the truck in the 3rd column). Additional visualizations are Appendix D and H
RELATED WORK
OOD detection for classification can be broadly categorized into post hoc and regularization-bant based approaches. In Bendale & Boult (016)) the OpenMax score is developed for OOD detection based
8

---

Published as a conference paper at ICLR 2022
Figure 4: Visualization of detected objects on the OOD images (from MS-COCO) by vanilla Faster-RCNN (top) and vOs (bottom). The in-distribution is BDD-100K dataset. Blue: Objects detected and classified as one of the ID classes. Green: OOD objects detected by vos, which reduce false positives among detected objects. on the extreme value theory (EVT). Subsequent work (Hendrycks & Gimpel 2017) proposed sim- ple baseline using maximum softmax probability. Improved algorithms have been proposed, such as ensembling [Lakshiinarayanan et 2017), ODIN (Liang et al. 2018), energy score (Liu al.) 2020a), Mahalanobis distance (Lee et al. 2018b), Gram matrices based score (Sastry & Oore 2020), and GradNorm score (Huang al. 2021 ). Very recently, al. (202I) showed that a simple acti- vation rectification strategy termed ReAct can significantly improve test-time OOD detection. Theo- retical understandings on different post-hoc detection methods are provided in (Morteza & Li 2022).
Different from [ee et al. (2018b), VOS performs dynamic estimation of class-conditional Gaussian during training, which shapes the uncertainty surface over time using our proposed loss.
Another line of approaches explore model regularization using natural outlier images (Hendrycks let al. 2019; Mohseni et al. 20200 Zhang et al.] (02T) or images synthesized by GANs (Lee et al, 2018a). However, real outlier data is often infeasible to obtain. Instead, VOS automatically syn- thesizes virtual outliers which allows greater flexibility and generality. Tack et al. (2020) applied
self-supervised learning for OOD detection, which we compare in Section 4 Blum al. (2021);
Jung et af. (2021); Besnier et al. (202T) proposed to detect outliers for semantic segmentation task. Greic al. (2021) trained generative model and synthesize outliers in the pixel space, which can- not be applied to object detection where a scene consists of both known and unknown objects. The
regularization is based on entropy maximization, which is different from VOS.
OOD detection for object detection is currently underexplored. Joseph et al. (202T) used energy score (Liu al., (220a) to identify the OOD data and then labeled them for incremental object detection. In contrast, VOS focuses on OOD detection and adopts a new unknown-aware train- ing objective with new test-time detection score. Our learning framework is generally applicable to both object detectors and classification models. Moreover, [Joseph et al. (2021) used the neg- ative proposals as unknown samples for model regularization, which is suboptimal as we show in Table 2 Harakeh & Waslander (2021); Riedlinger et al. (2021) focused on uncertainty estima- tion for the localization regression, rather than OOD detection for classification problems. Several works (Dhamija et al.) 20200] Miller et al. 2019 2018 Hall et al.) 2020; Deepshikha et al., (021) used approximate Bayesian methods, such as MC-Dropout (Gal & Ghahramani (2016) for OOD detection. They require multiple inference passes to generate the uncertainty score, which are com- putationally expensive on larger datasets and models.
Open-world object detection includes out-of -ot-of-domain generalization (Kim al. 202 Wang al., 2021 zero-shot object detection (Gu et al.) 2022
Rahman et al.] 2020) and incremental object
detection (Liu et al. 2020b Pérez-Rúa et al. 2020 Most of them either developed measures to
mitigate catastraphic forgetting (Joseph et al., 2020) or used auxiliary information (Rahman et al.,
2020), such as class attributes perform object detection on unseen data, which is different from our focus of OOD detection.
6 CONCLUSION
In this paper, we propose VOS, novel unknown-aware training framework for OOD detection. Different from methods that require real outlier data, vos adaptively synthesizes outliers during training by sampling virtual outliers from the low-likelihood region of the class-conditional distri- butions. The synthesized outliers meaningfully improve the decision boundary between the ID data and OOD data, resulting in superior OOD detection performance while preserving the performance of the ID task. VOS effective and suitable for both object detection and classification tasks. We hope our work will inspire future research on unknown-aware deep learning in real-world settings.

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
The authors of the paper recognize the importance and value of reproducible research. We summa- rize our efforts below to facilitate reproducible results:
1. Datasets. We use publicly available datasets, which are described in detail in Section 4.1
Section 4.2 and Appendix [
2. Baselines. The description and hyperparameters of the OOD detection baselines are ex- plained in Appendix E
3. Model training. Our model training on object detection is based on the publicly available Detectron2 codebase:
ron2 Hyperparamters are specified in Section with a thorough ablation study provided Appendix
4. Methodology. Our method is fully documented in Section with the pseudo algorithm detailed in Algorithm
5. Open Source. The codebase and the dataset will be released for reproducible research. Code is available at htttp::/// http:////eeepepee..ooo...o..///
ETHICS STATEMENT
Our project aims to improve the reliability and safety of modern machine learning models. Our study can lead to direct benefits and socictal impacts, particularly for safety-critical applications such autonomous driving. Our study does not involve any human subjects violation of legal compliance. We do not anticipate any potentially harmful consequences to our work. Through our study and releasing our code, we hope raise stronger research and societal awareness towards the problem of out-of-distribution detection in real-world settings.
ACKNOWLEDGEMENT
Research supported by Wisconsin Alumni Research Foundation (WARF). We sincerely thank Ziyang (Jack) Cai for helping with inspect the OOD datasets, and members in Li's lab for valuable discussions.
10

---

Published as a conference paper at ICLR 2022
REFERENCES
Abhijit Bendale and Terrance E. Boult. Towards open set deep networks. In Proceedings the IEEE conference on computer vision and pattern recognition, pp. 1563-1572, 2016.
the
Victor Besnier, Andrei Bursuc, David Picard, and Alexandre Briot. Triggering failures: Out-of- distribution detection by learning from local adversarial attacks in semantic segmentation. In Proceedings of the IEFF/CVF International Conference on Computer Vision (ICCV), pp. 15701- 15710, October 2021.
Hermann Blum, Paul-Edouard Sarlin, Juan I. Nieto, Roland Siegwart, and Cesar Cadena. The fishyscapes benchmark: Mcasuring blind spots in semantic segmentation. Int. Comput. Vis., 129(11):3119-1133 2021. doi: 10.1007/s 120000221112------11--1--
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pp. 3606-3613, 2014.
Kumari Deepshikha, Sai Harsha Yelleni, K. Srijith, and C. Krishna Mohan. Monte carlo dropblock for modelling uncertainty in object detection. CoRR, abs/2 102.0666 14, 2021.
Akshay Raj Dhamija, Manuel Günther, Jonathan Ventura, and Terrance E. Boult. The overlooked elephant of object detection: Open set. In IEEE Winter Conference on Applications of Computer Vision, WACV 2020, pp. 1010-1015 2020.
Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zis- serman. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2))303-338, 2010.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059.
PMLR, 2016.
Ross Girshick, Ilija Radosavovic, Georgia Gkioxari, Piotr Dollár, and Kaiming He. Detectron. 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-26828 2014.
Matej Greic, Petra Bevandic, and Sinisa Segvic. Dense open-set recognition with synthetic outliers generated by real NVP. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2021, Volume 4: VISAPP, Online Streaming. February 8-10, 2021, pp. 133-143. SCITEPRESS, 2021. doi:
10.5220001026000133011
Xiuye Gu, Tsung- Yi Lin, Weicheng Kuo, and Yin Cui. Zero-shot detection via vision and language knowledge distillation. In 10th International Conference on Learning Representations, ICLR 2022, 2022.
David Hall, Feras Dayoub, John Skinner, Haoyang Zhang, Dimity Miller, Peter Corke, Gustavo Carneiro, Anelia Angelova, and Niko Sünderhauf. Probabilistic object detection: Definition and evaluation. In IEEE Winter Conference on Applications of Computer Vision, WACV 2020, pp- 1020-1029, 2020.
Ali Harakeh and Steven L. Waslander. Estimating and evaluating regression predictive uncertainty in deep object detectors. International Conference on Learning Representations, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference computer vision, pp. 630-645. Springer, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and nut-of-distribute examples in neural networks. International Conference on Learning Representations, ICLR 2017, 2017.
11

---

Published as a conference paper at ICLR 2022
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations, 2019.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out- of-distribution image without learning from out-of-disributioon data. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10951-10960, 2020.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern 2017.
recognition, pp.
Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distribu- tional shifts in the wild. In Advances in Neural Information Processing Systems, 2021.
K. J. Joseph, Jathushan Rajasegaran, Salman H. Khan, Fahad Shahbaz Khan, Vineeth Balasub- ramanian, and Ling Shao. Incremental object detection via meta-leaming. arXiv preprint arXiv:2003.08790 2020.
K. J. Joseph, Salman Khan, Fahad Shahbaz Khan, and Vineeth N. Balasubramanian. Towards open world object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021, 2021.
Sanghun Jung, Jungsoo Lee, Daehoon Gwak, Sungha Choi, and Jaegul Choo. Standardized max logits: A simple yet effective approach for identifying unexpected road obstacles urban-scene segmentation. In IEEE International Conference on Computer Vision, ICCV, 2021.
Dahun Kim, Tsung- Yi Lin, Anelia Angelova, In So Kweon, and Weicheng Kuo. Learning open- world object proposals without learning to classify. CoRR, abs/2108.06755 06653,, 2021.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Interna- tional Conference on Learning Representations, ICLR 2015, 2015.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical Report, 2009.
Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4. International Journal of Computer Vision, pp. 1-26, 2020.
Balaji Lakshninarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, pp. 6402-6413, 2017.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training wonfidence-calibraiee classifiers for detecting out-of--isttrr distrribbbt samples. In International Conference on Learning Representations, 2018a.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting diitr--distribution samples and adversarial attacks. In Advances in Neural Information Process- ing Systems, pp. 7167-7177, 2018b.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations, ICLR 2018, 2018
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755, 2014.
Weitang Liu, Xiaoyun Wang. John Owens, and Yixuan Li. Energy-based out-of-distribution detec- tion. Advances in Neural Information Processing Systems, 2020a.
Xialei Liu, Hao Yang, Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Continual uni- versal object detection. CoRR, abs/2002.053447, 2020b.
12

---

Published as a conference paper at ICLR 2022
Dimity Miller, Lachlan Nicholson, Feras Dayoub, and Niko Sünderhauf. Dropout sampling for robust object detection in apen-set conditions. In IEEE International Conference on Robotics and Automation, ICRA 2018, pp. 1-7, 2018. doi: 10. 10.1109/CRRA.2018..88880
Dimity Miller, Feras Dayoub, Michael Milford, and Niko Sünderhauf. Evaluating merging strategies for sampling-based uncertainty techniques in object detection. In International Conference Robotics and Automation, ICRA 2019, pp. 2348-2354, 2019.
Sina Mohseni, Mandar Pitale, JBS Yadawa, and Zhangyang Wang. Self-supervised learning for generalizable -it-r--distribution detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 5216-5225 2020.
Peyman Morteza and Yixuan Li. Provable guarantees for understanding out-of-distribution detec- tion. Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confi- dence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 427-436, 2015.
Guan-Manuel Pérez-Rúa, Xiatian Zhu, Timothy M. Hospedales, and Tao Xiang. Incremental few- shot object detection. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion, CVPR 2020, pp. 13843-13852, 2020. doi: 10.1 N..110/CCVR42260020000000
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. De- signing network design spaces. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, pp. (0425-1043) 2020.
Shafin Rahman, Salman H. Khan, and Fatih Porikli. Zero-shot object detection: Joint recognition and localization of novel concepts. International Journal of Computer Vision, 128(12):2979- 2999, 2020. doi: 10.1007/s1 1263-020-0135566
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances neural information processing systems, pp. 91-99,
Tobias Riedlinger, Matthias Rottmann, Marius Schubert, and Hanno Gottschalk. Gradient-based quantification of epistemic uncertainty for deep object detectors. CoRR, 02022007.05517 2021.
Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribution examples with gram matrices. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, volume 119, pp. 8491-8501, 2020.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activa- tions. In Advances in Neural Information Processing Systems, 2021.
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive
learning on distributionally shifted instances. In Advances in Neural Information Processing Systems, 2020.
Xin Wang, Thomas E. Huang, Benlin Liu, Fisher Yu, Xiaolong Wang, Joseph E. Gonzalez, and Trevor Darrell. Robust object detection via instance-level temporal cycle confusion. In Proceed- ings of the IEEE International Conference on Computer Vision (ICCV), 2021.
Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsoureing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755. 2015.
Fisher Yu, Ari Seff. Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: /506. 003600 2015.
13

---

Published as a conference paper at ICLR 2022
Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Mad- havan, and Trevor Darrell. BDD100K: A diverse driving dataset for heterogeneous multitask learning. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2020,
pp. 2633-2642, 2020.
Sergey Zagoruyko and Nikos Komodakis. arXiv:1605.07116. 2016.
Wide
residual networks.
arXiv preprint
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empiri- cal risk minimization. In 6th International Conference on Learning Representations, ICLR 2018, 2018.
Jingyang Zhang, Nathan Inkawhich, Yiran Chen, and Hai Li. Fine-grained out-of-distribution de- tection with mixup outlier exposure. CoRR, abs/2106.03917. 2021.
Bolei Zhou, Ágata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions Puttern Analysis and Machine Intelligence. 40(6): 1452-146, 2018. doi: (0.100//PAMM2201.22733000
14

---

Published as a conference paper at ICLR 2022
Supplementary Material
A EXPERIMENTAL DETAILS
We summarize the OOD detection evaluation task in Table The OOD test dataset is selected from MS-COCO and Openlmages dataset, which contains disjoint labels from the respective ID dataset. The PASCAL model is trained for a total of 18,000 iterations, and the BDD-100k model is trained for 90,000 iterations. We add the uncertainty regularizer (Equation 5 starting from 2/3 of the training. The weight f is set to 0.1. See detailed ablations on the hyperparameters in Appendix
Task
Task
ID train datasct
VOC train
BDD train
ID val dataset
VOC val
BDD val
OOD dataset
COCO and OpenImages val
COCO and OpenImages val
#ID train images
16,551
69,853
#ID images
4,952
10,000
#OOD images for COCO
930
1,880
#OOD images for Openlmages
1,761
1,761
Table 5: OOD detection evaluation tasks.
B SOFTWARE AND HARDWARE
We run all experiments with Python 3,8.5 and PyTorch 1.7.0, using NVIDIA GeForce RTX 2080Ti GPUs.
C EFFECT OF HYPERPARAMETERS
Below we perform sensitivity analysis for each important hyperparamete We use ResNet-50 as the backbone, trained on in-distribution dataset PASCAL-VOC.
Effect of Since the threshold can be infinitesimally small, instead choose e based on the t-th smallest likelihood in pool of 10,000 samples (per-class), generated from the class-conditional larger threshold c. As shown Table 6 smaller I yields good performance. We set / = 1 for all our experiments.
Gaussian distribution. larger corresponds
mAPt
FPR95 4 AUROC
AUPR
48.7
54.69
83.41
92.56
48.2
57.96
82.31
88,52
48.3
62.39
82.20
88.05
48.8
69.72
80.86
89.54
48.7
$7.57
78.66
88.20
6
48.7
74.03
78.06
91.17
48.8
60.12
79.53
92.53
10
47.2
76.25
74.33
90.42
Table 6: Ablation study on the number of selected outliers (per class).
Effeet of queue size Qul. We investigate the effect of ID queue size [Qxl in Table 7 where we vary [Q&l = (50, 100, 200, 400, 600, 800, 1000). Overall, larger [Q&] is more beneficial since estimation of Gaussian distribution parameters can be more precise. In our experiments, we set the queue size [Qx to 1, 000 for PASCAL and 300 for BDD-100K The queue size is smaller for BDD because some classes have limited number of object boxes.
Effeet of 3. As shown in Table a mild value of 3 generally works well. As expected, a large value (e.g., B 0.5) will over-regularize the model and harm the performance.
Note that our sensitivity analysis uses the speckle noised PASCAL VOC validation dataset OOD data, which is different from the actual OOD test datascts use.
15

---

Published as a conference paper at ICLR 2022
Q+1
MAPt
FPR95
AUROCT
AUPRT
48.6
68.42
77.04
92.30
100
48.9
59.77
79.96
89.18
200
48.8
57.80
80.20
89.92
400
48.9
66.85
77.68
89.83
600
48.5
57.32
81.99
91.07
800
48.7
51.43
82.26
91.80
1000
48.7
54.69
83.41
92.56
Table 7: Ablation study on the ID queue size [Q&l.
8
mAPt
FPR95 4 AUROC1
AUPR
0.01
48.8
59.20
82.64
90.08
0,05
48.9
57.21
83.27
91.00
0.1
48.7
54.69
83.41
92.56
0.15
48.5
59.32
77.47
89.06
0.5
36.4
99.33
57.46
85.25
Table 8: Ablation study on regularization weight J
Effect of starting iteration for the regularizer. Importantly, we show that uncertainty regulariza- tion should be added in the middle of the training. If added too early, the feature space is sufficiently discriminative for Gaussian distribution estimation. See Table for the effect of starting iteration Z. We use Z 12, 000 for the PASCAL-VOG model, which traindd for total of 18,000 iterations.
Z
MAPt
FPR95 +
AUROCT
AUPR
2000
48.5
60.01
78.55
87.62
4000
48.4
61.47
79.85
89.41
48.5
59.62
79.97
89.74
8000
48.7
56.85
80.64
90.71
10000
48.6
49.55
83.22
92.49
12000
48.7
54.69
83.41
92.56
14000
49.0
55.39
81.37
93.00
16000
48.9
59.36
82.70
92.62
Table 9; Ablation study the starting iteration Z. Model is trained for total of 18,000 iterations.
16

---

Published as a conference paper at ICLR 2022
D ADDITIONAL VISUALIZATION RESULTS
We provide additional visualization of the detected objects on different OOD datasets with models trained on different in nn-distribution datasets. The results are shown in Figures
Queens
Bromx
TRUCKS
Queens
STOP PLEE
STOP
Figure 5: Additional visualization of detected objects on the OOD images (from MS-COCO) by a vanilla Faster-RCNN (top) and vos (bottom). The in-distribution is Pascal VOC dataset. Blue: Objects detected and classified as one of the ID classes. Green: OOD objects detected by vos, which reduce false positives among detected objects.
E BASELINES
To evaluate the baselines, follow the original methods in MSP (Hendrycks & Gimpel 2017
ODIN (Liang et al., 2018), Generalized ODIN (Hsu et al. 2020). Mahalanobis distance (Lee et al. 2018b), CST (Tack al. 2020), energy score (Liu al. 2020a) and gram matrices (Sastry & Oore 2020 and apply them accordingly on the classification branch of the object detectors. For ODIN, the temperature is set be 7 1000 following the original work. For both ODIN and Mahalanobis distance Lee et al. (2018b), the noise magnitude is set to because the region-based object detector not end-to-end differentiable given the existence of region cropping and ROIAlign. For GAN (Lee et al. 2018a), we follow the original paper and use GAN to generate OOD images. The prediction of the OOD images/objects regularized to be close to uniform distribution, through a KL diver- gence loss with weight of 0.1. We set the shape of the generated images to be 100x 100 and resize them to have the same shape as the real images. We optimize the generator and discriminator using the rotations (0", 90" , 180 270") as the self-suueri task. We set the temperature in the contrastive loss 0.5. We use the features right before the classification branch (with the dimension to be 1024)
Adam (Kngmma & Ba 2015), with learning rate of 0.001. For CSI (Tack et al. 2020),
17

---

Published as a conference paper at ICLR 2022
Figure 6: Additional visualization of detected objects on the 0OD images (from OpenImages) by a vanilla Faster-RCNN (top) and vos (bottom). The in distrriibuuoon is Pascal VOC dataset, Blue: Objects detected and classified as of the ID classes. Green: OOD objects detected by vos, which reduce false positives among detected objects.
to perform contrastive learning. The weights of the losses that used for classifying shifted in- stances and instance discrimination are both set to 0.1 to prevent training collapse. For Generalized ODIN Hsu et al. (2020), replace and train the classification head of the object detector by the most effective Deconf-C head shown in the original paper.
F VIRTUAL OUTLIER SYNTHESIS USING EARLIER LAYER
In this section, investigate the effect of using VOS earlier layer within the network. Our main results Table [ are based on the penultimate layer the network. Here, we additionally evaluate the performance using the layer before the penultimate layer, with feature dimension of 1, 024. The results are summarized in Table IO As observed, synthesizing virtual outliers in the penultimate layer achieves better OOD detection performance than the earlier layer, since the feature representations are more discriminative at deeper layers.
G VISUALIZATION OF THE LEARNABLE WEIGHT COEFFICIENT IN GENERALIZED ENERGY SCORE
To observe whether the learnable weight coefficient wy in Equation|[ccppture dataset-specific statis- ties during uncertainty regularization, visualize wy w.r.t each in-distribution class and the number
18

---

Published as a conference paper at ICLR 2022
COOTIC POI
OCO
366
42%
463
030
000
Figure 7: Additional visualization of detected objects the OOD images (from MS-COCO) by vanilla Better-RNN (lop) and (bottom). The in-distribution BDD-100K dataset. Blue: Objects detected and classified as one of the ID classes. Green: OOD objects detected by vos, which reduce false positives among detected objects.
Models
FPR95]
AUROCT
mAP
PASCAL VOC
VOS-final
47.53
88.70
48.9
VOS-earlier
50.24
88.24
48.6
BDD-100k
VOS-final VOS-earlier
44.27
86.87 86.08
31.3
49.66
30.6
Table 10: Performance comparison of employing VOS on different layers. COCO is the OOD data.
of training objects of that class in Figure We use the BDD-100K dataset Yu et al. 2020) as the in- distribution dataset and the RegNetX-440Gft (Radosavovic et al. (020)) as the backbone network. As can be observed, the learned weight coefficient displays consistent trend with the number of train- ing objects per class, which indicates the advantage using learnable weights rather than constant weight vector with all 1s.
19

---

Published as a conference paper at ICLR 2022
AB
TRALL PIO DISSACT
1OS DEVEY
CANNOT
AB
Figure 8: Additional visualization of detected objects on the OOD images (from OpenImages) by a vanilla Faster-RCNN (top) and vos (bottom). The in-distribution is BDD-100k dataset. Blue: Objects detected and classified as of the ID classes. Green: OOD objects detected by vos, which reduce false positives among detected objects.
0.5
100000
Number objects
Energy weight
Number of abjects
80000
Energy weightt
0,3
60000
40000
75000
0.1
0.0
pedestrian
rider
truck
train
motorcycle
bicycle
traffic light sign
Ubsses
Figure 9: Visualization of learnable weight coefficient in the generalized energy and the num- ber of training objects per in-distribution class. The value of the weight coefficient averaged over three different runs.
H VISUALIZATION OF THE VIRTUAL OUTLIERS
In this section, we visualize the synthesized virtual outliers by VOS using UMAP in Figure I0 The in-distribution dataset is the Paseal VOC dataset with the backbone of ResNet-50. Note that
20

---

Published as a conference paper at ICLR 2022
cannot visualize virtual outliers in the pixel space since they are synthesized in low-dimensional feature space.
ID dbjects Virtual outtiers
Figure 10: UMAP visualization of the synthesized virtual outliers. The blue points denote the object features from the in-distribution class of Person. The green points denote the synthesized virtual outliers from the low-density space w.r. the features from that class.
From Figure T0 the virtual outliers reside the near-boundary region of the in-distribution feature cluster, which helps the model to learn compact decision boundary between ID and OOD objects.
T DISCUSSION ON THE DETECTED, REJECTED AND IGNORED OOD OBJECTS DURING INFERENCE
The focus of VOS is to mitigate the undesirable cases when an OOD object is detected and classified as in-distribution with high confidence. In other words, our goal is to ensure that "if the box is detected, it should be faithfully an in-distribution object rather than OOD". Although generating the bounding box for OOD data is not the focus of this paper, we do notice that VOS can improve the number of boxes detected for OOD data (+25% on BDD trained model compared to the vanilla Faster-RCNN).
The number of OOD objects ignored by RPN can largely depend on the confidence score threshold and the NMS threshold. Hence, we found it more meaningful to compare relatively with the vanilla Faster-RCNN under the same default thresholds. Using BDD100K as the in-distribution dataset and the ResNet the backbone, voS improve the number of detected OOD boxes by 25% (compared to vanilla object detector). VOS also improves the number of rejected OOD samples by 63%.
21


Reviews:
[{"review_id": "wZeA5sO0OjE", "reviewer": "Reviewer_rXCn", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n(1) This paper is well written and easy to understand. Especially,  the figures deliver the core idea of the whole paper.\n(2) The proposed method is straightforward but efficient. Also, the idea could be proved by a toy example.\n\nWeakness:\n(1) Some extreme cases are not clearly illustrated. For example, the sampling process is done online. In one batch, it is likely to have only one instance in some class. In that case, I am very concerned about the quality of estimation.\n(2) Too many ablations studies are arranged on supplementary files. \n\nAlso, I have a question about the unknown-aware training objective. In this part, I understand it like a binary-cross entropy with learnable weight. I am wondering about the influence of learnable weight. I checked the visualization in the supplementary file but do not find the performance effect.\n\n", "summary_of_the_paper": "This paper proposed a novel unknown-aware learning framework dubbed VOS (Virtual Outlier Synthesis), which optimizes the dual objectives of both ID task and OOD detection performance. The key of VOS is to sample outliers in low-dimension feature space instead of generated images. These outliers are used in the training process to help form the compact decision boundary between ID and OOD data.\n\nThe extensive experiments show the effectiveness of their method on both the PASCAL-VOC and the DeepDrive-100K datasets. ", "summary_of_the_review": "Overall, this paper could clear deliver the idea to the readers with well-written text and designed figures. The idea is somewhat novelty by the related work. The unknown-aware training object is similar to the dynamic focal loss with the virtual outlier. I agree that the motivation about sampling virtual outliers in the feature space instead of pixel space. Therefore, I vote this paper marginally above the acceptance threshold.\n\nGiven that I am not very familiar with OOD, I would like to check other reviewers' opinions.\n\n——————————After Rebutal ——————————\n\nThe author has addressed most of my concerns. In such a situation, I would like to raise my scores.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "eO8-k7vumuz", "reviewer": "Reviewer_XCrg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. This work looks solid, and the experiments in this paper are sufficient, which consider almost all key parameters or components of the proposed method and demonstrate the effectiveness of the method. Besides, the method achieves state-of-the-art performance on both object detection and image classification tasks.\n2. This paper is well motivated, which aims to utilize synthetic virtual outliers to regularize the model, thus avoiding the collection and cleaning of real OOD data.\n\nWeaknesses:\nA few details in the proposed method are not very clear. Specifically, what are the motivation and mechanism of the proposed generalized energy score? How to train the learnable weight coefficients? How to update the ID queue? It would be better to clarify these issues.\n\nOther concerns:\n1. Since the real outlier data may not be tightly distributed around the ID data in raw or feature space, why the performance of OE can be viewed as a performance upper bound of VOS? \n2. I wonder what virtual outliers look like. It might be better to visualize them.", "summary_of_the_paper": "This paper proposes an effective method for OOD detection and model regularization, which does not rely on real OOD datasets. Specifically, the proposed method synthesizes virtual outliers by sampling low-likelihood samples in the feature space of class conditional distribution, and adds a novel regularizer to the original ID training objective, which contrastively shapes the uncertainty space between the ID data and synthesized outlier data. Extensive and comprehensive experiments demonstrate the effectiveness of the proposed method.", "summary_of_the_review": "Overall, this paper is technically and experimentally sound, which conduct comprehensive experiments to verify the proposed method. The reasons to accept outweigh the reasons to reject (e.g., a few unclear details).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XEXdtJ3qo9S", "reviewer": "Reviewer_yx7X", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1. Considering synthesized OOD sample to expose model is more feasible compared to procuring real OOD samples. Authors synthesis the OOD samples in the low-dimensional feature space which is more efficient to generate than the actual image space. \n\n2. Unknown-aware training is also an effective idea to differentiate the in distribution samples from OOD samples.\n\n3.  Experiments are conducted on the benchmark dataset and results are impressive. Extensive experiments and ablation studies are conducted to evaluate various components of the proposed approach.\n\nWeaknesses:\n\n1. Overall, the novelty of the proposed approach is limited. Considering synthesized outliers to detect OOD or novel objects are explored in various previous approaches. Authors consider a multimodal Gaussian distribution to approximate in distribution samples and sample new OOD samples. However, the class boundaries in the high-dimensional space may not be Gaussian.\n\n2. Authors consider the object detector network for generating the bounding boxes for the OOD datasets. In this case, the object detector may miss a significant number of bounding boxes as some of the objects instances may not appear in the in-distribution data. In this case, these boxes will not appear for the OOD test. Do authors see this behavior and have any comments on this issue.\n\n3. In table 2, while comparing the other generative methods, do authors consider the same method for detecting OOD. For example, if the additional loss function is included in the approach, then the comparison may not reflect just the benefit of the proposed synthesis method. \n\n4. In table 1, authors are also encouraged to provide the backbone networks for the methods they compare against so that contribution can be better evaluated. \n\n5. What is the motivation for considering the synthetic OOD samples when real outlier data are abundant, easily available, and achieve better performance. ", "summary_of_the_paper": "This paper presents an approach for detecting out of distribution (OOD) samples by synthesizing outlier samples and considering an unknown-aware learning mechanism. The synthetic outlier samples are used to learn a tighter class boundary for in-distribution samples. The model is trained with an uncertainty-aware loss function that encourages a high probability score in distribution samples and a low probability score for OOD samples. \n", "summary_of_the_review": "The authors proposed an OOD detection approach the relies on synthetic outlier samples. The proposed approach is shown to be effective but the overall novelty is limited. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ROglELDnL0w", "reviewer": "Reviewer_jkoD", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\nS1. Sampling artificial training samples in the feature space appears effective.\n\nS2. There is some novelty in the formulations of uncertainty score (learned weights) and outlier probability (sigmoid of modulated uncertainty score).\n\nS3. The authors took time to implement and evaluate several prominent baselines. Experimental evaluation shows competitive performance.\n\nWeaknesses:\n\nW1. Reading the paper requires a lot of guesswork. Figure 2 suggests that the outliers are detected within RPN while in fact the paper considers only ROI classifier (it would be helpful to indicate the location of ROI-align and others layer of the ROI classifier). It is not clear why \\Sigma is used instead of \\Sigma_k. \n\nW2. The paper does not discuss the computational complexity of recovering \\Sigma. \n\nW3. The following work should be cited and discussed since it considers logit normalization for outlier detection:\n[R0] Sanghun Jung, Jungsoo Lee, Daehoon Gwak, Sungha Choi, Jaegul Choo. Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation. ICCV 2021.\n\nW4. The following work should be cited and discussed since it considers learning open-set recognition with artificial outliers:\n[R1] Zhi-Lin Zhao, Longbing Cao, Kun-Yu Lin. Revealing Distributional Vulnerability of Explicit Discriminators by Implicit Generators. CoRR abs/2108.09976 (2021)\n[R2] Matej Grcic, Petra Bevandic, Sinisa Segvic. Dense Open-set Recognition with Synthetic Outliers Generated by Real NVP. VISIGRAPP (4: VISAPP) 2021: 133-143\n\nSuggestions\n- Equation (8) could be clarified by using the notation from (9).\n- Would it make sense to include OOD AP in Table 1?\n- Propose some intuition on why the present method outperforms [liu20nips] in Table 3. Is it only due to w_k?\n- Quantify how many OOD objects are ignored by RPN-a, rejected by (10) and detected as in-distribution objects.\n\nPost rebuttal comment\n\nThe revision improves the presentation and clarifies some doubts. Hence I propose to raise my rating to accept.", "summary_of_the_paper": "The manuscript addresses open-set object detection by modifying the ROI classifier of Faster-RCNN. The probability p(O|x,b) is modeled as sigmoidal activation of a generalized energy score of the classification head logits. The method succeeds if i) RPN ignores the outlier object, or ii) RPN detects the outlier but the candidate gets rejected due to high p(O|x,b).\n\nThe generalized energy is formulated as log-sum-weighted-exp-logit in equation (6). The probability p(outlier|x) is modeled as a sigmoid of modulated generalized energy in equations (8) and (9). The classification head is trained to output high energy in inliers and low energy in artificial outliers through equation (8). The artificial outliers are sampled from low-likelihood regions of per-class Gaussians which are fitted to latent features of groundtruth ROIs. These Gaussians are periodically updated during training. \n\nIt appears that the authors consider ROI-wide representations of RPN candidates. These representations are likely produced by applying ROI-align and two fully-connected layers to the convolutional features of the RPN. These Gaussian distributions are modelled according to per-class means (\\mu_k) i and overall covariance (\\Sigma). Virtual outliers are sampled so that their likelihood with respect to \\mu_k and \\Sigma is less than \\epsilon. \n", "summary_of_the_review": "The manuscript proposes a reasonable baseline for open-set object detection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to synthetize virtual outliers by sampling from low-likelihood regions of the feature space of a class conditional distribution, in order to make more robust predictions via a regularization loss term.

In the reviewing phase certain criticisms were raised by reviewers: namely that i) the paper was not clear w.r.t. its goal, motivation and position in the literature of OOD detection for bounding boxes; ii) details about the energy-based formulation and covariance definitions and iii) experimental setting and metric used were missing. During rebuttal the authors answered to all the above criticisms up to a satisfying extent and were able to increase two reviewers' scores.

The paper is accepted conditioned on the fact that the camera-ready includes the additional details and discussions that arose in the comments with a specific emphasis on properly framing (and limiting) the motivation of OOD detection for open-set object detection and it is expected to properly cite the literature of the more general OOD detection task as discussed in the comments.
Decision: Accept (Poster)