Boosting Out-of-distribution Detection with Typical Features
Authors: Yao Zhu
YueFeng Chen
Chuanlong Xie
Xiaodan Li
Rong Zhang
Hui Xue'
Xiang Tian
bolun zheng
Yaowu Chen
Venue: NeurIPS 2022 Accept (2022)
Keywords: out-of-distribution detection
uncertainty estimation

Abstract: Out-of-distribution (OOD) detection is a critical task for ensuring the reliability and safety of deep neural networks in real-world scenarios. Different from most previous OOD detection methods that focus on designing OOD scores or introducing diverse outlier examples to retrain the model, we delve into the obstacle factors in OOD detection from the perspective of typicality and regard the feature's high-probability region of the deep model as the feature's typical set. We propose to rectify the feature into its typical set and calculate the OOD score with the typical features to achieve reliable uncertainty estimation. The feature rectification can be conducted as a plug-and-play module with various OOD scores. We evaluate the superiority of our method on both the commonly used benchmark (CIFAR) and the more challenging high-resolution benchmark with large label space (ImageNet). Notably, our approach outperforms state-of-the-art methods by up to 5.11% in the average FPR95 on the ImageNet benchmark.  

Boosting Out-of-distributio Detection with Typical Features
Yao Zhu Yuefeng Chen? Chuanlong Xie
Xiaodan Li²
Rong Zhang²
Hui Xue? Xiang Tian 1.5 Bolun Zheng Yaowu Chen
ZZejiinn University, 2 Alibaba Group. Beijjng Normal University. "Hangzhou Dianzi University
ZZejjinng Provincial Key Laboratory for Network Multimedia Technologies Zhejiang University Embedded System Engineering Research Center, Ministry of Education of China
Abstract
(OOD) detection is critical task for ensuring the reliability and safety of deep neural networks in real-world scenarios. Different from most previous OOD detection methods that focus on designing OOD scores or intro- ducing diverse outlier examples to retrain the model, we delve into the obstacle factors OOD detection from the perspective of typicality and regard the feature's ss high-probabiliig region of the deep model as the feature's 's typical set. We propose rectify the feature into its typical set and calculate the OOD score with the typical features to achieve reliable uncertainty estimation. The feature rectification can be conducted as plug-and-play module with various OOD scores. evaluate the superiority of our method on both the commonly used benchmark (CIFAR) and the more challenging high-resolution benchmark with large label space (ImageNet). Notably, our approach outperforms state-of-the-an -fr methods by up 5.11% in the average FPR95 on the ImageNet benchmark
Introduction
Deep neural networks have been widely applied in various fields. Apart from the success of deep models, predictive uncertainty is essential in safety-critical real-world scenarios such as autonomous driving 21, medical [3) financial 44, etc. When encountering some examples that the deep model has not been exposed to during training, we hope the model raises alert and hands them over to humans for safe handling. Such challenge usually referred to as ofttt--diisribution (OOD) detection and has gained significant research attention recently (5
Most of the existing research (5 1 worked on designing suitable OOD scores for the pre- trained neural network, hoping to assign higher scores to the in-distribution (ID) examples and lower scores to the out-of aut-of-distribution (OOD) examples. However, these methods overlook the obstacle factors in OOD detection caused by the model's internal mechanisms. In this paper, we rethink the OOD detection from a perspective of feature typicality. We observed that the distribution of the deep features of the training dataset on different channels is approximately consistent with the Gaussian distribution (See examples in Appendix J). Accordingly, we divide these features into typical features (fall in the high-probability region) and extreme features (fall in the low-probability region). Extreme features rarely appear in training and attract less attention from the classifier than the typical features. We hypothesize the classifier can model the typical features better than the extreme features, and the extreme features may lead to ambiguity and imprecise uncertainty estimation. Given the potential
Yao Zhu is with the Zhejiang University. Hangzhou, China. 310013. (E-mail: www.yuuyyyudduu... 'Coreespondig authors: Xiang Tian. Bolun Zheng. (E-mail: tianx tianx @ww..edu.cnnnnnn.m blzheng@hduu.duu.co 'The code will be available at this https URL
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

negative impact, properly dealing with these extreme features is key to improving the performance of OOD detection.
In this paper, we propose to rectify the features into their typical set and then calculate the OOD score with these typical features. In this way, the model conservatively utilizes the typical features to make decisions and alleviates the damage caused by extreme features, which can be beneficial to the OOD scores derived from the pre-trained classifier (5 12). Then the problem is how estimate the feature's 's typical set on different channels since this requires sufficient number of in-distribution examples and is comecconsunnng. Luckily, the commonly used operation Batch Normalization can shed light on shortcut to selecting the feature's 's typical set and we name our approach Batch Normalization Assisted Typical Set Estimation (BATS). The Batch Normalization layer endeavors to normalize the features of the training dataset to Gaussian distributions, which can be used to estimate the typical set for ID features. Typical features are more common in training. while extreme features are rare, which leads to difficulties for the model to estimate extreme features well. We truncate the deep features with the guidance of the Batch Normalization, rectifying the extreme features to the boundary values of typical sets. We illustrate the distribution of the OOD scores for ID (ImageNet) Rectifying the features into the typical set with
and OOD (four different datasets) examples in Fig. [ our BATS contributes to improving the separability between ID and OOD examples.
223
ragete
Inagetac
0.12
O3eriee 43%
0.14
Overaan
5.83
150
35
20
0.0
30
25
30
2.33
16
15
30
25
35
10
14:
0:3
Pap
033
Prag
11%
024
8:24
24
8:35
8:33
535
175
2.51
(b) SUN
(c)
Pleaes
[0] Teaturnes
Figure 1: The distribution of the scores for ID (ImageNet) and OOD examples on ResNet-50. We use the energy score
(5] as the OOD score. "Energy" means calculating the OOD score with the original features. "Ours" means calculating the OOD score with the typical features.
Theoretically, we analyze the benefit of BATS and the variinar trade-off influenced by the strength of the hyperparameter. A proper strength of BATS contributes to improving the estimation accuracy of the reject region. Empirically, we perform extensive evaluations and establish superior performance on both the large-scale ImageNet benchmark and the commonly used CIFAR bench- marks. BATS outperforms the previous best method by large margin, with up to a 5. 5.11% 11% reduction in the false positive rate (FPR95) and a 1.43% improvement in AUROC. Morcover, BATS can also slightly improve the test accuracy and robustness of the pre-trained models. The main contributions of our paper are summarized as follows:
We provide novel insights into OOD detection from the perspective of typicality and propose to rectify the features into the typical set. We design a concise and effective approach to select the feature's 's typical set named Batch Normalization Assisted Typical Set Estimation (BATS).
We provide theoretical analysis and empirical ablation on the benefit of BATS from the perspective of bias-variance trade-off to improve the understanding of our approach.
Extensive experiments show that BATS establishes state-of-ther -htt performance among post-hoe methods on a suite of OOD detection benchmarks. Moreover, BATS can boost the performance of various existing OOD scores with typical features.
2 Related work
The literature related to OOD detection can be broadly grouped into the following themes: post-hoc detection methods [5-8 [H confidence enhancement methods 9 10, 19) and density-based
methods 25). Post-hoc detection methods focus on improving the OOD uncertainty estimation

---

by utilizing the pre- trained classifiers rather than retraining a model, which is beneficial for adopting OOD detection in real-world scenarios and large-scale settings. MSP [6] observes that the maximum softmax probability of ID examples can be higher than that of the OOD examples and provide
simple baseline for OOD detection. ODIN 7 introduces sufficiently large temperature factor and input perturbation to separate the ID and OOD examples. Liu et al. [5] analyze the limitations of softmax function in OOD detection and propose to use energy score as an indicator. The examples with high energy are considered OOD examples, and vice versa. ReAct [8] hypothesizes that the OOD examples can trigger the abnormal activation the model and propose to clamp the activation value larger than the threshold to improve the detection performance. GradNorm [12] shows that the gradients of the categorical cross-entropp loss can be an effective test statistic for OOD detection. Different from these methods, our BATS proposes to calculate the OOD scores with the typical features, which benefits the estimation of the reject region and can improve the detection performance.
3 Preliminaries
3.1 Out-of-distribution detection
In this section, we provide a summary of the out-of-distribution detection from the perspective of hypothesis testing [0). We consider classification problem with K classes and denote the labels as y = {1.2.... K}. Let X' be the input space. Suppose that the in-distribution data Din. = {[T, y;) is drawn from joint distribution Px,y defined over X' x Y. We denote the marginal distribution of Px,y for the input variable X by Po- Given a test input x E X, the problem of fiftr---distiiitiion detection can be formulated single-sample hypothesis testing task:
Ho : x Po.
Vs.
H1 : x Pa-
Here the null hypothesis Ho implies that the test input an in-distribution sample. The goal of OOD detection here is to design criteria based on Din to determine whether Ho should be rejected. OOD detection tasks need to determine a reject region R such that for any test input x E X, the null hypothesis is rejected if 1 E R. Generally, the reject region R is formulated by test statistic and a threshold. Let f X + RK be a model pre- pre-trate trained from Din. which is used to predict the class label of an input sample. One construct a test statistic T(x; f), where x is the test input. Then the reject region can be written as R (x: T(x; f) S 7). where y is the threshold.
use the model f part of f (e.g., feature extractor) to
3.2 OOD detection with energy score
For classifier f and a data point (x, y), we use [kx)[k] to represent the k*th output of the last layer.
With reference to [5] 31 32], the marginal density p(x)) of the classifier can be expressed as: p(z)
exp(-F(z))
> (xp(/(2)])k) where Z is the normalizing factor and is independent to x. E(x) 2
2
represents the energy of x and is modeled by neural network as E(x)
log Ek exp (x)
See Appendix C for details. Considering that Z is a constant and is independent to x, Liu et al. [
propose an energy score that uses the opposite of the energy F(x) as test statistic detect OOD
examples. A higher energy score means a higher marginal density p(x:).
4 Methods
In this paper, we delve into the obstacle factor for the post-hoc OOD detection from the perspective
of typicality, which aims to boost the performance of the existing OOD scores and is orthogonal to the methods of designing different OOD scores. Given that the energy score [5] is provably aligned with the density of inputs and performs well, we mainly use the energy score as the OOD score. (See Appendix for other OOD scores).
4.1 Motivation
For classifier trained on the ID data f= fw.b o g where y is feature extractor mapping input x to its deep feature Let d-dimensional vector [21, ..., za] g(x) denote the deep features of
3

---

extracted by 9, and 2; indicate the i-th element of z. fw,b(z) = w + b is a fully connected layer mapping the deep feature z to output logits. The energy can be expressed as:
E(x) - log exp(f(x)[]] = log Σ exp((w (w- .2 + +2+b)y b)(e))-
(2)
The test statistic can be expressed as T(x: f) == -==-B(x) = log Σk= exp((w + [k], which depends the extracted deep features and the mapping operation of the fully connected layer (FC). Assuming that the distribution of the deep features is consistent with the Gaussian distribution (see examples in Appendix J), there are seehh-bbbbbbltty regions and low-probability regions in deep features. We name the features that fall in high-probability regions as typical features, and the corresponding regions called feature's typical sets. In contrast, in low low-probability regions as extreme features. Extreme features are rarely exposed to the training process, which leads to difficulties for the classifier to model these features and unreliable estimations in the inference process, Reducing the influence of extreme features on test statistics can be a key to improving OOD detection performance.
regard the features that fall
4.2 Batch Normalization Assisted Typical Set Estimation
Instead of designing new OOD scores to detect the abnormality, we provide novel insight into OOD detection from perspective of typicality. We propose to rectify the features into the feature's 's typical set and then use these typical features to calculate the OOD score. Consider commonly used layer structure in deep convolutional networks:
z' + :,, ReLU - z,
(3)
where z' is the feature vector extracted from the convolutional layer of g. To identify the typical set of z' for each channel, we should apply its feature map sufficient number of IDD examples and further calculate the empirical distribution of z' over the ID examples. If the number of features is large, the inference procedure is consumnn. Here we propose a simple and effective post hoc approach that leverages the information stored in f to infer the typical set without estimating the distribution of z'. Suppose the pre-trained deep neural network uses batch normalization (BN). We denote the BN unit in f as:
BN(Z': (4 2'-R(2')
Std(z') +
(4)
where μ4, are two learnable parameters. After the pre-training, all the four parameters μ4, IE(z). Std(z') are known and stored in the weights of f The Batch Normalization normalizes features of the training dataset to distribution with a mean of and standard deviation of a, which means that the features fall in the interval [μ= - X * o, μ + * o appear more frequently in training than the features in the complement of this interval. The parameter À controls the range of the interval. Thus we use the information in the Batch Normalization to identify the in-distribution feature's 's typical set and rectify the features into the typical set before calculating the OOD score. The uncertainty estimated with the typical features can be more reliable.
In practice we propose truncated activation scheme to bound the output features of the BN unit. First, we introduce the truncated BN unit by:
I + Aa, BN(2';
if z' 1 2 Ao; if Ao < z' p Ao;
TrBN A,) (),,
(5)
| Ao,
if z' # S -
where À is a tuning parameter. We replace the BN unit in the layer structure (Eq.3) with the TrBN unit and write the rectified final features as z and the new classifier as f. Then test statistic with the energy score can be expressed as:
T(x; J = log Σ exp((w 2 + b)())
(6)
1=1
'In general, EEzz) and Std(z') are estimated on a mini batch of the training data, Finally, the pre-trained model outputs moving average estimators each iteration.

---

and take the reject region by R = T(x; {} We name our approach Batch Normalization Assisted Typical Set Estimation (BATS). In comparison to the standard BN, the outputs of TrBN are concentrated toward the feature's typical set of ID data. This makes an ID example less susceptible to being mistakenly detected as an OOD example and buffers the negative impact of the extreme features. Fig. [ compares the distribution of the OOD scores from the original energy score and the energy score with typical features.
4.3 Theoretical analysis
The truncation threshold À is a key hyperparameter. Our method reduces the variance of z and also introduces a bias term since it changes the distribution of BN( o). The variance reduction means that our method is robust to the rare ID examples, while the introduced bias can lead to degradation of the model performance. In this section, we assume z' follows normal distribution and analyze the bias-variance trade-off in our method.
4.3.1 Understanding the benefits of BATS from the perspective of variance reduction
The variance reduction happens at the BN step. The variance of BN(z'; fr, () is o² since the distribution of the ID features z' is rescaled to N (²) While the TrBN unit truncates the extreme values and the variance of TrBN(z'; M4 o, A) becomes:
A
v2
X² Aexp(-
o2((A)== 02
erf((
erf(
2) x1
erf(
V
VT
V2
where erf(x) (2/ Jo exp(-t² dt is the Gauss error function. The value of C()) represents
the degree of variance reduction. In Eq. (7). C(O) = 0, dC(A) > O, and C(A) - 1 as X + +00
Therefore, (()) is a monotonically increasing function and 0 S C()) < 1 for 0 s < to0. In
summary, the smaller X, the smaller the variance. See Appendix D for the proof.
OOD detection is dingle-sample hypothesis testing problem (in Eq.T),, and the in in-distribution Po is unknown. So the reject region is determined by the empirical distribution of the test statistic T(x: f) over the ID data. The extreme features increase the uncertainty and lead more unusual values of T(x: f). This implies that the reject region may be underestimated due to the heavy tail property of T(x: f'). Our BATS aids this problem by reducing the variance of the deep features, which contributes to constraining the uncertainty of f and T(x: f) and improving the estimation accuracy of the reject region.
4.3.2 The bias introduced by BATS
BATS rectifies the features into the typical set, which reduces the variance of the deep features. However, this operation can also introduce bias term, which can reflect the change in the distribution of the features. A large bias can damage the performance of the model. The distribution of the output feature ReLU(BN(z'' μr, o, A)) is a one-side rectified normal distribution over [0, +o0). The expectation of is:
E(z) = p + σ
(2zT (<xppp
1
(exp(
- H
(1 + erf(
V20 ))
H
(8)
20
2o
For our method, the distribution of the output feature z = ReLU u1 o, A)) is two-sided rectified normal distribution over [0, μ + λa] and the expectation of 2 is:
B(z) 1 a ( (/2L(
112 1 (cxp(- 20
exp(-
X2 ))
/ (1 + +ef((-
/
)) +
A (1 erf( X
{/11)) (9)
20
/200
2
Then the bias caused by the truncation is:
E 2 ((z) a( exp( 2
X² X
(1 erf(
1 ) -A-A0(A)) (()))
(10)
2
where (b) and (0-) are the probability density function (pdf) and cumulative distribution function (cdf) of the standard normal distribution. One can find that the bias term (E(2) - [Z()) converges to zero as À + oo. In other words, if À is large enough, the bias can be very small. Thus, there exists a bias- variance trade-off. See Appendix D for the proof.
5

---

A proper selection of À can improve the detection performance by significantly reducing the uncer- tainty (variance reduction) and slightly changing the distribution of the features (small bias). If is large, T(x; f') uses more extreme features in both the ID and OOD data. As À tends to infinity, T(x: converges to T(x: f). Then BATS is the same to the original energy detection. If is small, extreme features are removed from the test statistic T(x: f) while introducing a non-negligible bias. Because of the change in feature distribution, the detection method loses its power to identify OOD examples. Fig. 2 illlstrates the distribution of OOD scores with different X, which empirically verifies this trade-off.
MoniTollurall
2.30
Mecratrogeeict
Mount Testarea)
0.30
MannTollura Irupphick
Monnhhagelee
0.20
... Wearthrogefell PecrtToelanca iespahe:
0.3
Imagpled
2:30
Irugettss
0:20
0.2
Dverlep:50
5.30
Overisp 38 61%
0.30-
Overlian 20%
Overbb:36.005
0.1
0.0
10
15
20
25
30
15
20
30
10
15
20
25
30
0.00
20
15
20
25
35
35
(K)
An
(1) Am 3.00
Figure 2: Bias-variance trade-off in BATS. We illustrate the OOD score for ID (ImageNet) and OOD (Textures) examples. Smaller À contributes to variance reduction which benefits the estimation of the reject region. But smaller À causes larger bias, which can drastically alter the distribution of features and damage the performance of the model in distinguishing ID and OOD examples.
5 Experiments
In this section, we first introduce our experiment implementation. Then, we evaluate our methods both on the large-scale OOD detection benchmark [33] and the CIFAR benchmarks [6]. After that, the ablation studies compare the influence of applying rectification on different layers and show the influence of the hyperparameter. Moreover, our BATS can also slightly improve the test accuracy of the pre- pre-trained models (in Appendix H). We consider the out-of disttribbiiitno detection as a single-sample hypothesis testing task and only test one sample at time.
5.1 Implementation
Dataset. For evaluating the large-scale OOD) detection performance, ImageNet-1k 33) as
the in-distribution dataset and consider four outtt--distribution datasets, including (subsets of) the
fine-grained dataset iNaturalist (34), the scene recognition datasets Places (35) and SUN [36]. and
the texture dataset Textures [37] with non-overlapping categories to ImageNet-Ik.
As for the evaluation on CIFAR Benchmarks, we use the CIFAR-10 and CIFAR-100 [38 as the in-distribution datasets using the standard split with 50,000 training images and 10,000 test images. We consider four OOD datasets: SVHN [39]. Tiny ImageNet 40], LSUN [1 and Textures [37.
Baselines. We consider different kinds of competitive OOD detection methods as baselines, including Maximum Softmax Probability (MSP) ( ODIN 77]. Energy [5), Mahalanobis [1]. GradNorm I2 and ReAct 8 MSP is simple baseline for OOD detection and ReAct a state-of-the-ard method that achieves strong detection performance. All methods use the pre-trained networks post-hoc.
Metrics. FPR95: the false positive rate of OOD (negative) examples when the true positive rate of in-distribution (positive) examples is as high as 95%. Lower FPR95 indicates better OOD detection performance and vice versa. AUROC: the area under the receiver operating characteristic curve (ROC). Higher AUROC indicates better detection performance. See Appendix E for more details.
5.2
Evaluation on the large-scale OOD detection benchmark
We first evaluate method on a large-scale OOD detection benchmark proposed by Huang and Li [33. [33] revealed that OOD detection methods designed for the CIFAR benchmark might not effectively be adaptable for the ImageNet benchmark with a large semantie space. Recent literature [3] proposes to evaluate OOD detection performance on images that have higher resolution and contain more classes than the CIFAR benchmarks, which is more relevant real-world applications.
In Tab. T we compare our method with the existing methods and show the OOD detection per- formance for each OOD test dataset and the average over the four datasets. We consider different

---

Table 1: OOD detection performance comparison on different architectures: ResNet-550 (RN50) 22 DenseNet-121 (DN121) 4 and MobileNet-V2 (MNet) [44] We use the pre-trained models in PyTorch 45 trained on ImageNet. All methods are post hoc and can be directly used for pre-trae trairrd models. The best results are Bold. The up arrow indicates that the higher the value, the better the performance, and vice versa.
INaturalist
SUN
Places
Textures
Average
Model
Method
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
6
51.44
88.17
72.04
79.95
74.34
78.84
54,90
78.69
65.18
81.41
ODIN
41.07
91.32
64.63
84.71
68.36
81.95
50.55
85.77
56.15
85,94
Energy 5
46.65
91.32
61.96
84,88
67.97
82.21
$6.06
84.88
58.16
85.82
RN50
Mabslanobs 11
97.00
52.65
98.50
42.41
98.40
41.79
55.80
85.01
87.43
55.47
GraNNorm
1
23.73
93.97
42.81
87.26
55.62
81.85
38,15
87.73
40.08
87.70
ReAct g
17.77
96.70
25.15
94.34
34.64
91.92
51.31
88.83
32.22
92.95
OuTS)
12.57
97.67
22.62
95.33
34.34
38.90
92.27
27.11
94,28
MSP
47.65
89.09
69.95
79.64
72.53
78.74
69.69
77.06
64.96
81.13
ODIN
30.72
Energy 5
33.16
93.81
53.79
86.70
61.01
83.83
55.42
84.06
50.85
87.10
DN121
Mahalanobis
97.36
42.24
96.21
41.28
97.32
47.27
62.78
46.83
GradNorm P2
22.88
94,40
43.12
87.55
55.80
82.00
47.58
85.16
42.35
87.28
ReAc
15.93
96.91
40.41
90.13
48.87
87.98
36.58
92.48
35.45
91.88
OuTS(
14.63
97.13
30.45
93.03
41.35
89.24
31.72
93.40
29.54
93,20
MSP
63.09
85.71
79.67
76.01
81.47
75.51
75.12
76.49
74.84
78.43
ODIN 7
45.61
63.03
83.44
$2.45
$7.78
85,31
Energy [S
49.52
91.10
63.06
84.42
69.24
81.42
$8.16
84.88
60.00
85.46
MNet
Mahalanobis |
62.04
82.37
$4.79
86.33
53.77
88.72
37.28
72.42
GradNorm 12
33.70
92.46
42.15
89.65
56.56
83.93
34.95
90.99
41.84
89.26
ReAct
BATS(Ours)
31.56
94.33
41.68
90.21
52.43
86.26
38.69
90.76
41.09
90.39
architectures, including the widely used ResNet-50 [42.
43 and lightweight model MobileNet-v2 [44]] Compared with the Energy Score [5]. the difference in our approach is rectifying the features that deviate from the feature's typical set. Our method outperforms the Energy Score on ResNet-50 by 31.05% in FPR95 and 8.46% in AUROC. Furthermore, our method reduces FPR95 by 5.11% and improves AUROC by 1.33% compared to the state-of-the-ang method [8] on ResNet-50..
DenseNet-121
Here the models are pre-trained in a standard manner. We also show that BATS can boost the OOD detection when using the adversarially pre-trae pre-trained classifiers in Appendix F.
Simultaneously, we observe that existing methods have different performances on different archi- tectures. Specifically, the performance of the GradNorm [2] in FPR95 is 7.8 7.86% worse than that of ReAct 8 on the ResNet-50, but surpasses ReAct on MobileNet- V2 by 4.86%. Our method achieves the best performance different architectures. Appendix K shows that our method also outperforms the existing methods when choosing the natural adversarial examples [46 as OOD examples.
Vani
70
Vanilla Vanlll-RRaAct RtAct Vanilla BATS
Vani RaAct
Vani BATS
40
50-
40
30-
20
10-
MSP
06/N
GreaNorm
MSP
000
Energy
GredNorm
(b) MobilaNet-V2
Figure The FPR95 for different methods on ImageNet (lower better) on ResNet-50 and MobileNet-V2. We illustrate the average performance on four OOD datasets. "Vanilla" means the original method and "VanillaBBATS" means applying our BATS on the method.
Our experiments mainly use the energy score as the test statistic. In Fig. 3 we show that BATS is also compatible with various OOD scores and BATS can boost the performance various OOD scores. Applying our BATS on GradNorm [2] (a gradient-based OOD score) can even achieve better performance than "Fnergy+BATS" but this method needs to derive the gradients the model, which costs more than "Energy+BATS. See Appendix I for detailed performance.
7

---

Table 2: OOD detection performance on CIFAR-I0 and CIFAR-100 100 and can be directly used for pre-trained models. The best results are in Bold.
38 All methods are post hoc
SVHN
Tiny-Imagenet AUROC
LeUN_resize resize AUROC
Texture
Average
Dataset
Method
FPR95
AUROC
FPR95
FPR95
FPR95
AUROC
FPR95
AUROC
1
6
59.60
50.01
93.02
52.15
92.73
66.63
88.50
91.39
ODIN 7
59.71
88.52
10.95
98.08
9.24
98.25
52.06
89.16
32.99
93,50
CIFARIO RNI8
Energy
23.53
96.14
GradNorm
82.45
79.85
19.23
96.77
48.99
90.67
69.40
81.72
55.02
87.25
ReAc|
46.87
96.92
94,29
OArs(Ours)
38.42
93.53
17.75
96.91
19.85
96.59
43.81
92.32
29.96
94.84
MSP
39.57
54.60
44.31
91.00
ODIN
61.13
82.49
12.79
97.61
12.49
97.50
61.13
80.18
36.89
89.45
CI
Energy
56.05
96.99
28, 38.44
61.74
40.95
91.15
WRN
GradNorm
P
88.55
49.14
41.25
90.68
91.02
48.94
90.83
46.28
77.91
58,76
ReAc
58.35
86.67
18.85
96.62
16.52
97.04
50,89
89.27
36.15
92.40
OATS(Ours) 50.60
89,50
25.17
95.66
11.98
97.70
45.30
91,18
33,26
93,51
6SP
81.79
77.80
68.32
83.92
82.51
75.73
85.12
73.36
79.44
77.70
ODIN 7
40.82
93.32
69.34
86,28
79.62
82,12
83.61
72.36
68.35
83,52
CIFAR100 RNI8
Energy
81.24
84.59
40.12
93.16
73.56
82.98
85.87
74.94
70.20
83.92
GradNorm
P
57.65
87.77
25.77
95.12
89.60
63.25
79.08
68.89
63.03
78.76
ReAct R
70.28
88.25
45.62
91.02
55.57
89.32
61.01
87.57
58.12
89.04
OuTS(Ours)
61.48
90.63
44.41
91.27
52.68
90,04
52.36
89.72
52,73
90.42
6SP
78.43
77.74
61.33
87.46
81.69
72.69
85.07
75,46
76.63
78.34
ODIN 7
35.69
94.84
82.68
79.17
87.48
74.53
86,97
65.40
73.21
78.49
CIFARI00
Energy
75.57
83.05
40.87
92.99
65.90
82.78
87.98
71.21
67.58
82.51
GradNorm 12
83.24
72.55
ReAct
72.94
86.89
42.07
91.97
60.87
85.90
84.18
76.22
65.02
85,25
BATS(Ours)
71.01
87.50
91.97
57.01
88.04
80.46
5.3 Evaluation on CIFAR benchmarks
We further evaluate our method on CIFAR benchmarks and CIFAR-10 and CIFAR-100 38 as the in-distribution datasets respectively. Tab. compares our method with the baseline methods and shows the OOD detection performance for each OOD test dataset and the average over the four datasets. We evaluate our method on the ResNet-18 18 (RN18) [42] and WideResNet-28-- (WRN) 47). The models are trained for 200 epochs with batch size of 128. The starting learning rate is 0.1 and decays by a factor of 10 at epochs 100 and 150.
ODIN 7 performs the best in the baselines methods CIFAR- 10 with an FPR95 of 32.99% on ResNet-18 18. Our method outperforms ODIN by 3.03% and outperforms the simple baseline method MSP 66) by 27.14% in FPR95. As for using CIFAR-I00
the in-distribution dataset, ReAct [8] is the best baseline method. Our approach surpasses the ReAct by 5.39% in FPR95 on ResNet- 18. Our method achieves the best performance on both CIFAR-10 10 and CIFAR-100 Our approach is also effective when using the WideResNet model, outperforming the existing methods.
5.4
Ablation studies
5.4.1
Rectifying the features of the early layers
In our experiments, we rectify the features of the penultimate layer (the layer before the fully connected layer), which is convenient and efficient. However, what will happen if we rectify the features of the early layers with BATS? The early layers refer the layers close the input [48). In particular, the original ResNet-50 [42] consists of four residual blocks. Block1 is close to the input and Block4 is close to the output. In Tab. we show the influence of applying feature rectification on the output of different blocks. Applying feature rectification to the early blocks (from Block to Block3) has little effect on the performance of OOD detection, while the last block plays a vital role. Applying feature rectification all the blocks performs the best in our experiments, which is 0.99% higher than the "Block4" in FPR95 and 32.04% higher than "Without" in FPR95. Considering that the latest block has more significant impact on the OOD detection performance than the other blocks, we just rectify the features of the penultimate layer with BATS for the simplicity of the method.
To find out why the last block has significant influence on the OOD detection while the other blocks contribute little, we visualize the feature embeddings extracted by different blocks in ResNet-50 using t-SNE 49 in Fig. 4 We choose the iNaturalist as the OOD dataset and the ImageNet as the ID dataset. The features extracted by the early blocks of the ID and OOD examples are similar, which has little benefit in distinguishing the ID and OOD examples. In contrast, the last block can extract
8

---

Table 3: Ablation study of the influence of feature rectification different blocks. "Without" means applying no rectification on any blocks. "Block1-4" means applying rectification on all blocks.
iNaturalist AUROC
SUN
Places
Textures
Average
Blocks
FPR95
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
Without
46.65
91.32
61.96
84,88
67.97
82.21
56.06
84,88
58.16
85.82
Block1
49.72
90.73
62.67
84,62
68.30
82.03
55.62
85.04
59.08
85.61
Block2
41.78
92.36
63.73
84.67
69.45
81.95
55.53
85.45
57.62
86.11
Block3
40.76
92.55
58.37
86.56
64.78
83.82
51.45
86.77
53.84
87.43
Block4
12.57
97.67
22.62
95.33
34.34
91.83
38.90
92.27
27.11
94.28
Block1-2
43.63
91.92
63.22
84.61
69.28
81.84
53.67
85.72
57.45
86.02
Block1-3
38.05
93.04
59.47
86.47
66.30
83,49
49.72
87.50
53.39
87.63
Block1.4
12.76
97.54
21.15
95.51
33.01
91.91
37.55
92.54
94.38
perfectly separable features the ID and OOD examples. This may be due the fact that deep
neural networks focus on similar general features (edges, lines, and colors) in the early layers and pay more attention to specific features related classification in the late layers [88 50). The late layer can contribute more to the OOD detection than the early layer. See more in Appendix B.
(a)
(b) Block2
[d]
Figure 4: t-SNE visualizations. We illustrate the 1-SNE plots for the features of in-distribution
examples (ImageNet) and out-offdistribution examples (iNaturalist) from different blocks.
5.4.2 The influence of the hyperparameter
In Sec. theoretically analyze the bias-variance trade-off in method. Our proposed BATS can reduce variance, which benefits OOD detection, but can also introduce a bias. Here, we empirically show the influence of the hyperparameter À in Fig. 5 As À tends to infinity, BATS approaches to the Energy Score (the horizontal lines). Very small X will damage the performance.
90
FPROS
63
F9R05
82
F9RS2
80
FAR5S AUROC
614
AUROCO
30
63.5
300
3.5
(a) ImageNat
(b) CIFAR-100
Figure 5: (a) The influence of the apper-parameter À on the OOD detection on ImageNet. The model ResNet-50. We illustrate the average performance four OOD datasets. (b) The influence of the hyper-parameter parameter À on the OOD detection on CIFAR-100. The model is WideResNet. The horizontal line indicates the OOD detection performance without feature rectification.
Conclusion
In this paper, we provide novel insight into the obstacle factor in OOD detection from the perspective of typicality and hypothesize that extreme features can be the culprit. We propose rectify the features into the typical set and provide concise and effective post-hoc approach BATS to estimating the feature's typical set. BATS can be applied to various OOD scores boost the OOD detection performance. Theoretical analysis and ablations provide further understanding of our approach. Experimental results show that our BATS can establish state-of-the-art OOD detection performance on the ImageNet benchmark, surpassing the previous best method by 5.11% in FPR and 1.43% in AUROC. We hope that our findings can motivate new research into the internal mechanisms of deep models and OOD detection and uncertainty estimation from the perspective of feature typicality.

---

Limitations and societal impact. This paper proposes to rectify the feature into its typical set to improve the detection performance against OOD data and provides a plug-and-play method with the assistance of BN. The limitation of our method can be that the BN layers are required in the model architecture in our approach. BN layers are widely used in convolutional neural networks to alleviate covariate shifts, but there are also architectures without BN. A set of training images can contribute to selecting the feature's 's typical set and alleviate this limitation. We also anticipate some other information in the model is conducive to selecting the feature's ' typical set and improving the post-hoc OOD detection performance. We leave this as future work. Although truncating features into a typical set improve OOD detection, potential negative impact of the proposed process is that it inherently introduces a bias and causes some information loss which may be important to the model in real-world scenarios.
Acknowledgments
This work was supported in part by the Fundamental Research Funds for the Central Universities, by Alibaba Group through Alibaba Research Intern Program, and by the National Nature Science Foundation of China 62001 146. Dr. Xie's research work is partially supported by the Interdisciplinary Intelligence SuperComputer Center of Beijing Normal University at Zhuhai.
References
[1] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, pages 3354-3366 135--361. IEEE, 2012.
[2] Xiaowei Huang, Daniel Kroening, Wenjie Ruan, James Sharp. Youcheng Sun, Emese Thamo, Min Wu, and Xinping Yi. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability. Computer Science Review, 37:100270, 2020.
[3] Geert Litjens. Thijs Kooi, Babak Ehteshami Bejnordi, Arnad Arindra Adiyoso Setio, Francesco Ciompi. Mohsen Ghafoorian. Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. A survey on deep lcarning in medical image analysis. Medical image analysis, 42:60-88 88, 2017.
[4] Ahmet Murat Ozbayoglu, Mehmet Ugur Gudelek, and Omer Berat Sezer. Deep learning for financial applications: A survey. Applied Soft Computing, 93:106384, 2020.
[5] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advancex in Neural Information Processing Systems, 2020.
[6] Dan Hendrycks and Kevin Cimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. Proceedings of International Conference on Learning Representations, 2017.
[7] Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of distribbuti image detection in neural networks. In International Conference on Learning Representations, 2018.
[8] Yiyou Sun, Chuan Guo, and Yixuan Li. React:
detection with rectified activations. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information P Processing Systems, 2021.
[9] Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. Advunces in Neural Information Processing Systems, 32, 2019.
[10] Aristotelis- Angelos Papadopoulos, Mohammad Reza Rajati, Nazim Shaikh, and Jiamian Wang. Outlier exposure with confidence control for out-of-distribttion detection. Neurocomputing, 441:138-150, 2021.
[11] Kimin Lec, Kibok Lec, Honglak Lec, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018
[12] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. In Advances in Neural Information Processing Systems, 2021.
[13] Haoran Wang. Weitang Liu, Alex Bocchieri, and Yixuan Li. Can multi-label classification networks know what they don't know? In Advances in Neural Information Processing Systems, 2021.
10

---

[14] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 41-50, 2019.
[15] Julian Bitterwolf, Alexander Meinke, and Matthias Hein. Certifiably adversarially robust detection of out distribution data, Advances in Neural Information Processing Systems, 33:16085- 16095, 2020.
[16] Dan Hendrycks*, Norman Mu*. Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji I Laksh- minarayanan. Augmix: simple method improve robustness and uncertainty under data shift. In International Conference on Learning Representations, 2020.
[17] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023-6032, 2019.
[18] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations. 2018.
[19] Jiefeng Chen, Yixuan Li. Xi Wu, Yingyu Liang, and Somesh Jha. Atom: Robustifying out-of-distribttion detection using outlier mining. In Joint European Conference on Machine Learning and Knowledge Discovery Databases, pages 430-445. Springer, 2021.
[20] Ivan Kobyzov, Simon JD Prince, and Marcus A Brubaker. Normalizing flows: An introduction and review of current methods. IEEE transactions on pattern analysis and machine intelligence, 43(1 (3(11):36643397) 2020.
[21] Fy Zisselman and Aviv Tamar. Deep residual flow for out of distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 14994414003,, 2020.
[22] Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F Núñez, and Jordi Luque. Input complexity and ou--of-iitribution detection with likelihood-based generative models. In International Conference on Learning Representations. 2019.
[23] Zhisheng Xiao, Qing Yan, and Yali Amit. Likelihood regret: An out-of-distribution detection score for variational auto-encode aato--ncoder. Advances in neural information processing systems, 33:20685- 2222066662666,,,,66 2020.
[24] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? International Conference on Learning Representations. 2018.
[25] Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-of- distribution data. Advances in neural information processing systems, 33:20578- 20589, 2020.
[26] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-of- distribution inputs to deep generative models using typicality. arXiv preprint (.XXtv:996600999 2019.
[27] Amirhossein Ahmadian and Fredrik Lindsten. I Likelihond-free out-of-distribution detection with invertible generative models. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021.
[28] Matan Haroush, Tzviel Frostig, Ruth Heller, and Daniel Soudry. statistical framework efficient out distribution detection in deep neural networks. In International Conference on Learning Representations, 2022.
[29] Lily Zhang, Mark Goldstein, and Rajesh Ranganath. Understanding failures in out-oo--istribution detection with deep generative models. In International Conference Machine Learning, pages 12427-12436. PMI.R, 2021.
[30] Federico Bergamin, Picrre-Alexandre Mattci, Jakob Drachmann Havtom, Hugo Senctaire, Hugo Schmutz, Lars Maalge, Soren Hauberg, and Jes Frellsen. Model-agnostic out-of detection using combined statistical tests. In International Conference on Artificial Intelligence and Statisticx, pages 10753-10776. PMLR, 2022.
[31] Will Grathwohl, Kuan- Kuan-Chieh Wang. Jömn Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. International Conference Learning Representations, 2020.
11

---

[32] Yao Zhu. Jiacheng Ma, Jiacheng Sun. Zewei Chen, Rongxin Jiang, Yaowu Chen, and Zhenguo Li. Towards understanding the generative capability of adversarially robust classifiers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7728-7737, 2021.
[33] Rui Huang and Yixuan Li. Mos: Towards scaling distr---isttiuution detection for large semantic space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8710-8719,
2021.
[34] Grant Van Horn, Oisin Mac Aodha, Yang Song. Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pictro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8769 8778. 2018
[35] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40 -9,1442--464,, 2017.
[36] Jianxiong Xiao, J Hays, KA Ehinger, A Oliva, and A Torralba. Sun database: Large-scale scene recognition from abhey to zoo. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010.
[37] Mircca Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606-3613, 2014.
[38] Alex Krizhevsky et al. Learning multiple layers of fcatures from tiny images. 2009.
[39] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[40] Patryk Chrabaszez, Ilya Loshchilov, and Frank Hutter. A downsampled variant of imagenct alternative to the cifar datasets. arXiv preprint 1xXiv:170700855 2017.
[41] Fisher Yu, Ari Seff. Yinda Zhang. Shuran Song. Thomas Funkhouser, and Jianxiong Xiao. Lsun: Con- struction a large-scale image dataset using deep learning with humans in the loop. arXiv preprint 1506. 03365,, 2015.
[42] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[43] Gao Huang, Zhuang Liu. Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 17700-4778, 2017.
[44] Mark Sandler. Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Inverted residuals and lincar bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510-4520, 2018
Chieh Chen. Mobilenetv2:
[45] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32, pages 8024-8035, 2019.
[46] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15262-15271, 2021.
[47] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
[48] Johannes Mehrer, Courtney Spoerer, Nikolaus Kriegeskorte, and Tim Kietzmann. Individual differences among deep neural network models. Nature Communications, 01 2020. doi: 10.1101/202001. 08.898288.
[49] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9:2579-2605, 2008
[50] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? Advances in neural information processing systems, 27, 2014.
12


Reviews:
[{"review_id": "lPvDOez8Vt5", "reviewer": "Reviewer_3fd9", "summary": "The authors propose an OoD method that does not require the retraining of the model, and which relies on replacing the last BN layer by their proposed TrBN at inference time. The main concept of the new layer is that it clamps the most extreme features to be within their variance. By correcting those features (compressing the extreme features), the outputs of the model become less susceptible to OoD miss-classifications. The authors provide comparable or better results to existing state-of-the-art methods under different OoD scenarios.", "questions": "Could you provide a bit more explanation on the differences with ReAct, that truncates the values of the activations? I assume by the results that the activations targeted by both ReAct and BATS are not the same, but is there any insight on that? Are the abnormal activations of ReAct closely correlated with the extreme features of BATS?\n\nMost reported results are basically an upgrade on top of Energy [5], which can be applied to other methods (as shown in Fig. 3, and Appendix I). Therefore, wouldn't it make more sense to report it in most tables/figures as Energy+BATS? Considering the huge gap between the Energy and the Energy+BATS results, why use that combination and not another one? Even with the added cost of GradNorm, the results are much better. Also, that would raise the question to why would BATS work better in some methods than others.\n\nMinor comments:\n- check for typos and revise manuscript (e.g. line 9 play-and-plug --> plug-and-play, line 98 energe --> energy)\n- Table 2, CIFAR100 WRN, ODIN, that should be in bold (check for other cases)\n- Figure 5 is not mentioned in the text.", "limitations": "It is correctly stated in the limitations that this method is proposed as a post-hoc method, however, it has the limitation of the model a BN layer before the last FC layer.\n\nThere does not seem to be any potential negative societal impact generated from this work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Strentghs*\n\nThe idea is of interest to the community and the method is easy to implement.\n\nThe theoretical analysis seems sound and well based.\n\nThe analysis of the trade-off from lambda and the bias introduced by the proposed TrBN looks correct.\n\nThe ablation study provides nice insight and answers a question I was already thinking of while reading the method section.\n\n*Weaknesses*\n\nThe comparison with the related work is brief and there is little discussion of the similarities and differences. Methods such as ReAct seem to be close to the proposed approach, since they target \"anomalous features\".\n\nThe presentation could be improved. It requires a revision to correct some typos and grammatical errors. Some images and their texts could be more readable (e.g. Figs. 1-2).\n\nThe best results of the proposed method are in the supplementary, which is a bit odd. I would expect those to be discussed in the main paper, and also provide some insight on why the proposed method might work better in some of those methods. With the results shown in the main paper, the improvements in some of the scenarios are quite marginal in relation to other methods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "QmWTM9Mmsux", "reviewer": "Reviewer_bMZr", "summary": "The paper presents a post hoc method for OOD detection in classification models. The key to the proposed approach is to determine the typical feature set from the data and the trained neural network, and use it to compute an existing OOD detection score, such as the energy score. The typical set is computed by truncating the output from a batch normalization error. The truncation is controlled by a hyperparameter that determines the bias-variance trade-off for the method. The proposed BATS method outperforms baseline methods consistently across a variety of experimental settings.", "questions": "In addition to addressing the above listed weaknesses, I have the following questions for the authors.\n\n1. How well do you believe that OOD detection can work post hoc without retraining the model? How much do poorly calibrated softmax uncertainties hinder post hoc method effectiveness?\n2. Do you have a hypothesis for why BATS performance is lower than baseline methods on the Tiny-Imagenet OOD dataset in Table 2?\n3. In the limitations, it is mentioned that 'some other information in the model' may be conducive to selecting the feature's typical set. Do you have a hypothesis for what this information might be?", "limitations": "The authors have done a good job in listing limitations of the BATS method. However, the addition of some potential negative societal implications would be helpful. For example, by truncating the features, certain biases in the data learned by the pre-trained model may be amplified. Furthermore, the process of truncation will inherently cause some information loss which may be crucial to model performance during deployment (building on the hyperparameter tuning discussion in the paper).", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* I really enjoyed reading this paper. It is well written and the ideas are easy to follow. \n* The problem is well-motivated and the literature review does a good job at contextualizing the paper in prior work.\n* The proposed approach is simple, yet appears to be highly effective. The theoretical analysis further provides intuition for the BATS method.\n* Overall, the empirical evaluation of the method is extensive and convincing. The analysis and discussion is thoughtfully constructed, and the effects of the hyperparameter thoroughly ablated.\n* The figures are informative and effectively illustrate the benefits of the proposed approach.\n* The analysis in the appendix was extensive and interesting, providing further support for the claims made in the main paper.\n\nWeaknesses:\n* In lines 3-4, the statement is not necessarily true as there are also methods that focus on better distribution calibration during training (some of these ideas are mentioned later in the paper as well). I would recommend softening the message of this sentence to be more precise.\n* Language like 'Obviously' (line 47) and 'It is easy to see' (line 168) should generally be avoided in academic writing.\n* At the end of Sec. 2, it would be helpful to have a one or two sentence discussion to contextualize the proposed BATS method in the described related work.\n* Variations on the phrase 'We propose to rectify the features into the feature's typical set and then use these typical features to calculate the OOD score.' are repeated frequently throughout the paper.\n* The $\\lambda$ hyperparameter was not introduced in line 142 when it was first used.\n* The decreased BATS performance on Tiny-Imagenet OOD detection in Table 2 is not mentioned or discussed.\n\nThe paper needs to be proofread for typos. The following is a non-exhaustive list of the typos I found:\n\n1. Line 2: 'which raises the attention on out-of-distribution (OOD) detection' is awkward phrasing.\n2. Lines 74-75: 'large sufficiently' should read 'sufficiently large'.\n3. Line 98: 'energe score' should read 'energy score'.\n4. Line 109: 'is provable aligned' should read 'is provably aligned'.\n5. Line 130: 'common-used layer' should read 'commonly used layer'.\n6. Footnote 1: I did not grammatically understand the phrase: 'the pre-training outputs moving average estimators during iterations'. Maybe something like: 'The pre-trained model outputs moving average estimators at each iteration.'?\n7. Line 184: 'a two-side rectified normal distribution' should read 'a two-sided rectified normal distribution'.\n8. There should be a space between the abbreviation and the number in references (i.e., Fig. X, Table Y, Sec. Z).\n9. Line 197: 'Fig.2 illustrate' should read 'Fig. 2 illustrates'.\n10. Line 220: 'verse vice' should read 'vice versa'.\n10. Line 225: 'Recent researches propose' should read 'Recent literature/work proposes'.\n11. Line 235: 'models are standard pre-trained' is grammatically awkward. Maybe something like: 'models are pre-trained in a standard manner'?\n12. Line 238: 'In specific,' should read 'Specifically,'.\n13. Line 246: 'which cost more' should read 'which costs more'.\n14. Line 252: 'The start learning rate' should read 'The starting learning rate'.\n15. Line 284: 'Our BATS can reduce the variance that benefit the OOD detection but also introduce a bias.' should read 'Our proposed BATS method can reduce variance, which benefits OOD detection, but can also introduce a bias.'.\n16. Line 285: 'Energy Score (The horizontal lines).' should read 'Energy Score (the horizontal lines)'.\n17. Fig. 5: x-axis labels are missing. This is also the case for some figures in the appendix.\n18. Sec. 6: 'Limitation and societal impact' should read 'Limitations and societal impact'. \n19. Sec. 6: batch normalization is referred to in three different ways in the last paragraph (Batch Normalization, Batch-Norm, BN).\n20. Line 693: 'our method surpass' should read 'our method surpasses'.\n21. The references should be proofread (e.g., to ensure the year is not entered twice in a citation, the conference venue is listed instead of ArXiv when available, etc.).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "IntM5JJIdZ9", "reviewer": "Reviewer_juXE", "summary": "This paper proposed Batch Normalization Assisted Typical Set Estimation (BATS) for enhancing Out-of-distribution Detection methods. BATS is a truncated activation scheme to bound the output features of the BN unit. The paper demonstrated that BATS could improve several OOD detection methods, such as Energy, Gradnorm, ODIN, when applying it to rectify the features of the penultimate layer. The proposed approach looks simple and effective.", "questions": "Besides the concerns mentioned above, I don't understand why BATS can boost OOD detection theoretically.\n\nSection 4.3 shows that BATS reduces the variance of output features and also introduces a bias term. In lines 175 - 177, the authors stated that \"Our BATS aids this problem by reducing the variance of the deep features, which contributes to constraining the uncertainty of f and T(x; f) and improving the estimation accuracy of the reject region.\"\n\nHowever, why variance reduction boosts OOD detection isn't clear. Does it generally improve classifiers or specifically solve OOD detection problem?", "limitations": "The discussion of limitations by authors is adequate. As said in Lines 299 - 300, \"The limitation of our method can be that the Batch-Norm layers are required in the model architecture in our approach.\" I would encourage the authors to continue to improve the method and find a more general formulation, as normalization technique (especially unified formulation) have been extensively studied.", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1) The proposed approach is simple to implement\n2) The proposed approach is effective on several OOD detection methods with BN in the architecture demonstrated by experiments\n3) The paper is well-written and easy to understand.\n\nWeaknesses:\n1) My first impression of this paper is that the proposed approach looks like a slight modification of ReAct.\nAlthough BATS outperforms ReAct in experiments, the operation and the mathematical analysis look very similar.\nIn my understanding, BATS adaptively estimates the parameter c in ReAct.\n\n- The authors explain the motivation in the perspective of typical features, but typicality is not novel in OOD detection, e.g.,\n\n[r1] Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality.\n\n[r2] WAIC, but Why? Generative Ensembles for Robust Anomaly Detection.\n\n2) The major limitation of the proposed approach is that it can be applied only on BN (please correct me if I misunderstand.)\nThis can bring two major concerns:\n- As previous research showed that BN increases adversarial vulnerability [r3, r4], whether the proposed approach solved the problem of BN or had the ability of detecting OOD samples is not clear.\n- Modern archiectures utilize better normalization methods other than BN, such as LN and GN, and thus interest and impact of the proposed method is greatly limited.\n\n[r3] Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective. ICCV 2021.\n\n[r4] Batch Normalization is a Cause of Adversarial Vulnerability.\n\n3) The authors didn't compare with SOTA of OOD detection on large-scale ImageNet, e.g.,\n\n[r5] Out-of-distribution detection with deep nearest neighbors. ICML 2022.\n\n[r6] VIM: Out-of-distribution with virtual-logit matching. CVPR 2022.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2l43FfUKsw", "reviewer": "Reviewer_NEkg", "summary": "The paper proposes a replacement of the batch normalization layer (BATS) to rectify deep model features into its typical set to improve OOD detection performance. The authors provide theorectical analysis and ablation studies to look deeper into BATS, and achieves state-of-the-art performance using it as a play-and-plug module.", "questions": "1.Uncertainty measurement is important in OOD cases. Although the authors used FPR95, AUROC and test accuracy (in appendix) to show the effectiveness of BATS, it would be interesting to analyze the results of the pretrained models from the perspective of uncertainty quantification, such as comparing Brier score or expected calibration error (ECE)[1].\n\n2.What are the choices for the reject region threshold $\\gamma$, is it always the best value or a fixed value? \n \n[1] Guo C , Pleiss G , Yu S , et al. On Calibration of Modern Neural Networks, 2017.", "limitations": "The authors have properly addressed the limitations and potential negative societal impacts.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\nOriginality: The idea is new and intriguing. The authors propose a novel insight for OOD detection from the perspective of feature typicality, and divides deep features into typical features and extreme features. They also propose a novel replacement for Batch Normalization, which can be integrated into existing model structrues and OOD scores.\n\nQuality: From the perspective of typicality, and under the assumption that extreme features are harmful for model training, the authors provide thourough ablation studies on the proposed BATS approach, and theoretically prove its bias-variance trade-off. The proof is in detail and experiments are discussed well.\n\nClarity: The paper is well-written and easy to follow.\n\nSignificance: The paper provides a novel perspective into OOD detection, and introduces a simple yet effective method for post-hoc detection. The method is evaluated on both small and large real-world datasets.\n\n**Weaknesses**\n\nI have not identified major weaknesses of this paper, while I do have some minor concerns that are listed in the “Questions” part.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept