Boosting Out-of-distribution Detection with Typical Features
Anonymous Author(s)
AffiliationAddress
Abstract
Out-of-distribution (OOD) detection is a critical task for ensuring the reliabilityand safety of deep neural networks in real-world scenarios.
Different from mostprevious OOD detection methods that focus on designing OOD scores or introducing diverse outlier examples to retrain the model, we delve into the obstaclefactors in OOD detection from the perspective of typicality and regard the feature’shigh-probability region of the deep model as the feature’s typical set.
We proposeto rectify the feature into its typical set and calculate the OOD score with the typicalfeatures to achieve reliable uncertainty estimation.
The feature rectification can beconducted as a plug-and-play module with various OOD scores.
We evaluate thesuperiority of our method on both the commonly used benchmark (CIFAR) and themore challenging high-resolution benchmark with large label space (ImageNet).
Notably, our approach outperforms state-of-the-art methods by up to 5.11 % in theaverage FPR95 on the ImageNet benchmark.
Introduction
Deep neural networks have been widely applied in various fields.
Apart from the success of deepmodels, predictive uncertainty is essential in safety-critical real-world scenarios such as autonomousdriving [1, 2], medical [3], financial [4] etc.
When encountering some examples that the deep modelhas not been exposed to during training, we hope the model raises an alert and hands them overto humans for safe handling.
Such a challenge is usually referred to as out-of-distribution (OOD)detection and has gained significant research attention recently [5–8].
Most of the existing research [5–7, 9–12] worked on designing suitable OOD scores for the pretrained neural network, hoping to assign higher scores to the in-distribution (ID) examples and lowerscores to the out-of-distribution (OOD) examples.
However, these methods overlook the obstaclefactors in OOD detection caused by the model’s internal mechanisms.
In this paper, we rethink the
OOD detection from a perspective of feature typicality.
We observed that the distribution of the deepfeatures of the training dataset on different channels is approximately consistent with the Gaussiandistribution (See examples in Appendix J).
Accordingly, we divide these features into typical features(fall in the high-probability region) and extreme features (fall in the low-probability region).
Extremefeatures rarely appear in training and attract less attention from the classifier than the typical features.
We hypothesize the classifier can model the typical features better than the extreme features, and theextreme features may lead to ambiguity and imprecise uncertainty estimation.
Given the potentialnegative impact, properly dealing with these extreme features is a key to improving the performanceof OOD detection.
In this paper, we propose to rectify the features into their typical set and then calculate the OODscore with these typical features.
In this way, the model conservatively utilizes the typical features tomake decisions and alleviates the damage caused by extreme features, which can be beneficial to the
OOD scores derived from the pre-trained classifier [5–7, 12].
Then the problem is how to estimatethe feature’s typical set on different channels since this requires a sufficient number of in-distributionexamples and is time-consuming.
Luckily, the commonly used operation Batch Normalizationcan shed light on a shortcut to selecting the feature’s typical set and we name our approach B atch
Normalization A ssisted T ypical S et Estimation ( BATS ).
The Batch Normalization layer endeavors tonormalize the features of the training dataset to Gaussian distributions, which can be used to estimatethe typical set for ID features.
Typical features are more common in training, while extreme featuresare rare, which leads to difficulties for the model to estimate extreme features well.
We truncate thedeep features with the guidance of the Batch Normalization, rectifying the extreme features to theboundary values of typical sets.
We illustrate the distribution of the OOD scores for ID (ImageNet)and OOD (four different datasets) examples in Fig. 1.
Rectifying the features into the typical set withour BATS contributes to improving the separability between ID and OOD examples.
[Figure]
O u r s Theoretically, we analyze the benefit of BATS and the bias-variance trade-off influenced by thestrength of the hyperparameter.
A proper strength of BATS contributes to improving the estimationaccuracy of the reject region.
Empirically, we perform extensive evaluations and establish superiorperformance on both the large-scale ImageNet benchmark and the commonly used CIFAR benchmarks.
BATS outperforms the previous best method by a large margin, with up to a 5.11 % reductionin the false positive rate (FPR95) and a 1.43 % improvement in AUROC.
Moreover, BATS can alsoslightly improve the test accuracy and robustness of the pre-trained models.
The main contributions of our paper are summarized as follows:
• We provide novel insights into OOD detection from the perspective of typicality and proposeto rectify the features into the typical set.
We design a concise and effective approach to
select the feature’s typical set named B atch Normalization A ssisted T ypical S et Estimation( BATS ).
• We provide theoretical analysis and empirical ablation on the benefit of BATS from theperspective of bias-variance trade-off to improve the understanding of our approach.• Extensive experiments show that BATS establishes a state-of-the-art performance among
post-hoc methods on a suite of OOD detection benchmarks.
Moreover, BATS can boost the
performance of various existing OOD scores with typical features.
Related work
The literature related to OOD detection can be broadly grouped into the following themes: post-hocdetection methods [5–8, 11–13], confidence enhancement methods [9, 10, 14–19], and density-basedmethods [20–25].
Post-hoc detection methods focus on improving the OOD uncertainty estimationby utilizing the pre-trained classifiers rather than retraining a model, which is beneficial for adopting OOD detection in real-world scenarios and large-scale settings.
MSP [6] observes that the maximumsoftmax probability of ID examples can be higher than that of the OOD examples and provide a simple baseline for OOD detection.
ODIN [7] introduces a sufficiently large temperature factor andinput perturbation to separate the ID and OOD examples.
Liu et al.
[5] analyze the limitations ofsoftmax function in OOD detection and propose to use energy score as an indicator.
The exampleswith high energy are considered OOD examples, and vice versa.
ReAct [8] hypothesizes that the OOD examples can trigger the abnormal activation of the model and propose to clamp the activationvalue larger than the threshold to improve the detection performance.
GradNorm [12] shows that thegradients of the categorical cross-entropy loss can be an effective test statistic for OOD detection.
Different from these methods, our BATS proposes to calculate the OOD scores with the typicalfeatures, which benefits the estimation of the reject region and can improve the detection performance.
Preliminaries3.1 Out-of-distribution detection
In this section, we provide a summary of the out-of-distribution detection from the perspectiveof hypothesis testing [26–30].
We consider a classification problem with K classes and denotethe labels as Y = { 1 , 2 , . . .
, K } .
Let X be the input space.
Suppose that the in-distribution data
D in = { ( x i , y i ) } ni =1 is drawn from a joint distribution P X,Y defined over X × Y .
We denote themarginal distribution of P X,Y for the input variable X by P 0 .
Given a test input x ∈ X , the problemof out-of-distribution detection can be formulated as a single-sample hypothesis testing task:
[Equation]
Here the null hypothesis H 0 implies that the test input x is an in-distribution sample.
The goal of
OOD detection here is to design criteria based on D in to determine whether H 0 should be rejected.
OOD detection tasks need to determine a reject region R such that for any test input x ∈ X , thenull hypothesis is rejected if x ∈ R .
Generally, the reject region R is formulated by a test statisticand a threshold.
Let f : X
(cid:55)→ R K be a model pre-trained from D in , which is used to predict theclass label of an input sample.
One can use the model f or a part of f (e.g., feature extractor) toconstruct a test statistic T ( x ; f ) , where x is the test input.
Then the reject region can be written as
R = { x : T ( x ; f ) ≤ γ } , where γ is the threshold.
3.2 OOD detection with energy score
For a classifier f and a data point ( x , y ) , we use f ( x )[ k ] to represent the k th output of the last layer.
With reference to [5, 31, 32], the marginal density p ( x ) of the classifier can be expressed as: p ( x ) =
[Equation]
, where Z is the normalizing factor and is independent to x .
E ( x )represents the energy of x and is modeled by neural network
as E ( x ) = − log (cid:80)
Kk =1 exp( f ( x )[ k ]) .
See Appendix C for details.
Considering that Z is a constant and is independent to x , Liu et al.
[5]propose an energy score that uses the opposite of the energy E ( x ) as a test statistic to detect OODexamples.
A higher energy score means a higher marginal density p ( x ) .
Methods
In this paper, we delve into the obstacle factor for the post-hoc OOD detection from the perspectiveof typicality, which aims to boost the performance of the existing OOD scores and is orthogonal tothe methods of designing different OOD scores.
Given that the energy score [5] is provably alignedwith the density of inputs and performs well, we mainly use the energy score as the OOD score (See
Appendix I for other OOD scores).
4.1 Motivation
For a classifier trained on the ID data f = f w,b ◦ g where g is a feature extractor mapping input x toits deep feature z .
Let d -dimensional vector z
=
[ z 1 , ..., z d ] ⊤ = g ( x ) denote the deep features of xextracted by g , and z i indicate the i -th element of z .
f w , b ( z ) =
w · z + b is a fully connected layermapping the deep feature z to output logits.
The energy can be expressed as:
[Equation]
The test statistic can be expressed as T ( x ; f ) := − E ( x ) = log (cid:80)
Kk =1 exp(( w · z + b )[ k ]) , whichdepends on the extracted deep features and the mapping operation of the fully connected layer (FC).
Assuming that the distribution of the deep features is consistent with the Gaussian distribution (seeexamples in Appendix J), there are high-probability regions and low-probability regions in deepfeatures.
We name the features that fall in high-probability regions as typical features, and thecorresponding regions are called feature’s typical sets.
In contrast, we regard the features that fallin low-probability regions as extreme features.
Extreme features are rarely exposed to the trainingprocess, which leads to difficulties for the classifier to model these features and unreliable estimationsin the inference process.
Reducing the influence of extreme features on test statistics can be a key toimproving OOD detection performance.
4.2 Batch Normalization Assisted Typical Set Estimation
Instead of designing new OOD scores to detect the abnormality, we provide a novel insight into OODdetection from a perspective of typicality.
We propose to rectify the features into the feature’s typicalset and then use these typical features to calculate the OOD score.
Consider a commonly used layerstructure in deep convolutional networks:
[Equation]
where z ′ is the feature vector extracted from the convolutional layer of g. To identify the typical setof z ′ for each channel, we should apply its feature map to a sufficient number of ID examples andfurther calculate the empirical distribution of z ′ over the ID examples.
If the number of features islarge, the inference procedure is time-consuming.
Here we propose a simple and effective post hocapproach that leverages the information stored in f to infer the typical set without estimating thedistribution of z ′ .
Suppose the pre-trained deep neural network uses batch normalization (BN).
Wedenote the BN unit in f as:
[Equation]
where µ , σ are two learnable parameters.
After the pre-training, all the four parameters µ , σ , E ( z ′ ) ,
Std ( z ′ ) are known and stored in the weights of f. 1 The Batch Normalization normalizes features ofthe training dataset to a distribution with a mean of µ and standard deviation of σ , which means thatthe features fall in the interval [ µ − λ ∗ σ, µ + λ ∗ σ ] appear more frequently in training than thefeatures in the complement of this interval. The parameter λ controls the range of the interval.
Thuswe use the information in the Batch Normalization to identify the in-distribution feature’s typicalset and rectify the features into the typical set before calculating the OOD score.
The uncertaintyestimated with the typical features can be more reliable.
In practice, we propose a truncated activation scheme to bound the output features of the BN unit.
First, we introduce the truncated BN unit by:
[Equation]
where λ is a tuning parameter.
We replace the BN unit in the layer structure (Eq.(3)) with the TrBNunit and write the rectified final features as ¯ z and the new classifier as ¯ f .
Then test statistic with theenergy score can be expressed as:
[Equation]
and take the reject region by R = { x : T ( x ; ¯ f ) ≤ γ } .
We name our approach B atch Normalization
A ssisted T ypical S et Estimation (BATS).
In comparison to the standard BN, the outputs of TrBN areconcentrated toward the feature’s typical set of ID data.
This makes an ID example less susceptibleto being mistakenly detected as an OOD example and buffers the negative impact of the extremefeatures.
Fig. 1 compares the distribution of the OOD scores from the original energy score and theenergy score with typical features.
4.3 Theoretical analysis
The truncation threshold λ is a key hyperparameter.
Our method reduces the variance of z ′ and alsointroduces a bias term since it changes the distribution of BN ( z ′ ; µ, σ ) .
The variance reduction meansthat our method is robust to the rare ID examples, while the introduced bias can lead to degradationof the model performance.
In this section, we assume z ′ follows a normal distribution and analyzethe bias-variance trade-off in our method.
4.3.1 Understanding the benefits of BATS from the perspective of variance reduction
The variance reduction happens at the BN step.
The variance of BN ( z ′ ; µ, σ ) is σ 2 since thedistribution of the ID features z ′ is rescaled to N ( µ, σ 2 ) .
While the TrBN unit truncates the extremevalues and the variance of TrBN ( z ′ ; µ, σ, λ ) becomes:
[Equation]
where erf ( x ) = (2 / √ π ) (cid:82) x 0 exp( − t 2 ) dt is the Gauss error function.
The value of C ( λ ) representsthe degree of variance reduction.
In Eq.(7), C (0) = 0 , dC ( λ ) /dλ > 0 , and C ( λ ) → 1 as λ → + ∞ .
Therefore, C ( λ ) is a monotonically increasing function and 0 ≤ C ( λ ) < 1 for 0 ≤ λ < + ∞ .
Insummary, the smaller λ , the smaller the variance.
See Appendix D for the proof.
OOD detection is a single-sample hypothesis testing problem (in Eq.(1)), and the in-distribution
P 0 is unknown.
So the reject region is determined by the empirical distribution of the test statistic
T ( x ; f ) over the ID data.
The extreme features increase the uncertainty and lead to more unusualvalues of T ( x ; f ) .
This implies that the reject region may be underestimated due to the heavy tailproperty of T ( x ; f ) .
Our BATS aids this problem by reducing the variance of the deep features,which contributes to constraining the uncertainty of f and T ( x ; f ) and improving the estimationaccuracy of the reject [-region.4.3.
-]region.
2 The bias introduced by BATS
BATS rectifies the features into the typical set, which reduces the variance of the deep features.
However, this operation can also introduce a bias term, which can reflect the change in the distributionof the features.
A large bias can damage the performance of the model.
The distribution of the outputfeature z = ReLU ( BN ( z ′ ; µ, σ, λ )) is a one-side rectified normal distribution over [0 , + ∞ ) .
Theexpectation of z is:
[Equation]
For our method, the distribution of the output feature ¯ z = ReLU ( TrBN ( z ; µ, σ, λ )) is a two-sidedrectified normal distribution over [0 , µ + λσ ] and the expectation of ¯ z is:
[Equation]
Then the bias caused by the truncation is:
[Equation]
[Equation] where ϕ ( · ) and Φ( · ) are the probability density function (pdf) and cumulative distribution function(cdf) of the standard normal distribution.
One can find that the bias term E (¯ z ) − E ( z ) converges tozero as λ → ∞ .
In other words, if λ is large enough, the bias can be very small.
Thus, there exists abias-variance trade-off.
See Appendix D for the proof.
A proper selection of λ can improve the detection performance by significantly reducing the uncertainty (variance reduction) and slightly changing the distribution of the features (small bias).
If λis large, T ( x ; ¯ f ) uses more extreme features in both the ID and OOD data.
As λ tends to infinity,
T ( x ; ¯ f ) converges to T ( x ; f ) .
Then BATS is the same to the original energy detection.
If λ is small,extreme features are removed from the test statistic T ( x ; ¯ f ) while introducing a non-negligible bias.
Because of the change in feature distribution, the detection method loses its power to identify OODexamples.
Fig. 2 illustrates the distribution of OOD scores with different λ , which empirically verifiesthis trade-off.
[Figure]
Experiments
[Figure]
In this section, we first introduce our experiment implementation.
Then, we evaluate our methodsboth on the large-scale OOD detection benchmark [33] and the CIFAR benchmarks [6].
Afterthat, the ablation studies compare the influence of applying rectification on different layers andshow the influence of the hyperparameter.
Moreover, our BATS can also slightly improve the testaccuracy of the pre-trained models (in Appendix H).
We consider the out-of-distribution detection asa single-sample hypothesis testing task and only test one sample at a time.
5.1 Implementation
Dataset.
For evaluating the large-scale OOD detection performance, we use ImageNet-1k [33] asthe in-distribution dataset and consider four out-of-distribution datasets, including (subsets of) thefine-grained dataset iNaturalist [34], the scene recognition datasets Places [35] and SUN [36], andthe texture dataset Textures [37] with non-overlapping categories to ImageNet-1k.
As for the evaluation on CIFAR Benchmarks, we use the CIFAR-10 and CIFAR-100 [38] as thein-distribution datasets using the standard split with 50,000 training images and 10,000 test images.
We consider four OOD datasets: SVHN [39], Tiny ImageNet [40], LSUN [41] and Textures [37].
Baselines.
We consider different kinds of competitive OOD detection methods as baselines, including
Maximum Softmax Probability (MSP) [6], ODIN [7], Energy [5], Mahalanobis [11], GradNorm [12]and ReAct [8].
MSP is a simple baseline for OOD detection and ReAct is a state-of-the-art methodthat achieves strong detection performance.
All methods use the pre-trained networks post-hoc.
Metrics.
FPR95: the false positive rate of OOD (negative) examples when the true positive rate ofin-distribution (positive) examples is as high as 95 % .
Lower FPR95 indicates better OOD detectionperformance and vice versa.
AUROC: the area under the receiver operating characteristic curve(ROC).
Higher AUROC indicates better detection performance.
See Appendix E for more details.
5.2 Evaluation on the large-scale OOD detection benchmark
We first evaluate our method on a large-scale OOD detection benchmark proposed by Huang and
Li [33].
[33] revealed that OOD detection methods designed for the CIFAR benchmark might noteffectively be adaptable for the ImageNet benchmark with a large semantic space.
Recent literature[8, 12, 33] proposes to evaluate OOD detection performance on images that have higher resolution andcontain more classes than the CIFAR benchmarks, which is more relevant to real-world applications.
In Tab. 1, we compare our method with the existing methods and show the OOD detection performance for each OOD test dataset and the average over the four datasets.
We consider differentarchitectures, including the widely used ResNet-50 [42], DenseNet-121 [43] and a lightweight model MobileNet-v2 [44].
Compared with the Energy Score [5], the difference in our approach is rectifyingthe features that deviate from the feature’s typical set.
Our method outperforms the Energy Score on ResNet-50 by 31.05 % in FPR95 and 8.46 % in AUROC.
Furthermore, our method reduces FPR95 by5.11 % and improves AUROC by 1.33 % compared to the state-of-the-art method [8] on ResNet-50.
Here the models are pre-trained in a standard manner.
We also show that BATS can boost the OODdetection when using the adversarially pre-trained classifiers in Appendix F.
Simultaneously, we observe that existing methods have different performances on different architectures.
Specifically, the performance of the GradNorm [12] in FPR95 is 7.86 % worse than that of
[Table]
89.26 ReAct[8] 37.
88.05 BATS(Ours) 31.
ReAct [8] on the ResNet-50, but surpasses ReAct on MobileNet-V2 by 4.86 % .
Our method achievesthe best performance on different architectures.
Appendix K shows that our method also outperformsthe existing methods when choosing the natural adversarial examples [46] as OOD examples.
[Figure]
Our experiments mainly use the energy score as the test statistic.
In Fig. 3, we show that BATS isalso compatible with various OOD scores and BATS can boost the performance of various OODscores.
Applying our BATS on GradNorm [12] (a gradient-based OOD score) can even achieve betterperformance than "Energy+BATS" but this method needs to derive the gradients of the model, whichcosts more than "Energy+BATS.
" See Appendix I for detailed performance.
5.3 Evaluation on CIFAR benchmarks
We further evaluate our method on CIFAR benchmarks and use CIFAR-10 and CIFAR-100 [38] asthe in-distribution datasets respectively.
Tab. 2 compares our method with the baseline methodsand shows the OOD detection performance for each OOD test dataset and the average over the fourdatasets.
We evaluate our method on the ResNet-18 (RN18) [42] and WideResNet-28-10 (WRN)[47].
The models are trained for 200 epochs with a batch size of 128.
The starting learning rate is 0.1and decays by a factor of 10 at epochs 100 and 150.
ODIN [7] performs the best in the baselines methods on CIFAR-10 with an FPR95 of 32.99 % on
ResNet-18.
Our method outperforms ODIN by 3.03 % and outperforms the simple baseline method
MSP [6] by 27.14 % in FPR95.
As for using CIFAR-100 as the in-distribution dataset, ReAct [8] [Table] is the best baseline method.
Our approach surpasses the ReAct by 5.39 % in FPR95 on ResNet-18.
Our method achieves the best performance on both CIFAR-10 and CIFAR-100.
Our approach is alsoeffective when using the WideResNet model, outperforming the existing methods.
5.4 Ablation studies
5.4.1 Rectifying the features of the early layers
In our experiments, we rectify the features of the penultimate layer (the layer before the fullyconnected layer), which is convenient and efficient.
However, what will happen if we rectify thefeatures of the early layers with BATS?
The early layers refer to the layers close to the input [48].
In particular, the original ResNet-50 [42] consists of four residual blocks.
Block1 is close to the inputand Block
is close to the output.
In Tab. 3, we show the influence of applying feature rectificationon the output of different blocks.
Applying feature rectification to the early blocks (from Block1 to
Block3) has little effect on the performance of OOD detection, while the last block plays a vital role.
Applying feature rectification on all the blocks performs the best in our experiments, which is 0.99 %higher than the "Block4" in FPR95 and 32.04 % higher than "Without" in FPR95.
Considering that thelatest block has a more significant impact on the OOD detection performance than the other blocks,we just rectify the features of the penultimate layer with BATS for the simplicity of the method.
[Table]
To find out why the last block has a significant influence on the OOD detection while the other blockscontribute little, we visualize the feature embeddings extracted by different blocks in ResNet-50using t-SNE [49] in Fig.
We choose the iNaturalist as the OOD dataset and the ImageNet as the IDdataset.
The features extracted by the early blocks of the ID and OOD examples are similar, which has little benefit in distinguishing the ID and OOD examples.
In contrast, the last block can extractperfectly separable features for the ID and OOD examples.
This may be due to the fact that deepneural networks focus on similar general features (edges, lines, and colors) in the early layers andpay more attention to specific features related to classification in the late layers [48, 50].
The latelayer can contribute more to the OOD detection than the early layer.
See more in Appendix B.
[Figure]
5.4.2 The influence of the hyperparameter
In Sec. 4.3, we theoretically analyze the bias-variance trade-off in our method.
Our proposed BATS can reduce variance, which benefits OOD detection, but can also introduce a bias.
Here, weempirically show the influence of the hyperparameter λ in Fig.
As λ tends to infinity, BATSapproaches to the Energy Score (the horizontal lines).
Very small λ will damage the performance. [Figure]
Conclusion
In this paper, we provide novel insight into the obstacle factor in OOD detection from the perspectiveof typicality and hypothesize that extreme features can be the culprit.
We propose to rectify thefeatures into the typical set and provide a concise and effective post-hoc approach BATS to estimatingthe feature’s typical set.
BATS can be applied to various OOD scores to boost the OOD detectionperformance.
Theoretical analysis and ablations provide a further understanding of our approach.
Experimental results show that our BATS can establish state-of-the-art OOD detection performanceon the ImageNet benchmark, surpassing the previous best method by 5.11 % in FPR and 1.43 % in
AUROC.
We hope that our findings can motivate new research into the internal mechanisms of deepmodels and OOD detection and uncertainty estimation from the perspective of feature typicality.
Limitations and societal impact.
This paper proposes to rectify the feature into its typical set toimprove the detection performance against OOD data and provides a plug-and-play method withthe assistance of BN.
The limitation of our method can be that the BN layers are required in themodel architecture in our approach.
BN layers are widely used in convolutional neural networks toalleviate covariate shifts, but there are also architectures without BN.
A set of training images cancontribute to selecting the feature’s typical set and alleviate this limitation.
We also anticipate someother information in the model is conducive to selecting the feature’s typical set and improving thepost-hoc OOD detection performance.
We leave this as future work.
Regarding potential negativeimpact, truncating the features into the typical set can improve the OOD detection but will introducea bias and inherently cause some information loss which may be important for the model in thereal-world scenario.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] See abstract and Section 1.
(b) Did you describe the limitations of your work?
[Yes] We describe the limitation of ourwork in our Conclusion section.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] We discuss
the potential negative societal impacts of our work in the Conclusion section.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 4.3. We theoretically analyze the bias-variance trade-off and state the full set of assumptions.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix D3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] We introduce
the implementation of our method in Section 5.1 and Appendix E. We plan to open the
source code to reproduce the main experimental results later
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See the Section 5.1 and Appendix E for details.
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix E.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We use the pre-trained model in PyTorch and cite the creators.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Type I error and type II error in OOD detection
In the preliminary section, we provide a summary for the out-of-distribution detection from theperspective of hypothesis testing.
As for the error of an OOD detection method, it can be evaluatedfrom two dimensions.
The mistaken rejection of an actually true null hypothesis H 0 is the type Ierror.
The significance level α is a predetermined scalar that bounds the type I error above:α > =
P ( x ∈ R|H 0 )
=
P ( T ( x ; f ) ≥ γ |H 0 ) =
{+P 
+}
( T ( x ; f ) ≥ γ ) .
By the Neyman–Pearson lemma [51], the threshold γ is determined by solving the equation α =
P
( T ( x ; f ) ≥ γ ) .
For a given significance level, the goal is to minimize the type II error: the failureto reject a null hypothesis that is actually false.
The probability of the type II error is denoted byβ
= P ( x / ∈ R|H 1 ) = P ( T ( x ; f ) <
γ |H 1 ) . In the literature on OOD detection, the type II error is also denoted by “FPR (1 − α ) ", which isshort for “the false positive rate of OOD examples when the true positive rate for ID examples is(1 − α )% . "
In the experiments, we follow the notation FPR (1 − α ) .
In our paper, we mainly show the superiority of our method on different datasets in the metrics of
FPR95 and AUROC.
Here we illustrate the OOD detection performance at different significancelevels (FPR(1- α )) in Fig.
The horizontal axis represents the significance level for FPR ("0.95"means FPR95).
Our method surpasses the existing methods at different significance levels on boththe large scale dataset (ImageNet) and the small scale dataset (CIFAR-10). [Figure]
B The influence of the early layers
In the paper, we show that the early layers of the model can hardly distinguish the feature embeddingsof the in-distribution examples (ImageNet-1k) and out-of-distribution examples (iNaturalist) for thatthe features of these examples extracted by the early layers are mixed up.
Rectifying the featuresof the early layers contributes little to OOD detection.
In this section, we illustrate the t-SNEvisualization for the feature embeddings of in-distribution examples and other out-of-distributionexamples (Places [35], SUN [36], and Textures [37]) from different blocks.
Their t-SNE visualizationresults are similar.
To be specific, the feature embeddings of the early blocks of the different datasetsare similar, while the last block shows differences.
In Tab. 3, we set the λ for Block1 and Block2 as5, the λ for Block3 as 2, and the λ for Block4 as 1, for that restricting the features of the early layersmay have a negative impact on the late layers.
C Energy and density in the classifier
To make our paper self-contained, we provide some details for the energy and density in the classifierwith reference to the previous works [32, 31, 5].
[Figure]
LeCun et al.
[52] show that any probability density p ( x ) for x can be expressed as
[Equation]
11) where E ( x ) represents the energy of x and is modeled by neural network, Z = (cid:82) exp
( − E ( x )) dx isthe normalizing factor which is also known as the partition function.
Similarly, p ( x, y ) can be defined as follows:
[Equation]
where ˜ Z = (cid:82) (cid:80)
[Equation]
Thus we also get p ( y | x ) expressed by E ( x ) and E ( x, y ) :
[Equation]
[Equation]
We denote f as a classification neural network.
Let x be a sample.
Then f ( x )[ k ] represents the k thoutput of the last layer and p ( y | x ) can be defined as:
[Equation]
(14) where n represents total possible classes.
From Eq.
(13) and (14), we define two energy functions asfollows:
[Equation] [Equation] [Equation]
And thus Z can be expressed as:
[Equation]
From Eq.
(15) and Eq.
(11), the marginal density p ( x ) for x can be expressed by the output of theclassifier as:
[Equation]
where Z is independent to x .
D Proofs of section 4.3
In this section, we prove the main results in Section 4.3.
Recall the layer structure in (3):
[Equation]
where z ′ is the feature vector extracted from the penultimate layer of g. We denote
[Equation]
Suppose z ′ is a Gaussian variable.
Then z 1 ∼ N ( µ, σ 2 ) and ¯ z 1 follows a Rectified Gaussiandistribution with lower bound µ − λσ and upper bound µ + λσ.
The cdf of ¯ z 1 is
[Equation]
where Φ( z ′ ; µ, σ 2 ) the cdf of a normal distribution with mean µ and variance σ 2 .
According to [53],
E (¯ [Equation] 1 ) = µ and Var (¯ z 1 ) =
σ [Equation] C ( λ ) , (21) where
[Equation] [Equation]
and erf ( x ) =
(2 / √ π ) (cid:82) x 0 exp( − t 2 ) dt is the Gauss error function.
It is easy to see
[Equation]
and
[Equation]
In addition,
[Equation] [Equation]
Therefore, as λ tends to + ∞ ,
[Equation]
[Equation] We obtain that C ( λ ) → 1 as λ → + ∞ .
Next we deal with the bias term.
To proceed further, we need more notations as follow:
[Equation]
Then we know that z follows a Rectified Gaussian distribution with lower bound 0 and upper bound+ ∞ and ¯ z is a Rectified Gaussian variable with lower bound 0 and upper bound µ + λσ.
Accordingto [53], their expectations are
[Equation]
27) and
[Equation] [Equation]
Therefore the bias term is
[Equation]
In addition,
[Equation]
Then we obtain that
[Equation]
E Experiments details
E.1 Details for metrics
FPR95: the false positive rate of OOD (negative) examples when the true positive rate of indistribution (positive) examples is as high as 95 % .
The t rue p ositive r ate (TPR) can be computed as:
[Equation]
(32) where TP denotes the true positive (correctly identify the in-distribution examples as in-distributionexamples) and FN denotes the False Negative (incorrectly identity the in-distribution examples asout-of-distribution examples).
The f alse p ositive r ate (FPR) can be computed as:
[Equation]
(33) where FP denotes the false positive (incorrectly identify the out-of-distribution examples as indistribution examples) and TN denotes the true negative (correctly identify the out-of-distributionexamples as out-of-distribution examples).
AUROC: the area under the receiver operating characteristic curve (ROC) which is the plot of TPRvs FPR.
If FPR = 0 and TPR = 1, it means that this is a perfect OOD detector, which identify allexamples correctly.
If FPR=1 and TPR=0, this is a terrible detector that can not make any correctprediction.
The closer the area under the ROC curve is to 1, the better the performance of the detector.
E.2 Details for datasets
E.2.1 Large-scale OOD detection
We use the subsets from the following datasets as OOD examples and follow the setting in [8] and[12].
The subsets are curated to be disjoint from the ImageNet-1k labels.
We set λ = 1 . 25 in ourapproach for ResNet-50 and DenseNet-121, and λ = 0 .
4 for MobileNet-V2.
During the evaluation,all images are resized to 3 × 224 × 224 .iNaturalist
[34]: iNaturalist contains 675,170 training and validation images from 5089 naturalfine-grained categories, including 13 major categories such as plants, insects, birds, and mammals.
We randomly sample 10000 images that are disjoint from ImageNet-1k for evaluation.
Places [35]: Places is a scene image dataset, which contains 10 million pictures and more than 400different types of scene environments.
We randomly sample 10000 images that are disjoint from
ImageNet 1k for evaluation.
SUN [36]: The Scene UNderstanding (SUN) contains 397 well-sampled categories to evaluate theperformance of scene recognition algorithms.
We randomly sample 10000 images that are disjointfrom ImageNet-1k for evaluation
Textures [37]: Describable Textures Dataset (DTD) is a texture dataset, including 5640 images,which can be divided into 47 categories according to human perception.
We use the entire Texturesdataset for evaluation.
E.2.2 CIFAR OOD detection
We use the CIFAR-10 and CIFAR-100 [38] as the in-distribution examples respectively.
CIFAR-10consists of 60,000 images in the shape of 3 × 32 × 32 , including 10 categories (aircraft, cars, birds,cats, deer, dogs, frogs, horses, boats, and trucks).
CIFAR-100 contains 100 categories of images, andeach category has 600 images in the shape of 3 × 32 × 32 .
We evaluate our approach on four common
OOD datasets.
We set λ = 3 in our approach for CIFAR-10 and λ = 1 . 5 and for CIFAR-100 on
ResNet-18.
As for WideResNet-28-10, we set λ = 0 .
7 for CIFAR-10 and λ = 1 . 0 for CIFAR-100.
During the evaluation, all images are resized to 3 × 32 × 32 .
SVHN [39]: Street View House Number (SVHN) consists of the house numbers extracted from
Google Street View images.
We use the entire of its test set as OOD examples (26032 images).
Tiny ImageNet [40]: Similar to ImageNet, Tiny ImageNet is an image classification dataset, whichcontains 200 categories, and each category contains 50 test images.
We randomly crop the images to3 × 32 × 32 .
LSUN [41]: LSUN is a scene understanding dataset, which mainly includes scene images ofbedrooms, fixed houses, living rooms, classrooms, etc.
We randomly sample 10000 images asout-of-distribution examples and resize the images to 3 × 32 × 32 .
Textures [37]: Describable Textures Dataset (DTD) is a texture dataset, including 5640 images,which can be divided into 47 categories according to human perception.
We use the entire Texturesdataset for evaluation.
E.3 Hardware
Our experiments are implemented by PyTorch [45] and runs on RTX-2080TI.
F Boosting the OOD detection on the robust classifiers
The models used in our experiments (Tab. 1) are standard pre-trained.
Salman et al.
[54] showthat adversarially robust models with less accuracy often perform better than their standard-trainedcounterparts in transfer learning.
In Fig. 8 we evaluate the OOD detection performance on eightadversarially pre-trained ResNet-50 trained with ℓ 2 perturbation of different strength ϵ [54] on the
ImageNet benchmark.
The horizontal axis represents the perturbation strength in training the model,e.g. "0.05" represents the robust model trained with ℓ 2 perturbation ϵ = 0 .
The strength ofperturbation has a great influence on GradNorm [12], but little influence on other methods.
Our BATSsurpasses all the existing methods on different robust models.
[Figure]
G Related literature to OOD detection
OOD detection has received wide attention because it is critical to ensuring the reliability and safetyof deep neural networks.
The literature related to OOD detection can be broadly grouped into thefollowing themes.
Our paper briefly reviews the literature related to post-hoc detection methods.
Here, we provide a more comprehensive review.
Post-hoc Detection Methods.
Post-hoc methods focus on improving the OOD uncertainty estimationby utilizing the pre-trained classifiers rather than retraining a model, which is beneficial for adopting
OOD detection in real-world scenarios and large-scale settings.
In this paper, we mainly focuson the post-hoc OOD detection methods.
Hendrycks and Gimpel [6] observe that the maximumsoftmax probability of In-Distribution Examples (ID) can be higher than the Out-of-Distribution(OOD) samples and provide a simple baseline for OOD detection.
ODIN [7] introduces a largesufficiently temperature factor and input perturbation to separate the ID and OOD examples.
Liu et al.[5] analyze the limitations of softmax function in OOD detection and propose to use energy scoreas an indicator.
The examples with high energy are considered as OOD examples, and vice versa.
Wang et al.
[13] propose to use joint energy score which take labels into consideration to enhance the
OOD detection.
ReAct [8] hypothesize that the OOD examples can trigger the abnormal activationof the model, and propose to clamp the activation value larger than the threshold value to improvethe detection performance.
Lee et al.
[11] use the mixture of Gaussians to model the distribution offeature representations and propose using the feature-level Mahalanobis distance instead of the modeloutput.
GradNorm [12] shows that the gradients of the categorical cross-entropy loss contains usefulinformation for OOD detection.
Confidence Enhancement Methods.
To enhance the sensitivity to OOD examples, some methodspropose to introduce the adversarial examples into the training process.
Hein et al.
[14] endowlow confidence predictions to the examples far away from the training data through an adversarialtraining optimization technique.
Moreover, Bitterwolf et al.
[15] enforce low confidence in an l2ball around the OOD examples.
Proper data augmentation also contributes to OOD uncertaintyestimation [16, 17, 9].
Some methods take advantage of a set of collected OOD examples to enhancethe uncertainty estimation, which are named outlier exposure methods [18, 10, 19].
The correlationsbetween the collected and real OOD examples can largely affect the performance of outliers exposuremethods [55].
Density-based Methods.
Directly estimating the density of the examples can be a natural approach,this kind of methods explicitly model the distribution of ID examples with probabilistic models anddistinguish the OOD/ID examples through the likelihood [20–23].
However, some works show thatthe probabilistic models may assign higher likelihood to OOD examples than ID examples and fail todistinguish OOD/ID examples [24, 25].
H Test accuracy
In this section, we show that with a proper hyper-parameter λ , our feature rectification methodcan slightly improve the test accuracy of the pre-trained models both on the clean images and thecorrupted images.
Here we set λ = 3 .
As shown in Tab. 4, we evaluate the test accuracy of the normalpre-trained models and the pre-trained models with our feature rectification method on the cleanimages and the corrupted images.
We choose some image corruption methods used in [56], includingsalt-and-pepper noise (SP(0.2)), cropout (Crop(0.8)), JPEG compression (JPEG(50)), Gaussian blur(GB) and Gaussian noise (GN).
Fig. 9 shows some examples that can be classified correctly by ourfeature rectified ResNet-50 but classified wrongly by the original ResNet-50.
[Table]
[Figure]
I BATS on other detection methods
In our paper, we provide a concise and effective approach BATS to improve the performance ofthe existing OOD detection methods.
We mainly show the effectiveness of applying our featurerectification (BATS) on Energy Score [5].
In Tab. 5 we show that out method is compatible withvarious OOD detection test statistics (including output-based methods [6, 5, 7] and gradient-basedmethod [12]) and can bring improvements to different methods.
Applying our BATS on GradNormcan even achieve better performance than "Energy+ BATS " but this method needs to derive thegradients of the model which cost more than "Energy+ BATS ".
OOD detection methods hope to assign higher scores to the in-distribution (ID) examples and lowerscores to the out-of-distribution (OOD) examples.
The advanced detection method (GradNorm[12]) can assign better scores to ID and OOD examples than the simple baseline method (MSP [6]).
However, there still exists an overlap in the distribution of the scores. As shown in Fig. 10, our BATSreduces the variance of the scores and makes the scores of the ID and OOD examples more separable,which can improve the performance of the OOD detection methods. We think combining our methodwith a better OOD score can achieve better performance.
“BATS+Energy" has already achievedstate-of-the-art performance on large-scale and small-scale benchmarks.
[Table]
89.26 GradNorm+ReAct 25.
92.24 GradNorm+ BATS 21.
[Figure]
Fig. 11 illustrates the feature distribution on different channels of in-distribution examples (ImageNet) and the out-of-distribution examples on ResNet-50.
These features are extracted by the lastconvolution block.
We name the region where features are concentrated as the typical set of features.
These regions receive more attention during training, and the model is more familiar with the featuresin these regions than those in extreme regions.
For better visual presentation, we illustrate featuresbefore ReLU.
K Detection on natural adversarial examples
In our paper, we follow the settings of the existing research, choosing iNaturalist, Places, SUN and
Textures as out-of-distribution datasets and choosing ImageNet-1k as the in-distribution dataset.
Herewe consider a much more challenging task: detecting natural adversarial examples [46].
Hendryckset al.
[46] introduce natural adversarial examples ImageNet-O, which are naturally occurring examplesin the real world but significantly degrade the deep model performance.
We use the ImageNet-O[46] which contains anomalies of unforeseen classes as the out-of-distribution examples. Fig. I m age N e t i N a t u r a li s t
T e x t u r e s
S UN P l a c e s [Figure] shows that our method surpasses the existing methods by a large margin in FPR(1- α ) with differentsignificance levels.
Our method significantly improve the AUROC from 56.68 % to 64.48 % . [Figure]
L Class activation mapping
In this section, we use the Smooth Grad-CAM++[57] to generate the heat map for different images.
As shown in Fig.
13, the heat map of our rectified model aligns better with the objects in the imagethan that of the original model.
We use the pre-trained ResNet-50 in PyTorch [45].
The heat map ofour rectified model for Fig.
13(b)
(the mud turtle) shows that the head of the turtle dominates the decision while the original model pays more attention to the neck of the turtle.
The rectified modeltakes more object-relevant parts into consideration, which may contribute to its slightly better testaccuracy (in Appendix H). [Figure] M Additional Analysis for Performance Degradation Case in Tab.2
In Tab. 2, we show that the average performance of our BATS surpasses the baseline method Energy,but the performance degrades in the case using Tiny-Imagenet as the OOD dataset. We hypothesizethat this performance degradation is due to the bias introduced by BATS. By truncating the features, BATS can reduce the variance of the in-distribution examples which benefits the estimation of thereject region but inherently cause some information loss which may reduce the performance of thepre-trained models.
To validate our hypothesis, we tune the bias-variance trade-off by the hyperparameter λ .
As shown in
Fig.
14, BATS can indeed reduce the variance of the OOD scores.
With a proper λ , BATS can reducethe overlap between the ID and OOD examples and reduce the FPR95, while a small λ hinders theperformance of OOD detection.
For example, using larger λ = 8 , BATS can achieve better FPR95performance 15.10 % on detecting Tiny-Imagenet using ResNet-18, which is 2.65 % better than λ = 3in our Tab. 2.
For the practicability of our method, we set the same hyperparameter to test different
OOD datasets, without adjusting for specific OOD datasets. [Figure] N The difference between BATS and ReAct The similarity between our BATS and ReAct is that these methods are used to improve the performance of the existing OOD scores.
As follows, we discuss the difference between BATS and ReActfrom three aspects.
First, the motivation between our BATS and ReAct is different.
ReAct hypothesizes that the meanactivation of OOD data has significantly larger variations across units and is biased towards havingsharp positive values, while the activation of the ID data is well-behaved with a near-constant meanand standard deviation.
Thus, ReAct thinks that the truncation can rectify the activation of the OODexamples and preserve the activation of in-distribution data.
However, as shown in Fig. 15, themean activation of OOD data does not always have significantly larger variations than the ID data,which means this hypothesis does not always hold.
The distribution of the deep features after batchnormalization is consistent with the Gaussian distribution.
Our BATS hypothesizes that deep modelsmay be hard to model the extreme features but can provide reliable estimations on the typical features. This is because extreme features are exposed to the training process with a low probability. Wepropose to rectify the features into the typical set and calculate the OOD scores with the typicalfeatures. Second, the mathematical analysis between our BATS and ReAct is different.
ReAct theoreticallyanalyze that if the OOD activations are more positively skewed, their operation reduces mean OODactivations more than ID activations.
We analyze the benefit of BATS from the perspective of thebias-variance trade-off. BATS can reduce the variance of the deep features, which contributes toconstraining the uncertainty of the test static T ( x ; f ) and improving the estimation accuracy of thereject region. Our method hopes to estimate the reject region better, and we do not assume whether OOD data is positively skewed.
Third, our method surpasses the ReAct in both the large-scale benchmark (ImageNet) and thesmall-scale benchmark (CIFAR).
[Figure]
O Selecting features’ typical set without assistance of BN
In our paper, we mainly analyze that rectifying the features in the typical set can improve theperformance of the existing OOD scores.
We provide a concise and effective method to select thetypical set with the assistance of the BN layers and achieve a state-of-the-art performance amongpost-hoc methods on a suite of OOD detection benchmarks.
Here, we provide another method to select the features’ typical set, which directly uses a set oftraining images to estimate the mean µ and the standard deviation σ of the features (extracted bythe penultimate layer of the model) at each dimension. Then we rectify the features into the interval[ µ − λ ∗ σ, µ + λ ∗ σ ] and use these typical features to calculate the OOD scores. We named thismethod as T ypical F eature E stimated M ethod (TFEM). This method does not require the BN layersin the model but needs to use a set of training images.
In Tab. 6, we compare the OOD detection performance of the OOD detection methods with andwithout our TFEM.
In this experiment, we randomly choose 1500 images from the training datasetof the ImageNet.
The λ is set to 1.
The experiment is performed on the ImageNet benchmark.
The models are pre-trained ResNet-50 and ViT. ViT (Vision Transformer) [58] is a transformer-basedimage classification model which treats images as sequences of patches and does not have BN layers.
We use the officially released ViT-B/16 model, which is pre-trained on ImageNet-21K and fine-tunedon ImageNet-1K. Rectifying the features into the typical set with TFEM can greatly improve theperformance of the existing OOD detection methods both on the model with BN layers (ResNet-50)and the model without BN layers (ViT).
This experiment demonstrates the effectiveness of the typical features in OOD detection, which isconsistent with the analysis in our paper.
We believe there exists a method that can estimate thefeatures’ typical set better.
In this paper, BATS has already established state-of-the-art performanceon both the large-scale and small-scale OOD detection benchmarks.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
MSP
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
52 ODIN 41.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
41 Energy 46.
[Table]
[Table]
[Table]
[Table]
[Table]
85.82 Energy
[Table]
[Table]
[Table]
[Table]
[Table]
.55 GradNorm 23.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
P Comparison between BATS and two latest detection methods
In this section, we compare our BATS with two latest OOD detection methods KNN [59] and ViM[60].
KNN is a nearest-neighbor-based OOD detection method, which computes the k-th nearestneighbor (KNN) distance between the embedding of test input and the embeddings of the training setto determine if the input is OOD or not.
ViM combines the class-agnostic score from feature spaceand the In-Distribution class-dependent logits to calculate the OOD score.
As shown in Tab. 7, our
BATS outperforms the existing methods by a large margin.
KNN explores and demonstrates theefficacy of the non-parametric nearest-neighbor distance for OOD detection, but its performance isworse than GradNorm and ReAct.
ViM performs well on the OOD dataset Textures, but when using
SUN as the OOD dataset, its performance is even worse than the simple baseline MSP
[Table]
[Table]
Q Benefits of BATS on calibration
The outputs of a classifier are often interpreted as the predictive confidence that this class wasidentified.
Deep neural networks are often not calibrated which means that the confidence always does not align with the misclassification rate.
Expected Calibration Error (ECE) is a metric to measurethe calibration of a classifier.
For a perfectly calibrated classifier, the ECE value will be zero.
We use the reliability diagram to find out how well the classifier is calibrated in Fig.
The model’spredictions are divided into bins based on the confidence value of the target class, here, we choose 20bins.
The confidence histogram shows how many test examples are in each bin.
Two vertical linesrepresent the accuracy and average confidence, and the closer these two lines are, the better the modelcalibration is.
BATS can improve the calibration of the pre-trained model and reduce the ECE of thepre-trained model from 3.56 % to 2.12 % .
[Figure]