Boosting Out-of-distribution Detection with Typical Features
Anonymous Author(s)
AffiliationAddress email
Abstract
Ensuring the reliability and safety of deep neural networks in real-world scenariosis a crucial issue, which raises the attention on out-of-distribution (OOD) detection.
Existing OOD detection methods primarily work for designing OOD scores orintroducing diverse outlier examples to retrain the model. We delve into theobstacle factors in OOD detection from the perspective of typicality and regardthe feature’s high-probability region of the deep model as the feature’s typical set.
We propose to rectify the feature into its typical set and calculate the OOD scorewith the typical features to achieve reliable uncertainty estimation.
The featurerectification can be conducted as a play-and-plug module with various OOD scores.
We evaluate the superiority of our method on both the commonly used benchmark(CIFAR) and the more challenging high-resolution benchmark with large labelspace (ImageNet).
Notably, our approach outperforms state-of-the-art methods byup to 5.11 % in the average FPR95 on the ImageNet benchmark.
Introduction
Deep neural networks have been widely applied in various fields.
Apart from the success of deepmodels, predictive uncertainty is essential in safety-critical real-world scenarios such as autonomousdriving [1, 2], medical [3], financial [4] etc.
When encountering some examples that the deep modelhas not been exposed to during training, we hope the model raises an alert and hands them overto humans for safe handling.
Such a challenge is usually referred to as out-of-distribution (OOD)detection and has gained significant research attention recently [5–8].
Most of the existing research [5–7, 9–12] worked on designing suitable OOD scores for the pretrained neural network, hoping to assign higher scores to the in-distribution (ID) examples and lowerscores to the out-of-distribution (OOD) examples.
However, these methods overlook the obstaclefactors in OOD detection caused by the model’s internal mechanisms.
In this paper, we rethink the
OOD detection from a perspective of feature typicality.
We observed that the distribution of the deepfeatures of the training dataset on different channels is approximately consistent with the Gaussiandistribution (See examples in Appendix J).
Accordingly, we divide these features into typical features(fall in the high-probability region) and extreme features (fall in the low-probability region).
Extremefeatures rarely appear in training and attract less attention from the classifier than the typical features.
We hypothesize the classifier can model the typical features better than the extreme features, and theextreme features may lead to ambiguity and imprecise uncertainty estimation.
Given the potentialnegative impact, properly dealing with these extreme features is a key to improving the performanceof OOD detection.
In this paper, we propose to rectify the features into their typical set and then calculate the OODscore with these typical features.
In this way, the model conservatively utilizes the typical features tomake decisions and alleviates the damage caused by extreme features, which can be beneficial to the
OOD scores derived from the pre-trained classifier [5–7, 12].
Then the problem is how to estimatethe feature’s typical set on different channels since this requires a sufficient number of in-distributionexamples and is time-consuming.
Luckily, the commonly used operation Batch Normalizationcan shed light on a shortcut to selecting the feature’s typical set and we name our approach B atch
Normalization A ssisted T ypical S et Estimation ( BATS ).
The Batch Normalization layer endeavors tonormalize the features of the training dataset to Gaussian distributions, which can be used to estimatethe typical set for ID features.
Typical features are more common in training, while extreme featuresare rare, which leads to difficulties for the model to estimate extreme features well.
We truncate thedeep features with the guidance of the Batch Normalization, rectifying the extreme features to theboundary values of typical sets.
We illustrate the distribution of the OOD scores for ID (ImageNet)and OOD (four different datasets) examples in Fig.1.
Obviously, rectifying the features into thetypical set with our BATS contributes to improving the separability between ID and OOD examples.
[Figure]
Theoretically, we analyze the benefit of BATS and the bias-variance trade-off influenced by thestrength of the hyperparameter.
A proper strength of BATS contributes to improving the estimationaccuracy of the reject region.
Empirically, we perform extensive evaluations and establish superiorperformance on both the large-scale ImageNet benchmark and the commonly used CIFAR benchmarks.
BATS outperforms the previous best method by a large margin, with up to a 5.11 % reductionin the false positive rate (FPR95) and a 1.43 % improvement in AUROC.
Moreover, BATS can alsoslightly improve the test accuracy and robustness of the pre-trained models.
The main contributions of our paper are summarized as follows:
• We provide novel insights into OOD detection from the perspective of typicality and proposeto rectify the features into the typical set.
We design a concise and effective approach to
select the feature’s typical set named B atch Normalization A ssisted T ypical S et Estimation( BATS ).•
We provide theoretical analysis and empirical ablation on the benefit of BATS from the perspective of bias-variance trade-off to improve the understanding of our approach. • Extensive experiments show that BATS establishes a state-of-the-art performance among
post-hoc methods on a suite of OOD detection benchmarks.
Moreover, BATS can boost the
performance of various existing OOD scores with typical features.
Related work
The literature related to OOD detection can be broadly grouped into the following themes: post-hocdetection methods [5–8, 11–13], confidence enhancement methods [9, 10, 14–19], and density-basedmethods [20–25].
Here we briefly review the literature related to post-hoc detection, which is themost relevant to our work.
Post-hoc detection methods focus on improving the OOD uncertaintyestimation by utilizing the pre-trained classifiers rather than retraining a model, which is beneficialfor adopting OOD detection in real-world scenarios and large-scale settings.
Hendrycks and Gimpel
[6] observe that the maximum softmax probability of ID examples can be higher than that of the OOD examples and provide a simple baseline for OOD detection.
ODIN [7] introduces a largesufficiently temperature factor and input perturbation to separate the ID and OOD examples.
Liuet al.
[5] analyze the limitations of softmax function in OOD detection and propose to use energyscore as an indicator.
The examples with high energy are considered OOD examples, and vice versa.
ReAct [8] hypothesizes that the OOD examples can trigger the abnormal activation of the modeland propose to clamp the activation value larger than the threshold value to improve the detectionperformance.
GradNorm [12] shows that the gradients of the categorical cross-entropy loss can be aneffective test statistic for OOD detection.
Preliminaries3.1 Out-of-distribution detection
In this section, we provide a summary of the out-of-distribution detection from the perspectiveof hypothesis testing [26–30].
We consider a classification problem with K classes and denotethe labels as Y = { 1 , 2 , . . .
, K } .
Let X be the input space.
Suppose that the in-distribution data
D in = { ( x i , y i ) } ni =1 is drawn from a joint distribution P X,Y defined over X × Y .
We denote themarginal distribution of P X,Y for the input variable X by P 0 .
Given a test input x ∈ X , the problemof out-of-distribution detection can be formulated as a single-sample hypothesis testing task:
[Equation]
Here the null hypothesis H 0 implies that the test input x is an in-distribution sample.
The goal of
OOD detection here is to design criteria based on D in to determine whether H 0 should be rejected.
OOD detection tasks need to determine a reject region R such that for any test input x ∈ X , thenull hypothesis is rejected if x ∈ R .
Generally, the reject region R is formulated by a test statisticand a threshold.
Let f : X
(cid:55)→ R K be a model pre-trained from D in , which is used to predict theclass label of an input sample.
One can use the model f or a part of f (e.g., feature extractor) toconstruct a test statistic T ( x ; f ) , where x is the test input.
Then the reject region can be written as
R = { x : T ( x ; f ) ≤ γ } , where γ is the threshold.
3.2 OOD detection with energe score
For a classifier f and a data point ( x , y ) , we use f ( x )[ k ] to represent the k th output of the last layer.
With reference to [5, 31, 32], the marginal density p ( x ) of the classifier can be expressed as: p ( x ) = [Equation] − E
[Equation]
, where Z is the normalizing factor and is independent to x .
E ( x )represents the energy of x and is modeled by neural network
as E ( x ) = − log (cid:80)
Kk =1 exp( f ( x )[ k ]) .
See Appendix C for details.
Considering that Z is a constant and is independent to x , Liu et al.
[5]propose an energy score that uses the opposite of the energy E ( x ) as a test statistic to detect OODexamples.
A higher energy score means a higher marginal density p ( x ) .
Methods
In this paper, we delve into the obstacle factor for the post-hoc OOD detection from the perspectiveof typicality, which aims to boost the performance of the existing OOD scores and is orthogonal tothe methods of designing different OOD scores.
Given that the energy score [5] is provable alignedwith the density of inputs and performs well, we mainly use the energy score as the OOD score (See
Appendix I for other OOD scores).
4.1 Motivation
For a classifier trained on the ID data f = f w,b ◦ g where g is a feature extractor mapping input x toits deep feature z .
Let d -dimensional vector z
=
[ z 1 , ..., z d ] ⊤ = g ( x ) denote the deep features of xextracted by g , and z i indicate the i -th element of z .
f w , b ( z ) =
w · z + b is a fully connected layermapping the deep feature z to output logits.
The energy can be expressed as:
[Equation]
The test statistic can be expressed as T ( x ; f ) := − E ( x ) = log (cid:80)
Kk =1 exp(( w · z + b )[ k ]) , whichdepends on the extracted deep features and the mapping operation of the fully connected layer (FC).
Assuming that the distribution of the deep features is consistent with the Gaussian distribution (seeexamples in Appendix J), there are high-probability regions and low-probability regions in deepfeatures.
We name the features that fall in high-probability regions as typical features, and thecorresponding regions are called feature’s typical sets.
In contrast, we regard the features that fallin low-probability regions as extreme features.
Extreme features are rarely exposed to the trainingprocess, which leads to difficulties for the classifier to model these features and unreliable estimationsin the inference process.
Reducing the influence of extreme features on test statistics can be a key toimproving OOD detection performance.
4.2 Batch Normalization Assisted Typical Set Estimation
Instead of designing new OOD scores to detect the abnormality, we provide a novel insight into OODdetection from a perspective of typicality.
We propose to rectify the features into the feature’s typicalset and then use these typical features to calculate the OOD score.
Consider a common-used layerstructure in deep convolutional networks:
[Equation]
where z ′ is the feature vector extracted from the convolutional layer of g. To identify the typical setof z ′ for each channel, we should apply its feature map to a sufficient number of ID examples andfurther calculate the empirical distribution of z ′ over the ID examples.
If the number of features islarge, the inference procedure is time-consuming.
Here we propose a simple and effective post hocapproach that leverages the information stored in f to infer the typical set without estimating thedistribution of z ′ .
Suppose the pre-trained deep neural network uses batch normalization (BN).
Wedenote the BN unit in f as:
[Equation]
where µ , σ are two learnable parameters.
After the pre-training, all the four parameters µ , σ , E ( z ′ ) ,
Std ( z ′ ) are known and stored in the weights of f. 1 The Batch Normalization normalizes features ofthe training dataset to a distribution with a mean of µ and standard deviation of σ , which means thatthe features fall in the interval [ µ − λ ∗ σ, µ + λ ∗ σ ] appear more frequently in training than thefeatures in the complement of this interval.
Thus we use the information in the Batch Normalizationto identify the in-distribution feature’s typical set and rectify the features into the typical set beforecalculating the OOD score.
The uncertainty estimated with the typical features can be more reliable.
In practice, we propose a truncated activation scheme to bound the output features of the BN unit.
First, we introduce the truncated BN unit by:
[Equation]
where λ is a tuning parameter.
We replace the BN unit in the layer structure (Eq.(3)) with the TrBNunit and write the rectified final features as ¯ z and the new classifier as ¯ f .
Then test statistic with theenergy score can be expressed as:
[Equation]
and take the reject region by R = { x : T ( x ; ¯ f ) ≤ γ } .
We name our approach B atch Normalization
A ssisted T ypical S et Estimation (BATS).
In comparison to the standard BN, the outputs of TrBN areconcentrated toward the feature’s typical set of ID data.
This makes an ID example less susceptibleto being mistakenly detected as an OOD example and buffers the negative impact of the extremefeatures.
Fig.1 compares the distribution of the OOD scores from the original energy score and theenergy score with typical features.
4.3 Theoretical analysis
The truncation threshold λ is a key hyperparameter.
Our method reduces the variance of z ′ and alsointroduces a bias term since it changes the distribution of BN ( z ′ ; µ, σ ) .
The variance reduction meansthat our method is robust to the rare ID examples, while the introduced bias can lead to degradationof the model performance.
In this section, we assume z ′ follows a normal distribution and analyzethe bias-variance trade-off in our method.
4.3.1 Understanding the benefits of BATS from the perspective of variance reduction
The variance reduction happens at the BN step.
The variance of BN ( z ′ ; µ, σ ) is σ 2 since thedistribution of the ID features z ′ is rescaled to N ( µ, σ 2 ) .
While the TrBN unit truncates the extremevalues and the variance of TrBN ( z ′ ; µ, σ, λ ) becomes:
[Equation]
[Equation]
where erf ( x ) = (2 / √ π ) (cid:82) x 0 exp( − t 2 ) dt is the Gauss error function.
The value of C ( λ ) representsthe degree of variance reduction.
It is easy to see that C (0) = 0 , dC ( λ ) /dλ > 0 , and C ( λ ) → 1as λ → + ∞ .
Therefore, C ( λ ) is a monotonically increasing function and 0 ≤ C ( λ ) < 1 for0 ≤ λ < + ∞ .
In summary, the smaller λ , the smaller the variance.
See Appendix D for the proof.
OOD detection is a single-sample hypothesis testing problem (in Eq.(1)), and the in-distribution
P 0 is unknown.
So the reject region is determined by the empirical distribution of the test statistic
T ( x ; f ) over the ID data.
The extreme features increase the uncertainty and lead to more unusualvalues of T ( x ; f ) .
This implies that the reject region may be underestimated due to the heavy tailproperty of T ( x ; f ) .
Our BATS aids this problem by reducing the variance of the deep features,which contributes to constraining the uncertainty of f and T ( x ; f ) and improving the estimationaccuracy of the reject {+region.4.3.
+}
The bias introduced by BATS
BATS rectifies the features into the typical set, which reduces the variance of the deep features.
However, this operation can also introduce a bias term, which can reflect the change in the distributionof the features.
A large bias can damage the performance of the model.
The distribution of the outputfeature z = ReLU ( BN ( z ′ ; µ, σ, λ )) is a one-side rectified normal distribution over [0 , + ∞ ) .
Theexpectation of z is:
[Equation]
For our method, the distribution of the output feature ¯ z = ReLU ( TrBN ( z ; µ, σ, λ )) is a two-siderectified normal distribution over [0 , µ + λσ ] and the expectation of ¯ z is:
[Equation]
Then the bias caused by the truncation is:
[Equation]
where ϕ ( · ) and Φ( · ) are the probability density function (pdf) and cumulative distribution function(cdf) of the standard normal distribution.
One can find that the bias term E (¯ z ) − E ( z ) converges tozero as λ → ∞ .
In other words, if λ is large enough, the bias can be very small.
Thus, there exists abias-variance trade-off.
See Appendix D for the proof.
A proper selection of λ can improve the detection performance by significantly reducing the uncertainty (variance reduction) and slightly changing the distribution of the features (small bias).
If λis large, T ( x ; ¯ f ) uses more extreme features in both the ID and OOD data.
As λ tends to infinity,
T ( x ; ¯ f ) converges to T ( x ; f ) .
Then BATS is the same to the original energy detection.
If λ is small,extreme features are removed from the test statistic T ( x ; ¯ f ) while introducing a non-negligible bias.
Because of the change in feature distribution, the detection method loses its power to identify OODexamples.
Fig.2 illustrate the distribution of OOD scores with different λ , which empirically verifiesthis trade-off.
[Figure]
Experiments
In this section, we first introduce our experiment implementation.
Then, we evaluate our methodsboth on the large-scale OOD detection benchmark [33] and the CIFAR benchmarks [6].
Afterthat, the ablation studies compare the influence of applying rectification on different layers andshow the influence of the hyperparameter.
Moreover, our BATS can also slightly improve the testaccuracy of the pre-trained models (in Appendix H).
We consider the out-of-distribution detection asa single-sample hypothesis testing task and only test one sample at a time.
5.1 Implementation
Dataset.
For evaluating the large-scale OOD detection performance, we use ImageNet-1k [33] asthe in-distribution dataset and consider four out-of-distribution datasets, including (subsets of) thefine-grained dataset iNaturalist [34], the scene recognition datasets Places [35] and SUN [36], andthe texture dataset Textures [37] with non-overlapping categories to ImageNet-1k.
As for the evaluation on CIFAR Benchmarks, we use the CIFAR-10 and CIFAR-100 [38] as thein-distribution datasets using the standard split with 50,000 training images and 10,000 test images.
We consider four OOD datasets: SVHN [39], Tiny ImageNet [40], LSUN [41] and Textures [37].
Baselines.
We consider different kinds of competitive OOD detection methods as baselines, including
Maximum Softmax Probability (MSP) [6], ODIN [7], Energy [5], Mahalanobis [11], GradNorm [12]and ReAct [8].
MSP is a simple baseline for OOD detection and ReAct is a state-of-the-art methodthat achieves strong detection performance.
All methods use the pre-trained networks post-hoc.
Metrics.
FPR95: the false positive rate of OOD (negative) examples when the true positive rate ofin-distribution (positive) examples is as high as 95 % .
Lower FPR95 indicates better OOD detectionperformance and verse vice.
AUROC: the area under the receiver operating characteristic curve(ROC).
Higher AUROC indicates better detection performance.
See Appendix E for more details.
5.2 Evaluation on the large-scale OOD detection benchmark
We first evaluate our method on a large-scale OOD detection benchmark proposed by Huang and
Li [33].
[33] revealed that OOD detection methods designed for the CIFAR benchmark might noteffectively be adaptable for the ImageNet benchmark with a large semantic space.
Recent researches[8, 12, 33] propose to evaluate OOD detection performance on images that have higher resolution andcontain more classes than the CIFAR benchmarks, which is more relevant to real-world applications.
In Tab.1, we compare our method with the existing methods and show the OOD detection performancefor each OOD test dataset and the average over the four datasets.
We consider different architectures,including the widely used ResNet-50 [42], DenseNet-121 [43] and a lightweight model MobileNet-v2[44].
Compared with the Energy Score [5], the difference in our approach is rectifying the featuresthat deviate from the feature’s typical set.
Our method outperforms the Energy Score on ResNet-50by 31.05 % in FPR95 and 8.46 % in AUROC.
Furthermore, our method reduces FPR95 by 5.11 % andimproves AUROC by 1.33 % compared to the state-of-the-art method [8] on ResNet-50.
Here themodels are standard pre-trained.
We also show that BATS can boost the OOD detection when usingthe adversarially pre-trained classifiers in Appendix F.
Simultaneously, we observe that existing methods have different performances on different architectures.
In specific, the performance of the GradNorm [12] in FPR95 is 7.86 % worse than that of
[Table]
89.26 ReAct[8] 37.
88.05 BATS(Ours) 31.
ReAct [8] on the ResNet-50, but surpasses ReAct on MobileNet-V2 by 4.86 % .
Our method achievesthe best performance on different architectures.
Appendix K shows that our method also outperformsthe existing methods when choosing the natural adversarial examples [46] as OOD examples.
[Figure]
Our experiments mainly use the energy score as the test statistic.
In Fig.3, we show that BATS isalso compatible with various OOD scores and BATS can boost the performance of various OODscores.
Applying our BATS on GradNorm [12] (a gradient-based OOD score) can even achieve betterperformance than "Energy+BATS" but this method needs to derive the gradients of the model, whichcost more than "Energy+BATS.
" See Appendix I for detailed performance.
5.3 Evaluation on CIFAR benchmarks
We further evaluate our method on CIFAR benchmarks and use CIFAR-10 and CIFAR-100 [38]as the in-distribution datasets respectively.
Tab.2 compares our method with the baseline methodsand shows the OOD detection performance for each OOD test dataset and the average over the fourdatasets.
We evaluate our method on the ResNet-18 (RN18) [42] and WideResNet-28-10 (WRN)[47].
The models are trained for 200 epochs with a batch size of 128.
The start learning rate is 0.1and decays by a factor of 10 at epochs 100 and 150.
ODIN [7] performs the best in the baselines methods on CIFAR-10 with an FPR95 of 32.99 % on
ResNet-18.
Our method outperforms ODIN by 3.03 % and outperforms the simple baseline method
[Table]
MSP [6] by 27.14 % in FPR95.
As for using CIFAR-100 as the in-distribution dataset, ReAct [8]is the best baseline method.
Our approach surpasses the ReAct by 5.39 % in FPR95 on ResNet-18.
Our method achieves the best performance on both CIFAR-10 and CIFAR-100.
Our approach is alsoeffective when using the WideResNet model, outperforming the existing methods.
5.4 Ablation studies
5.4.1 Rectifying the features of the early layers
In our experiments, we rectify the features of the penultimate layer (the layer before the fullyconnected layer), which is convenient and efficient.
However, what will happen if we rectify thefeatures of the early layers with BATS?
The early layers refer to the layers close to the input [48].
In particular, the original ResNet-50 [42] consists of four residual blocks.
Block1 is close to the inputand Block
is close to the output.
In Tab.3, we show the influence of applying feature rectificationon the output of different blocks.
Applying feature rectification to the early blocks (from Block1 to
Block3) has little effect on the performance of OOD detection, while the last block plays a vital role.
Applying feature rectification on all the blocks performs the best in our experiments, which is 0.99 %higher than the "Block4" in FPR95 and 32.04 % higher than "Without" in FPR95.
Considering that thelatest block has a more significant impact on the OOD detection performance than the other blocks,we just rectify the features of the penultimate layer with BATS for the simplicity of the method.
[Table]
To find out why the last block has a significant influence on the OOD detection while the other blockscontribute little, we visualize the feature embeddings extracted by different blocks in ResNet-50using t-SNE [49] in Fig.4.
We choose the iNaturalist as the OOD dataset and the ImageNet as the ID dataset.
The features extracted by the early blocks of the ID and OOD examples are similar, whichhas little benefit in distinguishing the ID and OOD examples.
In contrast, the last block can extractperfectly separable features for the ID and OOD examples.
This may be due to the fact that deepneural networks focus on similar general features (edges, lines, and colors) in the early layers andpay more attention to specific features related to classification in the late layers [48, 50].
The latelayer can contribute more to the OOD detection than the early layer.
See more in Appendix B.
[Figure]
5.4.2 The influence of the hyperparameter
In Sec.4.3, we theoretically analyze the bias-variance trade-off in our method.
Our BATS can reducethe variance that benefit the OOD detection but also introduce a bias.
Here, we empirically show theinfluence of the hyperparameter λ .
As λ tends to infinity, BATS approaches to the Energy Score (Thehorizontal lines).
Very small λ will damage the performance of the model.
Conclusion
In this paper, we provide novel insight into the obstacle factor in OOD detection from the perspectiveof typicality and hypothesize that extreme features can be the culprit.
We propose to rectify thefeatures into the typical set and provide a concise and effective post-hoc approach BATS to estimatingthe feature’s typical set.
BATS can be applied to various OOD scores to boost the OOD detectionperformance.
Theoretical analysis and ablations provide a further understanding of our approach.
Experimental results show that our BATS can establish state-of-the-art OOD detection performanceon the ImageNet benchmark, surpassing the previous best method by 5.11 % in FPR and 1.43 % in
AUROC.
We hope that our findings can motivate new research into the internal mechanisms of deepmodels and OOD detection and uncertainty estimation from the perspective of feature typicality.
Limitation and societal impact.
This paper proposes to rectify the feature into its typical set toimprove the detection performance against OOD data and provides a plug-and-play method with theassistance of Batch Normalization.
The limitation of our method can be that the Batch-Norm layersare required in the model architecture in our approach.
BN layers are widely used in convolutionalneural networks to alleviate covariate shifts, but there are also architectures without BN.
A set oftraining images can contribute to selecting the feature’s typical set and alleviate this limitation.
Wealso anticipate some other information in the model is conducive to selecting the feature’s typical setand improving the post-hoc OOD detection performance.
We leave this as future work.
We anticipateno negative consequences of our work.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] See abstract and Section 1.
(b) Did you describe the limitations of your work?
[Yes] We describe the limitation of our work in our Conclusion section.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] We
anticipate no negative consequences of our work.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 4.3.We theoretically analyze the bias-variance trade-off and state the full set of assumptions.(b)
Did you include complete proofs of all theoretical results?
[Yes] See Appendix D
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] We introduce
the implementation of our method in Section 5.1 and Appendix E. We plan to open the
source code to reproduce the main experimental results later.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See the Section 5.1 and Appendix E for details.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix E.34.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We use the pre-trainedmodel in PyTorch and cite the creators.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Type I error and type II error in OOD detection
In the preliminary section, we provide a summary for the out-of-distribution detection from theperspective of hypothesis testing.
As for the error of an OOD detection method, it can be evaluatedfrom two dimensions.
The mistaken rejection of an actually true null hypothesis H 0 is the type Ierror.
The significance level α is a predetermined scalar that bounds the type I error above:
[Equation]
[Equation]
) = [Equation] 0 ( T ( x
[-P 
-][Equation]
[Equation]
By the Neyman–Pearson lemma [51], the threshold γ is determined by solving the equation α =
P
( T ( x ; f ) ≥ γ ) .
For a given significance level, the goal is to minimize the type II error: the failureto reject a null hypothesis that is actually false.
The probability of the type II error is denoted by
[Equation]
In the literature on OOD detection, the type II error is also denoted by “FPR (1 − α ) ", which isshort for “the false positive rate of OOD examples when the true positive rate for ID examples is(1 − α )% . "
In the experiments, we follow the notation FPR (1 − α ) .
In our paper, we mainly show the superiority of our method on different datasets in the metrics of
FPR95 and AUROC.
Here we illustrate the OOD detection performance at different significancelevels (FPR(1- α )) in Fig.6.
The horizontal axis represents the significance level for FPR ("0.95"means FPR95).
Our method surpasses the existing methods at different significance levels on boththe large scale dataset (ImageNet) and the small scale dataset (CIFAR-10).
B The influence of the early layers
In the paper, we show that the early layers of the model can hardly distinguish the feature embeddingsof the in-distribution examples (ImageNet-1k) and out-of-distribution examples (iNaturalist) for thatthe features of these examples extracted by the early layers are mixed up.
Rectifying the featuresof the early layers contributes little to OOD detection.
In this section, we illustrate the t-SNEvisualization for the feature embeddings of in-distribution examples and other out-of-distributionexamples (Places [35], SUN [36], and Textures [37]) from different blocks.
Their t-SNE visualizationresults are similar.
To be specific, the feature embeddings of the early blocks of the different datasetsare similar, while the last block shows differences.
In Tab.3, we set the λ for Block1 and Block2 as 5,the λ for Block3 as 2, and the λ for Block4 as 1, for that restricting the features of the early layersmay have a negative impact on the late layers.
C Energy and density in the classifier
To make our paper self-contained, we provide some details for the energy and density in the classifierwith reference to the previous works [32, 31, 5].
[Figure]
LeCun et al.
[52] show that any probability density p ( x ) for x can be expressed as
[Equation]
11) where E ( x ) represents the energy of x and is modeled by neural network, Z = (cid:82) exp
( − E ( x )) dx isthe normalizing factor which is also known as the partition function.
Similarly, p ( x, y ) can be defined as follows:
[Equation]
where ˜ Z = (cid:82) (cid:80)
[Equation]
Thus we also get p ( y | x ) expressed by E ( x ) and E ( x, y ) :
[Equation]
[Equation]
We denote f as a classification neural network.
Let x be a sample.
Then f ( x )[ k ] represents the k thoutput of the last layer and p ( y | x ) can be defined as:
[Equation]
(14) where n represents total possible classes.
From Eq.
(13) and (14), we define two energy functions asfollows:
[Equation] [Equation]
And thus Z can be expressed as:
[Equation]
From Eq.
(15) and Eq.
(11), the marginal density p ( x ) for x can be expressed by the output of theclassifier as:
[Equation]
where Z is independent to x .
D Proofs of section 4.3
In this section, we prove the main results in Section 4.3.
Recall the layer structure in (3):
[Equation]
where z ′ is the feature vector extracted from the penultimate layer of g. We denote
[Equation]
Suppose z ′ is a Gaussian variable.
Then z 1 ∼ N ( µ, σ 2 ) and ¯ z 1 follows a Rectified Gaussiandistribution with lower bound µ − λσ and upper bound µ + λσ.
The cdf of ¯ z 1 is
[Equation]
where Φ( z ′ ; µ, σ 2 ) the cdf of a normal distribution with mean µ and variance σ 2 .
According to [53],
E (¯ z 1 ) = µ and Var (¯ z 1 ) =
σ 2 C ( λ ) , (21) where
[Equation] [Equation]
and erf ( x ) =
(2 / √ π ) (cid:82) x 0 exp( − t 2 ) dt is the Gauss error function.
It is easy to see
[Equation]
and
[Equation]
In addition,
[Equation] [Equation]
Therefore, as λ tends to + ∞ ,
[Equation]
We obtain that C ( λ ) → 1 as λ → + ∞ .
Next we deal with the bias term.
To proceed further, we need more notations as follow:
[Equation]
Then we know that z follows a Rectified Gaussian distribution with lower bound 0 and upper bound+ ∞ and ¯ z is a Rectified Gaussian variable with lower bound 0 and upper bound µ + λσ.
Accordingto [53], their expectations are
[Equation]
27) and
[Equation] [Equation]
Therefore the bias term is
[Equation]
In addition,
[Equation]
Then we obtain that
[Equation]
E Experiments details
E.1 Details for metrics
FPR95: the false positive rate of OOD (negative) examples when the true positive rate of indistribution (positive) examples is as high as 95 % .
The t rue p ositive r ate (TPR) can be computed as:
[Equation]
(32) where TP denotes the true positive (correctly identify the in-distribution examples as in-distributionexamples) and FN denotes the False Negative (incorrectly identity the in-distribution examples asout-of-distribution examples).
The f alse p ositive r ate (FPR) can be computed as:
[Equation]
(33) where FP denotes the false positive (incorrectly identify the out-of-distribution examples as indistribution examples) and TN denotes the true negative (correctly identify the out-of-distributionexamples as out-of-distribution examples).
AUROC: the area under the receiver operating characteristic curve (ROC) which is the plot of TPRvs FPR.
If FPR = 0 and TPR = 1, it means that this is a perfect OOD detector, which identify allexamples correctly.
If FPR=1 and TPR=0, this is a terrible detector that can not make any correctprediction.
The closer the area under the ROC curve is to 1, the better the performance of the detector.
E.2 Details for datasets
E.2.1 Large-scale OOD detection
We use the subsets from the following datasets as OOD examples and follow the setting in [8] and[12].
The subsets are curated to be disjoint from the ImageNet-1k labels.
We set λ = 1 . 25 in ourapproach for ResNet-50 and DenseNet-121, and λ = 0 .
4 for MobileNet-V2.
During the evaluation,all images are resized to 3 × 224 × 224 .iNaturalist
[34]: iNaturalist contains 675,170 training and validation images from 5089 naturalfine-grained categories, including 13 major categories such as plants, insects, birds, and mammals.
We randomly sample 10000 images that are disjoint from ImageNet-1k for evaluation.
Places [35]: Places is a scene image dataset, which contains 10 million pictures and more than 400different types of scene environments.
We randomly sample 10000 images that are disjoint from
ImageNet-1k for evaluation.
SUN [36]: The Scene UNderstanding (SUN) contains 397 well-sampled categories to evaluate theperformance of scene recognition algorithms.
We randomly sample 10000 images that are disjointfrom ImageNet-1k for evaluation.
Textures [37]: Describable Textures Dataset (DTD) is a texture dataset, including 5640 images,which can be divided into 47 categories according to human perception.
We use the entire Texturesdataset for evaluation.
E.2.2 CIFAR OOD detection
We use the CIFAR-10 and CIFAR-100 [38] as the in-distribution examples respectively.
CIFAR-10consists of 60,000 images in the shape of 3 × 32 × 32 , including 10 categories (aircraft, cars, birds,cats, deer, dogs, frogs, horses, boats, and trucks).
CIFAR-100 contains 100 categories of images, andeach category has 600 images in the shape of 3 × 32 × 32 .
We evaluate our approach on four common
OOD datasets.
We set λ = 3 in our approach for CIFAR-10 and λ = 1 . 5 and for CIFAR-100 on
ResNet-18.
As for WideResNet-28-10, we set λ = 0 .
7 for CIFAR-10 and λ = 1 . 0 for CIFAR-100.
During the evaluation, all images are resized to 3 × 32 × 32 .
SVHN [39]: Street View House Number (SVHN) consists of the house numbers extracted from
Google Street View images.
We use the entire of its test set as OOD examples (26032 images).
Tiny ImageNet [40]: Similar to ImageNet, Tiny ImageNet is an image classification dataset, whichcontains 200 categories, and each category contains 50 test images.
We randomly crop the images to3 × 32 × 32 .
LSUN [41]: LSUN is a scene understanding dataset, which mainly includes scene images ofbedrooms, fixed houses, living rooms, classrooms, etc.
We randomly sample 10000 images asout-of-distribution examples and resize the images to 3 × 32 × 32 .
Textures [37]: Describable Textures Dataset (DTD) is a texture dataset, including 5640 images,which can be divided into 47 categories according to human perception.
We use the entire Texturesdataset for evaluation.
E.3 Hardware
Our experiments are implemented by PyTorch [45] and runs on RTX-2080TI.
F Boosting the OOD detection on the robust classifiers
The models used in our experiments (Tab.1) are standard pre-trained.
Salman et al.
[54] showthat adversarially robust models with less accuracy often perform better than their standard-trainedcounterparts in transfer learning.
In Fig.8 we evaluate the OOD detection performance on eightadversarially pre-trained ResNet-50 trained with ℓ 2 perturbation of different strength ϵ [54] on the
ImageNet benchmark.
The horizontal axis represents the perturbation strength in training the model,e.g. "0.05" represents the robust model trained with ℓ 2 perturbation ϵ = 0 .
The strength ofperturbation has a great influence on GradNorm [12], but little influence on other methods.
Our BATSsurpasses all the existing methods on different robust models.
[Figure]
G Related literature to OOD detection
OOD detection has received wide attention because it is critical to ensuring the reliability and safetyof deep neural networks.
The literature related to OOD detection can be broadly grouped into the following themes.
Our paper briefly reviews the literature related to post-hoc detection methods.
Here, we provide a more comprehensive review.
Post-hoc Detection Methods.
Post-hoc methods focus on improving the OOD uncertainty estimationby utilizing the pre-trained classifiers rather than retraining a model, which is beneficial for adopting
OOD detection in real-world scenarios and large-scale settings.
In this paper, we mainly focuson the post-hoc OOD detection methods.
Hendrycks and Gimpel [6] observe that the maximumsoftmax probability of In-Distribution Examples (ID) can be higher than the Out-of-Distribution(OOD) samples and provide a simple baseline for OOD detection.
ODIN [7] introduces a largesufficiently temperature factor and input perturbation to separate the ID and OOD examples.
Liu et al.[5] analyze the limitations of softmax function in OOD detection and propose to use energy scoreas an indicator.
The examples with high energy are considered as OOD examples, and vice versa.
Wang et al.
[13] propose to use joint energy score which take labels into consideration to enhance the
OOD detection.
ReAct [8] hypothesize that the OOD examples can trigger the abnormal activationof the model, and propose to clamp the activation value larger than the threshold value to improvethe detection performance.
Lee et al.
[11] use the mixture of Gaussians to model the distribution offeature representations and propose using the feature-level Mahalanobis distance instead of the modeloutput.
GradNorm [12] shows that the gradients of the categorical cross-entropy loss contains usefulinformation for OOD detection.
Confidence Enhancement Methods.
To enhance the sensitivity to OOD examples, some methodspropose to introduce the adversarial examples into the training process.
Hein et al.
[14] endowlow confidence predictions to the examples far away from the training data through an adversarialtraining optimization technique.
Moreover, Bitterwolf et al.
[15] enforce low confidence in an l2ball around the OOD examples.
Proper data augmentation also contributes to OOD uncertaintyestimation [16, 17, 9].
Some methods take advantage of a set of collected OOD examples to enhancethe uncertainty estimation, which are named outlier exposure methods [18, 10, 19].
The correlationsbetween the collected and real OOD examples can largely affect the performance of outliers exposuremethods [55].
Density-based Methods.
Directly estimating the density of the examples can be a natural approach,this kind of methods explicitly model the distribution of ID examples with probabilistic models anddistinguish the OOD/ID examples through the likelihood [20–23].
However, some works show thatthe probabilistic models may assign higher likelihood to OOD examples than ID examples and fail todistinguish OOD/ID examples [24, 25].
H Test accuracy
In this section, we show that with a proper hyper-parameter λ , our feature rectification methodcan slightly improve the test accuracy of the pre-trained models both on the clean images and thecorrupted images.
Here we set λ = 3 .
As shown in Tab.4, we evaluate the test accuracy of the normalpre-trained models and the pre-trained models with our feature rectification method on the cleanimages and the corrupted images.
We choose some image corruption methods used in [56], includingsalt-and-pepper noise (SP(0.2)), cropout (Crop(0.8)), JPEG compression (JPEG(50)), Gaussian blur(GB) and Gaussian noise (GN).
Fig.9 shows some examples that can be classified correctly by ourfeature rectified ResNet-50 but classified wrongly by the original ResNet-50.
[Table]
[Figure]
I BATS on other detection methods
In our paper, we mainly show the effectiveness of applying our feature rectification (BATS) on
Energy Score [5].
In Tab.5 we show that out method is compatible with various OOD detectiontest statistics (including output-based methods [6, 5, 7] and gradient-based method [12]) and canbring improvements to different methods.
Applying our BATS on GradNorm can even achieve betterperformance than "Energy+ BATS " but this method needs to derive the gradients of the model whichcost more than "Energy+ BATS ".
J The feature distribution on different channels
[Figure]
Fig.10 illustrates the feature distribution on different channels of in-distribution examples (ImageNet)and the out-of-distribution examples on ResNet-50.
These features are extracted by the last convolution block.
We name the region where features are concentrated as the typical set of features.
Theseregions receive more attention during training, and the model is more familiar with the features inthese regions than those in extreme regions.
For better visual presentation, we illustrate featuresbefore ReLU. [Table]
K Detection on natural adversarial examples
In our paper, we follow the settings of the existing research, choosing iNaturalist, Places, SUN and
Textures as out-of-distribution datasets and choosing ImageNet-1k as the in-distribution dataset.
Herewe consider a much more challenging task: detecting natural adversarial examples [46].
Hendryckset al.
[46] introduce natural adversarial examples ImageNet-O, which are naturally occurring examplesin the real world but significantly degrade the deep model performance.
We use the ImageNet-O[46] which contains anomalies of unforeseen classes as the out-of-distribution examples.
Fig.11shows that our method surpass the existing methods by a large margin in FPR(1- α ) with differentsignificance levels.
Our method significantly improve the AUROC from 56.68 % to 64.48 % .
L Class activation mapping
In this section, we use the Smooth Grad-CAM++[57] to generate the heat map for different images.
As shown in Fig.
12, the heat map of our rectified model aligns better with the objects in the imagethan that of the original model.
We use the pre-trained ResNet-50 in PyTorch [45].
The heat map ofour rectified model for Fig.
12(b)
(the mud turtle) shows that the head of the turtle dominates thedecision while the original model pays more attention to the neck of the turtle.
The rectified modeltakes more object-relevant parts into consideration, which may contribute to its slightly better testaccuracy (in Appendix H).
I m age N e t
i N a t u r a l i s t
T e x t u r e s
S UN
[Figure]
[Figure]
[Figure]