How Useful are Gradients for OOD Detection Really?
Authors: Conor Igoe
Youngseog Chung
Ian Char
Jeff Schneider
Venue: Submitted to ICLR 2023 (2023)

Abstract: One critical challenge in deploying machine learning models in real-life applications is out-of-distribution (OOD) detection. Given a predictive model which is accurate on in distribution (ID) data, an OOD detection system can further equip the model with the option to defer prediction when the input is novel and the model has low confidence. Notably, there has been some recent interest in utilizing gradient information in pretrained models for OOD detection. While these methods are competitive, we argue that previous works conflate their performance with the necessity of gradients. In this work, we provide an in-depth analysis of gradient-based methods and elucidate the key components that warrant their OOD detection performance. We further demonstrate that a general, non-gradient-based family of OOD detection methods are just as competitive, casting doubt on the usefulness of gradients for OOD detection

Under review as a conference paper at ICLR 2023
How USEFUL ARE GRADIENTS FOR OOD DETECTION REALLY?
Anonymous authors Paper under double-blind review
ABSTRACT
One critical challenge in deploying machine learning models real-life applica- tions is out of distribution (OOD) detection. Given a predictive model which is accurate on in distribution (ID) data, an OOD detection system can further equip the model with the option defer prediction when the input novel and the model has low confidence. Notably, there has been some recent interest utilizing gradient information in pre-trained models for OOD detection. While these methods are competitive, we argue that previous works conflate their performance with the ne- cessity of gradients. In this work, we provide an in-depth analysis and comparison of gradient based methods and elucidate the key components that warrant their OOD detection performance. We further demonstrate that general, non non-gradie gradient- based family of OOD detection methods are just as competitive, casting doubt on the usefulness of gradients for OOD detection.
INTRODUCTION
Recent advances in algorithms, models, and training infrastructure have brought about unprecedented performance machine learning (ML) methods, across wide range of data types and tasks. Despite their demonstrated potential on benchmark settings and domains, one obstacle which limits ML methods' applicability real-world applications is the uncertainty or confidence of the predictions. Without any deliberate mechanisms, ML models will output a prediction for any given input, and the question of whether this prediction can be trusted will be especially critical many high-rist decision-making settings (e.g. self-driving cars (Agarwal et al. (021), physical sciences (Char et al., 2021 Boyer al. 202Th, and healthcare (Zhou et | (020 This risk is further exacerbated with deep learning where the interpretability of models often limited (Rudin) 2019)).
is unrealistic for one to expect to train a model that has perfect predictions for all possible inputs, partly because real-world datasets are limited in their scope. Thus in lieu of trying to make predictions for all test inputs, one can attempt to first detect whether the input is covered by the support of the training data. This is the motivation behind OOD detection. Among the diverse approaches to OOD detection for image recognition, a recent line of work has suggested utilizing the information in gradients derive efficient and performant methods for OOD detection (Liang al. 2017
Lee &
AlRegib 22200 Agawwa et al. 2020 Lee & AlRegib 2021 Huang et al. 202 S un et al. 2022
Kokilepersaud et 2022).
We motivate our work by first exploring the claim that gradients are useful for OOD detection.
Through a comparison with various extensions of gradient-based scores, we analyze the key compo- nents that actually drive the performance of these methods, and we argue that gradient computations are not essential deriving performant post hoc OOD detectors. Rather, these methods ultimately rely on the magnitude of the learned feature embedding and the predicted output distribution. We thereby refute many of the intuitions that previous works motivate their methods with. Based on our analysis, advocate for the study of more general, non- con--radient--aan framework for producing performant score functions and provide a comprehensive empirical evaluation of various instantiations of the score within this framework.
The rest of this paper is structured as follows. We first provide a formal statement of the problem setting and the related works in Section 2 We then introduce existing and new gradient- baasdd detectors and discuss how both can be simplified into intuitive forms (Section 3). Following this, we perform empirical evaluations of the methods (Section and discuss their implications in Section 5

---

Under review as a conference paper at ICLR 2023
(1 p(
max, D (M)
Σx- 12
pl
Figure 1: Output Score Components The probability output components (V components from are shown a 3-class probability simplex where each of the 3 vertices signifies probability to a single class. From left to right, these are the output score terms used for EXGRAD, MSP, and GRADNORM. Lighter to darker shades indicate lower to higher values.
Section 2
2
PRELIMINARIES AND RELATED WORK
Problem Setting and Notation We focus on the classification setting where the task is to predict a class label y [C] given an input x E Rd, where C is the total number of possible classes and d is the dimensionality of the input. We assume we have access to training data {(x,, } in order to train the parameters @ of a deep neural network fo TRd -> RRC Here, is a collection of all of the weights from each layer in the network, thus, 0 (Wi I [Z]],, where denotes the index of each layer, and we assume the network has L layers in total.
Given an input x, the network predicts a probability vector, p, via the softmax function of the network exp e (x)/T)
Eee-, exp (x)/T) where the superscripts denote the index of the vector and T is the temperature. If not otherwise specified, will assume T T= 1. Although p(x) depends on both 0 and x, we will always exclude the former and often exclude the latter when is clear from context or unimportant. Lastly, we often abuse notation and use Y ~ pxx) to mean Y is sampled from the categorical distribution parameterized by p(x).
outputs: p (*) (x) P(Y k|x)
The OOD Detection Problem In a real-world scenario, the model may be given an input, x E Rd. during deployment that is substantially different from any of the datapoints in the training set. For example, a classifier trained identify numeric digits may be given an image of cat. Since the model's s prediction p(&) cannot be trusted, it would be advantageous to flag such instances and defer prediction. This is the problem that OOD detection addresses.
More formally, the goal in OOD detection is to derive binary classifier which labels whether a given input, X, is ID (in distribution) or OOD. This goal is commonly addressed by learning a score function S _ + E which quantifies the degree to which the input is OOD. Existing works have approached this problem of learning the mapping S from various perspectives, which generally vary by where the signal to generate S is extracted from. Here, we introduce broad groupings of methodologies to provide context for our work, and we refer the reader to Yang et al. (2021): Salehi et al. (2021) for an in-depth survey of the field.
One class of methods focuses on the input space and learns score functions based on characteristics that can be derived from the input features. For example, distance-based methods are based on the intuition that OOD data should lie "far away" from ID data and define S with distances between the input point and reference points that are representative of ID (Lee et al., 2018 Techapanurak et al. 2020: Van Amersfoort et al. 2020). Meanwhile, density- based methods utilize probabilistic models to describe the density ID data and argue that OOD points should occur in areas of low densities. Thus generative models are often used and the score function is often derived with the predicted likelihood of the test input point (Ren et al. 2019 Serrà al.. 2019 Zisselman Tamar 2020).
Another class of methods aims to directly influence predictive model's behavior to OOD data through explicit training. These methods often assume access to OOD examples during training and incorporates them into predictive model's training procedure to maximize the separability between ID and OOD inputs. This is usually achieved by setting aside actual samples from an OOD test distribution (Hendryckss et al. 2018 Yu & Aizawa 2019 Liu et al. 2020), if they not available, by synthesizing OOD examples via adversarial training, perturbations, or sampling from boundaries
or low density regions (Lakshminnrryynan al. 2016 Lee et al. 2017: Vernekar al. 2019)
2

---

Under review as a conference paper at ICLR 2023
Many works have instead turned attention to the information that can be extracted from predictive model that is fully trained on ID data. With the intuition that an ideal predicted distribution should have low uncertainty (heavily concentrated on the predicted class) for an ID point and have high uncertainty (close to a uniform distribution) for an OOD input, some methods focus on generating scores with the predicted distribution of pre-trained models Hendryeks & Gimpel (20T6); Linmans
ict al. (2020); Liu et al. (2020). Mcanwhile, some works have examined the information available when backpropagating gradients of a loss function. Liang et al. (2017); Agarwal et (2020) utilize
information in the gradient w.r.t. the input, while whilelLee& & AlRegib (2020) Huang et al. (2021) examine gradients w.r.t. the model parameters. Huang et al. (2021)) in particular, proposed GRADNORM, an efficient post hoc score function which simply measures the norm of model parameter gradients and thus requires no additional training or hyperparameter tuning. Despite this, GRADNORM achieves State-of-the-AAe (SotA) performance among post hoc methods. We explain GRADNORM in more detail in the next section.
GRADIENT BASED OOD METHODS
initially proposed using model gradients signal detect OOD data. Specifically, they provide a fully trained network with an input point and backpropagate the cross entropy loss between uniform probability distribution and the network's predicted class probabilities. We summarize their intuition for this algorithm in the following hypothesis:
Lee & AlRegib 2020)
Hypothesis 3.1. The Extractioon Hypothesis If learning a point (x, Ú) requires large change in a well-trained network, then X must have been novel, because most of a deep network is dedicated to feature extraction, which by assumption should already perform well on ID data.
Meanwhile, GRADNORM contends the opposite case: they argue that trying to fit the predicted distribution of ID data a uniform distribution label will necessitate higher gradients the model parameters than with OOD data. We note that this intuition relies on the predicted distributions actually displaying high entropy for OOD inputs and low entropy for ID inputs.
Despite conflicting intuitions, both works ultimately propose taking the KL divergence (identically. the cross-entropy loss) w.r.t. the uniform distribution and measuring a norm of the model gradients as a score function. Through extensive empirical ablations, GRADNORM hones in on the L1 norm of the last layer gradients of the Kl.-divergence w.r.t. the uniform distribution as the score function,
SGN (x) @D (u (softmax((f(()))
aWz
We can expand the divergence term and simplify this score into the following expression:
SGN (x) = uuii
Vw,, log p (X)
That is, GRADNORM measures the norm of the expected gradient according to a uniform labeling distribution. Note that Vw, denotes the gradient w.r.t. the parameters in the final layer of fo-
Another Gradient Approach We note that Fq is not unique measure that leverages gradients. To aid our analysis, we also consider the following variant:
SEc = Byx)(x)
V log 02
[1]] p
We will show later that this score also has interpretable properties, and helps us understand the central components of gradient- based score functions. Due to the outer positioning of the expectation, we refer to the score function in Eq. as EXGRAD. Two key differences in this score are 1) the label distribution of comes from the model's own predicted distribution, p, not the uniform distribution, and 2) this calculates the expected norm of the gradient, whereas Eq. 2 calculates the norm of the expected gradient. Other gradient-basee scores are readily plausible, and we visit a suite of scores with empirical comparisons in Section 4.1
3.1 DECOMPOSITION OF GRADIENT METHODS
To give further grounds for their method, Huang et al. (202)) analyze Eq. and show that it can be decomposed into two terms: one that is characterized by the size of the magnitude of the encoding
3

---

Under review as a conference paper at ICLR 2023
fed to the last layer of the network, and one that is characterized by the output of the network. In particular, they derive
C 1 |h|| Σ - (())
1 T
SGN (x)
(4)
where h is the encoding being fed to the last layer of the network. Following their notation, we will denote U to be the part of the score characterized by h and V to be the part characterized by the Eاald p and SaN = 2 UV.. Huang et al. (202T) show that U and V have the same trend where their values are large for iD points and lower for OOD points. While each can be used as OOD detectors alone, the product of the two terms results in stronger performance.
network output, i.e. here U | V 2/2
Although not mentioned in their analysis, the V term in GRADNORM's is score is simply the total variation (TV) distance between a discrete uniform distribution and the model's predicted distribution. This characterization of V gives insight as to why GRADNORM works. When the input image is in distribution, the network will likely have higher confidence, making the TV distance between p and the discrete uniform large.
Decomposition of EXGRAD Doing a similar analysis, our alternative gradient-based detector, EXGRAD, can be broken down in similar way. In particular,
SEG(x) 2 UV T
U [
V Σ P (1 (1-p' (k) k=1
(5)
The full derivation is shown in Appendix out to be an interpretable quantity. Let B3 (ernoulli(p(*) be the random variable corresponding to the event that x belongs to class k. Then, V = ->>> Var(Bz) Intuitively, when the input image single class, the variance of each Bernoulli random
Like with GRADNORM, the term for EXGRAD turns
in distribution and there is high confidence variable will be low. Note that this is the opposite trend of the TV distance and ||b]], A visual comparison of these scores can be seen in Figure !I
This general UV--style score will be of particular interest to us throughout this paper, and we will often refer it as an "Encoding- -uttinttt-tutt" composition as relies on both the networks encoding for the image (at least at the penultimate layer) and its output. We note that recent work on the "Familiarity Hypothesis" in DDitterrch Guyer (2022). as well as work on the role of the feature norm in open-set recognition and OOD detection in Vaze al. (2022) further motivates the study of U and suggests a plausible alternative theory to the gradient-base Treature-Extractinn Hypothesis".
4 EMPIRICAL EVALUATION
The previous section introduced variations to existing gradient-based scores and decomposed the scores into interpretable components. This naturally begs the questions, which scores actually perform well, and what should their favorable performance be attributed to? In this section, explore these questions by comparing the performance multiple gradient-basse based scores against simpler, non-gradient based approaches OOD detection tasks in image classification. Specifically, the goals of these experiments are as follows:
1. investigate whether different gradient-based scores are useful for OOD detection or whether only particular variants perform well (i.e. is it essential that the gradient- based detector is derived by taking the derivative of the KL divergence?);
2. to examine the plausibility of the feature- feature-extraction hypothesis in explaining the strong performance gradient-base based OOD detection methods;
3. to investigate the claim that gradient- based approaches offer unique performance advantages in the context of post hoc OOD detection tasks.
Indeed, we show that SotA OOD detection performance is achievable by leveraging information solely from the predicted class distribution and latent encoding, calling into question the additional utility gained from gradient based score functions. Furthermore, we find that there is significant variability in performance across tasks for different gradient-based score functions. Moreover,
4

---

Under review as a conference paper at ICLR 2023
show that muddeett-based methods often worse than computationally simpler approaches that require no backpropagation. Lastly, we perform experiments that challenge the plausibility of one emerging theory that attempts to explain the role of gradients in SotA OOD methods.
4.1 INVESTIGATING THE PERFORMANCE OF MLADIENT-BASED SCORE VARIANTS
Experimental Set Up A key motivation of our work is in the design of post hoc OOD detection mechanisms. In light of this, for each OOD method, we take a deep neural network pre-trained on particular dataset (the ID dataset) and use data from one of the remaining datasets as OOD data. These models were obtained from popular open-source source library | of pre-trained neural network image classifier models and achieve competitive performance on ID data.
We investigate the performance of seven gradient-based OOD scores, one of which has been previously described in the literature (GRADNORM) and another which we described in detail in Section 3 (ExGRAD). We include additional experimental results on natural variants that can be derived by simple design choices in implementation, specifically by interchanging norms and expectations, choice of norm and choice of distribution to generate the synthetic label at test time. Finally, we compare these gradient- gradient-based methods against two non-gradient based approaches inspired by Huang iet al. (202)) and our analysis in Section 3. We refer to these scores as "V term" scores.
We used MNIST Deng (2012), SVHN Netzer et al. (201T) and CIFAR-10 Krizhevsky (2009) as base datasets for the first experimental setup, and use 10,000 samples from the test split of each dataset form our IDD and OOD datasets. Each OOD method defines score function S(x) which we then use to define an OOD classifier [[x > e]. We then calculate AUROCs for each method, varying the threshold e. Table I describes the mean and standard deviation of the AUROC for each method across the six ID-OOD dataset combinations. Full experimental results showing performance for each ID-OOD combination are available in Appendix [ The columns "Deep" and "Shallow" refer to variants of each method that perform gradients w.r.t. all parameters of the network (Deep) or w.r.t. just the parameters of the final layer (Shallow), i.e. the weights of the layer that generate logits.
Analysis of Results We first observe that, on average over the 6 ID-OOD benchmark tasks, EXGRAD outperforms the previous SotA gradient- post hoc OOD method, GRADNORM. Moreover, no method consistently dominates all other methods across ID-OOD splits (see Appendix (A). This sug- gests that the singular focus in the literature on backpropagating KL divergence losses is unwarranted.
Our next observation is that, in instances where extending gradients to the entire network produces change in performance, such changes are at best modest gains, and can in fact hurt performance. This aligns with results found in Huang al. (2021). Given that the last layer of a deep neural network is merely taking linear combinations of learned features, this observation serves as further evidence against the Peature-Extraction Hypothesis.
Lastly, we note that the highest performing score functions are the non- con--grdientt-ae based approaches. In particular, the two score functions inspired from the decomposition analyses improve beyond their best gradient-based variants by 2.9 and 0.7 percentage points respectively.
These observations provide the basis for general framework achieve perffrerro post hoc OOD detectors that do not rely on calculating test-time gradients: combining a norm of learned encodings with a function of the predicted class distribution. In the following section, we provide additional experimental details that leverage this proposed template to further illustrate the ability to achieve high performance without relying on test-time gradients.
4.2 EXPLORING ENCODING-OUTE OUTPUT COMPOSITIONS
The ImageNet Benchmark In addition to the previous experimental set up, in this section we also evaluate on the large-scale ImageNet benchmark proposed by Huang & Li (202T). For this benchmark, the ImageNet-II Ik dataset (Deng et al., 2009 is used for the ID dataset. This ID dataset is different from MNIST, CIFAR 10, and SVHN in that it is composed of higher resolution images and has C = 1,000 classes instead of C = 10. The iNaturalist (Van Horn et al. 2017). SUN (Xio et al.) 2010), Places (Zhou et al. 2017), and Textures (Cimpoi et al. 2014) datasets used OOD datasets. This benchmark uses the 50,000 images in the ImageNet validation set as the ID data and
5

---

Under review as a conference paper at ICLR 2023
Score Expression
AUROC
Gradient Depth
[By-p
0
0.725 (# 0.086)
Deep
[By-p_
Vo log p 2
0.741 (+: 0.081)
Shallow
Ey-thileen
Va log p
0.825 (#: 0.120)
Deep
Volog
0.850 (++ 0,135)
Shallow
Ey-Ihittn
0.867 (d: 0.148)
Deep
Bu-tntten
Vologg
0.887 (+: 0,107)
Shallow
Vo
[logp[]]]]
(GRADDNRRM)
0.892 (+: 0.087)
Deep
V V
[ogp(>)]
(GRADNORM)
0.906 (#: 0.092)
Shallow
Ry-p
Ieep"" ply
VV Pog
0.910 (+ 0.090)
Shallow
Ey-p
V
0.919 (#: 0.041)
Shallow
Ey-p
|[ log 111
0.921 (+: 0.034)
Deep
Ey-p
logp
log
0.921 0.106)
Deep
Byap.
Woglogp011
(ExGRAD)
0.925 (#: 0.047)
Shallow
Ey
1og [1]]]
(EXGRAD)
0.926 0.053)
Deep
p
[
(0)
(ExGRAD V term)
0.933 (#: 0.063)
M/A
E
1
pd) (GRADNORM V term)
0.935 (+: 0.064)
N/A
Table I: AUROC for Gradient-Based OOD Detectors on Small-Scale Experiments The leftmost. column shows the definition each score function. The Gradient Depth column signifies whether gradients were taken with respect to all parameters ("Deep") or just the last layer's 's parameters ("Shallow").. For each row, the mean AUROC is reported with standard deviations in parentheses, both calculated across the 6 ID-OOD dataset combinations from {MNIST, CIFAR-10, 10, SVHN). Rows are sorted by mean AUROC. Note that the last two rows are the highest performing score functions and correspond scores that do not involve any gradient calculations.
Datasct
V Only
[b]], V
Gradient-Base
ID
0OD
MSP
Energy
VarSum
TV
MSP
Energy
VarSum
GradNorm
ExGrad
CIFARIO
0.971
0.957
0,972
0.974
0.907
0.928
0.867
0.896
0.963
MNIST
SVHN
0.987
0.989
0.988
0.990
0.978
0.980
0.949
0.971
0.975
MNIST
0.924
0.831
0.925
0.928
0.913
0.860
0.895
0.913
0.921
CIFARIO
SVHN
0.943
0.926
0.944
0.948
0.994
0.957
0.995
0.995
0.919
MNIST
0.813
0.807
0.813
0.811
0.748
0.740
0.637
0.735
0.840
SVHN
CIFARIO
0.955
0.953
0,956
0.955
0.936
0.914
0.844
0.926
0,931
Average
0.932
0.91
0,933
0.935
0.913
0.896
0.865
0.906
0.925
iNaturalist
0.876
0.885
0.884
0.885
0.892
0.917
0.748
0.904
0.769
SUN
0.782
0.852
0.790
0.830
0.818
0,910
0.787
0.890
0.666
ImageNet
Places
0.767
0.813
0.771
0.789
0.795
0,872
0.729
0.849
0.689
Textures
0.744
0.758
0.750
0.761
0.778
0.817
0.729
0.811
0.651
Average
0.792
0.827
0.799
0.816
0.821
0.879
0.748
0.864
0.694
Table 2: AUROC scores The table shows AUROC scores for several detectors grouped by category.
The leftmost group uses only the output of the network (V only), the second is the product of these measures with the 1-norm of the encoding passed to the last layer of the network ||h|| V. and the last grouping is detectors that leverage gradients of the last layer of the network. The highest AUROC found for each OOD task is bolded. We separately report the average of the ImageNet baselines and all of the other baselines. Note that we do show ||b]], TV in this table since it exactly GRADNORM.
10,000 images for each of the OOD datasets (except for Textures which uses 5,640). The pre-trained model is from Google BiT-Sa Kolesnikov et al. 2020) and uses the ResNetv2-101 architecture (He
https://gi htt

---

Under review as a conference paper at ICLR 2023
al. 2016). Our code for these experiments was built top of the code from Huang et al. (2021 and more details about this baseline can be found in their paper.
Experimenting with the Soooding-Output Composition In what follows, test variety of detectors that adhere to the encoding-outppr output composition described Section In particular, each of the methods we test here
product of a U term, which is function of the encoding h, and a V term, which a function of the outputted probability vector p. We test all combinations of UE [1, ] and V [Energy, VarSum, MSP. TV}.
Here, U = denotes not using any encoding of the input and only using the output score (i.e. V term). For the V term, TV is the TV distance between p and discrete uniform distribution (V term from Eq. 4). The scores for Energy (Liu et al. 2020) and MSP (Hendrycks & Gimpel 2016) are as follows:
Senergy = T log Σ (x)/T
SMPP
max p
(c)
RE [C
Note that the energy score we use here is the negative version of the one originally introduced in Liu
al. 2020). Like before, assume that T 1 for all experiments.
VarSum is a term inspired by the decomposition of EXGRAD (i.e the V term from Eq. 5).
Because
) is anti-correlated with every other score, we make VarSum a correlated version of this. particular, VarSum here are visualized in Figure with the exception of Energy, since logits cannot be deduced from
V == = p (A1(1 (1- 11ppp (*))
Ek (1) p (.) Each of the V terms explored
probabilities alone.
Table displays the OOD detection performance for each of these methods in both the small-scale benchmark setting (with MNIST, CIFARIO, SVHN datasets) and the large-scale benchmark setting with the ImageNet dataset. The small-scale experiments assume the same setting as Section 4.1
Analysis of Results is immediately clear that there is significant difference between the ImageNet baseline and the other baselines, and such, we average the AUROCs separately. Starting with the small-scale setting, we find the best performers to be methods which look at the outputted predicted distribution only. Interestingly, the two best scores appear to be VarSum and the TV distance, which to the best of our knowledge, have not been recommended for OOD detection by previous works.
Literature generally warns against exclusively using the model's predicted distribution for OOD detection (Hein et al., 2019; Kirsch et al., 202T) since even OOD points can produce confident predictions with probabilities that are highly concentrated that this issue is milder in smaller scale models (MNIST, SVHN), and exacerbated for large models which are highly
single class. These results indicate
pereetarmmetee (CIFARIO, 10, ImageNet). This result is generally in line with observations in calibration, which point out the over-confidence of neural network models as they become more over over-paramette wve-pparameterized (Guo et al. 2017 Wang et al.) 2021).
Following this, it seems that information about the penultimate layer encoding is important for high performing encoders in the ImageNet baselines. Besides VarSum, every possible V improves across all ImageNet baselines when multiplied by ||b]|]- It is unclear why VarSum and EXGRAD do not follow this trend and have subpar performance; however, we believe it may be related how the sum of variance landscape changes with dramatic increase in classes.
We believe that at the time of writing this paper GRADNORM is the current state-of-the-art post hoc method for all considered benchmarks. However, importantly, we find that | | x ENERGY a strictly better detector than GRADNORM on the ImageNet benchmark. These results further strengthen our claim that gradients do not necessarily provide unique benefits for OOD detection performance. It is perhaps only the encoding-outpur decomposition that results in strong performance, rather than some unique property of gradients.
4.3 EXPLORING ENCODING CHOICES
While the previous section limited the feature encoding to the L1 norm, in this section, we study the effect of encoding design choice by varying the norm applied. Although Huang et al. (2021) do an

---

Under review as a conference paper at ICLR 2023
VarSum MSP TV Energy
0.820
0.824
0.917
0.913
0.897
0.879
0.768
0.689
0.651
0.634
0.625
0.611
D.803
0.779
0.892
0.894
0.881
0.864
0.739
0.656
0.617
0.600
0.592
0.579
0.794
0.837
0.851
0.843
0.830
0.821
0.785
0.765
0.755
0.750
0.748
0.742
0.380
0.619
0.764
0.774
0.766
0.748
0.548
0.410
0.379
0.378
0.383
0.409
0
0.1
0.3
0.5
0.8
2
6
Inf
Norm
Figure 2: Average AUROC over ImageNet Benchmark Each cell of the heatmap shows the average AUROC for different configuration of probability output score (y-axis) and order of the norm the encoding fed to the last layer (x-axis).
ablation which changes the order of the norm, they do this ablation on the gradients themselves. By focusing on the norm of h, we are able have more control over the score and the resulting detector.
Figure F displays the average AUROC performance in the ImageNet benchmark for 48 different pairs of UV scores, where U E ( [lbllp p E {0, 0.1, 0.1, 0.5, 0.8, 1, 2,3: 3, 4, 5, 6, 2o } and V E (Energy, TV, MSP, VarSum). Heatmaps for every experiment are shown in Appendix C The results indicate that the choice of encoding does have significant effect in score's OOD detection performance. In particular, ||h] 0.3 x Energy achieves an AUROC of 0.917, which is a drastic improvement over any score in Table 2
It is also better than some previously proposed non-post hoc methods such MOS (Huang & Li 2021) which achieves an average AUROC of 0.901.
We note that this result is not exactly fair since the parameter test set. Nevertheless, this is an encouraging result for what could possibly be achieved by method that considers both image encoding and network output. In particular, we believe that devising more sophisticated U terms than the naive ones tested here could be promising direction for improving OOD detection performance within this framework.
done over the benchmark
4.4 CHALLENGING THE FEATURE- EXTRACTION HYPOTHESIS
As described above, one emerging theory that attempts to explain the role of gradients in OOD detection is the Feature-Extractiin Hypothesis. Quoting Lee & AlRegib (020).
Gradient-base optimization involves larger updates when there is a larger gap between predictions and correct labels for given inputs. It implies that the model requires more significant adjustments to its parameters, as it has not learned enough features to represent the inputs relationships between learned features and classes for correct prediction.
We challenge the plausibility of the Ceature-Extraction Hypothesis in explaining the role of gradients with the following observation: the gradient calculated at a singleton test point x can be associated with learning problem that requires feature learning in order to minimise the loss. In particular, we note that previous gradient based scores involve calculating gradients for optimisation problems of the form ming Ey[e(Y p())). Notably, degenerate mappings lie in the solution space for such optimisation problems, which by definition cannot extract features from input data. In order to investigate the plausibility of the Feature Extraction Hypothesis, define score function that by construction excludes degenerate mappings in its solution space. Specifically, we define the score
)!!]
V log (x())
SBC(x) = Ey-p
+
Σ log O
(x!
ID
(6)
where (x!P, Y!D) is an in distribution training point belonging to class i. We refer to the OOD classifier based on this score function as BATCHGRAD, In particuar, we note that the loss function inside the parentheses of equation O defines an optimisation problem that, for sufficiently expressive networks, cannot have degenerate mappings as its solution. This is because the 2nd term inside
NNoe that by "true mapping", mean mapping that is not degenerate, and by "degenerate mapping" we mean mapping that maps the value for all inputs.
8

---

Under review as a conference paper at ICLR 2023
Deep BATCHGRAD 0.787 (#0.224)
Shallow BATCHGRAD
AUROC
0.925 (#0.044)
Table 3: BATCHGRAD Small-Scale Experimental Results This table shows AUROC results, calculated across the 6 ID-OOD dataset combinations from (MNIST, CIFAR-10, SVHN).
the parentheses penalizes choices of 0 that parameterise a degenerate mapping. Note that this is in contrast to other loss functions, such as those used in EXGRAD and GRADNORM, where degenerate mappings are in the solution spaces of their associated optimisation problems. The motivation for the BATCHGRAD loss function is to design experiment where gradients guaranteed to point towards a true (i.c. -on-degenerate) mapping, giving a better chance of observing the Feature Extraction Hypothesis in action, should it be true. In particular, if the Feature Extraction Hypothesis were true, then we would be especially likely to see a reduction in OOD detection performance when restricting gradients the final layer of BATCHGRAD. However, shown in the results in Table 3 we find that the opposite holds: the variant of BATCHGRAD with gradients restricted to just the last layer is more informative for OOD detection than when using the gradients w.r.t. the whole network. This observation is consistent with previous experiments showing the sufficiency, and at times improvement, of restricting gradients to final layer parameters. This experimental result, in combination with analysis demonstrating the significance of the last layer gradients, leads us to conclude that the Feature-Extraction Hypothesis is not an appropriate explanation for the high performance of gradient-based OOD detection methods.
5 DISCUSSION
In this work, we experimentally investigated gradient-based OOD detection methods. While using gradient-based approaches can result in strong performance, we find previous explanations attributing performance to gradients unsatisfactory. Although prior works have focused on the interpretation of taking the gradient with respect to the KL divergence between the outputted distribution and a discrete uniform distribution (Lee & AlRegib 2020 (Huang al. 202T). find that other gradient- based scores that do not have this interpretation also perform well, especially on smaller scale problems.
We also question the idea that the gradient space of the neural network holds key information used for OOD detection. Our experiments provide evidence against the hypothesis that gradient-baat based methods are informed by large changes needed in the network capture unseen, OOD images. Moreover, we show that we can derive better performing detectors that are agnostic to gradients and only use the recoding-output decomposition discussed in [Huang et al. (202T). As such, we believe the strength of GRADNORM comes not from its leverage of gradients, but solely from the fact that it fuses information about network encodings and outputted distributions. Hence, while it is possible that gradients contain useful information for OOD detection, we do not believe that previous methods leverage information from gradients that cannot be derived more easily through other means.
Future Work Our hope is that the insights provided in this work can be used to further improve OOD detection. In particular, we believe that more advanced methods can be developed that fuse together information from the network encoding and scores derived from the model's predicted distribution. For example. we evaluated the choice of input encodings by varying the order of the norm applied on the last hidden layer outputs. Investigating the utility of other hidden layer features or auxiliary encoders and studying what properties of an encoding are helpful in detecting OOD data could provide further insights to devising stronger OOD detectors.
Another interesting direction for investigation is what role task difficulty and model capacity play in how these detectors perform. We found that when the in distribution task was easier, top performing detectors only depended on network outputs; however, it was essential to use both outputs and input encodings for the more complex ImageNet tasks. As mentioned in Section 4.2 existing works in uncertainty quantification and calibration have noted the unreliability of a deep model's predicted class distribution. While calibration is orthogonal to the scope of this work, we believe there could be fundamental ties in determining the boundary between solely relying on the output predicted distribution for OOD detection (i.e. utilizing the V term only), and additionally requiring extracted information from the input space via input encodings (utilizing the U term). We leave investigating this direction for future work.

---

Under review as a conference paper at ICLR 2023
Repraduciblity Statement Our experimental setup follows closely that of main baseline Huang et al. 2021). We have followed their code as provided in the public github htppsttp:::. /tt:://itthu com http://eepparattgggsssss/gg/aaaa granorr/ _ood_ which includes the pre-trained models and data processing. For the "small scale" experiments (any experiments involving MNIST, CIFAR I0, SVHN datasets), we the pre-trained models they are provided in the public github repo com/aaron
and follow the rest of
the protocol as provided in the earlier repo.
10

---

Under review as a conference paper at ICLR 2023
REFERENCES
Agarwal, C., D'souza, D., and Hooker, S. Estimating example difficulty using variance of gradients. arXiv preprint arXiv:2008. 1I600, 2020.
Agarwal, T., Arora, H., and Schneider, J. Learning urban driving policies using deep reinforcement learning. In 202/ IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 607-614. IEEE, 2021.
Boyer, M., Wai, J., Clement, M., Kolemen, E., Char, I., Chung, Y., Neiswanger, W., and Schneider, J. Machine learning for tokamak scenario optimization: combining accelerating physics models and empirical models. Bulletin of the American Physical Society, 2021.
Char, I., Chung, Y., Boyer, M., Kolemen, E., and Schneider, A model-based reinforcement learning approach for beta control. In APS Division of Plasma Physics Meeting Abstracts, volume 2021, pp. PP11-150, 2021.
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3606-3613, 2014.
Deng, J.. Dong, W., Socher. R., Li, L.-J,, K., and Fei-Fei, Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 255-255. leee, 2009.
Deng, L.
The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, (9(6):141-11) 2012.
Dietterich, T. G. and Guyer, A. The familiarity hypothesis: Explaining the behavior of deep open set methods. Pattern Recognition, 132:108931, 2022.
Guo, C., Pleiss, G., Sun, Y., and Weinherger, Q. On calibration of modern neural networks. In International Conference Machine Learning, pp. 1321-1330. PMLR, 2017.
He, K., Zhang, X., Ren, S., and Sun, J. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645. Springer, 2016.
Hein, M., Andriushchenko, M., and Bitterwolf, J. Why relu networks yield high-confidence pre- dictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Hendrycks, D. and Gimpel, K. baseline for detecting misclassified and duito--distribution examples in neural networks. arXiv preprint 1rXiv:1610.0211 2016.
Hendrycks, D., Mazeika, M., and Dietterich, T. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 1446666 2018.
Huang, R. and Li, Y. Mos: Towards scaling out-of-distributtin detection for large semantic space. In Proceedings of the IEEE/CVE Conference Computer Vision and Puttern Recognition, pp. 8710-8718 2021.
Huang, R., Geng, A., and Li, Y. On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34, 2021.
Kirsch, A., Mukhoti, J., Amersfoort, J. Torr, P. H., and Gal, Y. On pitfalls in ood detection: Predictive entropy considered harmful. 2021.
Kokilepersaud, K., Prabhushankar, M., AlRegib, G., Corona, S. T., and Wykoff, C. Gradient-based severity labeling for biomarker classification in oct. In 2022 IEEE International Conference Image Processing (ICIP), pp. 3416-3420. IEEE, 2022.
Kolesnikov, A., Beyer, L., Zhai, X.. Puigcerver, J., Yung, J., Gelly, S.,, and Houlsby, N. Big transfer (bit): General visual representation learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pp. 491-507. Springer, 2020.
11

---

Under review as a conference paper at ICLR 2023
Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, 2009.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint arXiv: 1rX.v:1612.01474, 2016.
Lec, J. and AlRegib, G. Gradients as a measure of uncertainty in neural networks. 2020 IEEE International Conference Image Processing (ICIP), pp. 2416-2420. IEEE, 2020.
Lee, J. and AlRegib, G. Open-set recognition with gradient- brasde representations. In 2021 IEEE International Conference on Image Processing (ICIP), pp. 469-473 IEEE, 2021.
Lee, K., Lee, H., Lee, K., and Shin, J. Training confidencee-cllivaaeed classifiers for detecting out-of-distribution samples. arXiv preprint arXiv:1711.09333) 2017.
Lee, K., Lee, K., Lee, H., and Shin, J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Liang, S., Li, Y., and Srikant, R. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint 1rXiv:/706.022
2017.
Linmans, J., der Laak, J., and Litjens, G. Efficient outoo--distribution detection in digital pathology using multi-head convolutional neural networks. In MIDL, pp. 465-478, 2020.
Liu, W., Wang, X., Owens, J. D., and Li, Y. Energy-based out-of-distribution detection. arXiv preprint 201V:20010003799 2020.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Reading digits in natural images with unsupervised feature learning. 2011.
Ren, J. Liu, P. J., Fertig, E., Snoek, J., Poplin, R., DePristo, M. A., Dillon, J. V., and Lakshmi- narayanan, Likelihood ratios for diittri-iitiittiion detection. arXiv preprint arXiv: 1906. 02845, 2019.
Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206-22 215, 2019.
Salchi, M., Mirzaei, H., Hendrycks, D., Li, Y., Rohban, M. H., and Sabokrou, M. A unified survey on anomaly, novelty, open-set, and out-of-distribution detection: Solutions and future challenges. arXiv preprint arXiv:2 12.1//.4440 1405 2021.
Serrà, J., Álvarez, D., Gómez, V., Slizovskaia, O., Núñez, J. F., and Luque, J. Input complex- ity and out-of-distribution detection with iikelihood-based generative models. arXiv preprint arXiiv:9099 1480, 2019.
Sun. J., Yang, L.. Zhang, J., Liu, F.. Halappanavar, M., Fan, D., and Cao, Y. Gradient- based novelty detection boosted by self-supervised binary classification. Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8370-8377, 2022.
Techapanurak, E., Suganuma, M., and Okatani, T. Hyperparameter-fie free out-of distriibutt detection using cosine similarity. In Proceedings of the Asian Conference on Computer Vision, 2020.
Van Amersfoort, J., Smith, L., Teh, Y. W., and Gal, Y. Uncertainty estimation using a single deep deterministic neural network. In International Conference on Machine Learning, pp. 9690-9700. PMLR, 2020.
Van Horn, G., Mac Aodha, O., Song. Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie. S. The inaturalist species classification and detection dataset-suppleementary material. Reptilia, 32(400):1-3, 2017.
Vaze, S., Han, K., Vedaldi, A., and Zisserman, A. Open-set recognition: good closed-set classifier is all you need. In International Conference Learning Representations, 2022. URL https: //openreview net. 2
Vernekar, S., Gaurav, A., Abdelzad, V. Denouden, T., Salay, R., and Czarnecki, K. Out-of-distribution detection in classifiers via generation. arXiv preprint arXiv: 191.09424, 2019.
12

---

Under review as a conference paper at ICLR 2023
Wang, D.-B., Feng, L.. and Zhang, M.-L. Rethinking calibration of deep neural networks: Do not be afraid of overconfidence. Advances in Neural Information Processing Systems, 34, 2021.
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A. Sun database: Large-scale scene recognition from abbey zoo. In 2010 IEEE computer society conference on computer vision and paltern recognition, pp. 3485-3492. IEEE, 2010.
Yang, J., Zhou, K., Li, Y., and Liu, Z. Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2 10.11334, 2021.
Yu. Q and Aizawa, K. Unsupervised out-of-distribution detection by maximum classifier discrepancy. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), October 2019.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence. 40(6): 1452-1464, 2017.
Zhou, H., Cheng, C., Lipton, Z. C., Chen, G. H., and Weiss, J. C. Mortality risk score for critically ill patients with viral or unspecified pneumonia: Assisting clinicians with covid-19 eemo planning. In International Conference on Artificial Intelligence in Medicine, pp. 336-347. Springer, 2020.
Zisselman, E. and Tamar, A. Deep residual flow for out of distribution detection. In Proceedings of the JEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13994-1455 14003, 2020.
13

---

Under review as a conference paper at ICLR 2023
A
ADDITIONAL EXPERIMENTAL DETAILS
The tables below shows detailed breakdowns of AUROC results for the small-scale experiments. Note that the top row describes the ID dataset, and the two entries beneath describe the OOD datasets.
SVHN
MNIST
CIFAR
Score Expression
Gradient
MNIST
CIFAR
SVHN
CIFAR
SVHN
MNIST
[V log 113
Deep
0.595
0.730
0.839
0.786
0.669
0.733
[By-
[Vo log 11
Shallow
0.651
0.796
0.838
0.775
0.635
0.749
[[ log [1]
Deep
0.692
0.816
0.790
0.717
0.993
0.940
By Uniioo
[ log
[1]]
Shallow
0.606
0.817
0.939
0.857
0.995
0.886
-Unito
[ log
11
Deep
0.591
0.895
0.986
0.957
0.954
0.819
By Uniiom
V log
Shallow
0.709
0.919
0.976
0.903
0.994
0.819
V
Uliifom
log
Deep
0.759
0.937
0.911
0.814
0.989
0.943
VaEy VaEy~ ULaffrm
log
Shallow 0.735
0,926
0.971
0.896
0.995
0,913
By=p
Velog
Shallow
0.749
0,933
0.983
0,928
0.994
0.870
p(n)
By
[|[ log p
Shallow
0.848
0.908
0.964
0.955
0.915
0.925
By
[[ log (
Deep
0.862
0.936
0.956
0.947
0.916
0.910
Togp (t)
By-p
p
Vo log
Deep
0.710
0.926
0.993
0.968
0.984
0.947
By-p
loVologp (1]]
Shallow
0.840
0.931
0.975
0.963
0.919
0.921
Byap
(1Vologp(>))
Deep
0.830
0.947
0.977
0.965
0.921
0.918
DC
p (b) (())
N/A
0.813
0.956
0.988
0.972
0.944
0.926
Σ
(+)
N/A
0.811
0.955
0.990
0.974
0.948
0.928
ID Dataset
SVHN
MNIST
CIFAR
OOD Dataset
MNIST
CIFAR
SVHN
CIFAR
SVHN
MNIST
Deep BATCHGRAD
0.843
0.947
0.974
0.950
0.505
0.502
Shallow BATCHGRAD
0.851
0.922
0.975
0.963
0.920
0.921
B DERIVATION OF EXGRAD DECOMPOSITION
We will rewrite Eq. 3 into more digestible form. Assume that only take the derivative with respect to the last layer of the network, assume that W E Cx are the parameters for the last layer, and let the output of the last layer be (o) Wh, where is the encoding taken in by the last layer. Although we have not considered bias terms in this formulation, note that they can easily be included by adding another dimension with the value For readability, we write p instead of p(h) and p(k) instead of (k) (h).
First, fix a class k and note that
i
0f(A) (k) log
log
e
3 0
(k)
T
k==
elaa
1
T
Σn el 1
(1
kkk)
14

---

Under review as a conference paper at ICLR 2023
For class k # k.
θ log
e IT
0f 0f
k
Σ elo 1(k's
Σx Exeto 1T
p(A))
T
Building on this,
100 /T
1
log
h h 1pp()
-p(2)
- (8),
aw
ΣC k==
et0
T
- (()
(1)
(1 p (2)
p h (2)
p (2)
(1 - p ()
T
The L1 norm of the gradient with respect to W is just the sum of the absolute values of each entry.
(1-p) ( (1)
[bsl
Σ
Σ (1- (8) h,l + p (
W
= 2(1 -p (k)) ||ا|||
Putting all together,
etour IT
S(x) ENND
Vw log
Σ k/ et
2
(1 (1-p(()) [|bl]]]
Rkmp
C
2
|bl Σ
p (8) (1- (A)
T
c=1
C ADDITIONAL HEATMAPS
In this section, we present additional heatmaps (similar Figure 2 which indicate the AUROC for each of the methods evaluated in Section 4.3 separately for each of the ID-OOD dataset settings tested in the experiments section. The title of each plot indicates the ID and OOD dataset in order. i.e. "ID OOD". Each cell of the heatmap shows the AUROC for different configuration of probability output score (y-axis) and order of the norm on the encoding fed the last layer (x-axis).
SVHN x MNIST
0,766
0.591
0.685
0.715
0.734
0.740
0.752
0.756
0.757
0.757
0.757
0.756
0.753
0.567
0.665
0.703
0.727
0.735
0.751
0.754
0.755
0.755
0.755
0.747
MSP
0.760
0.587
0.686
0.720
0.741
0.748
0.761
0.764
0.765
0.765
0.765
0.758
VarSum
0.656
0.1
0.3
0.5
0.8
1
4
5
inf
15

---

Under review as a conference paper at ICLR 2023
SVHN CIFAR10
Energy
0.909
0.706
0.839
0.880
0.905
0.914
0.930
0.936
0.938
0.939
0.939
0.935
VarSum MSP
0.932
0.730
0.873
0.911
0.930
0.936
0.947
0.950
0.951
0.952
0:952
0.950
0.659
0.616
0.714
0.774
0.824
0.844
0.888
0.901
0.906
0.907
0.906
0.871
0.1
0.3
0.8
4
Inf
Norm
MNIST SVHN
0.983
0.983
0.983
0.81
0.496
0.847
0.933
0.965
0.971
0,980
0.981
0.981
0.980
0.980
0.974
VerSum MSP
0.860
0.553
0.893
0.953
0.973
0.978
0.984
0. 0.984
0.984
0.984
D.984
0.979
0.294
0.411
0.777
0.688
0.937
0.949
0.965
0.967
0.966
0.965
0.964
0.955
0.1
0.3
0.8
4
inf
Norm
MNIST x CIFAR10
Energy
0.947
0.696
0.890
0.914
0.924
0.928
0.933
0.933
0.933
0.932
0.932
0.926
0.749
0.497
0.781
0.852
0.886
0.896
0.911
0.913
0.912
0.911
D.909
0.894
VarSum MSP
0.811
0.907
0.722
0.890
0.1
0.3
0.5
0.8
4
5
inf
Nerm
16

---

Under review as a conference paper at ICLR 2023
CIFARIO
Energy
0.951
0.975
0.971
0.965
0.960
0.957
0.948
0.944
0.941
0.940
0.938
0.933
VarSum MSP
0.989
0.994
0.995
0.995
0.995
0.994
0.988
0:984
0.980
0.976
0:973
0.947
0.993
0.993
0.995
0.995
0.995
0.995
0.994
0.992
0.989
0.986
0.983
0.938
0.1
0.3
0.8
4
Inf
Norm
CIFAR10 MNIST
0.914
0.860
0.834
0.809
0.763
0.977
0.967
0.959
0.948
0.928
0.913
0.834
0.758
0.687
0.623
0.568
0.318
VarSum MSP
0.964
0.369
0.960
0.948
0.927
0.913
0.844
0.780
0.722
0.670
D.626
0.434
0.974
0.965
0.954
0.940
0.915
0.895
0.787
0.688
0.586
0.503
0.436
0.197
0.1
0.3
0.8
4
inf
Norm
ImageNet x INatural
0,861
0.816
0.929
0.932
0.926
0.917
0.823
0.728
0.678
0.654
0.643
0.625
0.863
0.280
0.534
0.732
0.753
0.756
0.748
0.534
0.331
0.333
0.375
63
65
Momy
17

---

Under review as a conference paper at ICLR 2023
ImageNet SUN
Energy
0.821
0.785
0.911
0.917
0.915
0.910
0.849
0.778
0.735
0.712
0.700
0.674
MSP
0.776
0.794
0.830
0.829
0.823
0.818
0.792
0.774
0.763
0.757
0.754
0.747
VarSum
0.254
0.547
0.744
0.769
0.783
0.787
0.715
0.563
0.501
0.482
0.477
0.481
0.1
0.3
0.8
4
inf
Norm
ImageNet Places
Energy
0.881
0.872
0.801
0.735
0.699
0.681
0.651
0.740
0.697
0.842
0.853
0.854
0.849
0.777
0.704
0.666
0.647
D.637
0.617
MSP
0.762
0.776
0.806
0.806
0.600
0.795
0.772
0.755
0.745
0.739
D.736
0.729
VerSum
0.313
0.535
0.692
0.713
0.726
0.729
0.650
0.524
0.484
0.474
0.473
0.480
0.1
0.3
0.8
4
5
inf
Norm
ImageNet x Textures
Energy
0.808
0.932
0.944
0.920
0.865
0.817
0.600
0.515
0.491
0.486
0.487
0.493
0.822
0.922
0.942
0.920
0.867
0.811
0.536
0.450
0.434
0.434
0.438
0.453
VarSum MSP
0.865
0.668
0.294
0.1
0.3
0.5
0.8
4
inf
Nerm
18


Reviews:
[{"review_id": "cLSKiN4gWuo", "reviewer": "Reviewer_ACRA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper evaluates different methods for OOD detection which use and do not use gradient information. They decompose several methods as in [1] into two components related to the encoding and output probability vectors. \nThrough their decompositions they propose different OOD detection methods not based on gradients which outperform benchmarks such as grad norm [1] in some settings.\n\n[1] Huang et al. 2021", "summary_of_the_review": "\nThe paper is very interesting and beautifully written. However, some statements and results could benefit from additional clarity. I would be glad to see this paper accepted if the authors can address all the reviewer’s questions (above) as it has an interesting take on ongoing OOD detection discussions in the literature.  ", "strength_and_weaknesses": "\n- The paper is written clearly and the contributions are clearly stated.\n- Some conclusions are not fully supported/could use additional explanations\nQuestions/comments\n- The results have high variance (cf. Fig 2), in this case the paper would greatly benefit from studying the performance of the proposed OOD methods on additional datasets. Conversely, are the authors proposed that using ||h||_{0.3} x energy should be the OOD score to use across different tasks/models?\n- Eq 6 requires a more through explanation. Why does it imply learning a true mapping? \n- Can the authors elaborate on their explanation why varsum and exgrad have subpar performance? \n- The authors don't include AUPR results which should be included. While AUC-ROC shows us the tradeoff between true positive and false positive predictions, and the AUC-PR shows us the tradeoff between precision (low false positive) and recall (low false negative) predictions.\n- Do the authors believe that the feature extraction / familiarity hypothesis [1] holds for the last layer? \n\n[1] Dietterich and Guyer 2022", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "M-W2QRd1TTw", "reviewer": "Reviewer_ipBL", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The submission primarily aims at refuting ideas expressed in the NeurIPS21 paper “On the importance of gradients for detecting distributional shifts in the wild” (Huang et al.), by suggesting that gradients are actually not intrinsically required for the high performance in that paper; it’s more to do with the nature of the contributing terms once the GradNorm score is viewed with the decomposition in Huang et al. While this analysis was already made in Huang et al., the submission proposes variants of the individual terms, showcasing improvements over GradNorm with alternative choices.", "summary_of_the_review": "The submissions takes a thorough look at GradNorm, following up on the initial analysis provided in Huang et al. about the benefits lying in the separated components of the score. However, in my view, the analysis and main takeaway (that gradient-based scores are not a requisite for high performance; and one can look for other ways to develop OOD-scores using activations and predictive distributions) are less relevant given the plethora of high-performing non-gradient based methods that already take such messages to heart. Therefore, my recommendation at this time is rejection.", "strength_and_weaknesses": "Strengths:\n\nHuang et al. had discussed how the gradient-based method might work, through a product of two terms that are to do with the predictive distribution and the latent encoding separately. The submission performs a fairly detailed analysis of possible alternatives to these terms, not necessarily decomposed from a gradient-based score. The conclusions are sensible — gradient-based methods don’t seem essential to high performance, and alternatives can be formed with different heuristic choices for an OOD score. The paper is very well-written, which made it a pleasure to read.\n\nWeaknesses:\n\nMy main comment is about relevance and significance to ICLR. The submission claims that GradNorm is SOTA at the time of writing, which would justify investing effort in rigorously analyzing the method, exposing limitations, and suggesting alternatives. However, there seem to be at least a couple other papers with equally good and better numbers around, for example, [1, 2, 3], all published reasonably earlier than the ICLR deadline (not all of them report all datasets in the submission, but without an evaluation, one cannot disregard them). These methods are not gradient-based, and seem to be well-aware that gradient-based methods aren’t the best way to go. In general, the narrative in the submission makes it appear as if the community is currently of the widespread opinion that gradient-based methods are top-choice, which needs to be corrected. In my view, past and contemporary literature does not indicate this, thus making the contributions of this submission less significant. There are, to my knowledge, only the two works of note (in the ocean of papers on OOD detection) that use gradients — Huang et al. (2021), and Lee and alRegib (2020). Lee and alRegib has been discussed at length in Huang et al., in particular the point that training a binary classifier on the test-distributions is poor methodology, and this paper is generally not well-cited — I only mention this to counter the narrative in the submission that there is an “emerging theory” in the literature that needs challenging (at a top-tier venue), with substantial portions dedicated to debunking the “feature-extraction hypothesis”. In my opinion, this sort of challenge is best-suited for a “Letter to the Editor” section, as in other scientific journals; unfortunately, our field does not seem to have allocated space for similar back-and-forth between author and challenger. \n\nGiven that no method is universally better, one cannot really draw any strong conclusions. Using a network’s “properties” (activation strengths and predictive distributions) implies that results are strongly dependent on the network and training choices. Using different underlying choices might change trends significantly. This is not a problem unique to this submission, but it somewhat comes in the way of the sort of analysis it seeks to provide: reporting numbers for several different modifications of the score might be reporting results that are “overfitted” on the specific underlying frameworks. The fact that the new alternative, ExGrad (and VarSum variants) underperform on the ImageNet benchmarks also hurts what might have been a positive point for technical novelty.\n\n\n[1] ReAct: OOD Detection with Rectified Activations, NeurIPS 2021\n\n[2] ViM: Out-Of-Distribution with Virtual-logit Matching, CVPR 2022\n\n[3] Out-of-Distribution Detection with Deep Nearest Neighbors, ICML 2022", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "GIpOKzK8_MP", "reviewer": "Reviewer_RRxz", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThis paper investigates the role of gradient-based methods in OOD detection. Specifically, the author tries to answer the following questions and provide answers correspondingly: \n\n1. Is it essential to use the scoring function strictly derived from the gradient?\nNo. The recombination of different components also works fine.\n\n2. How do we align the feature-extraction hypothesis with the strong performance of gradient-based OOD detection methods?\nFeature-Extraction Hypothesis is not an appropriate explanation.\n\n3. Does gradient-based approaches offer unique performance advantages for post hoc OOD detection?\nNo. Essentially the same reason as question 1. ", "summary_of_the_review": "The paper is performing an extensive and in-depth study of the gradient-based method. However, I don't share the excitement upon reading the finding provided in the paper, since I find the observation listed is common and not surprising. Therefore I lean toward rejecting the paper at this point. ", "strength_and_weaknesses": "\nStrength: \n\n1. This is the first paper systematically explore the role of different components for gradient-based methods in OOD detection. \n2. Some new insights are provided like the V component in GradNorm can be regarded as total variation and the V component in ExGrad can be regarded as the variance of a Bernoulli variable.\n3. The related work is extensive. \n\nWeakness: \n\n1. The paper does not offer a finding that is interesting and surprising enough to readers. Several observations are listed in a scattered way, which also makes the paper distractive. For example:\n\n\ta) The author finds that |h|_1 × ENERGY is a strictly better detector than GRADNORM. My feeling is that you can do any recombinations of different OOD scoring functions and find a specific combination (B + D) that performs specifically well. However, such a finding is very likely due to an \"overfitting\" of the test setup. As evidence, |h|_1 × ENERGY does not perform well on small-scale settings as the author showed. \n\n\tb) The author claims that a strict derivation from the gradient form is not needed. That is true, but not surprising. The GradNorm is simply a combination of U and V as pointed out in the GradNorm paper. Both U and V have the discriminative power for ID vs OOD. You can change U or V to other discriminative scoring functions and can still work. A more interesting question is that why U and V synthetically work better than U and V alone. \n\n\tc) The author claims that the Feature-Extraction Hypothesis is not applicable to the explanation of GradNorm. That is also understandable because they are in nature, not conflict. GradNorm is working based on the assumption for input sample x alone: OOD data are closer to a more flat probability space. Feature-Extraction Hypothesis is applied on both input x and label y: if you offer the input sample with a wrong label, the gradient will be large. I don't see why it is essentially counterintuitive. \n\n2. The writing needs to be improved especially in Section 3 and Section 4. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "AMll9Tl9La", "reviewer": "Reviewer_YM1z", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper questions a recent assumption in the OoD detection literature: that gradient information is useful for OoD detection. Specifically, the authors break down the norm of the gradient as 'U * V' (following the GradNorm paper, a SoTA OoD detection method), where U represents the feature norm and V is some function of the output layer.\n\nThe authors conduct both theoretical and empirical analysis under this framework to conclude that the gradient information provides little extra information that could not be extracted in another way. In particular, they find that the feature norm plays an important role in the high performance of the GradNorm method. They also develop a new scoring rule, ExGrad, which empirically outperforms GradNorm on small-scale OoD detection benchmarks, and provide analysis on both large and small-scale OoD detection benchmarks.", "summary_of_the_review": "This is a good paper which provides clear and focussed analysis into a narrow research question. I think it will be useful to the OoD detection community.\n\nUPDATE AFTER AUTHOR RESPONSE:\n\nAfter reading the authors' responses to mine and other reviewers, I re-iterate my initial stance that this is a good paper which would be valuable to the community. I believe that focussed investigation on existing research trends, as carried out in this paper, is just as valuable as the proposals of novel methods. \n\nI maintain my rating of '8, Good Paper'.", "strength_and_weaknesses": "Overall, I enjoyed reading this paper (though not all parts were entirely clear to me). This paper provides a clear and focussed investigation into the question posed in the title, which I believe is useful for researchers and practitioners in the OoD detection community. \n\nStrengths: \n* This paper provides an in-depth analysis of GradNorm and related gradient-based OoD detection methods, which have recently gained traction in the OoD community. The paper analyses GradNorm (a SoTA OoD detection method) and its variants through both an empirical and theoretical analysis to draw conclusions. They find that many choices of the 'V' term could be equally useful for identifying OoD samples, even ones that are not explicitly related to a gradient computation.\n* The authors investigate a 'Feature Extraction Hypothesis', an intuitive assumption that the gradient of the cross-entropy loss (with a uniform label) for an OoD sample would high, as a large change in the features is required to model the OoD sample. They authors conduct empirical and theoretical analysis and find evidence *against* this claim.\n* The authors show a promising direction forward for OoD detection, which focusses on the norm of the extracted features. Though tuning on the test set, the authors show that SoTA OoD detection on the ImageNet-scale evaluation can be achieved simply by tuning the P-norm of the extracted features, and combining with the Energy scoring rule.\n\nWeaknesses:\n* The authors heavily use Energy as scoring rule in the 'V' term, implicitly treating it as independent of the feature norm in U. However, the energy function is itself impacted by the feature norm. This should be discussed.\n* Perhaps I have misunderstood, but the authors highlight the opposite intuitions behind GradNorm and the Feature Extraction Hypothesis. Specifically, one suggests that the gradient norm should be high for ID examples, while the other says it should be high for OoD samples. If this is the case, should one of them not perform very poorly for OoD detection (AUROC <<50, given that they are anti-correlated)? If this is the case, is the Feature Extraction hypothesis not trivially verifiable or falsifiable? Perhaps the authors could clarify.\n\nMisc:\n* [1] discusses the role of the feature norm in open-set recognition and OoD detection. They provide intuitions as to why low-feature norms may occur for OoD samples, and further suggest the maximum logit score (MLS) as an alternative to MSP for open-set scoring (as the former is sensitive to feature norm). The authors should discuss this. \n* The authors have used different parameters with respect to which to take the gradient in Eqs 2 and 3. This makes things less clear, so authors should either change this or discuss the difference. \n\n[1] Open-set Recognition: a Good Closed-Set Classifier is All You Need?, Vaze et al., ICLR 22", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject