Optimal Parameter and Neuron Pruning for Out-of-Distribution Detection
Authors: Chao Chen
Zhihang Fu
Kai Liu
Ze Chen
Mingyuan Tao
Jieping Ye
Venue: NeurIPS 2023 poster (2023)
Keywords: Out-of-Distribution Detection
Parameter Sensitivity
Parameter Pruning
Neuron Pruning
TL;DR: An Optimal Parameter and Neuron Pruning (OPNP) approach is proposed to improve the out-of-distribution performance

Abstract: For a machine learning model deployed in real world scenarios, the ability of detecting out-of-distribution (OOD) samples is indispensable and challenging. Most existing OOD detection methods focused on exploring advanced training skills or training-free tricks to prevent the model from yielding overconfident confidence score for unknown samples. The training-based methods require expensive training cost and rely on OOD samples which are not always available, while most training-free methods can not efficiently utilize the prior information from the training data. In this work, we propose an \textbf{O}ptimal \textbf{P}arameter and \textbf{N}euron \textbf{P}runing (\textbf{OPNP}) approach, which aims to identify and remove those parameters and neurons that lead to over-fitting. The main method is divided into two steps. In the first step, we evaluate the sensitivity of the model parameters and neurons by averaging gradients over all training samples. In the second step, the parameters and neurons with exceptionally large or close to zero sensitivities are removed for prediction. Our proposal is training-free, compatible with other post-hoc methods, and exploring the information from all training data. Extensive experiments are performed on multiple OOD detection tasks and model architectures, showing that our proposed OPNP consistently outperforms the existing methods by a large margin.

Optimal Parameter and Neuron Pruning for Out-of-Distribution Detection
Chao Chen', Zhihang Fu': Kai Liu?, Ze Chen', Mingyuan Tao', Jieping Ye'
Alibaba Cloud 27hejiang University
(ercong.co, cc, zhihang fzh, yejieping/@aalbatte inc. com
Abstract
For a machine learning model deployed in real world scenarios, the ability of de- tecting cut-of-distribution (OOD) samples is indispensable and challenging. Most existing OOD detection methods focused on exploring advanced training skills training-fio feining--rreeeee tricks to prevent the model from yielding overconfident confidence score for unknown samples. The training-bann based methods require expensive training cost and rely on OOD samples which are not always available, while most training- free methods can not efficiently utilize the prior information from the training data. In this work, propose Optimal Parameter and Neuron Pruning (OPNP) approach, which aims to identify and remove those parameters and neurons that lead to over-fitting. The main method is divided into steps. In the first step, we evaluate the sensitivity of the model parameters and neurons by averaging gradients over all training samples. In the second step, the parameters and neurons with exceptionally large or close to zero sensitivities are removed for prediction. Our proposal is training-free compatible with other post-hoe methods, and exploring the information from all training data. Extensive experiments are performed on multiple OOD detection tasks and model architectures, showing that proposed OPNP consistently outperforms the existing methods by a large margin.
Introduction
Over the past decade, deep neural networks have achieved dramatic performance gains in computer vision 77.. natural language processing (38], and AI for Science (25]. However, when deploying those deep learning models in real world scenarios, the model will provide false prediction result for unseen categories, which will lead to serious security issues. promising approach address this problem is Out-of- Distribution (OOD) detection [51], which aims to distinguish whether the given sample is from training class or unknown category. The deep learning models with good OOD detection ability know what they don't know and can be used safely in real world applications.
Recently, large number of works have been proposed address the OOD detection problem 21 26). Early representative methods utilized the Maximum Softmax Probability (MSP) [21] or Mahalanobis distance [26] as score function. The main challenge that modern overparameterizeed deep neural networks can easily produce overconfident predictions on OOD samples, making the in-distribution (ID) data and OOD data inseparable. To alleviate the overconfident problem, some training-
wrining-bbssed based methods and post-hoc methods have been proposed. The training- based methods mitigate the overconfident problem by incorporating OOD samples in training process [22] or synthesizing virtual outliers to regularize the model's decision boundary [9] The post-hoc methods
43. to make the ID and OOD samples more separable. The training- based methods are controllable and interpretable, but they rely on expensive training cost and additional OOD samples, which are not always available.
mainly focused on optimizing score function (2 29 31], or rectifying activations
CCoreepondingg Author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Parameter sensitivity distribution of ResNet50
Parameter sensitivity distribution of VIT-8/16
17.5
avg=0.063
avg=0.052
15.0
Denstyy
12.5
Density
30
10.0
7.5
20
10
2.5
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2 0.4
0.8 1.0
Normalized parameter sensitivity
Normallize parameter sensitivity
(a) ResNet50
(b) ViT-B/16
Figure 1: Illustraiion of parameter sensitivity distribution for (a) ResNet50 and (b) ViT-B/16. The parameters are selected from the last fully-connected layer for both ResNet50 and ViT-B/16. The dotted line in red indicates the average sensitivity and the maximum sensitivity is normalized to 1.
In contrast, the post-hoe methods training-free low-cost and plug and play, however they effectively leverage the prior information from the training data and trained models.
It has been widely observed that overparameterized deep neural networks often suffer from redundant parameters and neurons, which consequently result overconfident predictions. Conversely, precisely pruned sub-network is capable of achieving comparable performance 17 30 46). This motivates a straightforward question: Can we identify the parameters and neurons that lead to overconfident outputs by leveraging the prior information from the off-th the-shelf models and training samples! ? To answer this question, we first demonstrate the parameter sensitivity of the last fully-
connected (FC) layer for ResNet50 [8] and Vision Transformer (ViT-B/16) [7] in Fig. [ As be observed, the distribution of parameter sensitivity is heavily positively skewed. More specifically, a significant proportion of parameter sensitivities are close to zero, while only a few parameters exhibit exceptionally high sensitivities. The gap between the maximum and minimum sensitivity values is more than 200 times.
The above observation naturally inspires a simple yet surprisingly effective method Optimal Parameter and Neuron Pruming (OPNP) for OOD detection. The motivations of our OPNP are in two aspects: (1) The parameters and neurons with sensitivities close to zero are redundant and can lead to overconfident predictions 23 result in a sharp landscape, which hurts the model generalization [ 55 54]. Therefore, in this study, present empirical and theoretical evidences show that the OOD detection performance can be significantly improved by pruning parameters and neurons with exceptionally large or close to zero sensitivities. The main contribution of this paper are follows:
(2) The parameters with exceptionally large sensitivity
A gradient based approach is present to estimate the sensitivity of parameters and neurons in deep model. Building upon this approach, we introduce OPNP A simple yet effective training-frre free method, which significantly improves OOD detection performance by removing weights and neurons with exceptionally large or close to zero sensitivities.
We evaluate OPNP on different OOD detection tasks and model architectures, including ResNet and VIT. Compared to the baseline model, OPNP achieves 32.5% FPR95 reduction on large-scale ImageNet- benchmark, and outperforms existing state-of-the-art post-hoc OOD detection method by 5.5% in FPR95.
Extensive ablation experiments are performed to reveal the insight and effectiveness of the proposed method. We show that OPNP is compatible with other post-hoc methods and can benefit model calibration. We believe our insights can inspire and accelerate future research in related tasks.
2 Related Work
General OOD Detection. OOD detection is highly relevant several early research tasks, including outlier detection (OD), anomaly detection (AD) [40] and open-set recognition (OSR) [14]. All

---

these tasks are aim to identifying the OOD samples in the open world scenario [5T]. The most promising OOD detection methods can be divided into five categories, including density-based
methods [39], distance-based methods [26 45 outlier exposure methods [22 28
2 virtual OOD
synthesis methods 9 47 and post-hoe methods [1 Besides, some recent advances also pay
attention to exploring large-scale pretrained vision- rision-language model for OOD detection [35] 10], and
extending OOD detection from classification to other learning tasks, such as object detection [8] and segmentation 200] In this study, we mainly focus on post-hoc OOD detection, which is training-free freee approach that does not require any OOD samples. For a more comprehensive understanding of the general OOD detection task, we suggest referring to [51] for further details.
Post-hoe OOD Detection. Recently, the post-hoc hoc OOD detection methods have achieved promising B1 29]. To alleviate the over-confident prediction caused by the softmax function, ODIN 29 introduces a temperature scaling strategy to make the softmax scores between ID and OOD images more separable. Liu et al. [31 replace the MSP score with energy score, which is theoretically aligned with the probability density and less susceptible to the overconfident problem. In huang et. al [24], GradNorm is presented which utilizes the magnitude of gradients as OOD score. Another line of work relieve overconfident problem by rectifying typical features in the penultimate layer 43 57]. In particular, ReAct [43 truncates features with a global threshold, which is chosen to preserve the activations for ID data while rectifying that of OOD data. ASH [6] shows that simply removing lower activations or binarizing representations leads to promising OOD detection performance. The most relevant method to our proposal is DICE [. which ranks weights based measure of contribution, and selectively use the most salient weights to derive the output for OOD detection. Both DICE and our proposal alleviate over-fitting by model pruning. DICE selects the most important connections while our proposal removes the most sensitive and insensitive parameters and neurons. Besides, the method used measure the weight importance or sensitivity is different.
performance and drawn increasing attention [44 43 57
Parameter and Neuron Pruning. In deep neural networks, parameter and neuron pruning is widely
Dropout is the most well well-knownn method to improve the robustness of deep models, which randomly drops dropout, Gomez et al. [15] introduce targeted dropout, which drops units and weights with low magnitude. They experimentally demonstrated that target dropout benefit to post-hoc pruning of units and weights. Besides, parameter and neuron pruning have also been widely used in model compression. Han et al. [17] indicate that the weights with low magnitude is less important and propose to remove those weights for model compression. In contrast, Li et al. (27] propose to drop the feature maps based on weight norms, the model performance is well preserved after pruning more than 30% units. In 30 3 the authors demonstrate the effectiveness of ensembling multiple sparse networks, which are trained from scratch with dynamic sparsity constraint, for OOD detection. The aforementioned researches have demonstrated that the weights and neurons are significantly redundant in deep networks, which inspires us to prune the parameters and neurons that result in over-fitting for OOD detection. We believe optimal post-hoc parameter and neuron pruning is a promising approach for OOD detection.
used for reducing over-fitting [23] 22 13 5 and model compression 33 17 27
some neurons [42] or connections [48] in training time. Based
3 Method
3.1 Problem statement
Assume that we have an in-distribution dataset Din of pairs (Iin Vin) and out-of-distribation dataset Dout of pairs (Xout, )out), ). where Tin, Tout E X' denote the input feature vector of ID and OOD samples, yin Yo. == {1, 2, .... K} denotes the ID class label. and your E Vout denotes the output class label, V:., Jout = 0. Given classification model f(;; 0) trained from in-distribution dataset D.n. The goal of post-hoc OOD detection to design binary classifier ga (z) which is able to distinguish whether the test sample is from ID or OOD distribution. Therefore, the challenge of OOD detection is to find an optimal score function S(x) such that for given test sample x E X,
ID, S(x) 2 X OOOD S(x) <
-x(x)
where samples with higher score S(x)) are classified as ID and vice versa, is the threshold which usually set to ensure 95% ID samples correctly classified. Maximum softmax probability [21]
3

---

and energy score [3 are the most widely used score functions in post-hoc OOD detection. We follow previous post-hoe methods 4S 44 to utilize energy score OOD detection metric, which consistently outperforms MSP score.
We denote (t(=:) TR. the feature representation from the penultimate layer, denote W R and b E RR the output weights and bias of the last FC layer. Then, the output logit can be given as
f(x; 0) = (T((2) +
The energy score function 31 maps the output logit a energy value by,
E(e; 0) loo Σ exp(Ji(x))
where fx) denotes the logit output for class i. The energy softmax function, but the overparameterized connection weights W in the last fully-connected layer may still cause overconfident logit output. Therefore, in the following section, aim provide optimal parameter and neuron pruning strategy to reduce over-fitting.
reduces over-fitting caused by
3.2 Parameter sensitivity estimation
In this section, we propose to estimate parameter sensitivity by measuring how sensitive the output energy score to a small change of parameters. For a given sample æ and corresponding energy output Exx;; 0), small change ò,j is added to the parameter Ozy, which results in change in the output energy score,
E(ag;
0 ( E(xx;; 0) = Σ
Pis )oij
(4)
i
aF(ax; 0)
9ig (xx)
(5)
a0.;
Here, Psy (æx)) denotes the gradient of model output to the parameter 0:g at data point 2t Since 8, a small constant, the parameter sensitivity to model output can be measured by the magnitude of the gradient Pij. In this respect, given batch of samples {æx Im the parameter sensitivity can be estimated by accumulating the gradients over all input samples,
1 m M,j
m Σ gi (xx)
where My denotes the sensitivity of parameter O.j. It's worth noting that utilize the change in energy score as the sensitivity measure, which better aligned with the OOD detection metric. Practically, the parameter sensitivity
also be measured by other model output, such the change of logit norm |[ (x; f)]|2. which has been exploited in lifelong learning
In Fig. I we illustrate the sensitivity of parameter W for two representative deep networks ResNer50 18 and ViT-B/16 77, where the maximum sensitivity normalized to 1. can seen that maximum parameter sensitivity of the ResNet50 and ViT-B/16 is nearly 20 times than the average sensitivity. addition, compared with ViT-B/16. there are more parameters with sensitivity close to zero in ResNet50, which indicates that the last FC layer of ResNet50 has more redundant parameters. Based on the intuition that the parameters and neurons with exceptionally large sensitivity or with sensitivity close to zero tend to result in overconfident prediction, optimal parameter and neuron pruning strategy is introduced to improve the OOD detection performance.
3.3 Optimal parameter and neuron pruning
In this section, we mainly introduce how to prune the connection weights and neurons in the last fully-connected layer, which directly result in overconfident results. The pruning strategy other layers can be achieved in the same manner.

---

sensitivity Distribution
h(x)
h(x)
w 0)
w f(;
OPNP
Neuron sensitivity Distribution
Before OPNP
After OPNP
Before OPNP
(a) Optimal parameter and neuron pruning
(b) Sensitivity distribution
Figure 2: (a) Illustration of the last fully- colnnecddee layer before and after OPNP, the connections and neurons in grey color represent the pruned ones. (b) The first row illustrate the parameter sensitivity before and after pruning and the second row illustrate the neuron sensitivity before and after pruning.
Parameter Pruning. Given the connection weights W that maps the feature representation to logit, the corresponding parameter sensitivity M can be computed according Eq. As mentioned above, the parameters with exceptionally large or close to zero sensitivity tend to result in over-fitting. Therefore, simple threshold function can be utilized remove the risky connections,
M.j
min
W
O,
M.j Qma
W
other
where W ij denotes the weights after pruning, Ω and Q mag denote the minimum and maximum thresholds. To align with previous post-hoc methods [43] 44 6]. we obtain the threshold by percentile p. which indicates that the threshold is set to the pth-percentile of the entire sensitivity matrix. For example, Pmaz = 1% represents that 9 is set to the 1% largest sensitivity value in M, and pmin 10% represents that 1 is set to the 10% smallest sensitivity value in M. In Fig. 2(b) the first row illustrates the sensitivity distribution of parameter W in ResNet50 before and after parameter pruning. After pruning, the connection weights that larger than D or smaller than 1 are removed, and will not contribute to the output logit, which reduces the risk of over-fitting.
Neuron Pruning. In additional to parameter pruning, also propose prune the neurons in the pre-logit layer, which has also been shown to mitigate over-fitting [2 15]. For the i-th neuron the pre-logit layer, it contributes to all output neurons, therefore the sensitivity of the i-th neuron should be defined based on the sensitivity of the connection weights between the i-th hidden neuron and all E norm of the weights usually used measure the
output neurons. Considering that the importance of the units in deep networks [41 [5). We define the neuron sensitivity as the average sensitivity of weights that connected with the neuron, which equivalent to l norm of the weight sensitivity, and reflects the average sensitivity to all classes, i.c..
O,
M
(8)
where O, denotes the sensitivity of i-th neuron pre-logit layer, represents number of output neurons. In this respect, similar threshold function as Eq. 7 to remove the risky neurons,
0,
O < Do Pnn
0. O >> Ωo D
(x)
other
where T.' (x) denotes the output feature from the i-th pruned neuron in the pre-logit layer, D and 2o Do represent the minimum and maximum sensitivity thresholds determined by the pruning percentage Pr and P The second row Fig. 2 illustrates the sensitivity distribution of the hidden neurons in ResNet50 before and after neuron pruning, where the maximum sensitivity is
5

---

normalized to 1. As observed, after neuron pruning, the redundant neurons (with sensitivity close to zero) and risky neurons (with sensitivity far above the average) are removed and the distribution of sensitivity across neurons becomes more uniform, which potentially reduces over-fitting.
3.4 Insight Justification
The following remarks are provided to explain why OPNP improves OOD detection performance.
Remark 1. Parameter and neuron pruning avoid overconfident predictions. The over- parameterized deep neural networks tend to generate overconfident predictions even for OOD samples [9 Therefore, most existing methods improve OOD performance by avoiding overconfident predictions 29 31 44]. For deep network, the last fully connected layer can be regarded as linear classifier. The most widely used technique to prevent a classifier from overfitting is to employ a & or f2 regularization, which can be formulated ming ((z.)>>D | h((=) - e + (R))). where R(0) represents f1- or ly-norm of 0. As the parameter pruning is able to reduce R(0), it can be regarded as an effective post regularization technique that reduces the model complexity and avoids overconfident predictions. Besides, the neuron pruning is similar target dropout [15] which has also been demonstrated to reduce overfitting. Therefore, the proposed OPNP avoids overconfident predictions and potentially improves the OOD detection performance.
Remark Pruning the least sensitive parameters and neurons improve separability between ID and OOD samples. We denote fz (z) the logit output of j-th class, after pruning the least sensitive parameters, the logit reduction of the j-th class can be estimated as
(x)
Σ
M
Wial hx(x)
(10)
Mj R
shows that the logit reduction is positively correlated with the average sensitivity of the pruned weights. As the parameter sensitivity is computed over the training II) set, the least sensitive parameters on ID distribution should be more sensitive for OOD samples on average, i.e.,
Σ
MOOD
Σ
M
(11)
MyR
Therefore, the logit reduction OOD samples is larger than on ID samples (Zow)) Af which leads to better separability between ID and OOD samples, and improves OOD detection performance. We also show the parameter sensitivity distribution (on ID and OOD sets) of the pruned weights in Fig. 9 which experimentally verifies Eq. I1
Remark 3. Pruning the most sensitive parameters and neurons improves generalization. We follow 54 to define the first-order flatness as
(O) & p
|Af((')]
40 e θ
(12)
0'1l where Af")") denotes the derivative point 0', ((0, p) (0': 00- < p} denotes the open ball of radius p centered at the point 0 in the Euclidean space and denotes the perturbation radius that controls the magnitude of the neighbourhood. The flatness (g) describes how flat the function landscape is (5). It has been demonstrated that a flatter landscape could lead to better generalization 53 Eq. 12 indicates that the first-order flatness is determined by the largest gradient, therefore, our proposed method pruning the most sensitive parameters is able to improve the flatness of the function landscape and lead better generalization. However, according Remark pruning the most sensitive parameters and neurons may also hurt the separability between ID and OOD samples. Therefore, there a trade-off between better generalization and better ID-OOD separability. This explains why OOD performance improves with very few sensitive parameters pruned and drops with large pruning ratio, as demonstrated in Fig.
[1 15
Experiments
In this section, we describe our experimental setup and implementation details, then evaluate the effectiveness of the proposed OPNP method in different model architectures and OOD detection benchmarks, followed by extensive ablation studies.

---

Table 1: OOD detection results on ImageNet- Ik benchmark with ResNet50 model. OPP, ONP and OPNP represent only using optimal parameter pruning, only using optimal neuron pruning and using both parameter and neuron pruning, respectively. All numbers are percentages.
Average
Method
INatualist
SUN
Places
Texture
FPR951
AUROCT
FPR95
AUROCT
FPR95]
AUROCT
FPR95,
AUROCT
FPR95,
AUROC1
[2
54.05
87.43
73.37
78.03
72.98
78.03
68.85
79.06
67.31
80.64
ODIN 29
47.66
89.66
60.15
84.59
67.89
81.78
50.23
85.62
56.48
85.41
Mahalanobis
26
97.00
52.65
98.50
42.41
98.40
41.79
55.80
85.01
87.43
55.47
Energy 31
55.72
89.95
59.26
85.89
64.92
82. 86
53.72
85.99
58.41
86.17
13
94.20
DICE
25.63
94.49
35.15
90.83
46.49
87.48
31.72
90.30
34,75
90.77
MICE+ReAce
96.24
25.45
93.94
93.40
OPP
23.58
95.41
30.40
93.17
40.76
90.65
41.27
92.10
34.00
92.83
ONP
OPNP
18.89
96.03
18.50
95.62
30.14
93.46
36.17
91.70
25,93
94.20
ReAPtReAct
95.65
30.23
4.1 Experimental Setup
Datasets. Following prior works 43 44
we utilize ImageNet-1K, CIFAR-10 and CIFAR-100 as ID dataset. For ImageNet- IK benchmark, 50000 test samples are used as test ID samples, and four datasets are used as test OOD data, which are from (subset of) iNaturalist, SUN, Place and Texture [4]] For CIFAR benchmark, the standard split with 50,000 training ID images and 10,000 test ID images are utilized for training and evaluation. We follow
4 evaluate six common OOD
datasets including iSUN [0) and Textures
[22 LSUN-Crop 52,, SVHN (6), Places365 [56].
Models. We utilize two representative deep models, ResNer50 [8] and ViT-B/16 77. to evaluate our proposal. For ResNet50, the model weights provided in Torch Vision 34 is utilized. For ViT-B/16, we utilize the model weights provided in Pytorch Image Models (timm) library (4)). For both models, only estimate the parameter and neuron sensitivity in the last fully-connected (FC) layer. The number of neurons in th pre-logit layer are 2048 and 768 for ResNet50 and ViT-B/16 respectively.
Evaluation Metric and Baselines. We utilize FPR95 and AUROC as evaluation metrics, which are the most important metrics in OOD detection 22 31. FPR95 is short for FPR @ TPR95 which represents the false positive rate when the positive rate is 95%. AUROC denotes the area under the receiver operating characteristic curve, which is threshold -eree and reflects the average performance under different thresholds. Note that following [44) do not report the ID classification accuracy since our proposal is training-free and only revises the last FC layer. We can always use the original FC layer for classification, which ensures an identical classification accuracy unpruned model. We compared our OPNP with the most competitive post-hoe OOD detection methods, including MSP 21], ODIN 29 Mahalanobis distance [26]. Energy Score [II, ReAct [43 and DICE (44).
Implementation Details. Implementation of this work based on Pytorch library We use all training images in ImageNet-IK to obtain the parameter sensitivity by using the autograd function achieved by energy www.scorrrbccc. energy_score.bace backwardc At test time, all images are resized to 224 x 224. The hyperparameters Pi Pmnaz Pruin Pinez experimentally determined in the validation set, which includes 50000 test ID samples and 50000 test OOD samples that are selected from images- 21k. We utilize grid search determine the optimal pruning percentage, where we vary Pm (0. 5, 10, 20, ... ,60), Plaa = {0, 0.1, 0.3. 0.5, 1,3,5, 5). Pmiin = (0, 5. 10, 20, ... ,50), Praz = (0. 0.5, 1,5, 10, 20, 50). The same hyperparameters adopted in the same model.
4.2 Main Results.
Evaluation on Image benchmark. In Table T we compare our proposal with other competi- tive post-hoe OOD detection methods based on ResNet50. 1 denotes larger values are better and denotes smaller values are hetter. The results for those comparison baselines are directly taken from
7

---

Table 2: OOD detection results on ImageNet- Ik benchmark with ViT-B/16 model. All numbers are percentages.
OOD Datasets
Average
Melhod
iNatualist
SUN
Places
Texture
FPR954
AUROCT
FPR954
AUROCT
FPR95]
AUROCT
FPR95
AUROCT
FPR95,
AUROCT
M2
21.28
91.63
51.96
85.08
50.63
84,92
50.57
43.61
87.28
Energy 31
7.61
98.23
40.30
90.77
46.89
88.62
33.54
93.21
32.09
92.71
ReAc
2.28
99,42
30.68
93.94
35.32
91.40
37.08
92.84
26.34
94.40
DI
32.43
93.30
37.46
93.91
DICE+ReAc 44
2.65
99.38
29.45
93.52
38.45
91.17
33.78
93.27
26.08
94,34
OPP
3.12
99.18
25.
93,99
34.00
91.43
35.56
92.21
24.49
94,20
ONP
3.87
99.13
29.63
93.08
35.68
90.73
35.60
91.54
26.20
93.62
OPNP
3.16
99.38
24.32
93.86
34,52
38.76
25.19
94.16
OPP+ReAct
2.52
99.35
23.96
94.50
32.80
92.10
36.03
91.77
23,83
94.43
Table 3: Changes in ID classification accuracy by varying pruning percentage in ResNet50 model.
Percentage p" = 0 Pmaz = 0.1 Pmax = 0.3
Pmax
Pmax
Piin 5 Pimx 10
Panin
Pmin
ID Acc
76.13
75.14
74.72
71.60
$6.37
76.06
75.88
75.86
75.06
which utilizes the same experimental setup as ours. OPP, ONP and OPNP represent only utilize optimal parameter pruning, only utilize optimal neuron pruning and utilize both parameter and neuron pruning, respectively. The results in Table reveal several interesting observations: (1) Optimal parameter pruning (OPP) achieves similar performance DICE which also removes
45
unimportant connections for OOD detection, and significantly outperforms MSP (11. Mahalanobis
distance (26) and Energy (31) baselines. (2) Optimal neuron pruning (OPN) achieves much better
performance than OPP, outperforms DICE [4] by large margin and outperforms SOTA feature rectification method (ReAct) 443 by 1% in FPR95 and 0.7% in AUROC. (3) Compared OPP and ONP, combining parameter pruning and neuron pruning (OPNP) also reduces FPR95 by 3.3% and improves AUROC by 0.6% based on ONP. (4) OPNP outperforms both SOTA weights pruning method (DICE) and SOTA feature rectification method (ReAct) by large margin (more than 5% in FPR95). (5) Our proposed OPNP does not outperform ReAct and DICE in Texture dataset, which can be compensated by combining ReAct.
compare our proposal with competitive post-hoc OOD detection methods based ViT-B/16 model. We note that the performance of MSP (1 and Energy Score (31 in previous work g is worse than our implementation based on the same model, therefore, we reported the performance reproduced by ourselves. The results show that: (1) Both OPP and ONP outperforms Energy baseline (31 by a large margin, which demonstrates the effectiveness of our proposal in ViT model. (2) OPP outperforms ONP and combining parameter and neuron pruning (OPNP) does not further improve the OOD performance, which is different from the results ResNet50. We think this is because the number of neurons in ViT-B/16 (768) is much less than in ResNet50 (2048), besides, there relu layer before pre-logit layer in ResNet, which results in more risky parameters and neurons in ResNet50 model. (3) OPP outperforms ReAct by 1.85% and outperforms DICE by 3.91 in FPR95, combining OPP and ReAct brings additional improvement.
In Table 2
Evaluation on CIFAR benchmark. The main results on CIFAR1O 10 and CIFAR benchmarks are show in Table 5 and Table 6 As can be seen, we consider the three most commonly used post-hoc methods and compare the OOD detection performance with and without applying the OPNP. The results show that: (1) On both CIFARIO and CIFAR 100, using OPNP consistently outperforms the counterpart without OPNP, which indicates that our proposal is compatible with other post-hoc methods. (2) The performance improvement brought by OPNP on CIFAR100 is more significant than on CIFARI6 10 and less significant than on ImageNet benchmark. We believe this is because the FC layer on CIFARIO classification model is much less overparameterized than CIFAR 100 and ImageNet classification models. (3) The OPNP achieves similar performance ReAct On CIFAR10 benchmark, and outperforms ReAct on CIFAR 100 benchmark. Besides, utilizing OPNP and ReAct jointly reduces FPR95 by 1.55% and 4.49% in CIFAR10 and CIFAR100 benchmark, respectively.

---

45.3
SUN Faces
SUN
10
4415
40
25.2
32.5
40.0
202
17.5
33.3
03
25
1.3
3.0
55
23
30
40
50
20
25.
10
25
10
55
Pia
(a) Sensitivity of Pmaz
(b) Sensitivity of Pmin
Sensitivity Pmaax
(d) Sensitivity of Pimin
Figure 3: Effect of varying pruning percentage parameters in ResNet50 model. (a) Effect of varying
Pmazi (b) Effect of varying Pmin when set Piaa = 0.5; (c) Effect of varying Pmazi (d) Effect of varying Pmin when set Pmaa 30. All numbers percentages.
Table 4: OOD detection performance with different parameter and neuron pruning methods. We ImageNet- 1K as ID) data, SUN and Places as OOD data. FPR95 performance ResNet50 is reported.
Method
RPP
TPP
OPP
RNP
TNP
ONP
43.36
Places
54.40
51.86
40.76
53.92
52.84
32.69
Confidence Histogram
Confidence
Confidence Relabiliy
0.
Confidence
ECE-3.6475
0.8
ECE 2.14%%
0.6
0.6
0.1
0.2
00
0.2
0.6
1.0
0.0
6.0
0.4
0.0
0.0
0.0
0.4
0.8
(a) Distribution
(b) Uncalibrated
(c) After OPNP
Figure 4: Illustration of confidence reliability diagrams. (a) Sample distribution histogram in different confidence bins. (b) Confidence reliability diagrams (CRD) the original calibrated model. (c) CRD in the model with optimal parameter and neuron pruning.
4.3 Ablation Studies.
Effect of pruning percentage. In Fig. we demonstrate the impact on OOD detection performance
by varying pruning percentages in two different tasks. Fig. 3(a) shows that pruning only 0.5% high sensitivity parameters brings significant improvement. In Fig, 3 we observe considerable performance improvement with large pruning percentage for low sensitive parameters. Fig. 3(c)
and Fig. 3(d) suggest that pruning high sensitive neurons more effective than pruning low sensitive neurons, which brings marginal improvement. While OPNP achieves the SOTA performance, the performance could be significantly improved by pruning only the weights or neurons, which much simple to determine the thresholds. Besides, the performance is improved and insensitive in wide range of pruning ratio. For example, the optimal pruning ratio can be set to P E [10, 30) and p [0.5, 3] across different OOD sets. In Tabel 3 we demonstrate the impact of different pruning percentages on ID classification accuracy. As observed, pruning only 1.0% high sensitive parameters decreases the ID accuracy by 4.53%. In contrast, pruning 40% low sensitive parameters only reduces ID accuracy by 1.07%. This highlights the effectiveness of our sensitivity estimation method, and also demonstrates the significant parameter redundancy in deep networks.
Ablation on pruning methods. In this ablation, we compare the proposed sensitivity guided parameter and neuron pruning method with other pruning method, including: (1) Random parameter pruning (RPP) 48 (2) Target parameter pruning (TPP) [15), which prunes weights with low magnitude; (3) Random neuron pruning (RNP) 42): and (4) Target neuron pruning (TNP) [15] which prunes neurons with low feature norm. For the comparison methods, we try different pruning percentages and report the best results. The ablation results in Table. reveal that: (1) pruning

---

Stribation
deribcatoon
Stribction
1701
10 Dingganne-u
Ewrg Scoms
120 Grange Sccme
Erwgy SO0E
dae Every Scons
(a) Energy-SUN
(b) OPNP-SUN
(c) Energy- Placcss
(d) OPNP-Places
Figure 5: Illustration of OOD score distributions in two tasks with the Energy baseline and our proposed OPNP. (a) Energy baseline in SUN benchmark. (b) OPNP in SUN benchmark. (c) Energy
baseline in Places benchmark. (d) OPNP in Places benchmark.
parameters and neurons with low magnitude outperforms random parameter and neuron pruning. (2) The introduced optimal parameter and neuron pruning outperforms other pruning methods by margin margin, with 16.68% improvement in SUN dataset and 18.95% in Places dataset.
OPNP benefits model calibration. A well calibrated model should have better OOD detection performance [16]. In this ablation, we explore how OPNP influences model calibration. In Fig. 4 we evaluate model calibration performance with Confidence Reliability Diagrams (CRD) and Expected Calibration Error (ECE), which was introduced [6]. As observed Fig. 4(b) for an uncalibrated model, the confidence obviously exceeds accuracy, which indicates overconfident confidence. Fig. 4(c) illustraas the effect of utilizing OPNP, which shows that the consistency between confidence and accuracy is improved, and the ECE is reduced by 1.5%. It demonstrates that OPNP is beneficial to model calibration, which also explains why OPNP improves OOD detection performance.
How OPNP changes the score distribution In Fig. we illustrate the OOD score distribution with the Energy baseline [T and our proposed OPNP. We utilize SUN and Places benchmarks with ResNet50 model exhibit how the OPNP changes the OOD score distributions. From the illustration, several interesting observations are: (1) Utilizing OPNP increases OOD scores for both ID and OOD samples. (2) The utilization of OPNP has a significant impact the score distribution of OOD samples, resulting in a more condensed distribution. (3) The OOD score distributions of ID and OOD samples become more separable after applying OPNP, which validates the effectiveness of optimal parameter and neuron pruning.
Conclusion
Conclusion and future work. In this paper, we propose simple yet effective post-hoc OOD detection method. In particular, gradient-based method is proposed to estimate the sensitivity of model parameters and neurons. show that the OOD detection performance could significantly improved by simply removing the connection weights and neurons with exceptionally large or close to zero sensitivities. Extensive experiments and ablations are performed demonstrate the effectiveness of proposal. Compared to energy score baseline, our OPNP reduces FPR95 by 32.5% in ImageNet- IK benchmark. Besides, the OPNP outperforms the SOTA feature rectification method by 5.5% in FPR95 and outperforms the SOTA weight pruning method by 8.8% in FPR95. We believe the optimal parameter and neuron pruning is promising direction for OOD detection tasks, and hope findings bring ideas and breakthroughs other researchers. In our future work, we will explore other post-hoc hoc model pruning and quantization method, as well as low rank decomposition of model parameters for OOD detection.
Limitation and societal impact. The main limitation of this work is lack of theoretical guarantee. Therefore, we call for further application and explanation of the sensitivity guided parameter and neuron pruning method for OOD detection. This work aims to improve the safety of modern deep learning models, which tends to benefit a wide range of applications in social life, such as Al for medical, smart city and driverless system. We hope to provide a plug-and-play tool for Al model users to reduce the false recognition caused by OOD samples in the real world.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the National Key R&D Program of China under Grant 2020AAA0103902.
References
[1] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pages 139-154, 2018.
[2] Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Atom: Robustifying out-of- distribution detection using outlier mining. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference. ECML PKDD 2021, Bilbao, Spain, September 13-17, 2021, Proceedings, Part III 21, pages 430-445. Springer, 2021.
[3] Zhen Cheng, Fei Zhu, Xu-Ya YaoYao Zhang, and Cheng-Lin Liu. Average of pruning: Improving performance and stability of out-of-distribution detection. arXiv preprint arXiv:2303.01201, 2023.
[4] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606-3613, 2014.
[5] Guneet S Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton, Jeremy D Bernstein, Jean Kossaifi, Aran Khanna, and Animashree Anandkumar. Stochastic activation pruning for robust adversarial defense. In International Conference on Learning Representations.
[6] Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distributtio detection. arXiv preprint 2tti:::2209098555 2022.
[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations.
[8] Xuefeng Du, Xin Wang, Gabriel Gozum, and Yixuan Li. Unknown-aware object detection: Learning what you don't know from videos in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13678-13688, 2022.
[9] Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't 't know by virtual outlier synthesis. arXiv preprint arXiv:2202.01 197, 2022.
[10] Sepideh Esmaeilpour, Bing Liu, Eric Robertson, and Lei Shu. Zero-shot out-of-distribution detection based on the pre-trained model clip. In Proceedings of the AAAI conference artificial intelligence, volume 36, pages 6568-6576, 2022.
[11] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-await min- imization for efficiently improving generalization. In International Conference on Learning Representations, 2020.
[12] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of cut-of-distribution detection. Advances in Neural Information Processing Systems, 34:7068-7081, 2021.
[13] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803. 180.:036333335 2018.
[14] Chuanxing Geng, Sheng-jun Huang. and Songcan Chen. Recent advances in open set recogni- tion: survey. IEEE transactions on pattern analysis and machine intelligence, 43(10))3614- 3631, 2020.
[15] Aidan N Gomez, Ivan Zhang, Siddhartha Rao Kamalakara, Divyam Madaan, Kevin Swersky, Yarin Gal, and Geoffrey E Hinton. Learning sparse networks using targeted dropout. arXiv preprint
2019.

---

[16] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. International conference on machine learning, pages 1321-1330. PMLR, 2017.
[17] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015.
[18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[19] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 41-50, 2019.
[20] Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joseph Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling cuttof-ddstributiion detection for real- world settings. In International Conference on Machine Learning, pages 8759-8788 8773. PMLR, 2022.
[21] Dan Hendryeks and Kevin Gimpel. baseline for detecting misclassified and out-of distribbuti examples in neural networks. In International Conference on Learning Representations.
[22] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations.
[23] Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks. The Journal of Machine Learning Research, 1(1):10882-11005 2021.
[24] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34:677- 689, 2021.
[25] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873)583-588, 2021.
[26] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-f--distriuution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
[27] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. In International Conference Learning Representations.
[28] Yi Li and Nuno Vasconeelos. Background data resampling for outlier-aware classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13218-13227, 2020.
[29] Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In 6th International Conference Learning Representations, ICLR 2018, 2018
[30] Shiwei Liu, Tianlong Chen. Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, and Decebal Constantin Mocanu. Deep ensembling with no overhead for either training or testing: The all-round blessings of dynamic sparsity. In International Conference on Learning Representations, 2021.
[31] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-hased dittof-iistiibttion detection. Advances neural information processing systems, 33:21464-21475, 2020.
[32] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations.
12

---

[33] Christos Louizos, Max Welling, and Diederik P Kingma. Learning sparse neural networks
through 1 regularization. In International Conference on Learning Representations.
[34]
Torch maintainers and contributors. Torchvision: Pytorch's computer vision library. https://github. htmp//yyoocc//iviioooo 2016.
[35] Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of- distribution detection with vision-language representations. In Advances Neural Information Processing Systems.
[36] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[37] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427-436, 2015.
[38] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-traing pre-training. 2018.
[39] Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. Advances in neural information processing systems, 32, 2019.
[40] Lukas Ruff, Jacob R Kauffmann, Robert A Vandermeulen, Grégoire Montavon, Wojeiech Samek, Marius Kloft, Thomas Dietterich, and Klaus-Robert Müller. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE, 109(5):756-795, 2021.
[41] Simone Scardapane, Danilo Comminiello, Amir Hussain, and Aurelio Uncini. Group sparse regularization for deep neural networks. Neurocomputing, 241:81-89, 2017.
[42] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting The journal of machine learning research, 15()):1929-1958, 2014.
[43] Yiyou Sun, Chuan Guo, and Yixuan Li. React: Outtof-distributtoo detection with rectified activations. Advances in Neural Information Processing Systems, 34:14 34:144-157, 2021.
[44] Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision, 2022.
[45] Yiyou Sun, Yifei Ming. Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning, pages 20827-20840. PMLR, 2022.
[46] Hidenori Tanaka, Daniel Kunin, Daniel I. Yamins, and Surya Ganguli. Pruning neural networks without any data by iteratively conserving synaptic flow. Advances in neural information processing systems, 33:6377-6389, 2020.
[47] Leitian Tao, Xuefeng Du, Jerry Zhu, and Yixuan Li. Nan-parametric outlier synthesis. The Eleventh International Conference on Learning Representations.
[48] Li Wan, Matthew Zeiler, Sixin Zhang, Yann Le Cun, and Rob Fergus. Regularization of neural networks using dropconnect. In International conference on machine learning, pages 1058-1066. PMIR, 2013.
[49] Ross
Wightman. Pytorch image models. 2019.
httpp::/giithab..oo/rriiggg
pytorch-image-motte
[50] Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsoureing saliency with webcam based eye tracking, arXiv preprint 1rXivv15004.000
2015.
[51] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of--istribution detection: A survey, arXiv preprint 2rXiv:2110.11332. 2021.
13

---

[52] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprini arXiv: 1506.03365, 2015.
[53] Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. In International Conference on Learning Representations, 2019.
[54] Xingxuan Zhang, Renzhe Xu, Han Yu, Hao Zou, and Peng Cui. Gradient norm minimiza- tion seeks first-order flatness and improves generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 20247-20257, 2023.
[55] Yang Zhao, Hao Zhang, and Xiuyuan Hu. Penalizing gradient norm for efficiently improving generalization in deep learning. In International Conference on Machine Learning, pages 26992. PMLR, 2022.
[56) Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 4056)11452-144, 2017.
[57] Yao Zhu, YueFeng Chen, Chuanlong Xie. Xiaodan Li, Rong Zhang, Hui Xue, Xiang Tian, Yaowu Chen, al. Boosting duttt--distributtin detection with typical features. arXiv preprint arXiv:2200.0420, 2022.
14

---

A Evaluation on CIFAR Benchmarks
A.1 Experimental setup
We additionally evaluate our proposal on the widely used CIFAR benchmarks with CIFARIO and CIFAR 100 as ID datasets. The standard split with 50,000 training ID images and 10,000 test ID images are utilized for training and evaluation. We following 43 to evaluate on six common OOD datasets including iSUN [50], LSUN-Resize [52), LSUN-Crop [52] SVHN [6), Places365 [56, and Textures [4. Please refer to 44 for the details of the datasets. For both CIFAR1O and
CIFARION I00 datasets, train standard ResNet50 [18] model for 100 epochs. The learning rate is set to 0.01 and cosine annealing [32) is utilized for learning rate decay. The final accuracy of CIFARIO
and CIFARIO0 classification model is 96.44% and 85.27% respectively.
A.2 Main results
The main results on CIFARIO and CIFARIO0 benchmarks are show in Table Table As can be seen, we consider the three most commonly used post-hoe methods and compare the OOD detection performance with and without the applying OPNP. The results show that: (1) On both CIFARIO 10 and CIFARION 100, using OPNP consistently outperforms the counterpart without OPNP, which indicates that our proposal is compatible with other post-hoc methods. (2) The performance improvement brought by OPNP on CIFAR 100 is more significant than CIFAR1O and less significant than on ImageNet benchmark. We believe this is because the output layer on CIFARIO classification model, which has fewer connection weights, is much less overparameterized than CIFAR 100 and ImageNet classification models. (3) The OPNP achieves similar performance as ReAct On CIFARIO benchmark, and outperforms ReAct on CIFAR benchmark. Besides, utilizing OPNP and ReAct jointly reduces FPR95 by 1 1.55% and 2.49% in CIFARIO and CIFAR100 benchmark, respectively.
Table 5: OOD detection results on CIFAR10 10 benchmark with ResNet50 model. All numbers are percentages. The performances are reported as FPR95/AUROC.
OOD Datasets
Method
Average
iSUN
LSUN-C
LSUN-R
SVHI
Places365
Textures
M2
18.54/33.80 80
11.40996.52
15.75/4.25
6.81/98.07
24.47/22110
18.54/93.77
15.92/94.75
MSP+OPNP
11,52/95.44 44
7,27/98.99
12.59/96. 10
3.32/98.87
19,41/94.40
14,84/9.100
11.49/96.57
Energy 31
9.44/97.60
4.29/99.04
7.77/97.79
155099.67
18.60/95.33
13.22/96998
14/4777774
Energy-OPNP
7.24/98. 18
3,91/99.04
6,38/98.45
O. 86/99.82
13.65/96673
7,40988311
6.61/98.42
ReAct [3]
8.81/97.94
4.94/98.92
6.29/98.34
2.3
11 11.24/97.1
5.93/98.48
6.59/98.41
ReAct=OPNP
5.54/98.32 32
3.62/99.25
4.48/99.00
0.40/99.90
9.34/97.64
6,88/98.20
5.04/98.72
Table 6: OOD detection results CIFAR 100 benchmark with ResNet50 model. All numbers
percentages. The performances are reported as FPR95/AUROC.
OOD Datasets
Method
Average
ISUN
LSUN-C
LSUN-R
SVHN
Places365
[2
51.22/83.16
49.73/86.08
45.77/84.78
31.859144 48
40.76/87.17
40.31/87.79
43.27/86.74
MSP=OPNP
63.68/66.2 20
13.96/87.13
38.44788.67
24.65/92994
33.80/91.04
31.68/91.33
35,86/89 54
Energy [1]
38.39/89.70
30.77/93.18
32.00/1.554
15.21/96.94
32.96/93.06
25.43/93.56
29. 39.13/93.00 00
OPPP
32.39/91.88 86
26, 26,24/9436
27.19/94.36
11,23/97 88
27.03/94.38
23.96/95.10
24.67/94.66
ReAct 43
30.53/92.00
30.86/3.110
31.15/92.57
14.69/7.15
28.04/9.001
21.03/95.27
26.05/94.02
ReAct+OPNP
28.78/92.11
46.4274.14
29.42/93.84
16.71/96.68
22,71/9.007
17.36/96.38
23.56/94.71
B More Ablation Studies
B.1 Perform OPNP on different layers
In our main experiments, we apply the OPNP in the last FC layer in both ResNet50 and ViT-B/16. In this ablation, we also utilize the proposed OPNP in different intermediate layers of ResNet50.
hhuns://tthub.ccmmdeellemmnggggu
15

---

As illustrated in Table. we utilize the OPNP in four different convolutional layers, Layerl- Layer4, which represent the kernel weights the last x convolutional layer in four ResBlocks of ResNet50. The results show that utilizing the OPNP in the last FC layer significantly outperforms the performance by using OPNP in the intermediate convolutional layers, which indicates that the overconfident predictions mainly comes from the last fully connected layer. Besides, we also estimate the parameter sensitivity of the whole model and pruning the weights with two globally pruning strategies, including (I) Global Threshold Pruning: pruning all the weights in Convolution layer and FC layer a global threshold. (2) Layer-wise Pruning: pruning the weights in different layer individually with the same pruning ratio. We find that both global pruning methods perform than only pruning the FC layer. To help understand what happens when weights are pruned globally, we illustrate the average parameter sensitivity of different layers in Fig. 6 It shows that the sensitivity magnitude across different layers differs greatly, the sensitivity of the FC layer is less than To of the shallow Convolution layer. Therefore, it is unreasonable to utilize global threshold for all layers. might be work if we use different thresholds for different layers, but it is too tricky to determine the optimal thresholds.
Table 7: Ablation study of applying OPNP different layers of ResNet50. All numbers percentages. The performances are reported as FPR95/AUROC.
Where to OPNP
SUN
Places
Layers
54.98/85.44
59.63/83.91
Layers2
54.30/85.78
58.77/84.10
Layers3
46.51/86.55
53.12/85.33
Layers4
37.06/88.63
48.45/87.16
Global Threshold Pruning
39.74/91.91
49.13/88.32
Layer-wise Pruning
40.48/92.35
44.92/90.27
FC Layer
18.50/95.62
30. 10.14/93.46
Average sensitivity different layers ResNet50
Average layer sensibiiity
500000
300000
#30000
320000
L00000
Layer index
Figure 6: Average parameter sensitivity of different layers in ResNet50.
B.2 Parameter sensitivity estimation with different number of training samples.
To reduce the cost for parameter sensitivity estimation, it is advisable compute the parameter sensitivity based on a subset of the training set rather than the entire set. To demonstrate the
effectiveness of using subset for sensitivity estimation. We randomly select 1 1%, 5%, 20% and 100% training samples for sensitivity estimation, and perform parameter pruning based on the sensitivities.
The experimental results are shown Table which demonstrate that using only 1% of the training set (ImageNet- IK) also achieves promising performance.
B.3 Neuron sensitivity estimation with different statistics
In the main experiments, we utilize the average sensitivity (equivalent to l norm ) as the neuron sensitivity. It is worth noting that other statistics, such &g norm, Variance, Maximum (Max),
16

---

Table 8: Comparison of experimental results when estimating parameter sensitivity with different numbers training samples. The performance of OPP ResNet50 is reported. w/o indicates Energy baseline without parameter pruning. Numbers are percentages.
Sampling Ratio
w/o pruning
1%
5%
20%
100%
SUN
59.3/85.9
32.57/92.83 42.30/90.00
32.06/92.78 41.96/90.08
30.92/93.05 41.21/90.33
30.40/93.17
Places
64.9/82.9
40.76/90.65
Table 9: Comparison of experimental results when estimating neuron sensitivity with different statistics. The performance ResNet50 is reported. Numbers are percentages.
Statisties
Mean
Max
Min
Median
Eg Norm
Variance
SUN
26.7/94.7
32.4/93.1
26,0/95.1
26.3/94.6 33.4/92.5
31.2/92.7 34.5/91.2
46, 1/88.5
Places
32.7/92.9
39.8/91.2
32.6/92.8
51.1/87.4
Minimum (Min) and Median may also feasible statistics measure the neuron sensitivity. Therefore, we compare the performance of using different statistics as the neuron sensitivity. The results are presented in Figure which show that using the Mean, Min and Median of the weights sensitivity as neuron sensitivity achieve similar performance, and using the Variance statistic performs worst. As the Mean value is more robust and less likely to be susceptible by noisy connections, suggest the users utilize the Mean statistic parameter sensitivity for neuron pruning.
C Sensitivity Distribution Visualization
C.1 Parameter and neuron sensitivity distribution on CIFAR benchmarks.
In Fig. andd Fig. we illustrate the parameter and neuron sensitivity distribution for CIFARIO and CIFARTOO classification models, which reveal several interesting insights. First, on both CIFARIO and CIFARIO 100 classification models, there are a few parameters that exhibit exceptionally high sensitivities and close to zero sensitivities, which is consistent with the sensitivity distribution the ImageNet benchmark. Second, there are more parameters with close to zero sensitivities and more neurons with abnormal sensitivities CIFAR 100 classification model, which explains why OPNP brings more performance gains on CIFAR 100 benchmark.
Parameter sensitivity distribution CIFAR10
Parameter sensitivity distribution in CIFAR10D
12
10:
Density
Density
4.
3.
2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Normalized parameter sensitivity
Normalized parameter sensitivity
(a) CIFAR10
(b) CIFARIO0
Figure 7: Illustration of parameter sensitivity distribution of classification models on (a) CIFARIO and (b) CIFARI00. The parameters taken from the last fully-connected layer in ResNet50, which are 10 2048 and 100 x 2048 for CIFARIO and CIFAR 100 classification model respectively. The dotted line in red indicates the average sensitivity and the maximum sensitivity is normalized to I.
C.2 Parameter Sensitivity Distribution of the Pruned Weights
The parameter sensitivity distribution (on ID and OOD set) the pruned weights illustrated in Fig. The results show that the pruned weights contain many high sensitive connections for OOD set.
17

---

Neuron sensitivity distribution CIFAR10
Neuron sensitivity distribution in CIFAR100
Density
4
Density
5
3-
2-
1-
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
Normalized neuron sensitivity
Normalized sensitivity
(a) CIFARIO
(b) CIFARI00
Figure 8: Illustration of neuron sensitivity distribution on (a) CIFARIO and (b) CIFAR100. The neurons from the pre-logits layer in ResNet50. The maximum sensitivity is normalized to
Besides, the average sensitivity on OOD (0,00024) is larger than the average sensitivity on ID set (0.00018), which experimentally verifies Eq.
Sensitivity distribution ID and OOD samples
OOD
Frequency
10.-
207-
10
0 0,0000 0,0025 0.0050 0.0075 0.0100 0.125 0.0150 0.0755
Parameter sensitivity
Figure Sensitivity distribution on ID and OOD samples. The top 20% sensitive parameters (measure on ID set) are illustrated. The average sensitivity on OOD samples is 0.00024 and the average sensitivity on ID samples 0.00018.
D Discussion
The whole OOD detection pipeline is presented in Algorithm In the main experiments, integrate parameter pruning and neuron pruning a unified framework, which actually can be used individually. The effectiveness of parameter and neuron pruning may vary for different model architectures and ID tasks. Therefore, we recommend readers to attempt different pruning percentage combinations for different tasks. Energy score KT, ReAct 43 DICE
44 ans ASH [6] are the most relevant researches to our work. The difference between our OPNP and the baseline method Energy Score 31 is that we remove some output connections and hidden neurons according to pre-computed parameter and neuron sensitivities. Different from ReAct [43), which clips features with abnormal magnitude, our OPNP removes the parameters and neurons with abnormal sensitivity. The differences between our method and DICE 44 are: (1) DICE obtains the weight importance by averaging the contribution of weights to logit output, while our method measures the parameter and neuron sensitivity to energy score by averaging gradient; (2) DICE only removes the weights with low importance, while our OPNP not only removes non-sensitive connections and neurons but also remove the most sensitive ones. Besides, ASH clips or binarizes the features based on the magnitude to improve OOD performance, whereas our OPNP prunes both weights and neurons based on sensitivity. Compared to those methods that also explore sparsity to improve OOD detection performance, the
18

---

introduced sensitivity metric measured by output energy is directly coupled with OOD detection tasks, and the sensitivity guided pruning is more intuitive, comprehensive and user-friendly.
Algorithm 1 OPNP- Optimal Parameter and Neuron Pruning
Input: A trained model f(æ; 0), training samples { pruning percentage Pinaxr Pmiin Pminn
Pmaa test sample {atg =i=1;
Output: Energy score of test sample (E(x))]k=1
1: for fori= 1 to do
2:
compute gradient g(æ;);
3: end for
4: M
[(xx)]:
5: O,
p=1 Mips i 1= L;
6: Get threshold 2 Smmi Dmaz by ranking M;
7: Get threshold Ω Ω by ranking O:
8: WMM Mazz O and W[M 9 Qminl = O:
9: for k = to do
Compute the pre-logit embedding h(xx);;
11: h(æx)) 0 > 2
0 and h (O < Dminl = 0;
f(xx) W h(()))
13:
F((gg) = - log E exx()
K
14: end for
15: Return {E(x*))
19


Reviews:
[{"review_id": "WK7lbmNGoN", "reviewer": "Reviewer_Dntp", "summary": "This paper proposes to adopt weight and neuron pruning for OOD detection. The proposed method is able to be combined with training-based approaches, demonstrating SOTA performance.", "questions": "See the above limitation.", "limitations": "n/a", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The motivation of the method is reasonable and sound. \n\n2. The results of OPNP+ReAct is strong. ", "weaknesses": "1. It is not new for the ML community that sparsity can help improve OOD detection. Numerous related works have been proposed to adopt sparsity/pruning to improve OOD detection. While might having different pruning approaches, it is necessary to discuss and compare them in the submission.  \n\n[1] Cheng, Zhen, et al. \"Average of pruning: Improving performance and stability of out-of-distribution detection.\" arXiv preprint arXiv:2303.01201 (2023). \n\n[2] Djurisic, Andrija, et al. \"Extremely simple activation shaping for out-of-distribution detection.\" arXiv preprint arXiv:2209.09858 (2022).\n\n[3] Sun, Yiyou, and Yixuan Li. \"Dice: Leveraging sparsification for out-of-distribution detection.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n\n[4] Liu, Shiwei, et al. \"Deep ensembling with no overhead for either training or testing: The all-round blessings of dynamic sparsity.\" arXiv preprint arXiv:2106.14568 (2021).\n\n2. The pseudocode of Algorithm 1 looks very trivial, and can be significantly improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment**\nWe thank Reviewer Dntp (R1) for the feedback and suggestions. Here, we respond to the concerns point by point.\n\n> **W1:**  It is not new for the ML community that sparsity can help improve OOD detection. Numerous related works have been proposed to adopt sparsity/pruning to improve OOD detection [1,2,3,4]. While might having different pruning approaches, it is necessary to discuss and compare them in the submission.\n\n**Ans:**  This is a good question and the recommended papers are related to our work, we will improve Section 3.4 according to the suggestion.\n\nIn our global response, we provided three remarks to explain the insight of our work, and discussed the relationship/difference between our method and these related methods.\n\n We acknowledge that sparsity and pruning are common strategies in machine learning, and some previous work have adopted sparsity/pruning to improve OOD detection [1,2,3,4]. However, Our proposal differs significantly from those methods: \n\n(1) The proposed OPNP is training free while Deep Ensemble ans AoP rely on training multiple sparse network from scratch with dynamic sparse regularization [1,2].  \n\n(2) We propose a gradient based sensitivity metric for post pruning, whereas those methods utilize the magnitude of features [2] or weights [1,3] as pruning metric. The sensitivitiy based pruning metric is more intuitive and technical sound compared to those magnitude based metric.\n\n (3) Our proposal prunes both weights and neurons and prunes both the most and the least sensitive units for OOD detection, whereas the existing methods only prunes the weights or features with small magnitude [1,2,3,4]. \n\n(4) Our proposal is simple, intuitive, effective, and demonstrate much better performance than DICE [4] (see in table1 and table 2 of our paper).\n\n (5) We explain why the sensitive based metric is reasonable to improve OOD detection performance, and explains why prunes both the most and the least sensitive units works.\n\nBesides, AoP (released to ArXiv in March) is a contemporaneous work as ours.\n\nBased on our insights and contributions, we believe our work is novel enough and should be known by the community. \n*******************\n\n> **W2:** The pseudocode of Algorithm 1 looks very trivial, and can be significantly improved.\n\n**A:** Thank you for your suggestion. We have improved the pseudocode (see the pdf), and will revise the submission accordingly.\n\n******************\nWe have carefully provided our response to your questions and concerns. We are looking forward to your reply to see whether our response has resolved your concerns. If your have any other questions, please let us know and we are happy to provide more explanations. We will feel grateful if you could boost our paper.\n\n******************\n**Reference**\n\n[1] Deep ensembling with no overhead for either training or testing: The all-round blessings of dynamic sparsity. ICLR 2022\n\n[2] Extremely simple activation shaping for out-of-distribution detection. ICLR 2022\n\n[3] Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection. Arxiv 2023.\n\n[4] Dice: Leveraging sparsification for out-of-distribution detection. ECCV 2022.\n"}, {"review_id": "sMC2bZcQ8A", "reviewer": "Reviewer_8qxy", "summary": "This submission proposes a post-hoc method for detecting out-of-distribution samples, by pruning the final classification layer, using a sensitivity metric based on the gradients of the energy scores. The proposed method is mainly validated on residual networks and visual transformers based on the ImageNet dataset.\n", "questions": "- Suggestion: I would advise the authors to go through the submission and correct the typos and grammar issues, for the next revision\n- I believe Equation (4) is an approximation, rather than an identity. Have the authors considered what would be a theoretical explanation that would connect the approximation in Equation (4) and the removal of least and most sensitive weights?\n- What is the norm in Equation (6)? Should it be an absolute value instead?\n- Can the authors provide more explanations on the intuition that “parameters and neurons with exceptionally large sensitivity can easily lead to overconfidence” (51-52)?\n- Lines 145-147: it is hard to see these differences from Figure 1, I would suggest adding numbers\n- Lines 181-182: this is not really a Gaussian (e.g it is not symmetric)\n- Can you provide more details on how OPNP is combined with ReAct?\n- In Table 2, can you provide results for DICE + ReAct?\n- Some of the plots and tables do not include information about the model and dataset used. For example, what is the model used in Table 3 or Table 4? What is the model and dataset from Figure 4?\n- The authors show results when (independently) pruning other layers in a ResNet50 model (Appendix Table 3). What happens if the weights are pruned globally, instead of each layer at a time?\n- Have the authors considered second order information for computing the sensitivity metric? For example, a second order Taylor approximation could be used instead of Equation (4), and since pruning is only done for the final layer, the Hessian could be more easily calculated. This is a similar approach to Optimal Brain Surgeon (Hassibi et al.)\n- Can you clarify what is meant by \"(subset of) iNaturalist, SUN, Place and Texture\" (line 211)? Are the same corresponding datasets used for evaluation when comparing with the other methods from Tables 1 and 2?\n- Can you please confirm that the ID test set used for validation (and determining the pruning hyperparameters and OOD threshold) was different than the one used for the final testing?\n- Can you please clarify the differences between Algorithm 1 and the text which specifies that the parameter and neuron sensitivities are computed based on the entire training set? In line 5 from Algorithm 1, it sounds like neuron pruning is done individually for each sample, but from the text the neuron sensitivities are computed on the entire training set. In my understanding, the model is pruned only once, and the same sparse model is used for evaluation on different OOD tasks. Can you please confirm whether my understanding is correct?\n", "limitations": "While the authors have addressed some of the limitations of their work in Section 5, I believe the proposed method, in its current form, has some technical limitations that would make it more difficult to use “plug-and-play” as the authors hope. For example, the method relies quite heavily on the optimal hyperparameters (thresholds) for pruning and it can have a substantial negative impact on the ID classification accuracy for the original task. I would advise the authors to consider how the search of the pruning hyperparameters could be automatized, or at least made more efficient. \n\n--------------------------------\n**Edited after rebuttals**\n\nAfter reading the authors' answer, I have raised my score from 4 to 5.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The submission proposes an interesting application using pruning to out-of-distribution detection, the latter being an important research problem\n- The proposed method is simple, and can be applied with a fairly small computational overhead\n- The results suggest that the proposed method, also when used in conjunction with other methods, can achieve state-of-the-art results on several tasks.\n\n", "weaknesses": "- The submission is not very well written, particularly the introduction, with quite a few typos and grammar issues (e.g. using “post-hot” instead of “post-hoc” in quite a few places). This makes reading a bit difficult. \n- The proposed method is not very novel, as other OOD detection methods have used pruning (e.g. references 20, 35 seem quite similar)\n- The performance of the method depends quite heavily on the pruning thresholds (see Figure 3) and can substantially decrease the accuracy on the original ID classification task (see Table 3). This makes the method less likely to be used “out of the box”.\n- While the authors introduce their method as \"post-hoc\", in fact it has a similar cost to training the model for one epoch, since the gradients for all the samples have to be computed. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment** We thank Reviewer 8qxy (R2) for the careful reviews and insightful suggestions, which really helped us improve our paper. Due to space constraints, part of the responses can be seen in our global response.\n\n***\n\n> **W1:**  Typos and grammar issues.\n\n**A:** We have corrected the typos in the revised paper, and will polish the whole paper carefully in the next version. \n\n***\n\n>  **W2:** The proposed method is not very novel, as other OOD detection methods have used pruning (e.g. Ref 20, 35)\n\n**A:** In our global response, we further explained our insight through 3 remarks and explained the novelty of our proposal . \n\n***\n\n> **W3:** The performance depends heavily on the thresholds (Fig.3)  and can substantially decrease the ID accuracy (Tab.3).  \n\n**A:** (1) According to Fig.3, of all the hyperparameters, only parameter $\\rho_{max}^w$ is relatively sensitive, which can be explained by **Remark 3** (See the global response). (2) The performance is consistently improved in a wide range of pruning ratio. The optimal pruning ratio can be set to $\\rho_{min}^w\\in[10,30]$ and $\\rho_{max}^w\\in[0.5,3]$ across different OOD set (see in Fig. 3).（3）As we only modified the last FC layer, we can always use the original FC layer for classification, which ensures an identical classification accuracy as unpruned model. Existing ReAct and DICE also rely on using the original FC layer for classification. \n\n***\n\n>  **W4:** The authors introduce their method as \"post-hoc\",  but it has a similar cost to training the model for one epoch\n\n**A:** The \"post-hoc\" is relative to the training based method. The cost of parameter sensitivity estimation is much cheaper than training the model, since we only compute the gradient of the last FC layer, and the parameter sensitivity could be estimated in a subset of the training data (See the global response) . Therefore, the training cost is less than 1% compared to the training based methods (even for one epoch).\nAlthough our method is more costly than MSP and Energy, it has similar cost as other popular post-hoc methods (ReAct and DICE), while achieves better performance. \n\n***\n\n> **Q1:** Equation (4) is an approximation, rather than an identity. Any theoretical explanation of removing the least and most sensitive weights?\n\n**A:** Thans for your seggestion, we have revised Eq. 4 as \n\n$E(x_k; \\theta+\\delta) - E(x_k; \\theta) \\approx \\sum_{i,j}g_{ij}(x\\_k)\\delta_{ij}$\n\nWe have provided three remarks in the global response to explain why sensitivity based pruning works.\n\n***\n> **Q2:**  What is the norm in Equation (6)? Should it be an absolute value instead?\n\n**A:** Yes, it should be absolute, we have revised it as\n\n$\\mathbf{M}\\_{ij} = \\frac{1}{m}\\sum_\\{k=1}^m\\lvert g_{ij}(x_k)\\lvert$\n\n***\n\n> **Q3:** Provide more explanations on the intuition that “parameters and neurons with exceptionally large sensitivity can easily lead to overconfidence” (51-52)?\n\n**A:** We have provided three remarks to explain the insight in our global response. The function landscape is flat if the model output does not change drastically in the neighbourhood of the model parameter. Many existing works have shown that a flat landscape lead to better generalization. We have revised the sentence as \"As the parameters with exceptionally large sensitivity can lead to a sharp landscape, which has been proved to hurts model generalization [1]\"\n\n***\n\n> **Q4:**  Lines 145-147: it is hard to see these differences from Figure 1, I suggest adding numbers\n\n**A:** We have revised Fig.1 as suggested. \n\n***\n\n> **Q5:** Lines 181-182: this is not really a Gaussian\n\n**A:** The sentence has been revised as \"As observed, before neuron pruning, there are several risky neurons which has typical large sensitivities.\"\n\n***\n\n> **Q6:** More details on how OPNP is combined with ReAct?\n\n**A:** We have provided the details of OPNP+ReAct in the global response. \n\n***\n\n> **Q7:** In Table 2, can you provide results for DICE + ReAct?\n\n**A:** We have added the results for DICE+ReAct in Table 2 as suggested. Results are as follows.\n\nFPR95/AUROC are reported.\n\n| OOD Dataset | iNaturalist |     SUN     |    Places   |   Texture   |   Average   |\n|:-----------:|:-----------:|:-----------:|:-----------:|:-----------:|:-----------:|\n|  DICE+ReAct |  2.65/99.38 | 29.45/93.52 | 38.45/91.17 | 33.78/93.27 | 26.08/94.34 |\n\n***\n\n> **Q8:** Some of the plots (Fig. 4) and tables (Table 3, 4) do not include information about the model and dataset used. \n\n**A:**  ResNet50 model is used in both Table 3, Table 4 and Fig. 4. In Fig.4,  we utilize the prediciton results in ImageNet-1K to evaluate the calibration performance. We have added the information in the revised paper.\n\n***\n\n> **Q9** What happens if the weights are pruned globally?\n\n**A:** Please see the **Ablations on global pruning** in our global response.\n\n***\n\n> **Q10:** Have the authors considered second order information for computing the sensitivity?\n\n**A:** Yes, we have considered using the eigenvalues of the hessian matrix to measure parameter sensitivity. However, for the ResNet50 in ImageNet-1K, the hessian matrix is 2,048,000x2,048,000, which needs about 8TB Memroy in float16.\n\n***\n\n> **Q11:** Can you clarify what is meant by \"(subset of) iNaturalist, SUN, Place and Texture\" (line 211)?  Please confirm that the ID test set used for validation was different than the one used for testing?\n\n**A:** Please see the **Evaluation Setting** in our global response.\n\n***\n\n> **Q12:** In line 5 from Algorithm 1, the neuron pruning is done individually, but from the text the neuron sensitivities are computed on the entire training set.\n\n**A:** The pruned parameter and neuron index is computed once, which are obtained based on the training set.  But the neuron pruning for different test samples is done individually by setting the correspinding feature to zero in inference time. (See the revised pseudocode in PDF)\n\n[1] Sharpness-aware minimization for efficiently improving generalization"}, {"review_id": "e1nZHAHdRR", "reviewer": "Reviewer_tCft", "summary": "This paper proposes a parameter and neuron pruning strategy to enhance out-of-distribution detection. The approach involves removing near-zero- and high-sensitivity parameters, which are measured by the average gradient corresponding to all training in-distribution samples. Empirical results demonstrate the superior performance of the proposed algorithm.", "questions": "Although empirical evidence indicates performance gains when pruning the largest sensitivity value (as seen in Figure 3), understanding its behavior and identifying optimal values require further investigation. Users would benefit from discussions clarifying such cases and receiving intuitive suggestions to aid in parameter selection. Additionally, exploring the impact of using a fixed threshold instead of a percentile for pruning sensitivity values may lead to improved performance.\n\nThe reliance on the average sensitivity of connection weights for neuron pruning lacks an intuitive explanation. Other statistics, such as min, max, or median, could also be considered as potential alternatives.\n\nAn explicit explanation of how to combine OPNP with ReAct should be provided.\n\nTo enhance efficiency, it is advisable to consider pruning based on a subset of the training set rather than the entire set.\n", "limitations": "The authors have addressed some limitations of their work, and there are additional suggestions for improvement in the 'Paper Weakness'' part and \"Questions\" part.\n", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The proposed algorithm is characterized by its simplicity and remarkable effectiveness in out-of-distribution detection tasks, exhibiting consistently high performance.\n", "weaknesses": "\nWhile the proposed principle shows promising results, providing theoretical explanations for its success would be valuable. Additionally, selecting the appropriate hyper-parameters beforehand presents challenges.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment** We thank Reviewer tCft (R3) for the insightful questions and suggestions, which really helped us improve our paper. Here, we respond to the questions and suggestions point by point. \n\n> **W1:** While the proposed principle shows promising results, providing theoretical explanations for its success would be valuable.  Users would benefit from discussions clarifying such cases and receiving intuitive suggestions to aid in parameter selection\n\n***\n\n**A:** Thanks for your suggestion. We provided three remarks to explain the insight and explanations of why the gradient based sensitivity works. Please refer to the response to **Review qR61** for the details of the remarks. \n\n**Remark 1: Parameter and neuron pruning avoid overconfident predictions**\n\n**Remark 2: Pruning the least sensitive parameters and neurons improve separability between ID and OOD samples**\n\n**Remark 3: Pruning the most sensitive parameters and neurons improves generalization**\n\n***\n\n> **Q1:** Selecting the appropriate hyper-parameters beforehand presents challenges. Exploring the impact of using a fixed threshold may lead to improved performance.\n\n**A:** As we don't know the distribution of sensitivities, it is difficult to determine a fixed threshold. We refer to ReAct [3] and DICE [4] to use a prunning ratio, and explore the ratio on the validation set. The results in Tabel 1 and 2 show that the selected pruning ratio works for different OOD sets.\n\n\n***\n> **Q2:** The reliance on the average sensitivity of connection weights for neuron pruning lacks an intuitive explanation. Other statistics, such as min, max, or median, could also be considered as potential alternatives.\n\n**A:**  It is difficult to prove theoretically which statistic works best. When we design the sensitivity metric for neuron pruning, we have tried different statistics, such as mean,  variance and norm, the results showed that average works best.\n\nFor the $i\\text{-th}$ neuron in the penultimate layer, it contributes to all output neurons, therefore the sensitivity of the $i\\text{-th}$ neuron should be defined based on the sensitivity of the connection weights between the $i\\text{-th}$ hidden neuron and all output neurons. Considering that the $\\ell_1$ or $\\ell_2$ norm of the weights are usually used to meansure the importance of the units in deep networks [1, 2]. We define the neuron sensitivity as the average sensitivity of the connections, which is equivalent to to using $\\ell_1$ norm of the weight sensitivity, and reflects the average sensitivity to all classes. \n\n**Ablation study on different statistics**\n\nIn the main experments, we utilize the average sensitivity (equivalent to $\\ell_1$norm ) as the neuron sensitivity. It is worth noting that other statistics, such as $\\ell_2$ norm, variance, max, min, median may also a feasible statistics to measure the neuron sensitivity. Therefore, we perform a comparision experiment to show the performence of using different statistics as the neuron sensitivity. The results are presented as follows, which show that using the Mean, Min and Median of the weights sensitivity as neuron sensitivity achieve similar performance, and using the Variance statistic performs worst. As the Mean value is more robust and less likely to be susceptible by noisy connections, we suggest the users utilize the Mean sensitivity for neuron pruning. \n\nFPR95 and AUROC  in SUN and Places datasets are reported, ResNet50 model is utilized. \n\n| Statistics |    Mean   |    Max    |     Min    |    Median   |    Norm   |   Variance  |\n|:----------:|:---------:|:---------:|:----------:|:-----------:|:---------:|:-----------:|\n|     SUN    | 26.7/94.7 | 32.4/93.1 |  26.0/95.1 |  26.3/94.6  | 31.2/92.7 |  46.1/88.5  |\n|   Places   | 32.7/92.9 | 39.8/91.2 | 32.6/92.8 |   33.4/92.5 | 34.5/91.2 |   51.1/87.4 |\n\n\nWe will revise the submission (lines 172-174) to add more intuitive explanation and discuss the results of using different statistics for neuron pruning in the appendix.\n\n***\n\n> **Q3:** An explicit explanation of how to combine OPNP with ReAct should be provided.\n\n**A:** Thanks for your suggestion. ReAct is a clipping operation that truncates activations above c to limit the effect of noise. It can be defined as $\\overline h(x) = \\min(h(x), c)$, where $c$ is the clipping threshold.  Therefore, ReAct can be integrated into our method easily by clipping the neuron outputs after our neuron pruning stage. The clipping ratio is set to 10% in ResNet50 and 5% in ViT-B/16. \n\n We will add the information in the revised paper as suggested.\n\n***\n> **Q4:** To enhance efficiency, it is advisable to consider pruning based on a subset of the training set rather than the entire set.\n\n**A:** Good suggestion. That is actually what we do when we try different strategies to get the parameter sensitivity.\n We performed comparision experiments by using different number of training samples for parameter sensitivity estimation. The results are presented as follows, which shows that estimating the sensitivity with only 1% training samples also achieved promising performance. The ablation results will be added to the appendix of our paper.\n\n\nFPR95 and AUROC  in SUN and Places datasets are reported, ResNet50 model is utilized. w/o pruning denotes baseline without pruning.\n\n| Sampling Ratio |       w/o pruning | 1%          | 5%          | 20%         | 100%        |\n|----------------|-------------------|-------------|-------------|-------------|-------------|\n| SUN            | 59.3/85.9         | 32.57/92.83 | 32.06/92.78 | 30.92/93.05 | 30.40/93.17 |\n| Places         | 64.9/82.9         | 42.30/90.00 | 41.96/90.08 | 41.21/90.33 | 40.76/90.65 |\n\n***\n\n **Reference**\n\n[1] Pruning Filters For Efficient ConvNets. ICLR 2017. \n\n[2] Learning sparse networks using targeted dropout. Arxiv 2019.\n\n[3] React: Out-of-distribution detection with rectified activations. NeurIPS 2021.\n\n[4] DICE: Leveraging Sparsification for Out-of-Distribution Detection. ECCV 2022.\n"}, {"review_id": "z6l5x5qMS3", "reviewer": "Reviewer_qR61", "summary": "This paper contributes a new parameter and neuron pruning methods for OOD detection. Built upon the energy-based score, this paper defines the sensitivity of a parameter (neuron) wrt the energy-score by using gradient.", "questions": "$\\bullet$ What are the techniques (feature ensemble or/and input preprocessing) used in Mahalanbis score in comparison?\n\n$\\bullet$ This pruning method seems to be very promising. I understand this was discussed as limitation and future work, but is it possible to share the insights why the sensitivity term(s) will work?", "limitations": "The limitations are fully discussed with explanations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The arguments are clear and easily understood. The method is well motivated by removing the weights and neurons. The experiments are comprehensive and clear, and the results are promising. ", "weaknesses": "$\\bullet$ The major weakness is the theoretical explanation between the sensitivity and OOD performance, but this is clearly pointed out in the paper.\n\n$\\bullet$ The usage of the sensitivity is based on intuition, while this is a good technique for solving OOD problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment** We thank Reviewer qR61 (R4) for the helpful suggestions.  \n\n> **W:** The major weakness is the theoretical explanation between the sensitivity and OOD performance, but this is clearly pointed out in the paper. The usage of the sensitivity is based on intuition, while this is a good technique for solving OOD problems.\n\n**A:** We provide three remarks to explain the insight of our work\n\n## Insight Justification\n**Remark 1: Parameter and neuron pruning avoid overconfident predictions**\n\nThe over-parameterized deep neural networks always generate overconfident predictions even for OOD samples [1,2,3]. Therefore, most exsiting methods improve OOD performance by avoiding overconfident predictions [4,5]. For a deep network, the last fully connected layer can be regarded as a linear classifier. To prevent the classifier from overfitting, the most widely used technique is $\\ell_1$ and $\\ell_2$ regularization,  which can be formulated as \n\n$\\min_{\\theta}\\mathbb{E}_{(x,y)\\sim D}\\Vert\\theta^\\top\\cdot h(x)-y\\Vert_2^2 + \\lambda\\mathcal R(\\theta)$\n\nHere $\\mathcal{R}(\\theta)$ represents $\\ell_1\\text-$ or $\\ell_2\\text{-norm}$ of $\\theta$. As sensitivity based parameter pruning is able to reduce $\\mathcal{R}(\\theta)$,  it can be regarded as an effective post reguralization technique that reduces the model complexity and avoids overconfident outputs. Besides, the neuron pruning is similar to dropout technique [6] which has also been demonstrated to reduce overfitting. Therefore, the proposed OPNP can avoid overconfident predictions and potentially improve the OOD detection performance.\n\n**Remark 2: Pruning the least sensitive parameters and neurons improve separability between ID and OOD samples**\n\n We denote $f_j(x)$ the logit output of $j\\text{-th}$ class, denote $\\mathbf{W}$ the output weights, denotes $\\mathbf{M}$ the sensitive matrix of $\\mathbf{W}$. After pruning the least sensitive parameters, the logit reduction of the j-th class can be estimated as \n\n$\\Delta f_j(x) = \\sum_{\\mathbf{M}\\_{jk}<\\Omega\\_{min}^w}\\mathbf{M}\\_{jk}\\cdot\\lvert\\mathbf W_{jk}\\rvert\\cdot h_k(x)$\n\nwhere $L$ denotes the number of hidden neurons, and $\\Omega_{min}^w$ denotes the sensitivity threshold. Eq.2 shows that the logit reduction is positively correlated with the average sensitivity of the pruned weights. As the parameter sensitivity is computed over the training ID set, the least sensitive parameters on ID distribution should be more sensitive for OOD samples on average, i.e.,\n\n$\\sum_{\\mathbf{M}\\_{jk}<\\Omega\\_{min}^w}\\mathbf{M}\\_{jk}^{OOD} > \\sum_{\\mathbf{M}\\_{jk}<\\Omega\\_{min}^w} \\mathbf{M}_{jk}^{ID}$\n\nTherefore, the logit reduction on OOD samples is larger than on ID samples, which tends to improve the separability between ID and OOD samples, and leads to better OOD detection performance. \n\nWe also show the parameter sensitivity distribution (on ID and OOD set) of the pruned weights in Fig. 1 (see the PDF), which demonstrates that: (1) The pruned weights contain many high sensitive connections for OOD set. (2) The average sensitivity  (0.00024) on OOD set is larger than the average sensitivity (0.00018) on ID set. This experimentally verifies Eq. 3.\n\n**Remark 3: Pruning the most sensitive parameters and neurons improves generalization**\nWe follow [10] to define the first-order flatness as\n\n$R_\\rho(\\theta) \\triangleq \\rho\\cdot\\max_{\\theta'\\in B(\\theta,\\rho)}\\Vert\\Delta f(\\theta')\\Vert, \\quad \\forall\\theta\\in\\Theta$\n\nwhere $\\Delta f(\\theta')$ denotes the derivative at point $\\theta$, $B(\\theta,\\rho) = {\\theta': \\Vert\\theta-\\theta'\\Vert<\\rho}$ denotes the open ball of radius $\\rho$ centered at the point $\\theta$ in the Euclidean space and $\\rho$denotes the perturbation radius that controls the magnitude of the neighbourhood. The flatness $R_\\rho(\\theta)$ descripts how flat the function landscape is [9]. It has been demonstrated that a flatter landscape could lead to better generalization [7,8,9].  Eq. 4 indicates that the first-order flatness is determined by the largest gradient. Therefore, a smaller gradient norm leads to flatter landscape, and many recent works have been proposed to penalize the gradient norm to get better generalization performance [7,8]. Our proposed method pruning the most sensitive parameters, therefore, is able to improve the flatness of the function landscape and lead to better generalization. However, according to Remark 2, pruning the most sensitive parameters and neurons may also hurts the separability between ID and OOD samples. Therefore, there is a trade-off between better generalization and better ID-OOD separability. This explains why OOD performance improves with very few sensitive parameters pruned and drops with a large pruning ratio (see in Fig. 3 of our paper ).\n\n\n> **Q:** What are the techniques (feature ensemble or/and input preprocessing) used in Mahalanbis score in comparison?\n\n**A:**  For the results of Mahalanbis score, we follow [1,2] to report the performance in their paper since we utilize the same evaluation setting. The Mean and Covariance matrix are computed based on the training set. The implementation details of the Mahalanbis score can be seen in [3]\n\n[1] React: Out-of-distribution detection with rectified activations. NeurIPS 2021.\n\n[2] DICE: Leveraging Sparsification for Out-of-Distribution Detection. ECCV 2022.\n\n[3] A simple unified framework for detecting out-of-distribution samples and adversarial attacks. NeurIPS 2018.\n\n[4] Energy-based Out-of-distribution Detection. NeurIPS 2020.\n\n[5] DICE: Leveraging Sparsification for Out-of-Distribution Detection. ECCV 2022\n\n[6] Learning sparse networks using targeted dropout. Arxiv 2019.\n\n[7] Sharpness-aware minimization for efficiently improving generalization. ICLR 2021\n\n[8] Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning. ICML 2022\n\n[9] Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization. CVPR 2023"}]

Author rebuttal:
## Comment
We sincerely thank all the reviewers for their careful reviews and constructive suggestions, which helped us improve our submission. Here, we provide the response to several common concerns and suggestions raised by reviewers.

## Insight Justification
We provide three remarks to explain why OPNP improves OOD detection performance.

**Remark 1: Parameter and neuron pruning avoid overconfident predictions**

**Remark 2: Pruning the least sensitive parameters and neurons improve separability between ID and OOD samples**

**Remark 3: Pruning the most sensitive parameters and neurons improves generalization**

Due to space limitation, we put the details of the remarks in the response to **Review qR61 (R4)**



## Novelty Justification
We declare the contributions and highlights of our work and discuss the relationship between our proposal and other methods 

**Contributions and highlights**

(1) We introduce a gradient based approach to estimate the sensitivity of parameters and neurons in deep models and propose the sensitivity based pruning method for OOD detection. The sensitive based pruning is technically sound and we explain why it works

(2) We are the first to prune both the most sensitive and the least sensitive parameters and neurons for OOD detection, and explain the insight behind it.

(3) The whole method is intuitive,  training free, easy to implement and is able to be combined with different post-hoc approaches to achieve promising performance.

(4) Extensive experiments and ablations have been performed to show the effectiveness and efficiency of our proposal.

**The relationship between our proposal and other methods**

We have discussed the relationship between our proposal and Energy Score, ReAct and DICE in Section 3.4. Here, we also discuss the relationship between our proposal and Deep Ensemble [1],  ASH [2] and AoP [3] recommended by Review Dntp. Both Deep Ensemble and AoP rely on training multiple sparse network from scratch with dynamic sparsity constraint,  whereas our OPNP is training free and prunes both the most and the least sensitivity weights and neurons. ASH clips or binarizes the features based on magnitude to improve OOD performance, whereas our OPNP prunes both weights and neurons based on sensitivity. Compared to those methods that also explored sparsity to improve OOD detection performance, our proposal is more intuitive, comprehensive and user-friendly.


## Common Response

**Parameter sensitivity estimation with lower cost**

To reduce the cost for parameter sensitivity estimation, it is advisable to compute the parameter sensitivity based on a subset of the training set rather than the entire set. To demonstrate the effectiveness of using a subset for sensitivity estimation.  We utilize 1%, 5%, 20% and 100% training samples for sensitivity estimation, and perform parameter pruning based on the sensitivities. The experimental results are shown in Table 1, which demonstrate that using only 1% of the training set (ImageNet-1K) also achieved promising performance.

FPR95 and AUROC in SUN and Places datasets are reported, ResNet50 model is utilized. w/o pruning denotes baseline without pruning.
| Sampling Ratio |       w/o pruning | 1%          | 5%          | 20%         | 100%        |
|----------------|-------------------|-------------|-------------|-------------|-------------|
| SUN            | 59.3/85.9         | 32.57/92.83 | 32.06/92.78 | 30.92/93.05 | 30.40/93.17 |
| Places         | 64.9/82.9         | 42.30/90.00 | 41.96/90.08 | 41.21/90.33 | 40.76/90.65 |

**Implementation details of OPNP\+ReAct**

ReAct is a clipping operation that truncates activations above c to limit the effect of noise. It can be defined as $\overline h(x) = \min(h(x), c)$, where $c$ is the clipping threshold. Therefore, ReAct can be integrated into our method easily by clipping the neuron outputs after our neuron pruning stage. The clipping ratio is set to 10% in ResNet50 and 5% in ViT-B/16. 

**Ablations on global pruning**

To help understand what happens when weights are pruned globally, we also estimate the parameter sensitivity of the whole model and pruning the weights with different globally pruning stragery. The pruning stargegy are: (1) Global threshold pruning (GTP): pruning all the weights in a global threshold. (2) Layer-wise pruning (LWP): pruning the weights in different layer individually with the same pruning ratio. The results are as follows. We find that both global pruning methods performs worse than pruning the FC layer. We believe the reason is that the sensitivity magnitude across different layers is differs greatly, the sinsitivity of the FC layer is less than 1/10 of the previous Conv layer (See the Fig. 2 in the PDF). Therefore, it is unreasonable to utilize a global threshold for all layers. It might be work if we use different thresholds for different layers, but it is too tricky to determine the optimal thresholds.

| Pruning strategy |    SUN    |   Places  |
|:----------------:|:---------:|:---------:|
|        OPP       | 30.4/93.2 | 40.8/90.7 |
|        GTP       | 39.7/91.9 | 49.1/88.3 |
|        LWP       | 40.5/92.4 | 44.9/90.3 |


**Evaluation Setting**

we utilize the same evaluation setting as ReAct and DICE. The "(subset of) iNaturalist, SUN, Place and Texture" are selected by previous method and widely used for OOD detection evaluation. The hyperparameters is determined in the ID validation set and OOD validation set. 

## Reference

[1] Deep ensembling with no overhead for either training or testing: The all-round blessings of dynamic sparsity. ICLR 2022

[2] Extremely simple activation shaping for out-of-distribution detection. ICLR 2022

[3] Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection. Arxiv 2023.

Meta-review: All reviewers are voting towards a borderline or weak accept, as the work is technically solid and provide improvements upon evaluated methods. Once further results from the rebuttal (e.g. DICE+ReAct) are incorporated into the paper, it will certainly be stronger.
Decision: Accept (poster)