Extremely Simple Activation Shaping for Out-of-Distribution Detection
Authors: Andrija Djurisic
Nebojsa Bozanic
Arjun Ashok
Rosanne Liu
Venue: ICLR 2023 poster (2023)
Keywords: out-of-distribution
out-of-distribution detection
activation pruning
post hoc
sparsity
activation shaping

Abstract: The separation between training and deployment of machine learning models implies that not all scenarios encountered in deployment can be anticipated during training, and therefore relying solely on advancements in training has its limits. Out-of-distribution (OOD) detection is an important area that stress-tests a model’s ability to handle unseen situations: Do models know when they don’t know? Existing OOD detection methods either incur extra training steps, additional data or make nontrivial modifications to the trained network. In contrast, in this work, we propose an extremely simple, post-hoc, on-the-fly activation shaping method, ASH, where a large portion (e.g. 90%) of a sample’s activation at a late layer is removed, and the rest (e.g. 10%) simplified or lightly adjusted. The shaping is applied at inference time, and does not require any statistics calculated from training data. Experiments show that such a simple treatment enhances in-distribution and out- of-distribution sample distinction so as to allow state-of-the-art OOD detection on ImageNet, and does not noticeably deteriorate the in-distribution accuracy. Video, animation and code can be found at: https://andrijazz.github.io/ash.

Published as a conference paper at ICLR 2023
EXTREMELY SIMPLE ACTIVATION SHAPING FOR OUT- OF- OF-DIITRIBUTION DETECTION
Andrija Djurisic'
Nebojsa Bozanic!
Arjun Ashok'
Rosanne Liu
ABSTRACT
The separation between training and deployment of machine learning models implies that not all scenarios encountered in deployment can be anticipated during training. and therefore relying solely on advancements in training has its limits. Out-of-distribution (OOD) detection is an important area that stress-tests model's ability to handle unseen situations: Do models know when they don't know? Existing OOD detection methods either incur extra training steps, additional data or make nontrivial modifications to the trained network. In contrast, in this work, we propose an extremely simple, post-hoc, on-the-fly activation shaping method, ASH, where large portion (e.g. 90%) of a sample's s activation at late layer is removed, and the rest (c.g. 10%) simplified or lightly adjusted. The shaping is applied at inference time, and does not require any statisties calculated from training data. Experiments show that such a simple treatment enhances in-distribution and out-of- distribution distinction so to allow state-of-the-art OOD detection ImageNet, and does not noticeably deteriorate the in-distribution accuracy. Video, animation and code can be found https::/ancce jazz github. io/ash
INTRODUCTION
Machine learning works by iteration. We develop better and better training techniques (validated in closed-loop validation setting) and once model is trained, we observe problems, shorteomings, pitfalls and misalignment in deployment, which drive us to go back to modify or refine the training process. However, as we enter an era of large models, recent progress is driven heavily by the advancement of scaling, seen on all fronts including the size of models. data, physical hardware as well as team of researchers and engineers (Kaplan al. 2020; Brown et al.) 2020, Ramesh al.)
2022, Saharia et al. [022,] Yu et al. 20222 Zhang et al. (022). As a result, it is getting more difficult to conduct multiple iterations of the usual train-deployment loop; for that reason post hoc methods that improve model capability without the need to modify training are greatly preferred. Methods like zero-shot learning (Radford et al. (02T), plug-and-play controlling (Dathathri et al.) 22220) as well as feature post processing (Guo al. (017)) leverage post-hoc operations make general and flexible pretrained models more adaptive to downstream applications.
The out-of-distribution (OOD) generalization failure is of such pitfalls often observed in deploy- ment. The central question around OOD detection is "Do models know when they don't 't know?" Ideally, neural networks (NNs) after sufficient training should produce low confidence or high uncertainty measures for data outside of the training distribution. However, that's not always the case (Szegedy al.) 2013 Moosavi-Dezfooli et al. 2017 [Hendrycks & Gimpel 2017 Nguyen et al. 2015; Amodei Amodei et al. 2016). Differentiating OOD from in-distribution (ID) samples proves to be much harder task than expected. Many attribute the failure of OOD detection to NNs being poorly calibrated, which has led to an impressive line of work improving calibration measures (Guo et al.) 2017 [akshminarayaaan al., 2017 Minderer al. 2021). With all these efforts OOD detection has progressed vastly. however there's still room to establish a Pareto frontier that offers the best OOD detection and ID accuracy tradeoff: ideally, an OOD detection pipeline should not deteriorate
ID task performance, nor should it require a cumbersome parallel setup that handles the ID task and OOD detection separately.
'ML Collective. "Faculty of Technical Sciences, University of Novi Sad. *Google Research, Brain Team. Correspondence andri ja@ml col eec org.

---

Published as a conference paper at ICLR 2023
Feature extraction leyers
Feature representation
Classification layers
Input
Logits
Energy soore (for OOD detection)
Flatten
MiP
Sortmax Ohr prediction)
Extremely simplified representation
path, original
Forward path with Activation Shaping (ASH)
Figure I: Overview of the Activation Shaping (ASH) method. ASH is applied the forward path of an imput sample. Black arrows indicate the regular forward path. Red dashed arrows indicate our proposed ASH path, adding one additional step to remove large portion of the feature representation and simplify or lightly adjust the remaining, before routing back to the rest of the network. Note: we default to using the energy score calculated from logits for OOD detection, but the softmax score can also be used for OOD, and we have tested that in our ablation study.
A recent work, ReAct (Sun et al., (021), observed that the unit activation patterns of particular (penultimate) layer show significant difference between ID and OOD data, and hence proposed to rectify the activations at an upper limit in other words, clipping the layer output at an upper bound drastically improves the separation of and OOD data. A separate work, DICE (Sun & Li] 2022), employs weight sparsification on certain layer, and when combined with ReAct, achieves sate-of-theearr on OOD detection on a number of benchmarks. Similarly, in this paper. we tackle OOD detection by making slight modifications to a pretrained network, assuming knowledge of training or test data distributions. We show that an unexpectedly effective, new state-of-ther OOD detection can be achieved by a post hoc, one-shot simplification applied to input representations.
The extremely simple Activation SHaping (ASH) method takes an input's feature representation (usually from late layer) and perform two-stage operation: remove a large portion (e.g. 90%) of the activations based on simple top-K criterion, and 2) adjust the remaining (e.g. 10%) activation values by scaling them up, simply assigning them constant value. The resulting, simplified representation is then populated throughout the rest of the network, generating scores for classification and OOD detection usual. Figure fillustrates this process.
ASH is similar to ReAct (Sun et al. 202T) in its post-training, one-shot manner taken in the activation space in the middle of a network, and its usage of the energy score for OOD detection. And similar to DICE (Sun & Li| 2022), ASH performs sparsification operation. However, we offer number of advantages compared to ReAct: no global thresholds calculated from training data, and therefore completely post hoc; more flexible in terms of layer placement; better OOD detection performances across the board; better accuracy preservation on ID data, and hence establishing a much better Pareto frontier. As to DICE, make modification of the trained network whatsoever, and only operate in the activation space (more differences between ASH and DICE are highlighted in Section K in Appendix). Additionally, our method is plug-and-pll methods, including ReAct (results shown Table 5).
and can be combined with other existing
In the rest of the paper develop and evaluate ASH via the following contributions:
We propose an extremely simple, post-hoc and one-shot activation reshaping method, ASH, unified framework for both the original task and OOD detection (Figure I].
When evaluated across suite of vision tasks including 3 ID datasets and 10 OOD datasets (Table []]. ASH immediately improves OOD detection performances across the board, establishing new state of the art (SOTA), meanwhile providing the optimal ID-OOD trade-off, supplying a new Pareto frontier (Figure 2
We present extensive ablation studies on different design choices, including placements, pruning strength, and shaping treatments of ASH, while demonstrating how ASH can be
2

---

Published as a conference paper at ICLR 2023
readily combined with other methods, revealing the unexpected effectiveness and flexibility such a simple operation (Section
2 THE CUT-OF-DIITTRIBTTEE DETECTION SETUP
OOD detection methods are normally developed with the following recipe:
1. Train a model (e.g. a classifier) with some data- data-namely the in-distribution (ID) data. After training, freeze the model parameters.
2. At inference time, feed the model -ittr--distiittionn (OOD) data.
3. Turn the model into detector by coming up with score from the model's output, to differentiate whether an input is ID or OOD.
4. Use various evaluation metries to determine how well the detector is doing.
The highlighted keywords are choices to be made in every experimental setting for OOD detection. Following this convention, design choices for this paper are explained below.
Datasets and models (Steps 1-2) We adopt experimental setting representative of the previous SOTA: DICE (on CIFAR) and ReAct (on ImageNet). Table [
summarizes datasets and model architectures used. For CIFAR-10 and CIFAR-100 experiments, we used the 6 OOD datasets adopted
in DICE (Sun & Li 2022): SVHN (Netzer et al. 2011). LSUN-Crop Yu et al. 2015). LSUN- Resize (Yu et al. 2015 5), ISUN (Xu et al. 2013), Places365 (Zhou et al. 2017) and Textures (Cimpoi et al. 2014), while the ID dataset is the respective CIFAR. The model used is a pretrained DenseNet-
101 Huang et al. 2017). For ImageNet experiments,
inherit the exact setup from ReAct (Sun et al. 2021 where the ID dataset is ImageNet- Ik, and OOD datasets include iNaturalist ( (Van Horn et al.
2018). SUN (Xiao al.) 2010). Places365 (Zhou et al., 2017). and Textures (Cimpoi al. |01 4). We used ResNet50 (He et al.) 2016) and MobileNet (Sandler et al., 2018) network architectures. All networks are pretrained with ID data and never modified post- post-training: their parameters remain unchanged during the OOD detection phase.
ID Dataset
OOD Datasets
Model architectures
CIFAR-I0
SVHN, LSUN C, LSUN R, iSUN, Places365, Textures
DenseNet-101
CIFAR-100
SVHN, LSUN C, LSUN R, iSUN, Places365, Textures
DenseNet-101
ImageNet
iNaturalist, SUN, Places365, Textures
ResNet50, MobileNetV2
Table 1: Datasets and models in our OOD experiments. We cover both moderate and large scale OOD benchmark settings, including evaluations on up 10 OOD datasets and architectures.
Detection scores (Step 3) Commonly used score functions for OOD detection are the maximum/pre- dicted class probability from the Softmax output of the model (Hendrycks & Gimpel 2017), and the Energy score (Liu et al. 2020). As the Energy score has been shown to outperform Softmax (Liu et al. 2020 Sun al. [021]. we default using the former. In our ablation studies also experimented versions of our method combined with the Softmax score, as well as other methods (see Table 5). For given input x and a trained network function f. the energy function Exx; f) maps the logit outputs from the network, fxx), to a scalar: E(x; f) = log De= e where C the number of classes and fx( is the logit output of class The score used for OOD detection is the negative energy score, - E(x; f), so that ID samples produce a higher score, consistent with the convention. We compare our method with other scoring methods, e.g. Mahalanobis distance (Lee al. 2018 8), as well as other advanced methods that build upon them: ODIN (Liang et al. 2017) (built upon the Softmax score), ReAct (Sun al. 202I) and DICE (Sun Li 2022) (built upon the Energy score).
Evaluation metrics (Step 4) We evaluate method using threshold-fit metrics for OOD detection standardized in Hendryeks Gimpel (2017): (i) AUROC: the Area Under the Receiver Operating Characteristic curve; (ii) AUPR: Area Under the Precision-Recall curve; and (iii) FPR95:
3

---

Published as a conference paper at ICLR 2023
false positive rate- the probability that negative (e.g. OOD) example is misclassified as positive (e.g. ID)-when when the true positive rate is as high as 95% (Liang et al! 2017). In addition to OOD metrics, we also evaluate each method on their ID performance, which in this case is the classification accuracy on in-distribution data, e.g. Top-1 accuracy on the ImageNet validation set.
96
Average OOD AUROC (ocross 4 stask))
D+R
92-
90
B
ENERGY
8
SOFTMAX
ODIN
86-
REACT
DICE
84-
DICE+REACT
ASIL-P 65- 90%
ASH-S S 65 90%
ASH-B 65 90%
73
76
ID Accuracy
Figure 2: ID-OOD tradeoff on ImageNet. Plotted are the average OOD detection rate (AUROC: averaged across 4 OOD datasets iNaturalist, SUN, Places365, Textures) vs ID classification accuracy (Top-1 accuracy in percentage on ImageNet validation set) of all OOD detection methods and their variants used in this paper. Baseline methods "E", "S" and "O" lie the upper bound of ID accuracy (indicated by the dotted gray line) since makes no modification of the network or the features. "R", "D" and "D+R" improve on the OOD metric, but come with an ID accuracy drop. ASH (dots connected with dashed lines; smaller dots indicate lower pruning level) offers the best trade-off and form Pareto front.
ACTIVATION SHAPING FOR OOD DETECTION
trained network converts raw input data (e.g. RGB pixel values) into useful representations (c.g. a stack of spatial activations). We argue that representations produced by modern, over weerpparameterized deep neural networks are excessive for the task at hand, and therefore could be greatly simplified without much deterioration the original performance (e.g. classification accuracy), while resulting in surprising gain on other tasks (e.g. OOD detection). Such hypothesis is tested via an activation shaping method, ASH, which simplifies an input's feature representation with the following recipe:
1. Remove a majority of the activation by obtaining the pth-percentie of the entire representa- tion, f, and setting all values below to O.
2. For the un-pruned activations, apply one of the following treatments:
ASH-P (Algorithm) T): Do nothing. Pruning all we need. This used as a baseline to highlight the gains of the following two treatments.
ASH-B (Algorithm
2 Assign all of them to be positive constant the entire
representation becomes Binary.
ASH-S (Algorithm 3 Scale their values up by ratio calculated from the sum of activation values before and after pruning.
ASH is applied on-the-fly to any input sample's feature representation at an intermediate layer, after which it continues down the forward path throughout the rest of the network, depicted in Figure] Its output generated from the simplified representation is then used for both the original task (e.g. classification), or, in the case of OOD detection, obtaining score. to which thresholding mechanism applied to distinguish between ID and OOD samples. ASH is therefore a unified framework for both the original task and OOD detection without incurring any additional computation.
4

---

Published as a conference paper at ICLR 2023
Algorithm 1 ASH-P: Activa-
Algorithm 2 ASH-B: Activation
Algorithm 3 ASH-S: Activation
tion Shaping with Pruning
Shaping by Binarizing
Shaping with Scaling
Input: Single sample activation x, pruning percentile p Output: Modified activation
Input: Single sample activation x, Input: Single sample activation x,
pruning percentile p
pruning percentile
Output: Modified activation
Output: Modified activation x
I: Calculate the p-th percentile
1: Calculate the p-th percentile of 1: Calculate the y-th percentile of the input x threshold Calculate sum of x x
of the input threshold
the input - threshold & Calculate the sum
+ sl
2: Set all values x less than
3: Set all values in x less than to 0
3: Set all values x less than t to 0
to 0
4: Calculate number of norozeros in
4: Calculate the sum of x + s2
X + n
Set non-zero values in s/n
5: Multiply all non-zero values in with exp(a1/s2)
3: return x
6: return x
6: return x
Placement of ASH We can apply ASH at various places throughout the network, and the perfor- mance would differ. The main results shown in this paper are from applying ASH after the last average pooling layer, for ImageNet experiments, where the feature map size is 2048 x 1 x 1 for ResNet50, and 1280 x x 1 for MobileNetV2. For CIFAR experiments conducted with DenseNet- 101, we apply ASH after the penultimate layer, where the feature size is 342 x 1 x 1. Ablation studies for other ASH placements are included in Section 5 and Section A in Appendix.
The parameter ASH algorithms come with only one parameter, p: the pruning percentage. In experiments we vary p from 60 to 90 and have observed relatively steady performances (see Figure When studying its effect on ID accuracy degradation, we cover the entire range from 100 (see Figure 3). The SOTA performances are given by surprisingly high values of p. For ImageNet, the best performing ASH versions are ASH-B with p = 65, and ASH-S with p = 90. For CIFAR-10 and CIFAR- 100, the best performing ASH versions are ASH-S with p = 95 and p = 90, and comparably, ASH-B with p = 95 and = 85. See Section in Appendix for full details on the parameter choice.
4
RESULTS
4.1 ASH OFFERS THE BEST ID-OOD TRADEOFF
ASH unified pipeline for both ID and OOD tasks demonstrates strong performance at both. Figure Figure[]showe showws the ID-OOD tradeoff for various methods for ImageNet. On one end, methods that rely on obtaining a score straight from the unmodified output of the network, e.g. Energy. Softmax and ODIN (Liang et al., 2017), perfectly preserves ID accuracy, but perform relatively poorly for OOD detection. Advanced methods that modify the network weights or representations, e.g. ReAct (Sun) et 2021) and DICE (Sun& & Li 2022), come at compromise of ID accuracy when applied as unified pipeline. In the case of ReAct, the ID accuracy drops from 76. 13% to 73.75%.
ASH offers the best of both worlds: optimally preserve ID) performance while improving OOD detection. Varying the pruning percentage p. we can see Figure 2 that those ASH-B and ASH-S variants establish new Pareto frontier. The pruning-only ASH-P gives the same ID accuracy as ASH-S at each pruning level, while falling behind ASH-S on the OOD metric, suggesting that simply by scaling up un-pruned activations, we observe great performance gain at OOD detection.
4.2 OOD DETECTION ON BOTH IMAGENET AND CIFAR BENCHMARKS
ASH is highly effective at OOD detection. For ImageNet, while Figure 2 displays the averaged performance across datasets, Table llys of the two metrics: FPR95 and AUROC. The table follows the exact format as Sun et al. (2021). I), reporting results from competitive OOD detection methods in the literature, with additional baselines computed by (e.g. DICE and DICE+ReAct on MobileNet). As and ASH-S establish the new SOTA across almost all OOD datasets and evaluation metrics on ResNet, and perform comparably with DICE+ReAct while much more algorithmically simpler.
lays out detailed performances on each of them, with each
see, the proposed ASH-B
5

---

Published as a conference paper at ICLR 2023
ASH-P showcases surprising gains from pruning only (simply removing 60% low value activations), outperforming Energy score, Softmax score and ODIN.
On CIFAR benchmarks, followed the exact experimental setting as Sun & Li 2022): OOD datasets with pretrained DenseNet- 101. Table our method's performance (averaged across all 6 datasets) alongside all baseline methods. Detailed per-dataset performance is reported in Table and Table g in Appendix. All ASH variants profoundly outperform existing baselines.
OOD Datasels
Model
Methods
iNaturalist
SUN
Places
Textures
Average
EPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
1
Softmax scure
87.74
79.61
ODIN
47.66
89.66
(0. 15
84.59
67.89
81.78
50.23
85.62
56.48
85,41
Mahalanobis
Energy score
55.72
89.95
59.26
85.89
64.92
82. 86
53.72
85,99
58.41
86.17
RosNet
ReAct
96.22
24.20
33.85
47.30
31.43
92.95
DICE
25.63
94.49
35.
90.83
46.49
87.48
31.72
90.30
34.75
90.77
DICE ReAct
18.64
96.24
25.45
93.94
36.86
90.67
92.74
27.25
93.40
ASH-P (Ours)
44.57
92.51
$2.88
88.35
61.79
85.58
42.06
89.70
50.32
89.04
ASH-B (Ours)
14.21
97.32
22.08
95.10
33.45
92.31
21.17
95.50
22.73
95,06
ASH-S (Ours)
11.49
97.87
27.98
94.02
39.78
90.998
11.93
97.60
22.80
95.12
Softmax score
64.29
85.32
77.02
77.10
79.23
76.27
73.51
77,30
73.51
79.00
ODIN
85.88
85.81
Mahalanobis
62.11
81.00
47.82
86.33
$2.09
83.63
92.38
33.06
63.60
71.01
Energy score
NetileNet
ReAct
42,40
91.53
47.69
88. 16
$1.56
86.64
38.42
91.53
45.02
89.47
90.83
89.60
DICE + ReAct
32.30
93.57
31.22
92.86
46.78
88.02
16.28
96.25
31.64
92.68
ASH.P (Ours)
58.61
66.59
87.34
ASH-B (Ours)
31.46
94.28
38.45
91.60
51.80
87.56
20.92
95.07
35 66.66
92.13
(Ours)
43.62
90.02
13.12
38.67
90.95
Table 2: OOD detection results on ImageNet. We follow the exact same metries and table format as Sun et al. (202T). Both ResNet and MobileNet are trained with ID data (ImageNet- Ik) only. t indicates larger values are better and 4. indicates smaller values are better. All values are percentages. "DICE" and "DICE+ReAct" for MobileNet are implemented by us (refer to Section FF for hyperparameter choices), The rest of the table except for those indicated "Ours" are taken directly from Table in (un et al. (2021). For ResNet, ASH consistently perform better than benchmarks, across all the OOD datasets. In the case of MobileNet, ASH performs comparably with DICE+ReAct.
CIFAR-10
CIFAR-I00
Method
FPR95 1
AUROC 1
FPR95
AUROC
Softmax score
48.73
92.46
80.13
74.36
ODIN
24.57
93.71
58.14
84.49
Mahalanohis
31.42
89.15
$5.37
82,73
Energy score
26.55
94.57
68.45
81.19
ReAct
94.95
62.27
84.47
DICE
20.8344.55
95.24 4.0.44
49.72 .9
87. 4.0.7
ASH-P (Ours)
23.45
95.22
64.53
82,71
ASH-B (Ours)
20.23
96.02
48.73
88.04
ASH-S (Ours)
15.05
96.61
41.40
90.02
Table 3: OOD detection results CIFAR benchmarks. indicates larger values better and indicates smaller values are better. All values are percentages. Results are averaged accross 6 OOD datasets. Methods except for ASH variants (marked as "Ours") taken from Sun & Li (202
4.3 ON PRESERVING IN- IN-DISTRIBUTION ACCURACY
ASH preserves in-distribution performance, and hence can be used unified pipeline for ID and OOD tasks. As we can see in Figure 2 towards the low end of pruning percentage (65%) both ASH-S and ASH-P come with only slight drop of ID accuracy (ImageNet Top-1 validation accuracy;

---

Published as a conference paper at ICLR 2023
ImageNet benchmark FPR95
Table 4: An extreme variant of ASH: randomized activations. ASH-RAND sets un un-pruned activations to random val- ues between O and 10. All experiments are on ImageNet with ResNet-50, and the
Method
AUROC
AUPR
ID
ACC
ASH-RAND@65
45.37
90.80
98.09
12.16
ASH-RAND@
46.93
90.67
98.05
72.87
ASH-RAND@75
46.93
90.67
98.05
73.19
OOD performance is averaged across 4 surprisingly com- parable to, although not better than, ASH- and ASH-S. It consistently beats simple baselines like Energy, Softmax and ODIN.
ASH-RAND@sS 80
51.24
89.94
97.89
73.57
ASH-RAND@00
59.35
87.88
97.44
73.51
datasets. ASH-RAND
ASH-B @65
22.73
95.06
98.94
74.12
ASH-S@90
22.80
95.12
98.90
74.98
76.13% to 76.01%) The more prune, the larger drop of accuracy is seer At 90% that is, when 90% of activation values are Boaiinted-ASHt. and ASH-P maintain an ID accuracy of 74.98%. When experimenting with a wider range and granular pruning levels, shown in Figure 3 we observe that ASH-S and ASH-P do preserve accuracy all the way, until a rather high value of pruning (e.g. ID accuracy dropped to 64.976% 99% of pruning).
However, a reversed trend is observed in ASH-B: as seen in Figure 3 between 50-90% of pruning, the ID accuracy is trending up, while the best accuracy is achieved between 80% to 90% of pruned activation. The reason is that the rather extreme binarizing operation in ASH-B (setting all remaining activations to a constant) has a bigger impact when the pruning rate is lower (more values are being modified). To the extreme of 0% pruning, ASH-B simply sets all activation values to their average, which completely destroys the classifier (Figure left end of curve).
Accuracy Pruning Percentage
Aucuracy vs Pruming Percenlage (Zoomed-in)
ID Accuracy on ImageNNet.
0.8
0.6
3.76
0.4
ASH-B
ASH-B
ASHLIP
ASI P
0.2
ASTE-
ASII- 8
Proning
Proning
60
80
100
72
20
50
100
Pruning Percentage
Pruning Percentage
Figure 3: Accuracy degradation across pruning percentage. All three versions of ASH are applied to the penultimate layer a ResNet-50 pretrained on ImageNet. At test time the input samples from the ImageNet validation set being processed with ASH, and the Top-1 accuracy is reported across a range of pruming strengths. ASH-P and ASH-S have the exact same effect on ID accuracy, as expected. ASH-B fails when pruning percentage low, as the majority of the feature map would be converted to a constant value. The right plot is a zoomed-in version (on y axis) of the left.
4.4 ASH-RAND: RANDOMIZING ACTIVATION VALUES
Given the success of ASH on both ID and OOD tasks, especially ASH-B where the values of a whole feature map are set to either 0 or a positive constant, we are curious to push the extent of activation shaping even further. We experiment with a rather extreme variant: ASH-RAND, which sets the remaining activation values random nonnegative value between [o, 10) after pruning. The result is shown in Table It works reasonably well even for such an extreme modification of feature maps.
The strong results of ASH prompt us to ask: why does making radical changes to a feature map improve OOD? Why does pruning away a large portion of features not affect accuracy? Are represen- tations redundant to start with? We discuss useful interpretations of ASH Appendix Section F
ASH-P and ASH-S produce the exact accuracy on ID tasks, as linearly scaling the last-layer activation and hence logits, barring numerical instability, does not affect model's softmax output.
7

---

Published as a conference paper at ICLR 2023
5 ABLATION STUDIES
Global vs local threshold The working version of ASH calculates the pruning threshold t (accord- ing to fixed percentile see Algorithms 1-3) per-image and on the fly, that is, each input image would have pruning step applied with a different threshold. This design choice requires no global information about the network, training test data, but incurs slight computational overhead at inference. An alternative to calculate "global" threshold from all training data, assuming we have access them post training. There a 90% pruning level means gathering statisties of certain feature map from all training data and obtaining a value that reflect the 90% percentile.
We implement both ASH-S and ASH-B with both global and local thresholds; the difference between the two design choices are showcased in Figure ASH-S. ASH-B results are included in Section Appendix. As can see, with aligned pruning percentage, using local threshold always works better than setting global one, while the best overall performance is also achieved by local thresholds.
DORETY
Locd
Lool
Paxing Ammerage
Prasing Pemeragg
Priing Proorag
(a)
(b)
(c)
Figure 4: Global vs local threshold. We implement variant of ASH with global thresholds obtained from training data, and compare the results with the defaulted local threshold version. Shown are ASH-S with ResNet-50 trained on ImageNet, with local global curves FPR95, AUROC, and AUPR, respectively. Local thresholds, while incurring slight overhead at inference, perform much better and require access training data.
notice that a working placement of ASH is towards later layers of a trained network, e.g. the penultimate layer. We experimented how other placements affect its performance. In Figure 3 we show the effect performing ASH-S on different layers of network. We can see pruning becomes severe
Where to ASH
that the accuracy deterioration the network. For full OOD detection results from these placements, refer to Section A in Appendix.
move earlier parts of
ID accurity on ImogeNel
AS8I
Backk
C.
AS9I
Biock-
Prining
91.5
S1B
92.
95.5
100
Prosing Pereentage
ASH 60
ASH
ASH
ASH di
ASH
1020
AND POO
TVT
PALL
Con
Block
Block
Block
Block4
Panulimate
frist soew
Figure 5: Effect of ASH placements ID Accuracy. (Top) ID accuracy degradation curves across pruning percentage, each color indicating a different placement of ASH. (Bottom) Diagram of ResNet-50 architecture and indications of where ASH is applied.
Plug-and-play ASH to other methods ASH as a two-step operation is very much compatible with other existing methods. In Table 5 we demonstrate how it can be easily combined with the Softmax
8

---

Published as a conference paper at ICLR 2023
score, the Energy score, ODIN and ReAct to provide immediate improvement over them. Experiments are performed with CIFAR-10, 10, CIFAR- 100 and ImageNet. All methods are reimplemented by us.
See Section F in Appendix for implementation details.
CIFAR-10 AUROC
CIFAR-100 AUROC
ImageNet AUROC
Method
FPR95
AUPR
FPR95
AUPR
FPR95
AUPR
1
Softmax ScoTe
48.69
92.52
80.75
80.06
74.45
76.99
64.76
82.82
95.94
Softmax ScoTe + ASHI ir.
48.86
92.61
77.03
76.04
75.00
78.61
37.86
90.90
97.97
ODIN
25.71
94.72
95.60
64.87
82.43
84.85
50.80
87.57
97.19
ODIN + AS
15.38
96.41
96.62
38.54
90.49
91.51
28.59
93.34
98.40
Energy score
26.59
94.63
95.61
68.29
81.23
83.64
57.47
87.05
97.15
Energy score ASH lr.
15.05
96.61
96.88
41.40
90.02
91.23
22.80
95.12
98.90
ReAct
29.00
94.92
96. 14
69.94
82.07
85.43
31.43
92.95
98.50
ReAct ASH
16.35
96.91
97.41
41.64
88.93
90.14
24.88
94.27
98.66
Table 5: The ASH treatment (ASH tr.) is compatible with and improves on existing methods. CIFAR1O and CIFAR 100 results averaged across 6 different OOD tasks and ImageNet results are averaged across different OOD tasks. All methods and experiments are implemented by us. Note that "Energy score + ASH tr." " is the same as what's referred as " "ASH" elsewhere.
6 RELATED WORK
Post-hoc model enhancement Our method makes post-hoc modifications to trained representations of data. Similar practices have been incorporated in other domains. For example, Monte Carlo dropout (Gal & Ghahramani 2016) estimates predictive uncertainty by adding dropout layers at both training and test time, generating multiple predictions of the same input instance. In adversarial defense, randomized smoothing (Cohen et al. 2019) applies random Gaussian perturbations of an input to obtain robust outputs. The downside of these methods is that they all require multiple inference runs per input image. The field of model editing (Santurkar et al.) [202]; [Meng al.) 2022) and fairness (Alabdulmohsin & Lucic / al., 2019) also modify trained models for gains in robustness and fairness guarantees. However, they involve domain expertise as well as additional data, neither of which required in work. Temperature scaling (Guo et al. (017) rescales the neural network's logits by a scalar learned from a separate validation dataset. ODIN (Liang et al.) 2017) then combines temperature scaling with input perturbation. We have compared closely with ODIN our experiments and have shown superior results.
Sparse representations A parallel can be drawn between ASH and activation pruning, and the more general concept of sparse representations. Stochastic activation pruning (SAP) (Dhillon al. 2018) has been proposed as a useful technique against adversarial attacks. SAP prunes a random subset of low-magni activations during each forward pass and scales up the others. Adversarial defense is out of scope for this paper, but serves as a fruitful future work direction. Ahmad & Scheinkman (2019) used combinatories of random vectors show how sparse activations might enable units to use lower thresholds, which can have a number of benefits like increased noise tolerance. They use top-K K in their network as an activation function, in place of ReLU, The key difference from ASH the usage of top-K to create sparse activations even during training, and the removal of ReLU.
CONCLUSION
In this paper, we present ASH, an extremely simple, post hoc, on-the-fly. fly, and plug-and- and-play activation shaping method applied to inputs at inference. ASH works by pruning large portion of an input sample's activation and lightly adjusting the remaining. When combined with the energy score, it's shown to outperform all contemporary methods for OOD detection, on both moderate and large-scale image classification benchmarks. It is also compatible with and provides benefits to existing methods. The extensive experimental setup on 3 ID datasets, 10 OOD datasets, and performances evaluated on 4 metrics, demonstrates the effeetiveness of ASH across the board: reaching SOTA on OOD detection while providing the best trade-off between OOD detection and ID classification accuracy.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
This work was supported by the ML Collective compute grant funded by Google Cloud, and was selected by the ICLR 2022 DEI initiative (Liu & Maughan 2021 2022)
sponsored project. The authors would like to thank Marcus Lewis for proofreading and supplying connections to the sparsity literature, Dumitru Erhan for giving feedback on early drafts, and Milan Mísic for fruitful discussions and brainstorming in the early phase of the project. We thank Fargo the dog for his canine countenance. We are grateful for the ML Collective community and the ICLR CoSubmitting Summer community for the ongoing support and feedback of this research.
REFERENCES
Subutai Ahmad and Luiz Scheinkman. How can we be so dense? the benefits of using highly sparse representations. arXiv preprint arXiv: 1257, 2019.
Ibrahim M Alabdulmohsin and Mario Lucic. A near-optimal algorithm for debiasing trained machine learning models. Advances in Neural Information Processing Systems, 34:8072-8084, 2021.
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint (rXiv:1606.06565. 2016.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL https
//arxiv. wra.iiiorrraaaa http://bb/000..... 14165
L Elisa Celis, Lingxiao Huang, Vijay Keswani, and Nisheeth K Vishnoi. Classification with fairness constraints: A alga-algo with provable guarantees. In Proceedings of the conference on fairness, accountability, and transparency, pp. 319-328, 2019.
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describ- ing textures in the wild. In Proceedings of the IEEE conference on computer vision and puttern recognition, pp. 3606-3606666 3613, 2014.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. International Conference on Learning Representations, 2020. URL https
7/openreview net. wtwpreewwwwe.eee/oouue.e......dd d-t.
Guneet S Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial defense. arXiv preprint arXiv: /803.0/44, 2018
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty deep learning. In international conference on machine learning, pp. 1050-1059. PMLR, 2016.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and amples in neural networks. Proceedings of International Conference on Learning Representations, 2017.
10

---

Published as a conference paper at ICLR 2023
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2017.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020. htttthttt:/aaaattt
www.//g//sb....oo 22001 08361
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL
neurips Paperr
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribttion samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Chunyuan Li. Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC. Canada, April 30- May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. /ttptttss////peneeeee
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of outtof-distribution image detection in neural networks. arXiv preprint arXiv: (rXiv:706.006690 2017
Rosanne Liu and Krystal Maughan.
Broadening our call for participa-
tion to iclr 2022. 2021.
URL https://bloq... icl.r.iccl...omm wttpcccccc.///////
broaden loinng--o our- cal 11al1111 for-fo--paa--- por.-pprticippaioo-tooo
2022
Rosanne Liu and Krystal Maughan.
Reflection on the dei initiative
at iclr 2022, 2022.
URL https: www.riclrrroooo.e wclr.cc//022205/12
reflection -o--on--tto the dette--eiiiiittat
at-iccr- 10clr-2022
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distiibution detection. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt, 2022. URI. httttttss:/:/axxitt wttp://....ggaa/////o0// 05262
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems, 34, 2021.
Scyed-Mohsen Moosavi-Dezfoolli Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1765-1773, 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning, In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011. URL http: http://afldd.sstttfodd..... stanford. hoosennubeers/nips2211 hwwsenumbers.... pdf
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are casily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 427-42444444 2015.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, pp. 8763. PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents, 2022.

---

Published as a conference paper at ICLR 2023
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan. S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding, 2022. URL httpss:/aaaxvv.rr///
12205.124 41487
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Ching- Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
Shibani Santurkar, Dimitris Tsipras, Mahalaxmi Elango, David Bau, Antonio Torralba, and Aleksander Madry. Editing classifier by rewriting its prediction rules. In M. Ran- zato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 23359-233733 ciates, Inc., 2021. URL htttttt:p/poooceeeiiiii neuinnggss.uuuuueee neurips. ips. cccppaper//22//111e. 0bb49942baaa9ccccc31111100000010 Papees22dddddPaeeppppao Paper.
23373. Curran Asso-
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of distribution detection with rectified activations. Advances in Neural Information Processing Systems, 34:144-157, 2021.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. arXiv preprint arXiv:2204006507, 2022.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties neural networks. arXiv preprint arXiv:1312.619% 2013.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard. Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8769-8778, 2018.
Jianxiong Xiao, James Hays, Krista Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Lcage scene recognition from abbey to 700. In 2010 IEFE computer society conference on computer vision and pattern recognition, pp. 3485-3492. IEEE, 2010.
Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv: 1504.06555 2015.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large- scale image dataset using deep learning with humans in the loop. arXiv 2015.
preprint arXiv:1506.03::
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang. Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content-rich text-to-image generation, 2022.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zeutlemoyer. Opt: Open pre-trained transformer language models, 2022. URL https:ttpp:/attt http://aaxvvvvvv. absg/bb/ 2205..00 01068
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on paltern analysis and machine intelligence, 4066):1452-1464, 2017.
12

---

Published as a conference paper at ICLR 2023
APPENDIX: EXTREMELY SIMPLE ACTIVATION SHAPING FOR OUT-OF-DISTRIBUTION DETECTION
A FULL RESULTS ON ASH PLACEMENTS
The main results shown in the paper are generated by placing ASH at later layers of the network: the penultimate layer for ResNet-50 and MobileNet. Here we show results if place ASH at different locations the network: 1st, 2nd, 3rd and 4th Block of ResNet-50, at the end of the last convolution layer at each block before the activation function (ReLU). Figure (bottom) ( illustrates such placements, along with accuracy degradation curves (top). Here we show the OOD detection results from these ASH placements in Table
As we can see, indeed the penultimate layer placement gives the best result in terms of ID accuracy preservation and OOD detection. As we move ASH towards the beginning of the network, both accuracy and OOD detection rates degrade.
worth noting that results in Table adopt different version of ASH for each layer block. For the penultimate layer and 4th Layer, we use ASH-S @90 (the same setting as that in the main paper), while for 1-3 Layers we use ASH-P@90. The reason that ASH-S drastically degrades the performance. By simply adding scaling factor (changing from ASH-P ASH-S) on Layer 3, the ID accuracy drops from 75% to 5%.
ID: ImageNet; OOD: Naturallst, Places, Textures, Sun
ASH placement
FPR95
AUROC t AUPR
ID ACC
ASH-P@90 before last ReLU of 1st Block
93.55
59.46
89.89
28.57
ASH-P@90 before last ReLU of 2nd Block
70.45
81.45
95.93
62.78
ASH-P@90 before last ReLU of 3rd Block
63.38
85.83
96.91
75.36
ASH-S@90 before last ReLU of 3rd Block
98.51
41.39
83.59
5.21
ASH-S@ 70 before last ReLU of 3rd Block
97.72
49.50
87.27
19.32
ASH-S@90 before last ReLU of 4th Block
34.69
92.11
98.38
75.83
ASH-B@90 before last ReLU of 4th Block
33.74
92.37
98.48
75.70
ASH-S@90 after penultimate Layer*
22.80
95.12
98.90
74.98
No ASH (Energy alone)
58.41
86.17
96.88
76.13
Table ASH applied different places throughout ResNet-50, trained on ImageNet. OOD results are averaged across 4 different datasets/tasks: iNaturalist, Places, Textures, Sun. 1 indi- cates larger values are better and 4. indicates smaller values are better. All values are percentages. Experimental setup is described in Section
B DETAILED CIFAR RESULTS
Table 7 and Table 8 supplement Table 2 in the main text, as they display the full results on each of the OOD datasets for models trained CIFAR-IO and CIFAR- 100 respectively.
C ID-OOD TRADEOFF FOR ADDITIONAL ARCHITECTURES
While Figure 2 depicts the ID-OOD tradefoff for ImageNet dataset on ResNet-50 architecture, we supplement the figure with an additional architecture: MobileNetV2, whose OOD results are included in T As we can see in Figure both ASH-S (green) and ASH-B (blue) lines offer superior ID accuracy preservation, while offering comparable OOD performances with state the art (DICE+ReAct). Note that while on average, for this particular architecture, ASH is not outperforming DICE+ReAct on OOD metrics, it still offers number of advantages: algorithmically simpler, much lighter turning effort, precomputing cost, and preserving IDD accuracy.
13

---

Published as a conference paper at ICLR 2023
Places365 Zhou et al, 2017 LSUN- Crop (Yu et al
2015] LSUN- Resize et al. 2015). and ISUN [Xu et al. 2215). For each ID dataset, we use the same DenseNet pretrained on CIEAR- indicates larger values are better and indicates smaller values are better.
AUROC
95 24.24...42
92. 92.46
93.71
89 15
94.57
94.67
95 .2
96.02
96.61
Average
FPR95
20.83
48.73
24.57
34.25
31.42
26.55
23.45
20.23
15.05
89.15 13 20a3
AUROC
88.57
88.55
77.18
63.15
91.64
91.03
91.76
89.93
88 34
Places365
08.59tt..0
FPR95
63.02
52.85
39 39.40
40.44 40.46
48. 18.47
85.
48
Table Detailed results on six common OOD benehmark datasets: Textures Cimpoi et al. 2014 SVHN (Nezerr et al. 2011]
AUROC
K2.38
92.15
86.43
92.47
88 18118
K8.29
92.88
95.09
Textures
88,
441
FPR95
64.15
57 17.50
21.1
$6.12
53
50 KS
35.73
24
99 14 10.15
AUROC
94.52
98 90
91.75
25
98.01
9R 29
45
98 90
ISUN
EPROS
4.36
42.31
3,98
9.78
10.07
8.46
8.59
5.17
99.20 99.20
AUROC
94.51
99.02
97.09
98.12
98.33
98.54
98.92
LSUN-r
3.91 10.36
FPR95
42.10
3.09
9.14
9.28
11.46
7.97
8.13
4 .6
99 16.92*.00 1042
LSUN-c AUROC
95 54
98.86
86 96
99.15
99.34 34
99.48
99 73
98
0,26 11-1
FPR95
33.57
4.70
17.58
56.55
3.81
5.6
2,82
2.52
0.90
95.90 11.90 LM
AUROC
93.48
94.57
98.31
93.99
95. 29
96.86
98.6 65
SVHN
FPR95
47.24
25.29
6.08
6.42
40.61
30.14
17.92
6.51
25
Mahalamobis
Energy same
ASH-P ASH-P (Ours)
ASH-B (Ours)
Method
GODIN
ODIN
ReAct
DICE
ASH-S
14

---

Published as a conference paper at ICLR 2023
Places165 (Zouu et al, 2017] LSUN- Crop (Yu et al. 2015). LSUN-Resize Yu et al, 2015 and iSUN (Xu et al. 22055] For each ID dataset, we use the same DenseNet pretrained on CIEAR- 100. 1 indicates larger values are better and indicates smaller values are better.
AUROC
67.23tm.33
74,36 36.36
44.49
05.24
82.75 73
81.19
K2.77 21
83.04
90,02
Average
49,72
FPR95
80.13
58.14
55.37
6844
62.25
64.53
48.73
41.40
77.26*1.0 11M8
AUROC
74.31
76.84
61.39
77.72
77.26 T7.26
77.94
72.46
71.62
Places365
79. 09.88220 1234
FPR95
82.55
82.32
95.33
92.66
79.20
82.65
77.01
84.52
85 86
Table Detailed results on six common OOD benchmark datasets: Textures Cimpoi et al. 2014 SVHN (Neterr et al, 2011
AUROC
76.42
71.48
71.48
76.72
79.39
71.03
74,09
84.25
92.35
Textures
65.04 15.04hmm 2MAM
FPR95
84.79
82.34
62.39 39
84.15
77.78
78:24
53.71
34.02
90.08 11336
AUROC
70.17
83.84
93.21
78,95
86.55
80.81
91.09
91.30
ISUN
48.72 18.72-1.. 1.
FPRO5
85.99
67.05
31.38
74.54
65.27
70.90 90
47.82
46.67
91.04 11.04 11.40
AUROC
69.1 18
84.22
92.61
94.20
80 14
87 57.88
81.67
91.03
90. 12
LSUN-r
FPR95
85.24
65.22
23.07
70.65
49.40*1
67.56
48.38
51.33
LSUN-c AUROC
99 7.744900
85 60
97. 93
89.55
96.30
97.43
97.89
99. 17
98 94
0.93 KUI
FPR95
60.49
10.54
43.15
08.90
14.72
25 25.55
11,60
4.46
5.52
14.30.
AUROC
55.40
92.65
95.67
81.85
83.88 .6
90.27
95.76 76
SVHN
54.65 34.65-145
FPRO5
$1.70
41.35
36.74
22.44
87.46
$1.86
33.52
ASH-S 25.02
Mahallanbbis
Energy SaAR
ASH-P ASH-PPOOUN) (Ours)
ASH-B (Ours)
Method
GODIN
ODIN
ReAct
DICE
15

---

Published as a conference paper at ICLR 2023
Average COD AUROC (acrres taslos)
90-
D
48
E
ENERGY
REACT
DICE
84
D+R
DICE REACT
65- 90%
ASILS B5 90%
ASI-B 65 0076
ID Accuracy
Figure 6: ID-OOD tradeoff MobileNet architecture. Plotted are the average OOD detection rate (AUROC; averaged across 4 OOD datasets iNaturalist, SUN, Places365, Textures) vs ID classification aceuracy (Top-1 accuracy in percentage detection methods and their variants used in this paper on MobileNet architecture. Baseline methods "E" and "S" lie on the upper bound ID accuracy (indicated by the dotted gray line) since makes no modification of the network the features. "R", "D" and "D+R" improve on the OOD metric, but come with an ID accuracy drop. ASH (dots connected with dashed lines; smaller dots indicate lower pruning level) offers the best trade-off and form Pareto front.
ImageNet validation set) of all OOD
D MORE EXPERIMENTS ON GLOBAL VS LOCAL THRESHOLD
Data used to generate Figure can be seen in Table 9 Detailed results comparing local and global thresholds with a range of pruning percentage values and on three metries. Alternttivly, the comparison of global and local thresholds with ASH-B can be found in Table I0
Local threshold
Global threshold
Method
FPR95
AUROC
AUPR
FPR95
AUROC f
AUPR
1
t
ASH-S@99
40.49
88.23
97.01
44.24
88.94
97.37
ASH-S @98
34.72
90.30
97.54
45.03
89.53
97.64
ASH-S @97
30.88
91.77
97.93
46.99
89.41
97.63
ASH-S $99
28.34
92.94
98.26
48.53
89.17
97.57
ASH-S 9.
25.97
93.75
98.49
49.94
88.87
97.51
ASH-S@94
24.56
94.33
98.66
51.11
88.60
97.44
ASH-S@93
23.45
94.70
98.76
52.11
88.38
97.39
ASH-S@92
22.82
94,94
98.83
53.00
88.17
97.34
ASH-S@91
22.88
95.06
98.87
53.85
87.95
97.29
90H-S@90
22.80
95.12
98.90
54.74
87.75
97.25
ASH-S@85
24.28
94.85
98.86
58.28
86.97
97.07
ASH-S@80
26.59
94.33
98.75
60.84
86.42
96.96
ASH-S@75
29.32
93.82
98.64
63.16
85.98
96.88
ASH-S@T @70
31.64
93.33
98.53
64.68
85.66
96.83
Table 9: Global vs local thresholds: ASH-S. Shown ASH-S results with ResNet-50 trained ImageNet. Local threshold perform consistently better then global threshold.
16

---

Published as a conference paper at ICLR 2023
Local threshold
Global threshold
Method
FPR95
AUROC
AUPR
FPR95
AUROC
AUPR
1
t
t
ASH-B @99
45.43
89.09
97.55
41.70
89.75
97.55
ASH-B @98
39.59
91.22
98.08
41.64
90.57
97.87
ASH-B @97
36.54
92.17
98.30
42.73
90.57
97.88
ASH-B @96
34.26
92.79
98.44
43.55
90.43
97.85
ASH-B@95
32.64
93.20
98.52
44.38
90.20
97.79
ASH-B@9
31.09
93.51
98.59
45.39
89.99
97.75
ASH-B @93
30.03
93.76
98.64
45.77
89.86
97.72
ASH-B @92
29.01
93.95
98.68
46.29
89.71
97.68
ASH @@91
28.43
94.10
98.71
46.97
89.55
97.64
ASH 8
27.58
94.24
98.74
48.25
89.39
97.61
ASH-B@8S
25.26
94.65
98.83
51.04
88.77
97.48
ASH-B 880
24.04
94.88
98.88
53.63
88.24
97.37
SSH-@@75
22.95
95.02
98.91
56.66
87.63
97.25
ASH-B B @7
22.39
95.08
98.93
60.19
86.93
97.12
Table 10: Global vs local thresholds: ASH-B. Shown ASH-B results with ResNet-50 trained on ImageNet. Local threshold perform consistently better then global threshold.
E INTUITION AND EXPLANATION OF ASH
ASH post hoc regularization ASH can be thought of as simple "feature cleaning" step, a post hoc regularization of features. As we all now acknowledge that NNs are overparameterized. consequently, can think of the representation learned by such networks likely "overrepresentee." While the power of overparameterization shines mostly through making the training easier easier-adding more dimensions the objective landscape while the intrinsic dimension of the task hand remains constant (Li al. 2018), we argue that from the lens of representation learning, overparameterized networks "overdo" feature representation, i.e. the representation produced of an input contain too much redundancy. We conjecture that simple feature cleaning step post training can help ground the resulting learned representation better. Future work validate, or invalidate this conjecture would include testing if simplified regularized representation works well in other problem domains, from generalization, transfer learning and continual learning.
Connection to modified ReLU Another lens with which to interpret ASH is to see it as modified ReLu function, adapted on-the-fly per input sample. Since we operate on feature activations pre-ReLU, as shown in Figure 7 the most basic version, ASH-P, combined with the subsequent ReLU fuction, becomes simply an adjusted ReLU. Since the cut-off determined per-input and on-th on-the-fi the-fly. it is essentially an data-dependent activation function. The success of ASH highlights the need to look into more flexible, adaptive, data-dependent activation functions inference.
Magnitude vs value in pruning Lots of existing pruning methods rely on the magnitude of number (weights or activations). However in this work we use the direct values. That is, large negative will be pruned if it is within the p-percentile of value distribution. The reason is that we operate on activations either before ReLU, in which case all negative values will be removed subsequently, or on the penultimate layer where all values are already non-negative.
How ASH changes the score distribution One perspective interpret the effectiveness of ASH is to examine how it morphs the score distribution to maximize separation between ID and OOD. Since we default using Energy as the score function, change with different strength of ASH treatment, in Figure From no ASH treatment (pruning: 0%) from the left plot to full treatment (90%) on the right, we can see how both ID and OOD distributions are morphed to maximize separation.
demonstrate how the energy score distributions
Forr animated plots, see: https:://ante jazz wtw. htwp./giichub..oaa////ii...
17

---

Published as a conference paper at ICLR 2023
ReLU
ASH: Modified ReLU
Figure 7: ASH modified ReLU. Comparison of a regular ReLU activation funciton (left) with modified ReLU (right), which is equivalent to the ASH-P operation. The pruning threshold is input data dependent, which makes ASH-P equivalent modified ReLUU with adaptive thresholds.
Pruning percentage: 0% loU: 19.05%
Pruming percentage: 50%
Pruming percentage: 90%
600
600
600
IoU: 12.50%
loU: 7.97%
500
500
500
400
400
400
Frequency
ID
300
300
300
OOD
200
200
200
100
100
100
10
20
40
50
100
200
400
600
Energy score
Energy score
Energy score
Figure 8: Energy score distributions. ASH with increasing pruning strength from left to right morphs the distributions of ID and OOD data, improving the separation, indicated by the Intersection Over Union (IoU) measure. ID data is ImageNet, and OOD data iNaturalist. Shown are energy scores without ASH (left), with ASH-S at 50% pruning strength (middle) and ASH-S at 90% (right).
NOTES ON IMPLEMENTATION
All experiments are done with NVIDIA GTX1080Ti GPUs. Code to reproduce results is submitted alongside this appendix.
Notes ImageNet results (Table 2
For both ResNet and MobileNet results, "ASH-P (Ours)" is implemented with p 60, "ASH-B (Ours)" is implemented with p 65, and "ASH-S (Ours)" is implemented with p 90.
For the reimplementation of DICE with MobileNet, used DICE pruning threshold of 70%.
For reimplementing DICE ReAct on MobileNet, since DICE and ReAct each come with their hyperparameter,
tried grid search, where DICE pruning thresholds include {10% 15%, 70%) } and ReAct clipping thresholds {1.0, 1.5, 1.33). Rationals for choosing those values for grid search are: 10% and 15% are taken from the hyperparameter setup in
18

---

Published as a conference paper at ICLR 2023
the DICE codebase" and 70% is the recommended threshold of DICE when used alone. In the case of ReAct clipping thresholds, 1.0 and 1.5 are taken from the same codebase, while 1.33 is the 90% percentile calculated from training data, following the ReAct procedure. We report the best result out of all the hyperparameter combinations, given by 10% DICE pruning and 1.0 ReAct clipping.
Notes CIFAR results (Table
Table
Table 8
For CIFAR-10, 'ASH-P (Ours)" is implemented with p = 90, "ASH-B (Ours)" is imple- mented with p 95, and "ASH-S (Ours)" is implemented with p 95.
For CIFAR-100 'ASH-P (Ours)" implemented with p 80, "ASH-B (Ours)" is imple- mented with 85, and "ASH-S (Ours)" is implemented with = 90.
Results for other methods are copied from DICE.
Notes on compatibility results (Table 5)
"Softmax score + ASH" is implemented with ASH-P @ 95 for CIFAR- 10, ASH-B @ p 70 for CIFAR-100, and ASH-B @ p 65 for ImageNet.
"Energy score ASH" is implemented with ASH-S @ 95 for CIFAR-I0, ASH-S @ p == for CIFAR- 100, and ASH-S p = 90 for ImageNet.
"ODIN" and "ODIN ASH" pretrained DenseNet-101 for CIFAR, where the magni- tude parameter is 0.0028, and pretrained ResNet-50, where the magnitude parameter is 0.005.
"ODIN ASH" is implemented with ASH-S @ p 95 for CIFAR-10, 10, ASH-S @ p 90 for CIFAR-100, and ASH-S @ = 90 for ImageNet.
"ReAct" results CIFAR- and CIFAR-10 100 are implemented by us. are unable to replicate the exact results shown in the DICE paper supplementary Table 9.
"ReAct ASH" is implemented with ASH-S @ p 90 and ReAct clippign threshold of 1.0 for all of CIFAR-10, CIFAR-100, and ImageNet.
G ADDITIONAL ARCHITECTURES
In Table we used architectures (ResNet50 and MobileNetV2) for the ImageNet experiment. We included 2 more here in Table II VGG16 (Simonyan & Zisserman 2014) and DenseNet-122 (Huang ct al. 2017).
OOD Datasets
Model
Methods
INaturalist
SUN
Places
Textures
Average
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR9S
AUROC
Energy score
ASH- (Ours)
15.53
97.03
37.14
91.53
46.50
88.79
22.04
95.01
30.30
93.09
DenseNet- 121
ASH- (Ours)
34.05
92.28
42.98
89.11
55.09
84.90
56.21
83.10
47.08
87.35
ASH-B@90 (Ours)
18.22
96.36
35.17
92.48
45.38
89.15
22.75
95.38
30.38
93.34
51.35
90.30
57.54
87.55
64.20
84.83
44.24
89.98
54,33
88.17
ASH-B@
(Ours)
92.15
VGG-16
ASII$$90 (Ours)
47.14
91.34
55.20
88.41
61.58
85.0 $5.92
44.43
89 96
$2.09
88.91
ASH (Ours)
38.36
92.98
47.41
94.02
54.63
87.
44.87
90.47
ASH-S@99 (Ours)
29.30
93.18
43.59
90.17
49.46
87.79
43.60
86.36
41.49
89.38
Table 11: OOD detection results with other architectures on ImageNet. We follow the exact same metrics and table format as Sun et al. (2021). Both DenseNet-121 and VGG-16 are pretrained on ImageNet- 1k. indicates larger values are better and indicates smaller values are better. All values are percentages. For DenseNet-122 121, ASH-S @90 and ASH-B@R 66 have the best results. In the case of VGG-16, ASH-B@63 consistently perform better than benchmarks, across all the OOD datasets.
httpss://itthee http://..coomdeeeeeamm.a wttp///cc//////cccccccccccccp/
ce/bl / 80088 9cc970. addee879fe74bb2
demo www.mnttt........m.mmm..........
19

---

Published as a conference paper at ICLR 2023
H ADDITIONAL SCALING FUNCTIONS
ASH-S can be used with different scaling functions. Table 12 shows comparison of ASH-S perfor-
mance when used with Linear (f(z) = and Exponential scaling (f(x²) exp* functions.
We observed best performance when used with Exponential function.
OOD Datasets
Scaling functions
INaturalist
SUN
Places
Textures
Average
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95 L
AUROC
1
Lineur (AS
23.68 11,49
44.03
90.80
54.92
87.90
24.24
94.55
36.72
92.31
Exponential (ASH-S@90)
97.87
27.98
94.02
39.78
90.98
11.93
97.60
22,80
95.12
Table 12: Comparison of different scaling functions used with ASH-S.
I CIFAR 10 vs CIFAR 100
We used CIFAR 10 and CIFAR 100 as ID datasets in our experiments, while using six other, rather different datasets (SVHN, LSUN C, LSUN R, iSUN, Places365, Textures) as OOD. What happens if CIFAR 10 and CIFAR 100 two similar datasets but differ in complexity and OOD? Would ASH methods still work this case? We run ASH-S experiments with varying thresholds; results are seen in Figure We can see that ASH still works in this rather difficult case. are only including simple baselines like Energy and Softmax here.
are used as ID
92
OOD AUROC (CIFAR-10)) 100)
90-
B
ENERGY SOFTMAX
8
90%
34.0
94.2
94.4
96.6
94.8 96.0
ID Accuracy (CIFAR-10)
Figure 9: ID-OOD tradeoff on DenseNet-101 architecture. OOD detection rate (AUROC: on CIFAR-100) vs ID classification accuracy (Top-I accuracy in percentage on CIFAR- 10 validation set) of ASH-S variant used in this paper. Baseline methods "E" and "S" lie on the upper bound of ID accuracy (indicated by the dotted gray line) since makes no modification of the network or the features. ASH-S (dots connected with dashed lines; smaller dots indicate lower pruning level) improves OOD detection and comes with slight drop accuracy.
J ASH IMPROVES KNN FOR OOD
In 5 we showed that the ASH treatment improves upon existing methods: Softmax score, ODIN, energy score, and ReAct. In this section we experiment adding ASH to methodologically different detector, the K Nearest Neighbor (KNN). Following the setup in (Sun et al. 2022), we add ASH to a ResNet18, trained on CIFAR- 10. The OOD datasets SVHN, LSUN, iSUN, Textures, Places365 (same as used (Sun et al. )) final layer that generates feature vectors used for clustering, placed ASH on the layer before that,
). Since the original "penultimate layer of ResNet18" is now the
20

---

Published as a conference paper at ICLR 2023
right after the 4th Block of the ResNet18. As in Table L improves upon the baseline (KNN only) for a series of pruning percentages, more reliable for ASH-S than ASH-B.
KNN with ASH treatments
OOD Datasets
Method
SVHN
LSUN
ISUN
Textures
Places365
Average
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR95
AUROC
FPR05
AUROC
FPR95
AUROC
1
Baseline (KNN only)
27.97
95.48
18.50
96.84
24.68
95.52
26.74
94.96
44.56
90.85
28.49
94,73
ASH-Se65
27.38
95.55
96.96
34.68
95.53
26.45
95.00
44.42
94.78
26.26
ANH-8
25,86
V5.70
15.97
97.18
24.67
95. 90
26.22
94.97
43.46
91.04
27.24
94.88
ASH-S@80
25.06
95.79
14.84
97.31
25.02
95.38
26.21
94.88
43.35
91.10
26.90
94.89
ASH-S088
ASHI-5@90
24.26
95,93
13.49
97.53
30.11
94.59
28.09
94,34
44.97
90.87
28.18
94,65
ASH-B6-4
11.21
97.70
4.38
99.11
27.
95. 48
24.02
94.99
40.72
92. 22
21.53
95.90
ASH-B
ASH-B669
49.13
92.78
27.25
95.80
31.78
94.83
31.90
94.01
47.64
37.54
93.58
Table 13: ASH improves upon KNN for OOD detction. We add ASH treatments to the KNN algorithm for OOD detection, following the setup in (Sun et al. 2022). The network is a ResNet- 18 pretrained on CIFAR-10. ASH-S with pruning percentages for 65 90 consistently show improvements. ASH-B with a lower percentage (40) greatly improves upon the benchmark, while larger percentages do not.
K DIFFERENCES BETWEEN ASH AND DICE
Here we highlight the key differences between ASH (our work) and DICE (Sun & Li 2022), in both performance and methodology:
(Performance) ASH overall performs much better, and does not come with an ID accuracy degradation (Figure 2
2. (Performance) DICE alone does not perform that well, but only when combined with ReAct, when doubles the overhead of hyperparameter tuning
3. (Methodology) ASH is completely on-the-fly, and does require precomputation as DICE does
4. (Methodology) ASH make no modification of the trained network, with no necessary access training data
We consider DICE concurrent work as opposed to a precursor, which ASH comprehensively outperforms, albeit being philosophically similar.
L CALL FOR EXPLANATION AND VALIDATION
We are releasing two calls alongside this paper to encourage, increase, and broaden the reach of scientific interactions and collaborations. The two calls are an invitation for fellow researchers to address two questions that are not yet sufficiently answered by this work:
What are plausible explanations of the effectiveness of ASH, simple activation pruning and readjusting technique, on ID and OOD tasks?
Are there other research domains, application areas, topics and tasks where ASH (or a similar procedure) is applicable, and what are the findings?
Answers to these calls will be carefully reviewed and selectively included in future versions of this paper, where individual contributors will be invited collaborate. For each call we provide possible directions to explore the answer, however, we encourage novel quests beyond what's 's suggested below.
Pleasee follow instructions when submitting to the calls: https://andri wwr..gitthbe caluu..o//aaa o//as/
#ea 11s
21

---

Published as a conference paper at ICLR 2023
Call for explanation A plausible explanation of the effectiveness of ASH is that the knowingly overparameterized neural networks likely overdo representation learning generating features for data that are largely redundant for the optimization task at hand. It is both an advantage and peril: on the one hand the representation less likely to overfit to a single task and might retain more potential to generalize, but the other hand it serves poorer discriminator between data and unseen.
Call for validation in other domains We think adjacent domains that use deep neural network (or any similar intelligent systems) to learn representations of data when optimizing for training task would fertile ground for validating ASH. straightforward domain is natural language processing, where pretrained language models are often adapted for downstream tasks. Are native representations learned with those large language models simplifiable? Would reshaping of activations (in the case of transformer-bes based language models, activations can be keys, values queries) enhance damage the performance?
22


Reviews:
[{"review_id": "byD4Bv0uMAV", "reviewer": "Reviewer_kqRP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focuses on detecting OOD samples in the inference time by pruning a large portion of an input sample's activation and lightly adjusting the remaining. The method can be easily combined with previous OOD scores (like MSP, ODIN, Energy and ReAct). When combined with the energy score, it shows a significant improvement in OOD detection, on both moderate and large-scale image classification benchmarks.", "summary_of_the_review": "The paper focuses on detecting OOD samples in the inference time by pruning a large portion of an input sample's activation and lightly adjusting the remaining. The method can be easily combined with previous OOD scores (like MSP, ODIN, Energy and ReAct). When combined with the energy score, it shows a significant improvement in OOD detection, on both moderate and large-scale image classification benchmarks. However, the motivation of the proposed method is not clear and not convincing.", "strength_and_weaknesses": "Pros:\n\n1. The paper is well-written.\n\n2. The method is simple and easy to reproduce.\n\n3. The experiments are sufficient and performance are quite well on both imagenet and cifar benchmark.\n\n\nCons:\n\n1. In algorithm ASH-S, I think the scaling factor exp(s1/s2) is a hyperparameter. Are there other functions that can replace the exponential function? In other words, when the application scene changes, do the authors need to replace the exponential function to adapt to the current scene to obtain the optimum?\n\n2. The paper designs three algorithm and mostly ASH-S performs best (like in Table 1 and Table 2). Sometimes, ASH-B performs better than others, like Table 1 (MobileNet). So I wonder how to choose the algorithm when meeting a new dataset and a new model?\n\n3. As shown in Table 1, when the model is ResNet, ASH-S can achieve 95.12% AUROC and 22.18% FPR95. But when the model is MobileNet, the method can not outperform the SOTA. I think the generalization of these algorithms is not very good, and when the model changes, the performance difference is too large.\n\n4. The placement in Figure 5 and the results in Table 6 (Appendix A) show that the method can not work when it is deployed in 1-3th Block of ResNet. The result in 4 Block is not good enough and the highest performance is in the penultimate Layer. I think the results show that the method only works in the penultimate Layer, and the generalization of insertions in other positions in backbone is extremely poor.\n\n5. The placement of the method is in the penultimate Layer, which is the same with DICE. The method discards some features at the penultimate layer, which is actually partially equivalent to dropping out some connection weights at the penultimate layer (DICE's approach). So when this method is deployed to the penultimate layer, I think it has a high similarity to DICE.\n\n6. The theoretical proof of why the algorithm works is insufficient, as the operations of Algorithms 2 and 3 seem to be the result of empirical design rather than rigorous theoretical derivation. The authors may consider improving the interpretability and theoretical basis of this article.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "KDWq23wk_0", "reviewer": "Reviewer_Pwvn", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": " The authors hypothesize that over-parameterized DNNs extract too many features and can prune a large proportion of the features without drastically reducing ID accuracy, while significantly improving the OOD detection task. They propose a simple activation method to validate this hypothesis.  The proposed method prunes a large portion of a sample’s activation and rescaling the rest activation. The empirical evaluation on ImageNet shows that the improvement of the proposed methods is significant.", "summary_of_the_review": "I recommend acceptance. The experimental evidence is strong.", "strength_and_weaknesses": "Strength:\n\n- The proposed method is simple and effective.\n- The paper is well written.\n- This work presents extensive empirical evaluations.\n\nWeakness:\n\n1. The choice of hyperparameter $p$ is mentioned in both the main text and the appendix. The authors did not discuss how the ID influences the choice of $p$. If only ID data is available, I think the hyperparameter selection scheme is important. \n2. If the OOD detection task is CIFAR10 vs CIFAR100, does the ID-OOD trade-off still hold?  \n3. Table 5 shows that ASH improves on existing methods. Does ASH improve KNN? Does the choice of $p$ have a significant effect on the performance of KNN+ASH?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "DWQYYhFVUZ9", "reviewer": "Reviewer_CKHN", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is about weight pruning in a single layer and its relationship with out of distribution detection performance. The authors propose to prune activations at a single layer using the top p percentile, and candidate activations can either be set to zero (pruned), scaled, or binarized. Then the well known energy score is used to produce a score for out of distribution detection. This produces state of the art performance in out of distribution detection in all tested benchmarks.\n\nThe contributions of this paper are:\n- A post-hoc and simple method for OOD detection based on activation shaping/sparsification.\n- New state of the art results on three ID datasets and ten OOD datasets, including ImageNet, CIFAR10, and CIFAR100 as ID datasets over two different network architectures.\n- A good evaluation and ablation results that justify some of the chosen hyperparameters and activation shaping\n- A good study of the effect of activation shaping/sparsification and its relation to OOD performance. I think previous work only focuses on weight pruning/sparsification.\n", "summary_of_the_review": "This is overall a good and high quality paper, the proposed method is a good improvement for out of distribution detection without requiring large modifications to the model or as training procedure. The proposed method is well studied, with many comparisons to the state of the art in a good selection of in-distribution and out-of-distribution datasets (including ImageNet and CIFAR10/100), and an excellent selection of ablations to justify the decisions made by the authors.\n\nThe only issues I find in the paper are minor, I mentioned them in weaknesses, mostly about intuition or analysis of why activation sparsification improves OOD performance, and some effects of network architecture on ASH performance.\n\nAfter rebuttal, I can confirm that this paper should be accepted as the paper has improved.", "strength_and_weaknesses": "Strengths\n- The paper is very well written, it is easy to understand, I have no more comments about writing.\n- The proposed technique is very simple to understand and implement, prune activations with a threshold given by the p percentile, decide how to remove these activations, and then use the energy score to output a score for out of distribution detection.\n- Post-hoc methods are preferable, as there is no need to retrain the model or use special training methods, and this method can be applied to any pre-trained model.\n- The evaluation is solid, using multiple models, multiple ID (3) and multiple OOD datasets (10), multiple metrics (AUPR, AUROC, FP95), and a good selection of baselines, including many post-hoc methods. I do not have doubts about the validity of these results and conclusions. I think the only possible missing baselines are uncertainty-based method (Ensembles, Dropout, DUQ, etc), but I am not sure if these make sense given the post-hoc use of the proposed method.\n- I really like Figure 2, it shows the trade-off between in-distribution accuracy and out of distribution performance, with state of the art methods and the proposed method at different activation pruning thresholds, clearly showing the advantage of the proposed method, but also showing how these methods compare by themselves. It is known that OOD detection methods might reduce performance but it is the first time I see such comparison. It is clear from this result that the proposed method has the best ID/OOD performance trade-off (best in both accuracy and AUROC).\n- There is an improvement in out of distribution performance as measured by the AUROC and AUPR on ImageNet, in all benchmarks using ResNet, and in most benchmarks using MobileNets. For CIFAR10/100 using DenseNet, there is also a large improvement in all comparisons.\n- There is a good set of ablation results to show the effect of some hyper-parameters, in particular the p percentile used as a threshold to shape/sparsify the activations, and also on which layer this method should be applied, which justifies the final decision made by the authors.\n\nWeaknesses\nAfter rebuttal, I  see no weaknesses.\n\n~~- I think there is no theoretical justification provided in the paper on why this method works. By this I mean the paper does not do a theoretical analysis (which is fine), but the paper does not provide an intuition on why the method works. There is a good improvement in most out of distribution detection benchmarks, but then there is the question, why? Why does pruning or shaping activations over the p percentile improves OOD performance? Maybe the authors can give some intuition that guided the construction of the proposed method.~~\n\n~~- I see that in Table 2, when using MobileNet, DICE outperforms your method in most benchmarks, I think this deserves some analysis or acknowledgment, it seems that the proposed method has some degree of dependency with the network architecture. Could the authors comment on this?~~\n\nMinor Issues\n- I think there is a double blind leak in page 18, in a footnote, I do not include this link here but this should not happen, I think the link is not anonymized and leaks some information about the authors.\n\n~~- In Figure 2, please specify which OOD datasets were used to make this plot, the paper mentions four OOD datasets, but which ones?~~\n\n~~- Table 5 shows combinations of the proposed method with other OOD detection methods (very nice), but why is there an improvement between energy score and energy score + ASH? The proposed method ASH already uses the energy score, this combination seems very strange to me. Please clarify.~~", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "BT9mIyijR_", "reviewer": "Reviewer_UzHM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work presents a post-hoc activation shaping strategy for improved out-of-distribution (OOD) detection. The technique mainly consists of pruning the activation vector for a given sample in a chosen particular layer (favorably the penultimate layer). The paper proposes three algorithmic variants for the purpose of pruning and shaping the activations. The work is supported by empirical results on standard benchmark datasets (CIFAR-10/100 and ImageNet-1k) and detailed ablations. \n", "summary_of_the_review": "This work proposes a simple-to-use algorithm to prune activations in a post-hoc fashion to improve OOD detection. The method is empirically supported through experimental results on standard benchmarks but lacks reasoning and motivation behind the working principle of the algorithm. Hence, I recommend borderline reject. However, I am willing to increase my score based on the rebuttal.\n", "strength_and_weaknesses": "Strengths:\n\n1. Well written and easy to follow.\n2. Empirical analysis supports the effectiveness of the proposed algorithms.\n3. Simple post-hoc idea based on activation shaping.\n\nWeakness:\n\n1. Although the empirical results are quite significant, the paper lacks a proper justification for why simply shaping the activations provides an improvement in OOD detection. Previous works [1] and [2] have already shown that pruning the activations provide an improvement in OOD detection. To make the paper stronger, the authors should provide a clear justification on how scaling the activations is further helping.\n2. In Algorithms 2 & 3 (ASH-B, ASH-S), after pruning non-negative values are scaled to some constant value. However, proper reasoning on how this constant is chosen is missing. Can the constant be chosen as some arbitrary value and still get similar OOD performance?\n\n3. It is not clear how the value of the hyper-parameter p is chosen. I shall suggest the authors to provide more detailed explanation on this. \n\n4. It would be also interesting to see the performance on Hard OOD detection tasks such as CIFAR-100 vs CIFAR-10. Currently, the paper mainly shows results on OOD datasets that are significantly different from ID data and easier to detect.\n\nAlthough not a major concern, the related works section can be improved by adding some recently proposed post-hoc works such as [3].\n\n[1] Sun, Y., Guo, C. and Li, Y., 2021. React: Out-of-distribution detection with rectified activations. In NeurIPS 21.\n\n[2] Sun, Y. and Li, Y., 2022. Dice: Leveraging sparsification for out-of-distribution detection. In ECCV 22.\n\n[3] Sun, Y., Ming, Y., Zhu, X. and Li, Y.. Out-of-distribution Detection with Deep Nearest Neighbors. In ICML 22.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster