Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detection
Authors: Qinyu Zhao
Ming Xu
Kartik Gupta
Akshay Asthana
Liang Zheng
Stephen Gould
Venue: ICLR 2024 poster (2024)
Keywords: feature shaping
out-of-distribution detection
optimization problem
generalizability
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We develop an optimization framework for feature-shaping methods in out-of-distribution (OOD) detection that not only elucidates the workings of previous methods, but also paves the way for a more resilient approach.

Abstract: Feature shaping refers to a family of methods that exhibit state-of-the-art performance for out-of-distribution (OOD) detection. These approaches manipulate the feature representation, typically from the penultimate layer of a pre-trained deep learning model, so as to better differentiate between in-distribution (ID) and OOD samples. However, existing feature-shaping methods usually employ rules manually designed for specific model architectures and OOD datasets, which consequently limit their generalization ability. To address this gap, we first formulate an abstract optimization framework for studying feature-shaping methods. We then propose a concrete reduction of the framework with a simple piecewise constant shaping function and show that existing feature-shaping methods approximate the optimal solution to the concrete optimization problem. Further, assuming that OOD data is inaccessible, we propose a formulation that yields a closed-form solution for the piecewise constant shaping function, utilizing solely the ID data. Through extensive experiments, we show that the feature-shaping function optimized by our method improves the generalization ability of OOD detection across a large variety of datasets and model architectures. Our code is available at https://github.com/Qinyu-Allen-Zhao/OptFSOOD.

Published as a conference paper at ICLR 2024
TOWARDS OPTIMAL FEATURE-SHAPING METHODS FOR OUT-OF-DISTRIBUTHA DETECTION
Qinyu Zhao', Ming Xu', Kartik Gupta', A Akshay Asthana', Liang Zheng', Stephen Gould' The Australian National University
2 Seeing Machines Ltd
(qinyu. zhao, mi mingda. xu, liang. zheng, www.see...poootoeee gould] @anu edu. (kartik. sartta akshay as asthana @see pseeingmachines. com
ABSTRACT
Feature shaping refers to a family of methods that exhibit state-of-the-art perfor- mance for cut-of-distribution (OOD) detection. These approaches manipulate the feature representation, typically from the penultimate layer of a pre-trained better differentiate between in-distribution (ID) and OOD samples. However, existing Srature-shaping methods usually employ rules manually designed for specific model architectures and OOD datasets, which con- sequently limit their generalization ability. To address this gap. we first formulate abstract optimization framework for studying feature-shaping methods. We then propose a concrete reduction of the framework with a simple piecewise constant shaping function and show that existing feature-shaping methods approximate the optimal solution to the concrete optimization problem. Further, assuming that OOD data inaccessible, we propose a formulation that yields a closed-form solution for the piecewise constant shaping function, utilizing solely the ID data. Through extensive experiments, we show that the feature-shaping function optimized by our method improves the generalization ability of OOD detection across large variety datasets and model architectures.
deep learning model,
INTRODUCTION
Out-of-distribution (OOD) detection aims to identify test samples that fall outside the inherent training label space, given deep learning model pre-trained on an in-distribution (ID) training To detect OOD samples, OOD scores, such as maximum softmax probability (MSP) (Hendrycks. & Gimpel| 2016) and energy score (Liu et al. 2020) are computed using the logits estimated by the model, where a lower score indicates higher probability that the sample is OOD.
Feature shaping (Sun al. 2021; Djurisic et al. 2022, Xu & Lian 2023 Song al. 2022) refers to family of methods that manipulate the underlying feature representations, typically from the penultimate layer of a pre-trained model, such that OOD scores can more effectively distinguish between ID and OOD data. Representative methods like ReAct (Sun et al. 2021] and ASH-P (Djurisic ct al. 2022) employ element-wise feature clipping and percentile-based feature pruning, respectively.
Despite their notable achievements, existing feature-shaping methods usually rely on rules manu- ally designed for specific OOD datasets and model architectures, which consequently limit their generalization ability. Our observations indicate that variations in OOD datasets and model archi- tectures can cause significant performance degradation these methods. For instance, when using transformer-basee models, benchmarks, perform no better than random guessing. These observations emphasize the necessity for comprehensive understanding and comparison of the current feature-shaping methods and the development of a more generalizable approach.
approaches, despite achieving state-of-the-ant results established
To this end, we first formulate an abstract optimization problem that encapsulates all feature-shaping OOD detection methods. In nutshell, our formulation entails finding the shaping function that maximizes some performance objective for OOD detection.
Our code is available http://gtthh...oommmmm/ga/aaaaat

---

Published as a conference paper at ICLR 2024
75-
80
80
70
AUC Tracsoomee))
80 80-
ReAd
AtC
65-
60-
DIC
60-
40-
ASIE
55
50-
ASELP
ASIES,
ReAst
BFASt
ASHEP
ASHL-U
ASKS
VRA
Reli HrAs
ASH ASH-B ASHS
1 Gar
VRA
Redd
Beld
ASH
ASH-0
ASKSS
YEAR
83
AUC ConvNes)
(a)
(d)
Figure 1: Comparing our method with existing feature-shaping methods. The dashed lines denote the performance of our method for comparison. (a) ImageNet (ID) vs. iNaturalist (OOD) with ViT-B-16; (b) ImageNet (ID) vs. iNaturalist (OOD) with MLP-Mixer-B: (c) CIFARI00 (ID) vs. CIFARIO (OOD) with MLP-Mixer-Nano; (d) Average performance of different methods across eight OOD datasets with two ConvNets and with four transformer-bas based models.
To make the optimization problem concrete and tractable. propose piecewise constant shaping function and formulate an objective function that maximizes the expected difference between the maximum logit values for ID) and OOD data, assuming the availability of OOD training samples. An important benefit of this formulation is that highlights the operational mechanisms underpinning previous feature-shaping methods. Namely, the shaping functions used by existing feature-shaping methods including ReAct (Sun et al. 202T). BFAct (Kong & Li (023), and VRA-P (Xu & Lian 2023). approximate the optimal shaping function found by our approach.
Using our framework, we further propose feature-shaping method that can be optimized without access to OOD data. Specifically, through the analysis of the effect of OOD data on the optimal solution compared to ID data, we argue that it is feasible to remove the OOD-related term in the objective function, thereby deriving closed- form solution based on data only. Empirical observations in experiments support our claim.
Our experiments compare our proposed approach to existing feature-shaping methods, utilizing more comprehensive experimental setup than previous works. We use CIFAR10, CIFAR 100 (Krizhevsky (ct al. 2009), and ImageNet- 1k (Russakovsky et al. 2015) as ID datasets. For each ID dataset, we use evaluate all methods using wide variety of
eight OOD datasets for evaluation. Furthermore, models, including convolutional networks (ConvNets)) transformers and fully multi-layer perceptron (MLP) models, the latter two of which have not been extensively studied in prior works. As shown in Fig. compared with other feature-shand shappng methods, ours demonstrates more generalized OOD detection performance across broad spectrum of datasets and model architectures.
In summary, this paper covers three main points. First, we develop general optimization formulation for feature-shaping methods and show that concrete instance of this problem provides insight into the mechanisms of previous methods. Second, we propose novel approach for training a generalizable feature-shaping method for OOD detection without access OOD data. Last, our approach achieves more reliable OOD detection performance across various datasets and models.
2 A RECAP OF FEATURE SHAPING FOR OOD DETECTION
In this section, we define the problem of OOD detection for image classification, with particular emphasis on feature-shape shapien approaches, and subsequently, introduce our general optimization formulation of feature shaping for OOD detection.
2.1 OOD DETECTION FOR IMAGE CLASSIFICATION
Assume that we have an ID dataset, defined as a joint distribution D.P.y, over input observations. X' and discrete labels y for given model M. Furthermore, assume have OOD dataset DOOD. Previous works (Sun al. 2021 Sun & Li 2022 Yang et al. 2021) assume ) n n'y = -.. Inherent in this assumption is that the marginal distributions over observations for ID and OOD datasets do not coincide. That is, DID # DOOD
2

---

Published as a conference paper at ICLR 2024
The model, usually neural network, maps observations logits, M X RC, where C' = | denotes the number of classes. It is trained on dataset Stain {(x( y } comprised of N L..d. samples drawn from Dx.y- Furthermore, for typical architectures for image classification, we
can decompose the model .M- W o B into a backbone model, B X + RAM and linear classifier W IR-M -> IRC, B extracts penultimate layer feature representation E RR- of an input observation a, and W is linear layer that maps to logits & RC. Different from conventional machine learning practice, the test set Siest contains inputs sampled from the joint distribution D' DD* where D& and D denote the marginal distributions of Di,y and DX respectively.
The goal of OOD detection is to design a score function to indicate whether sample æ is from Di or D MSP, maximum logit score (MLS), and energy score serve as examples of scores for OOD detection. They are defined based on logits & as:
exp(&x)
SMSP max I.....C Σ exp(e))
MMLS(E) max li C....C
EEergy (e) log exp(l)), (I)
where & is the logit for the i-th class. For the scores discussed above, lower value indicates a higher likelihood of the sample being OOD. In practical applications, threshold. denoted by t. is utilized to
separate ID samples from OOD ones. If the score of sample surpasses this threshold, the sample will be classified as ID), and OOD otherwise.
2.2 FEATURE-SHAPING
feature- shapirngshaainggggg method uses a shaping function f RM + f=)). The goal of developing feature shaping method for OOD detection can be formulated as solving the following optimization problem,
RM manipulate features, i.e..
P { s(E)) s( 00D 0}+
maximize (over f) EgED D 10 DO0D
subject to
eDD = W(=
I [(B( 1))
(2)
2OOD_ W(O00D)
000D f(B(x
where P is given performance measure. For example, common measure is the area under the receiver operating characteristic curve (AUC), defined as PAuc(s1, s2) = 1 [s) > $2 Existing shaping functions f can be broadly categorized into two types: (i) element-wise, such ReAct (Sun) jtt al.) 202T) and VRA-P (Xu & Lian| 2023), which applies a global element- mapping to each individual feature, i.e., Σ, f(z),, (ii) phole-vector mappings such ASH (Djurisic et al. 2022 which manipulate the full feature vector x.
OPTIMAL FEATURE SHAPING
In this section, we propose a specific instance of the general formulation deseribed in Section 2 following previous element-wise feature-shaping methods (Sun et al.] 202T Xu & Lian 2023 Kong Li (023). To optimize the shaping function, we initially examine which feature value ranges are effective for OOD detection. A significant challenge arises from the continuous nature of feature values. To address this, we partition the domain of features into a disjoint set of intervals, which allows us to approximate the optimal shaping function with piece-wise constant function. With this approximation, we can formulate an optimization problem to maximize the expected difference between the maximum logit values of ID and OOD data. Subsequently, we show that the optimized shaping function using OOD samples bears a striking resemblance to prior methods, shedding light on the mechanics of these methods. Finally, propose formulation that does not require any OOD data to optimize and admits a simple, closed-form solution.
3.1 VALUE- AND INTERVAL-SPECHI FEATURE IMPACT
Given sample the maximum logit gmax (disregarding the bias term) is the dot product between the corresponding weight vector w max and its feature representation z, specifically,
M
mnax (z) = w max z
W max
(3)
3

---

Published as a conference paper at ICLR 2024
Assuming element-wise feature-shaping function R - R, define ((2) _ assuming the weights w do not change the maximum logit after shaping is:
flael
Then
mmax
f(z,)
Σ 8 w ! 2
(4)
To design (=) and understand which feature value ranges are effective for OOD detection, we compute the contribution of features with specific values the maximum logit. Concretely, define the value- speeifc feature impact (VSFI): V(=) = ΣM 1 {2: w 2,. However, is continuous variable, and real-world data only provide a countable number of features, which can gives overly sparse distributions of VSFIs. address this, we partition the domain of individual features into a disjoint set of intervals A that cover 24. Formally, the interval-speciffc feature impact (ISFI) on an interval [a, b) E A given by:
M
(z.
Σ I la S
max
Σ
u Z1.
max
(5)
Then, for single sample, the maximum logit is the summation all ISFIs, given
(2ax(2)
Σ (1a,)(z)..
(6)
(a.d)EA
In our experiments, we divide feature values into equal wiith width intervals. Given that feature distributions typically exhibit long-tailed characteristics (Sun et al. (021). we select the 0. and 99.9 percentiles of features in a training set as the lower a and upper B limits of feature values, respectively. We then partition the range [a. B) into K 100 equal-width intervals. This results in set of intervals A {lax, by) ) where ag + (A- 1)6, by the maximum logit can be expressed as:
k6, and 5 8-0 for k 1,..., K. Then,
gmax
Σ [(a,)(z)) = Σ Jo bu) (z),
(7)
[a,b]EA
and for each data sample, we can generate K-dim ISFI vector denoted as:
[(z) (F(os,b) (z), (ax. e RR
(8)
3.2 OPTIMIZING THE RESHAPING FUNCTION
We now describe how to optimize {(2), assuming ((2) is piecewise constant over the intervals A. For
an interval [os, ha), we estimate scalar 0, as constant approximation to 0(=) within the interval.
In the limit as the interval widths shrinks to zero, we recover the continuous, element-wise shaping function. We can approximate the maximum logit as,
mmax 2
While previous works (Sun al.) 2021 Xu & Lian 2023 evaluate their methods by computing the expected difference in maximum logits between and OOD data, optimizing this criteria solely does not improve OOD detection performance. To this, observe that simply uniformly scaling features by large factor increases the expected difference maximum logits without enhancing the differentiation between ID and OOD samples. This motivates introducing regularization the shaping function and leads us to the following problem
maximize (over 0) subject
Eg 0000 DOOD
0 (T= 0 (0200))
(10)
||0|
VK,
2ID
B(x,1D)
2DD OOD)
We validate this assumption in Section B.2
We constrain ||@|| to he equal ||1|||
instead of other values. Reasons found in Section A.2
4

---

Published as a conference paper at ICLR 2024
10.-
0.
8G5
-0.5-
(w/ UOD)
Ours
(w: (DD)
ReAct
BFA
VRA-P
Ours (Wio 00D)
(a)
(b)
(c)
(d)
1.3
8tzl
0.5
Ours (wi (OD)
Chrss (wr ODD)
Oues (wr (OD)
Ours (wi AND) ASH-Xand
-0.3
ASH-P
AS#-8
ASH-S
i Fesmue value
Feature value
Fenture value
Fenture value
(6)
(
(b)
Figure 2: Visualization of shaping functions. The blue lines (ours w/ OOD) derive from Eq. 10 while the green line (ours w/o OOD) from Eq. Red lines represent different existing methods, while shaded regions indicate estimated standard deviations. has been rescaled for the best visualization.
where due to the linearity expectation, the objective function can expressed as:
TT De [I(=D)]]
-0p0000 mages [T(20)))]
(11)
We extract ISFI vectors for the ImageNet- 1k training set (ID) (Russakovsky et al. 2015) and for the iNaturalist dataset (OOD) (Van Horn et al. 2018), using ResNet50 model pre-trained on ImageNet- Ik training set. The estimated 0g is plotted in Fig. 2
For completeness, we describe existing scature-shaping methods (Sun et al. 2021| Kong & Li 2023 Xu & Lian 2023) using our reparameterization of the feature shaping problem with O(z),
0
< low
(2) min(2,t)
OBFAa(())
OVRA.P(2)
low high
high 2,
V (5)2N'
high
(12)
where f, N, low, and high are hyperparameters. We then plot the resultant ((2) in Fig. 2 a-c.. We show that these prior works are empirically approximating function. A difference between our shaping function and VRA-P, is that VRA-P sets low-value features zero while ours flips the sign of these features to better utilize them for OOD detection.
optimal piecewise constant shaping
Then, we extend our analysis to other feature-shaping methods. ASH-P, -B, -S, and -Rand denote four simplistic yet effective methods introduced by Djurisic al. (2022). In contrast to ReAct, which employs a global threshold to clip features, ASH methods incorporate the entire feature vector and utilize local threshold to process the feature representation of each instance. Their shaping functions are not reducible to element- wemeet-wise functions.
We instead approximate the average shaping effect within each feature value interval. Specifically, we utilize ImageNet-Ik validation set (ID) and iNaturalist (OOD) with ResNet50. For an interval [ax, bx), we retrieve all features from ID and OOD datasets that fall within this range, along with their reshaped counterparts. Following this, the ratio of reshaped and original values computed as 0g for a specific data instance. Subsequently, we calculate the mean value and standard deviation of these 0g values over all samples on each interval. The results are depicted in Fig. 2(ch)).
3.3 OPTIMIZING THE SHAPING FUNCTION WITHOUT OOD SAMPLES
5

---

Published as a conference paper at ICLR 2024
Normally, have access to the ID training set, en- abling us to estimate ED [x= However, a significant challenge arises with the unknown term E D [(200b)] which renders the optimization problem defined in Eq. IO intractable.
optimized w/ OOD data
0 optimized w/o OOD data
To make the problem tractable without access to OOD data, omit the OOD-related term D2000_D000 [ [=000) by
Hard OOD
the following argument, which we will also sup-
ported empirically in Section 4.2
A diagram explain
intuition is shown in Fig.
Easy 0OD
Specifically, in scenarios where the OOD distribution closely resembles the ID (hard OOD), OOD samples likely have similar feature representations to ID samples, imply- ing both expectations share similar directions. Conversely, in cases where OOD samples significantly diverge from ID samples (easy OOD), the maximum logits of OOD samples tend to lower, thereby reducing the magnitude of 0000 Doon [[ compared to ID.
Figure 3: Diagram to show the intuition in deriving Eq. 13
Thus, the component of Egu ~ DooD
orthogonal to ID expectation small and we suggest omitting 000000 [0=)))) ), resulting in the following formulation:
[(z00D)
maximize (over 0) O E'Ez-Dy _ [x)) ||||| v K, z B()).
(13)
subject
The optimal solution to this new problem can be expressed in closed-form as:
K
F. Fa-DD [T=)].
(14)
We plot 6* calculated by Fq. 14 in Fig. dd, noting that is in close proximity to the optimal solution in the where OOD) samples available. In experiments, we will also show that method with only ID data performs almost on par with using the true T(000D) calculated from OOD data.
At inference time, for each sample, rescale its features within each interval [ag, by) by the corresponding entry in 0*, and calculate an OOD score based on the reshaped feature representation.
4 EXPERIMENTS
Datasets. We evaluate our proposed method using a large scale benchmark where ImageNet-
1k (Russakovsky et al. 2015) is the ID dataset, well as two moderate scale benchmarks where
CIFARIO and CIFAR 100 KKizhevsky et al., 2009) are used as the ID datasets. We evaluate eight
OOD datasets for the Net benchmark, comprised of Species (Hendrycks et al. (019b), iNatural- ist (Van Hor et al. 2018). SUN (Xiao et al. 20 Places (Zhou et al. 2017 7). OpenImage-O) [Wang]
al. 2022), Iet-ONet-O (Hendrycks al., 2021). Texture (Cimpoi al. 2014), and MNIST (Deng 2012). When CIFAR 100000 is the ID dataset, eight OOD datasets are exploited, comprised of
TinylmageNNet (Torralba al. 2008), SVHN (Netzer et al.) 201T), Texture (Cimpoi et al.
2014).
P65ces365 (Zhou et al. 2017). LSUN- (Yu et al. (015). LSUN- Yuu et al.
2015
iSUN (Xu al. (015). and finally CIFAR100/10.
Model architectures. We use pre-trained models provided by PyTorch and prior works in our experiments. For ImageNet- 1k, we use a pre-trained ResNet50 (He et al. (016) provided by PyTorch and MobileNet-v2 (Sandler et al.) (018)) provided by Djurisic al. | (2022). Additionally, we include models based on the transformer and fully-MLP architectures, which have been largely overlooked in previous studies. Specifically, we include ViT-B-16, 16 (Dosovitskiy et al. 2020). SWIN-S, (Tolstikhin et al. 2021). For the
SWIN-B (Liu et al., 2021), D), MLP-Mixer-B, B, and MLP-Mixer-L L CIFAR10 10 and CIFAR 100 benchmarks, we use NenseNet101 (Huang et al. (017) model provided by Sun Li 2022). a VIT-B-16 finetuned on CIFAR 100 consistent with Fort et al. (202)). and a MI.--ixer- Nano model trained on CIFAR 10/100 from seratch.

---

Published as a conference paper at ICLR 2024
Table 1: OOD detection results on ImageNet- Ik. 1 indicates larger values are better and & indicates smaller values are better. All values are percentages averaged over different OOD datasets. Bold numbers are superior results whereas underlined numbers denote the second and third best results.
Net5o
MohileNaV22
VIT-B-16
VIT-I-166
SWIN-S
SWIN-B
MLP-B
MLP-L
Average
Method
FPJ
AU1
FP4
AUT
FP4
AUT
FP4
AUT
FPI
AU1
FP.
AU1
FP4 AUT
FPL
ACT
FPL AU1
MSP
69.30
76.26
72.58
77.41
69.84
77.40
78.40
80.89
77.26
80.31
79.93
78.48
ODIN
61.56
80.92
62,91
82.64
69.25
72.60
70.35
74.51
62.35
78.67
70.90
68.36
65.00
82,99
67.90
81.86
66.28
77.82
Energy
60.97
81.01
61.40
82.83
73.96
67.65
74.89
70.11
66.56
75.95
79.53
60.14
87.86
78.42
84.60
79.01
73.72
74.39
DICE
45.32
83.64
49.33
84.63
89.68
71.32
72.38
67.08
78.84
49.06
77.89
50.10
57.12
80.60
61.98
80.75
66.57
70.90
ReAct
42.29
86.54
85.37
73.82
76.86
81.07
85
80.10
90.13
77.18
89.03
78.32
BFAc
43.87
86.01
52.87
85.78
77.64
80.16
84.02
81.12
64.76
85.57
68.40
84.36
96.04
67.87
96.00
70.59
72.95
80.18
ASH-P
55.30
83,00
59.41
83.84
99.36
21.17
99.18
20.27
99.09
21.78
98.83
22.11
98.19
35.85
98.91
29.69
88.53
39.71
ASHI-B
35.97
88.62
43.59
88.28
94.87
46.68
93,72
38.95
96.48
36.54
92.40
49.85
99.51
21.73
64.91
83.15
77.68
56.73
ASHLS
34.70
90.25
43.84
88.24
99.48
18.52
99.42
18.61
99.20
18.26
99
99.06
19:27
97.61
33,91
99.28
19.29
84.07
38.29
VRA-P
37.97
88.58
49.99 49.98
86.83
98 79
35.66
99.58
16.70
99.27
20.34
99.46
17 17.64
99.38
38
18.73
99.05
21.23
39
38.21
Ours (V)
41.84
88.11
53.31
86.36
69.
82.59
72.17
83.23
66.89
84.53
65.96
83.71
76.59
78.46
78.13
78.85
65.53
83.23
Ours (E)
39.75
88.56
51.77
86.62
69.52
82.66
78.57
82.94
66.28
84.99
66.20
84.21
83.54
75,94
85,50
78.11
67.64
83:00
"Choice of hyperparameters on Imagenet makes ODIN yield identical results to MLS.
Table 2: OOD detection results on CIFARI0/100. t indicates larger values are better and indicates smaller values are better. All values are percentages averaged over different OOD datasets. Bold numbers are superior results whereas underlined numbers denote the second and third best results.
CIFARIO
CIFARI00
Method
De
VIT-B-16
MI.P-N
Average
DenseNet101
VIT-B-16
MI.P-N
Average
FP.
AUT
FPI
AUT
FPI
AUT
FPI
AU1
FP1
AU1
FPJ
AU1
FP.
AU1
FPJ
AU1
MSP
52.66
91.42
25.47
95.61
67.01
82.86
48.38
89.96
80.40
74.75
61.
85.24
83.97
73.07
75.20
77.69
ODIN
32.84
91.94
24.93
92.84
69.20
69.53
42.32
84.77
62.03
82.57
56.91
80.86
78.71
66.47
68.88
76.63
MLS
12.79
97.16
64.50
82.97
36.41
70.71
80.18
52.31
74.52
68.48
Energy
31.72
93.51
12.40
97.22
63.95
82.84
36.02
91.19
70.80
80.22
51.93
88.82
79.90
75.30
67.54
81.45
DICE
93.27
26.84
92.41
96.64
17
51.05
59.56
82.26
95.18
95.78
44.48
ReAct
82.00
76.46
12.33
97.29
64,
81.85
52.89
85.20
77.00
78.30
52.25
88, 84
79.99
75.87
69.75
81.00
BFAct
84.40
74.39
12:34
97.21
78.02
72.68
58.2
81.43
80.27
73.36
51.23
80.05
76.58
70.52
79.66
ASH-P
29.39
93.98
20.20
95.31
84.39
66.93
44.66
85.41
68.21
81.11
53.69
87.66
86.73
65.27
69.54
78.01
ASILB
28:21
94.27
82
69.54
93.93
53.00
72.27
57.45
83.80
91.74
93.63
57.20
80.94
ASH-S
23.93
94.41
39.93
91 06
82.57
68.02
48.81
84.50
52.41
84.65
60.44
84,25
89.39
59.63
67.41
76.18
VRA-P
38.41
92.77
13.73
96.86
100.00
65.95
50.71
85.19
67.75
82.72
52.21
88.49
87.19
66.03
69.05
79.08
Ours (V)
30.12
93.97
13.13
97.07
77.25
73.86
40.17
88 30
69.01
81.42
52.76
88.60
83.88
73.65
68.53
81.22
Ours (E)
94.12
12.79
97.14
83.87
71.83
41.85
65.2
82.39
51.67
81.33
74.67
81.95
Evaluation metrics. et al. 2017 Sun et al.) 2021 Sun & Li (022):: the false positive rate (FPR95) when the true positive rate the ID samples 95% %, and AUC.
We utilize two standard evaluation metrics, following previous works (Liang
Implementation details. We evaluate two variants of our method, the vanilla method described in Eq. dennoted "Ours (V)", based on the maximum logit score, and a variant based on energy scores, denoted "Ours (E)". For comparison, we implement five baseline methods (top rows in results tables). including MSP (Hendrycks & Gimpel, 2016), ODIN (Liang et al. [2077). ,. ML. (Hendrycks et al.. 2019a). Energy (Liu et al. (020), and DICE (Sun & Li| 2022). and six feature-shaping methods (remaining rows in results tables), including ReAct (Sun et al. 202T). BFAct (Kong Li 2023), ASH-P, -B ASH-S (Djurisic et al., 2022), and VRA-P (Xu & Lian 2023). All comparison fcature-shaping methods are evaluated using the energy score. Hyperparameters are set consistent with the original papers and concrete hyperparameter settings can be found in Section of the Appendix. All experiments are run on a single NVIDIA GeForce RTX 4090 GPU.
4.1 OOD DETECTION BENCHMARK RESULTS
Experimental results on the ImageNet- Ik benchmark are shown in Table More detailed results can be found in Sections F and F of the Appendix. We observe that the comparison methods for
7

---

Published as a conference paper at ICLR 2024
Basc
Ours
Basc
Ours
FPR95
AUC
80
90
60
86
FPRO5
60
AUC
PPR99
AUC
80
31
50
84
40
10
12
70
40
MSP
MLS
Energy
DICE
ReAct
MSP
MLS
Energy
DICE
ReAct
3 10 30
100
300
Number of intervals (c)
(6)
Figure 4: Compatibility and sensitivity analysis. (a-b) Our method can improve other OOD scores and methods. "Base" denotes using the original OOD score or method, while "+Our" indicates combining the score or method with our feature-shaping function. (c) Our method's performance with different hyperparameter settings, i.e., numbers of intervals K.
feature shaping perform extremely well on the pretrained ResNet-50 model, with some ASH variants outperforming element-wist shaping methods like ReAct (Sun et al. (021), and yielding similar performance to our proposed methods. This not surprising, given that ResNet-50 was the model architecture used in the original papers. As we showed in Fig. the shaping functions proposed by these methods closely align with the optimal shaping function learned given actual OOD samples.
However, we observe that these methods do not generalize well to architectures outside of ResNet-50. We observe that for fcature-shaping methods more generally, stronger convolutional network archi- tecture performance is negatively correlated to performance on other network architectures. However, our proposed methods yield consistently strong performance across all network architectures, despite only having access to ID data for parameter tuning.
Results for the CIFARIO and CIFAR100 benchmarks are shown in Table An important observation regarding these results is that feature-shaping methods whole methods. This indicates that feature shaping appears to be ineffective on smaller-scale datasets such as CIFAR. While our methods do not outperform the baselines, are competitive and do outperform all of the comparison feature shaping methods, which is consistent with the ImageNet-I1 Ik benchmark.
underperform the baseline
4.2 FURTHER ANALYSIS
We conduct additional experiments using the ImageNet- Ik benchmark and employing ResNet50.
Extension other OOD scores. While our proposed method Eq. 14 is designed assuming MI.S as the OOD score, we also extend our method to use other OOD scores such energy and MSP. Furthermore, we combine our method with other OOD detection methods including DICE and ReAct. Results are averaged across the eight OOD datasets. As shown in Fig, 4(a-)), method yields consistent improvement across these OOD detection scores and methods.
Different numbers of intervals. We evaluate methods with different number of intervals. i.e., varying K. As shown in Fig, 4[c), most of the improvement in performance comes from K < 10. This indicates that the optimal feature shaping function has relatively simple form. This is consistent with prior methods (Sun et al. 2021 Xu & Lian! 2023 Kong Li 2023), which assume 2-3 intervals. In addition, the relatively large performance gap over the baseline motivates feature shaping more generally. Interestingly, we observe an extremely large jump between K = 0 ("Energy" bascline) with K 1, which effectively prunes the top and bottom 0.1% of feature values (using the thresholds computed based on the ID training data).
Empirical validation of Problem 13 In deriving Eq. 13 we remove the OOD-related term, assuming OOD data is inaccessible. We now empirically justify our intuition shown in Fig. 3 We estimate a Eu []] with ImageNet- k and
E F0 [000]] with different
|b||| of the two expectations. OOD datasets. We then compute the angles and the norm ratios, ||a
8

---

Published as a conference paper at ICLR 2024
As shown in Table challenging OOD datasets like Species and ImageNet-O exhibit smaller angles between the two expectations, while sim- pler OOD datasets like iNaturalist and MNIST demonstrate smaller norm ratios. Thus, Eq. [3 provides a reasonable approximation to 0 solved by Eq. T0 with access OOD data.
Table 3: Empirical validation of Problem a. and b denote the expectations of [(x) for ID and OOD, respectively. "AU (Real)" refers to the performance of solving Eq. TO with OOD data.
OOD dataset
||6||/||||| cos(a, b) AU (Real) AU (Ours)
Species INaturalist
0.72
0.99
78.69
78.59
Compare the formulation utilizing OOD data (Eq. 10) and our method (Eq.
0.48
0.97
96.53
SUN Places
0.55
0.98
92.90
92.84
(Eq.[4). [4). We opti- mize θ by solving Eq. [0 with each OOD dataset
0.59
0.99
90.06
90.15
OpenImage-O
0.56
0.98
92.31
92.54
and get the OOD detection performance on that dataset. The last two columns of Table com- com- pare the performance of 0 solved from Eq. [0] and that from Eq. 14 (ours). Notably, our method with only ID data yields performance comparable to the formulation with access to real OOD data.
ImageNet-O
0.91
0.99
70.19
59.33
Texture
0.51
MNIST
0.42
0.84
99.32
99.34
5 DISCUSSION
Here we first discuss core limitation to our framework that our proposed method assumes element- wise shaping functions, which fails to capture methods like ASH (Dosovitskiy et al., 2020). We then connect our proposed method to existing works.
General feature-shaping functions. As discussed previously, a feature-shaping function can represent arbitrary mappings f R-M RRM While our proposed methods handle mappings which can be defined element-wise over individual features, they fail to capture methods such as ASH (Dosovitskiy et al. 2020) which can only be defined over the full feature vector. An important direction for future work is in designing a tractable optimization problem where the family of functions f includes whole-vector methods such as ASH. We expect that the resultant mappings should also be relatively simple as in the element- wise case.
Feature contributions to the maximum logits. Prior research (unn & Li 2022 Dietterich & Guyer 2022) has explored the idea of feature contributions to the maximum logits. For example, DICE Sun & Li 2022) identifies the most significant feature indices for each class using ID data. During test-time, masks feature indices deemed insignificant before calculating OOD scores. However, our approach differs from theirs in that we consider feature value ranges instead of indices. We find our function yields significantly better performance in practice, partially attributed to the fact that it admits tractable optimization problem. Investigating the combination of these two classes of methods remains a promising direction for future work.
Classifier training for OOD detection. Prior studies (Liang et al. 2017 Sun & Li 2022) have leveraged surrogate OOD datasets, such as Gaussian and Uniform distributions, to train classifiers for distinguishing between ID and OOD samples. [Ling et al. (2017) primarily employ these datasets to assess their method's performance, while (Sun & Li (2022) use them for hyperparameter tuning. Despite these efforts, the approach is seldom applied several studies are probing into outlier synthesis (Du et al., 2022 He et al., (2022) to approximate real OOD distributions and train classifiers accordingly. Our optimization problem could benefit from these advancements, as our parameterized feature-shaping function could be optimized based on an ID dataset and synthesized outliers. We explore this possibility in Section B.I of the Appendix.
feature-shaping methods. Concurrently,
CONCLUSION
This paper formulates a general optimization problem for feature-shaping methods in OOD detection. Using this formalism, are able to better understand and explain the effectiveness of existing state-of-the- methods. In the context of our framework, we propose a novel feature shaping method which can be trained solely on ID data. Experimental results validate the superior performance and generalization ability of our approach compared to prior works, and motivate the development of new methods under framework further advance OOD detection methods.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
We would like to extend our deepest appreciation to Weijian Deng, Yanbin Liu, Yunzhong Hou, Xiaoxiao Sun, and all other lab colleagues for their invaluable support throughout this project. Their collaborative efforts, insightful discussion, and constructive feedback have been crucial in shaping and improving our paper.
This work was supported by an Australian Research Council (ARC) Linkage grant (project number I.P21020931)..
REFERENCES
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describ- ing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3606-3613, 2014.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 196)1141-142, 2012.
Thomas G Dietterich and Alex Guyer. The familiarity hypothesis: Explaining the behavior of deep open set methods. Puttern Recognition, 132:108931, 2022.
Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. arXiv preprint arXiv:2209.09858 2022.
Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Xuefeng Du Zhaoning Wang. Mu Cai, and Yixuan Li. Vos: Learning what you don't know by virtual outlier synthesis. arXiv preprint arXiv:2202.01197 2022.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of -iitr---istiibution detection. Advances in Neural Information Processing Systems, 34:7068-7088 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778, 2016.
Rundong He, Zhongyi Han, Xiankai Lu, and Yilong Yin. Ronf: reliable outlier synthesis under noisy feature space for -utto--distiibution detection. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 4242-4251, 2022.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136. 2016.
Dan Hendrycks. Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Sealing out-of-distribution detection for real-world settings. arXiv preprint arXiv: 191 1.1 1122 2019a.
Dan Hendrycks. Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of anttof-distributionn detection for real-world settings. arXiv preprint 1911:1191.11112. 2019b.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adver- sarial examples. In Proceedings of the IEEE/CVF Conference Computer Vision and Putlern Recognition, pp. 15262-152777 2021.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708. 2017.
10

---

Published as a conference paper at ICLR 2024
Haojia Kong and Haoan Li. Bfact: Out-of- detection with butterworth filter rectified activations. In Cognitive Systems and Information Processing: 7th International Conference, ICCSIP 2022, Fuzhou, China, December 17-18, 2022, Revised Selected Papers, pp. 115-129. Springer, 2023.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distributio image detection in neural networks. arXiv preprint arXiv: (706009900 2017.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy- out of-distribttton detection. Advances in neural information processing systems, 464-21475, 2020.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings the IEEF/CVF international conference on computer vision, pp. 10012-10022, 2021.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-252, 2015.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
Yue Song, Nicu Sebe, and Wei Wang. Rankfeat: Rank-1 feature removal for out-of-distribution detection. arXiv preprint arXiv:2209.08590 2022.
Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXIV, pp. 691-708. Springer, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. Advances in Neural Information Processing Systems, 34: 14:114415,, 2021.
Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et Mlp-mixer: An all-mip architecture for vision. Advances in neural information processing systems, 34:24261- 24272, 2021.
Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE transactions on pattern analysis and machine intelligence, 30(1 1955--1177 2008
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8769-8778. 2018.
Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual- logit matching. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2022.
Jianxiong Xiao. James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and patlern recognition, pp. 3485-3492. IEFE, 2010.
Mingyu Xu and Zheng Lian. Vra: Out-of-distribution detection with variational rectified activations, 2023.
11

---

Published as a conference paper at ICLR 2024
Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755, 2015.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. arXiv preprini
10.11344,. 2021.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.085: 506.0336,, 2015.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452 1464, 2017.
12

---

Published as a conference paper at ICLR 2024
A IMPLEMENTATION DETAILS
A.1 HYPERPARAMETER SETTINGS FOR EXPERIMENTS IN SECTION
For feature-shaping methods, we adhere to the hyperparameter settings reported in the original papers of the respective methods. These settings usually optimize the performance of the methods on existing benchmarks. For additional methods, we primarily rely on hyperparameter settings detailed in Sun's work (Sun & Li 2022). The detailed hyperparameter setting are listed in Table
Table 4: Hyperparameter settings for experiments in Section
Method
Hyperparameters
ImageNet- Ik
CTFARIO
CIF
MSP
No hyperparameters
T. temperature scaling noise level
ODIN
T = 1000, r 0.0
T = 1000, = 0.004
T 1000, 0.004
MLS
No hyperparameters
Energy
No hyperparameters
DICE
D. sparsity parameter
p= 0.7
p p= 0.9
= 0.9
ReAct
threshold percentile
90
90
90
N 2, thresh 95
N 2, Mhreeh m 95
N = 2, thresh m 95
BLAct
thresh. N
percentile training set
percentile the training set
percentile the training
ASH-P
p. pruning percentile
= 60
= 90
p 80
ASILBB
pruning percentile
= 65
95
85
ASH-S
p. proning pereentile
90
95
p 90
21 60 percentile of the training set
3.g percentile of the training set Ry 95 pereentile of the training set
VRA-P
21 22
21 0.5, 22 1.0
22 = 95 percentile the training set
K. the number of feature value intervals
Ours
K = 100
K 100
100
A.2 THE MAGNITUDE OF RESIIAPED FEATURES
In Section 3 we describe our method for learning a piecewise constant reshaping function subject to norm constraints. Scaling 0 has no effect on the vanilla version of method. However, OOD detection performance is sensitive to the scaling of 0 when our method is combined with other OOD scores with non-linear transformations such as MSP and the energy score.
Take the energy score as an example. If we multiply the feature representation z by a small positive factor q (0 < q < 1), ignoring the bias term in the last fully connected layer, we have:
Emergy (L) = log exp(ge,).
(15)
j=1
This is equivalent to apply a temperature scaling T 1/9 to the energy score, which will affect the OOD detection performance, as analyzed in previous paper (Liang et al.] 2017). Besides, considering a fully connected layer usually contains bias term, the scaling of 0 will further affect the influence of the bias. Thus, we need to consider the scale of 0 in our optimization problem.
We scale 0 by a factor of VK, motivated by the following analysis. Considering that the maximum logit is the sum of all ISFIs, we have:
K
pinax
Σ 1=1
= -
(16)
where 1 represents a K-dimensional vector of ones, which is trivial feature-shaping function with no effect. Therefore, we set ||0| = =|||| VK.
13

---

Published as a conference paper at ICLR 2024
Table OOD detection results of variants of our methods on ImageNet-Ik. 1 indicates larger values are better and &. indicates smaller values are better. All values are percentages averaged over different OOD datasets. Bold numbers are superior results whereas underlined numbers denote the second and third best results. "Real" means using real OOD dataset to resolve Problem (Eq. (0). "Gaussian" and "Uniform" indicate using Gaussian and Uniform distributions as surrogate OOD datasets in resolving Problem (Eq. (T), respectively. "Dynamic" denotes solving a non-convex problem with dynamic weight indices defined in Section B.2
ReaNer50
MobileNorV2
VI-B-166
VIT-L-16
SWIN-S
SWIN-B
MLP-B
MLP-L
Average
Method
AUT
AUT
AUT
PP4
AUT
FP4
AUT
FPL
AUT
HPC
AUT
FPI
AUT
FP]
ALT
MSP
69.30
76.26
72.58
77.41
69.84
77.40
70.59
78.40
65.12
80.89
69.45
77.26
70.66
80.31
71.63
79.93
69.90
78.48
ODIN
61.56
80.92
62,91
82.64
69.25
72.60
70.35
74.51
62.35
78.67
70.90
68.
65.00
82.99
67.90
81.86
66.28
77.82
Energy
60.97
81.01
61.40
82.83
73.96
67.65
74.89
70.11
66.56
75.95
79.53
60.14
87.86
78.42
84.60
79.01
73.72
74.39
DICE
83.64
71.32
67.08
49.06
50.10
57.12
80.60
61.98
80.75
66.57
ReAct
42.29
86.54
54.19
85.37
73.82
76.86
76.16
81.07
58.07
85.10
70.23
80.10
90.13
77.18
89.03
78.32
69.24
81.32
BFAct
43.87
86.01
52.87
85.78
77.64
80.16
84,02
81.12
64.76
85,57
68.40
84.36
96.04
67.87
96.00
70.59
72.95
80.18
PSII-P
$5.30
83.00
59.41
83.84
99.36
21.17
99.18
20.27
99.09
21.78
98.83
22.11
98.19
35.85
98.91
29.69
88.53
39.71
ASH-B
88.62
43.59
46.68
38.95
36.54
21.73
56.73
ASH-S
34.70
90.22
43.84
88
24
99.48
18.52
99.42
18.61
99.20
18.26
99.06
19.27
97.61
33.91
99.28
19.29
84.07
38.29
VRA.P
37.97
88.58
98.39
35.66
16.70
99.27
20.34
17.64
99.05
Real
42.09
88.13
55.05
85.77
73.69
81.65
75.36
83,10
76.06
83.22
73.54
83.11
81.66
78.34
78.82
79.20
69.53
82.82
Gaussian
42.05
88.20
55.35
85.67
82.01
73.99
83.28
83.99
83.37
82.09
77.58
76.40
79.79
67.98
82.99
Umffrm
42,36 88.02
55.12
85.73
73.14
81.82
74.30
K3.25
72.68
K3.74
70.57
83.43
80.84
76.88
76.41
79.74
68.18
82.83
Dynamic
53.29
69.33
83.23
66.90
84.52
65.98
83.71
76.87
78.35
78.85
65.56
83,22
Ours (V) 41.84 88. 11
53.31
86.36
69.33
82.59
72.17
83.23
66.89
84.53
65.96
83.71
76.59
78.46
78.13
78.85
65.53
83.23
Ours (E) 39.75 88.56
51.77
86.62
69.52
82.66
78.57
82.94
66.28
84.99
06.20
84.21
83.54
75.94
85.50
78.11
67.64
83.00
B VARIANTS OF OUR METHOD
In this section, we execute a series of experiments on variants of our proposed methods. Initially, we employ real and surrogate OOD datasets to address the optimization problem (Eq. 10) as alternatives to our ID only method. Subsequently, we evaluate an assumption that weight indices of maximum logit will not change after feature shaping. We calculate and report the ratio of changed weights after reshaping. Furthermore, we solve an optimization problem which allows for dynamic weight indices. In this scenario, it is possible that the weight indices of the maximum logits undergo changes after the reshaping exercise. Our experiments are based on the ImageNet- benchmark and the results are illustrated in T
B. EXPLOIT REAL AND SURROGATE OOD DATASETS
We draw comparisons between our ID only method and a feature-shaping function optimization approach that accesses both ID) and OOD datasets.
First, we utilize real OOD datasets to address Problem (Eq. For each model architecture, ImageNet- 1k training set (ID) and iNaturalist (OOD) are leveraged to optimze 0.
Second, we incorporate Gaussian and Uniform distributions as surrogate OOD datasets in resolving Problem (Fq. 10), aligning with prior studies (Liang et al. 2017, Sun & Li 2022). For the Gaussian distribution, each pixel is generated with a mean of 127.5 and standard deviation of 255. For the uniform distribution, we directly generate random pixel values within the range [0, 255J. Every generated image is of size 32 x 32, and is subsequently clipped and rounded to conform to the [0, 255] value range. For each surrogate OOD dataset, we generate total of 50,000 samples.
As illustrated in Table 5 our method, even without access to OOD data, performs comparably to methods that optimize 0 utilizing both ID and OOD datasets. Moreover, we observe that more robust objective function, such as Log Loss or Hinge Loss, could potentially outperform our method when given access real surrogate OOD datasets. This presents intriguing avenue for exploration.
14

---

Published as a conference paper at ICLR 2024
Table 6: Ratio of changed weight indices on ImageNet-lk. All numbers are percentages. We
randomly sample 10,000 samples from ImageNet- Ik training set, and is defined in Section B.2
ResNe150
MobileNetV2
VII-B-16
ViT-L-16
SWIN-S
SWIN-B
MLP-B
MLP-
Average
6.8
6.5
1.1
0.5
1.5
0.9
4.3
0.4
B.2 DYNAMIC WEIGHT INDICES
Our proposed method, elucidated in Sections 3.2 and 3.3 relies fundamental simplifying
assumption: the class index of the maximum logit remains unchanged after feature shaping.
We initially examine the validity of this assumption by calculating the ratio of weight indices that change after feature shaping. For given ID training set Swain of N samples, each sample x(() E Swain has weight indices corresponding to the maximum logits before and after feature shaping, as follows:
(4)) arg max w (f)
[ arg max w.
(17)
k
k
where wk denotes the weight vector corresponding to class &s in the final fully-connected layer, 2(() represents the feature representation for the sample x(9,, and z()) is the reshaped feature. Subsequently, we compute the ratio of changed weight indices as:
Σm 1 ( # A:00)
X 100%.
(18)
N
We randomly sample 10,000 samples from ImageNet- Ik training set. The ratios of changed weights with different models are presented in Table As observed, the ratios are markedly small, which further substantiates the credibility of our assumption pertaining to fixed weights.
Furthermore, we propose straightforward Alternating Optimization algorithm for dynamic weight indices. Each iteration involves three primary steps. Firstly, we optimize feature-shaping function by resolving the optimization problem (Eq. T0) given the weights relative to the maximum logits optimized feature-shaping function. Lastly,
for each sample. Next, reshape features utilizing we update the weights with reshaped features. To expedite the process, we randomly subsample 10,000 samples from the ID training set during each iteration and utilize them for optimization. Ten iterations are run for each model architecture. The results can be viewed Table 5
Predictably, this approach yields performance metries that closely mirror our initial method. Despite the incorporation of dynamic weight indices, the optimized feature-shaping function does not signifi- cantly differ from that obtained with fixed weights. This observation reinforces the validity of our original assumption: the class index of the maximum logit remains constant following reshaping.
C ROBUSTNESS OF OUR METHOD TO THE CHOSEN LOWER AND UPPER LIMITS OF FEATURES
Given that feature distributions typically ex- hibit long-tailed characteristics, we select the 0.1 and 99.9 percentiles of all features train- ing set as the lower and upper limits of feature values, respectively, to optimize our shaping function. Here we validate different choices of percentiles on the performance our method, on the ImageNet-Ile Ik benchmark with ResNet50. The results are shown Table[7]
Table 7: Performance our method with differ- ent choices of percentiles of all features the ImageNet- benchmark with ResNet50.
Lower percentile
Lower limit
Upper percentile
Upper limit
FPR95
AUC
0.0
0.00
100.0
20,65
42.42
87.90
0.01
0.00
99.99
5.64
42.22
87.97
0,05
0.00
99.95
4.40
41.97
88.04
Our method exhibits insensitivity to the chooses of lower and upper limits. Although in the main experiments 0.1 and 99.9 percentiles, exploring alternate set- tings reveals potential for slight improvements.
0.00
99.9
3.88
41.82
88.11
0.00
99.5
2.74
41.01
88.44
1.0
0.00
99.0
2.28
41.00
88.50
consistently use the
47.85
10.0
0.06
90.0
1.01
$7.45
83.60
15

---

Published as a conference paper at ICLR 2024
D EMPIRICAL ANALYSIS OF THE OPTIMAL SHAPING FUNCTION
Here we try to provide empirical insights into the specific form of our optimal shaping function based on ResNet50 model pretrained on ImageNet-Ik, lk, which is shown Fig. 2 Specifically, we feed ImageNet- val (ID) and iNaturalist (OOD) samples to the model, getting the feature representation and the weights w.r.t the maximum logit wax RM for each sample. The results are shown in Fig. 5 The red and green lines represent the mean weights assigned to different feature values for ID and OOD samples, re- spectively. The colored regions repre- sent standard deviations.
0.15
1.5
Weights the max logit
0.05
0.5
R
-0.05
-05
samples
00D samples
optimized 61z)
0.0
0.5
1.0
3.5
2.0
2.5
3.0
3.5
Feature value
Figure 5: Empirical analysis to explain a specific form of the optimal shaping function.
An interesting aspect is that our approach flips the sign of low-value features, contrary to previous techniques like ReAct and ASH. Our experiments revealed that low-value features (2 0.5) often align with negative weights (w < 0). Intriguingly. ID samples generally align these features with smaller weights than OOD samples (on average, wiD < maa < O). By flipping the sign of these low-value features, we can enhance the maximum logits more significantly for ID samples than for OOD samples, thereby boosting OOD detection performance.
However, note that the shape of the optimal shaping function is influenced by multiple factors. For instance, while ID samples are generally assigned larger weights for high-value features, OOD samples tend exhibit greater number of these features, observed in prior studies (Sun et al. 2021). This aspect reduces the impact of high-value features distinguishing between ID and OOD samples, leading to the decrease of 0 for those features.
Besides, note that our analysis above is grounded the context ResNet50 model pre-trained on ImageNet- 1k. The form of the optimal shaping function may differ when applied to other model architectures or datasets.
E DETAILED OOD DETECTION PERFORMANCE
We report additional results in Tables 8 and 10 to supplement the results provided in Table Specifically, we provide detailed performance for all eight test OOD datasets for the ImageNet- Ik benchmark for three representative models, including ResNet50, ViT-B-16, and MLP-B.
F STANDARD IMAGENET-IK BENCHMARK COMPARISON
Standard ImageNet- 1k benchmark uses subset of OOD datasets of ours, and these results can be found in Tables 8 and
Besides, we pick the results of standard ImageNet-Ik Ik benchmark in our experiments and calculated the new average performance. Our method still outperforms prior works this standard benchmark. Results are shown in T1
16

---

Published as a conference paper at ICLR 2024
Table 8: OOD detection results with ResNet50 on ImageNet-Ik, f indicates larger values are better and 4 indicates smaller values are better. All values are percentages averaged over different OOD datasets. Bold numbers are superior results whereas underlined numbers denote the second and third best results. "Real" means using real OOD dataset to resolve Problem (Eq. (0). "Gaussian" and "Uniform" " indicate using Gaussian and Uniform distributions as surrogate OOD datasets in resolving Problem (Eq. [O), respectively. "Dynamic" denotes solving non-convex problem with dynamic
weight indices defined in Section B.2
Method
Species
INaturalist
SUN
Places
OpenImage-O
ImageNet-O
Texture
MNIST
Average
FP4
AUt
FPL
AUT
FP1
AUT
FPL
AUt
FPL
AUt
FPL
AUt
FP1
AUT
FPL
AUt
FP.
AUT
MSP
79.53
75.15
52.77
88.42
68.58
81.75
71.57
80.63
63.58
84.98
100.00
28.64
66.13
80.46
52.25
90.02
69.30
76.26
ODIN
72.88
50.86
91.14
59.89
84.18
40.86
86.40
23.54
61.56
Energy
82.40
72.03
53.93
90.59
58.27
86.73
65.40
84.13
57.21
89.12
100.00
41.92
52:29
86.73
18.23
96,79
60.97
81.01
DICE
74.64
74.30
26.66
94.49
36.09
90.98
47.66
87.73
88.77
98.10
43.01
90.46
1.79
99.40
45.32
83.64
ReAct
68.64
77.41
19:55
96.39
24.00
94.41
33.43
91.93
43.68
90.53
98.05
$2.43
45.83
90.45
5.11
98.76
42 29
86.54
BFAct
77.76
20.53
20.94
95.24
92.62
96.20
54.15
43.87
86.01
ASH-P
80.09
73.19
44.57
92.51
52.88
38.35
61.79
85.58
50.45
90.67
100.00
45.99
42.06
89.70
10.57
97.99
$5.30
83.00
ASILB
80.33
14.21
97.32
22.08
95.10
33.45
92.31
37.63
91.97
93.05
56.82
21.17
95.50
0.06
99.58
35.97
88.62
ASH-S
64,61
81.45
11.49
97.87
27.98
94.02
39.78
90.98
32.74
92.75
89.05
67.51
11.93
97.60
0.01
99.80
34,70
90.25
VRA-P
69.61
76.93
15.70
97.12
26.94
94.25
37.85
91.27
36.37
92.93
95.55
60.79
21.47
95.62
0.30
99.70
37.97
88.58
Real
68.91
78.58
18.64
96.53
37.76
92.69
47.31
89.97
39.63
92.33
97.45
60.01
24.15
95,57
2,90
99.34
42.09
18.13
Gaussian
78.31
37.54
92.68
89.88
39.60
92.32
97.05
60.85
95.76
88.20
Uniform 69.34
78.49
18.83
96.51
38.71
92.51
47.98
89.81
39.51
92.46
97.70
59.49
24.24
95.54
2.58
99.37
42
88.02
Dynamic 68.81 78.58
18.33
96.63
36.87
92.87
45.94
90.17
39.34
92.54
91.80
59.44
24.70
95.50
2.68
99.34
41.81
88.13
Ours (V) 68.83 78.59
18.33
96.63
37.03
92.84
45.97
90.15
39.29
92.54
97.80
59.33
24.80
95.48
2.67
99.34
41.84
88.11
Ours (E) 68.46 78.51
15.90
97.00
34.00
93.28
43.61
90.50
37.07
92.82
96.25
60.70
21.61
95.99
0.85
99.66
39.75
88.56
Table 9: OOD detection results with ViT-B-16 on ImageNet- 1k. t indicates larger values are better and 4. indicates smaller values are better. All values are percentages averaged over different OOD datasets. Bold numbers are superior results whereas underlined numbers denote the second and third best results. "Real" means using real OOD dataset to resolve Problem (Eq. [0). "Gaussian" and "Uniform" " indicate using Gaussian and Uniform distributions as surrogate OOD datasets in resolving Problem (Eq. (0). respectively. "Dynamic" denotes solving non-convex problem with dynamic
weight indices defined in Section B.2
Method
Spocies
iNaturalist
SUN
Places
OpenImage-O ImageNet-O
Texture
MNIST
Average
FP4
AUf
FPL
AUf
FP1
AUT
FPL
AU1
FPL
AUt
FPL
AU1
FP4
AUt
FPI
AUt
FP.
AUT
MSP
80.56
72.91
51.52
88.16
66.54
80.93
68.67
80.38
59.93
84.81
90.95
58.80
60.23
82.99
80.29
70.23
69.88
77.40
ODIN
82.18
67.96
52,26
85.24
66.89
76.35
69.14
75.06
58.75
81.55
54.30
56.68
81.69
79.03
58.65
69.25
72.60
Energy
86.54
61.88
64.08
79.24
72.77
70.24
74,31
68.44
64.93
76.46
87.00
52.71
58.48
79.30
83.61
52.92
73.96
67.65
DICE
88.09
65.08
90.43
94.27
65.76
92.81
65.19
82.61
70.86
83.67
77.63
74.02
89.68
71.32
ReAct
86.82.
66.08
65
85.98
72.46
78.97
73.74
77.51
64.71
84.19
87.30
66.70
56.95
84.65
83.41
70.81
73.82
76.86
BFAct
90.52
64.43
80.01
84.62
77.86
77.86111
81.10
77.55
79.61
69.
85.45
87.15
73.84
57.61
85.67
80,76 76
86.53
64
16
ASH-P
98.38
28.99
99.95
10.42
99.69
19.83
99.66
21.02
99.57
18.91
99.15
30.70
98.
29.92
100.00
9.57
99.36
21.17
ASIL-B
53.78
92.36
52.35
52.18
92.74
93,90
99.96
94.87
ASH.S
98.11
28.50
99.99
6.71
99.73
16.70
99
18.33
99.84
14.07
99.40
28.51
99.11
24.15
100.00
11.20
99 48
18.52
VRA-P
98.14
39.19
99.72
06
98.83
30.67
89
32.59
98.28
37,46
65
43.67
94.57
52.22
100.00
26.40
35.66
Real
85.07
67.00
64.16
88.03
70.73
83.05
71.76
81.74
63.56
87.14
89.60
73.69
63.09
84.63
79.53
87.93
73.69
81.65
Gaussian
67.56
87.51
89,70
77.20
82.01
Uniform 84.69 67.25 62.56
85.44
70.60
83.27
71.37
81.93
64.54
87.41
90,20
73.59
61.60
84.94
79.54
87.71
73.14
81.82
Dynamic 83.20 69.47 56.06
89.91
66.63
84.17
68.48
82.69
60.21
87
89.50
71.82
56.81
73.75
88.08
09.33
82.59
Ours (V) 83.19 69.47
56.09
89.91
66.63
17
68.51
82.69
60.21
88.18
89.50
71.82
56.79
86.43
73.75
88.08
33
82,59
Ours (F) 84.56 68. 70
60.09
89.55
68.02
83
69.42
82.36
61.77
88.07
87.95
72.74
54.61
86.72
69.72
89.21
69.52
82.66
17

---

Published as a conference paper at ICLR 2024
Table 10: OOD detection results with MLP-B on ImageNet- Ik. 1 indicates larger values are better and J indicates smaller values are better. All values are percentages averaged over different OOD datasets. Bold numbers are superior results whereas underlined numbers denote the second and third best results. "Real" means using real OOD dataset to resolve Problem (Eq. TO). "Gaussian" and "Uniform" indicate using Gaussian and Uniform distributions as surrogate OOD datasets in resolving Problem (Eq. 10), respectively. "Dynamic" denotes solving non-convex problem with dynamic weight indices defined in Section B.2
Species
iNaturalist
SUN
OpenImage-O
ImageNet-O
Texture
MNIST
Average
Method
FPL
AUt
FPL
AUT
FP4
AUD
FP4
AUt
FP4
AUT
FPL
AUT
FPL
AUt
FPJ
AU1
FP1
AUT
MSP
82
73.21
58.99
86.84
72.98
80.92
75.52
79.83
72.42
82.59
90.95
65.84
69.95
80.27
42.36
92.95
70.66
80.31
ODIN
84.24
71.67
82.44
85.51
59.75
95.73
82.99
Energy
89.04
71.84
84.31
83.65
86.85
79.35
87.15
78.42
82.30
81.24
88.40
67.53
85.25
78.41
99.58
86.91
87.86
78.42
DICE
71.40
75.65
44.44
90.19
64.69
80.84
70.31
78.08
63.89
80.26
89.90
56.33
52.20
84.33
99.11
57.12
80.60
ReAct
91.20
70.81
91:47
80.11
91.83
77.31
90.99
76.82
85.66
79.60
87.85
67.74
87.36
77.20
94.65
87.89
90.13
77.18
BEAcI
67.50
70.00
94.61
72.19
67.87
ASILP P
97.25
46.50
99.22
32.82
98.72
40.24
97.74
43.07
98.67
38.34
94.55
52.65
99.38
31.38
100.00
1.81
98.19
35.85
ASH-B
99.67
100.00
15.86
99.71
19.84
74
21.36
99.82
24.19
97.25
46.39
99.86
100.00
99.51
21.73
ASH-S
96.50
42.89
98.42
32.35
98.03
38.00
97 97.90
38.63
97.65
37.03
93.50
50.19
98.90
30.41
100.00
1.80
97.61
33.91
VRA-P
98.88
29.11
99.92
11.57
99.73
17.54
99.41
20.71
99.77
16.82
97.80
37.15
99.50
16.49
100.00
0.44
99.38
18.73
Real
86.49
71.56
71.23
84.79
79.47
79.42
81.95
78.44
78.30
81.00
91.00
66.46
78
78.36
86.61
86.69
81.66
78.34
Gaussian 87.00
71.10
73.67
78
78.80
77.86
79.06
79.83
91.20
78.90
77.64
82.09
77.58
Uniform 86.92 70.35
75.45
81.81
80
67.69
82.25
77.07
79.29
78.56
90.70
65.63
78.05
76.45
73.59
87,51
84
76.88
Dynamic 87.24 70.88
76.95
82.17
83.83
77.26
84.75
76.12
78.65
80.31
90.70
65.68
76.76
79.17
36.06
95.19
76.87
78.35
Ours (V) 86.98 70.98
76.40
82.36
83.45
77.42
84.67
76.27
78.35
80.46
90.85
65.77
76.40
79.26
35.66
95.19
76.59
78.46
Ours (F) 91.67 69.17 92.48
76.83
91.83
73.88
91.86
73.34
87.14
77.34
90.40
66.02
89.29
75.33
33.61
95.61
83.54
75.94
Table 11: OOD detection results on ImageNet- Ik where OOD datasets are iNaturalist, SUN, Places, and Textures. 1 indicates larger values are better and indicates smaller values are better. Bold numbers are superior results whereas underlined numbers denote the second and third best results.
ResNet50
MobileNesV2
VIT-B-16
VIT-L-16
SWIN-S
SWIN-B
MLP-B
MLP-L
Average
Method
FP4
AUt
FP4
AUt
FPL
AUt
FPI
AU1
FPL
AUt
FPJ
AU1
FPL
AU1
FPL
AU1
FPI
AU1
MSP
64.76
82.82
70.47
80.67
61.74
83.12
65.22
81.75
59.68
83.75
62.79
81.38
69.36
81.97
76.01
80.04
66.25
81
ODIN
57.68
60.42
86.39
61.24
79.59
78.65
57.30
80.99
64.36
73.77
63.05
85.15
73.17
62.66
Energy
57.47
87.05
58.87
86.59
67.41
74.31
68.43
74.65
62.82
77.65
75.32
64.87
85.89
79.96
84.44
79.38
70.08
78.06
DICR
35.72
90.92
41.93
89.60
90.30
70.77
71.77
67.12
88.68
39.90
87.92
40.71
57.91
83.36
66.23
81.53
67.55
70.49
ReAct
30.70
93.30
50.09
88.81
67.08
81.78
69.58
83.50
$0.09
87.92
64.86
83.24
90.41
77.86
87.01
79.13
63.73
84.44
BFAct
31.41
48.35
89.19
73.26
82.75
82.69
57.20
65.44
86.62
46
96.36
68.74
ASII-P
50.33
89.04
57.15
87.34
99.45
20.30
99.42
18.37
99.11
19.21
99.08
20.59
98.77
36
99.04
29.20
87.79
40.11
ASH-B
22.73
95.06
35.66
92.13
94.33
49.14
94.08
89
96.42
30.00
91.47
47.38
99.83
19.14
66.05
84
75.07
56.88
ASH-S
22.80
95.12
38.67
90.95
16.47
99.55
16.72
99
14.98
99.35
17.75
34.85
99.36
18.65
83
38.18
VRA-P
25.49
94.57
45.53
89.85
98.02
34.64
99.62
14.99
99.38
18.15
99.65
15.51
99.64
16.58
99.23
19.49
83.32
37.97
Ours (V) 31.53 93.78
49.09
89.62
62.01
85,80
68.61
85.10
61.47
87.04
62.96
86.09
80.23
78,83
82.59
78.92
62.31
85.65
Ours (E) 28.78
94.19
46.92
89.90
63.04
85.64
74,96
84.69
60.67
87.50
63,50
86.55
91.37
74.85
88.49
78.22
64.71
85.19
18


Reviews:
[{"review_id": "W6feHlbgRk", "reviewer": "Reviewer_kmFm", "summary": "This paper proposes a feature-shaping method for general OOD detection without OOD data.\nThis paper uses maximum logit score and energy scores with the manipulated feature to find the OOD data.\nThis method yields a closed-form solution for the piecewise constant shaping function.\nExtensive experiments support the superiority and generalization of this approach.", "questions": "1. Is there a more intuitive explanation for this method that would make it more understandable?\n2. Will the code and data be open source?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method is generalizable across different benchmarks does not require OOD data, and is somewhat original.\nMultiple datasets and multiple backbones are evaluated.\n2. This paper is supported by theory as well as experimental validation and is of high quality.\n3. The overall description is relatively clear.\n4. The OOD task is of great significance. It makes sense to research it.", "weaknesses": "1. The reviewer didn't see any discussion or analysis about the robustness of the proposed method to thresholding.\nAdding this analysis would make the paper more complete.\n2. The reviewer believes that further explanation is needed as to why $E_{x^{OOD} \\sim D^{OOD}_{\\chi}} [I (Z^{OOD}) ]$ can be ignored.\nThe explanation in Fig. 3 is still not convincing enough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2kARfrCryA", "reviewer": "Reviewer_c5SY", "summary": "This work proposed the feature shaping method in OOD detection. By dividing K intervals, the vector of the K dimension is constructed, named the ISFI vector. Based on this vector, they optimize theta to increase the score of in-distribution and decrease the score of out-distribution. The authors showed the effectiveness of their approach on various architectures compared to existing feature shaping methods.", "questions": "Please refer to the weaknesses.\n\n==\n\nI raise my score from 3 to 6.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Compared to previous feature shaping methods, the proposed method is effective on various architectures, especially on MLP architectures.", "weaknesses": "- What is the motivation to introduce ISFI? For example, ReAct [A] observes the distribution of per-unit activation for In and out-distribution data.\n- I am confused about the meaning of Figure 2. Suppose that z of M dimension is given. Then, theta of ReAct [A] is also M dimension because ReAct conducts the operation on each element of feature z. theta of the proposed method is K dimension determined by the number of intervals. I am not sure whether it is possible to put the plots together.\n- theta is optimized over the proposed optimization problem that the closed form exists. However, other feature shaping methods [A, B] do not have the other parameters to be optimized.\n- Other feature shaping methods [A, B] said that their methods have comparable performance in distribution distribution after applying their methods. Can authors discuss this point?\n\n[A] React: Out-of-distribution detection with rectified activations, NeurIPS 2021\n\n[B] Extremely Simple Activation Shaping for Out-of-Distribution Detection, 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Im5gmheLTX", "reviewer": "Reviewer_bZUo", "summary": "The paper finds that the existing feature-shaping methods usually employ rules manually designed for specific model architectures and OOD datasets, which consequently limit their generalization ability. To address this gap, the authors first formulate an abstract optimization framework for studying feature-shaping methods. then they propose a concrete reduction of the framework with a simple piecewise constant shaping function and show that existing feature-shaping methods approximate the optimal solution to the concrete optimization problem. Further, assuming that OOD data is inaccessible, the authors propose a formulation that yields a closed-form solution for the piecewise constant shaping function, utilizing solely the ID data. Through extensive experiments, the authors show that the feature-shaping function optimized by the proposed method improves the generalization ability of OOD detection across a large variety of datasets and model architectures.", "questions": "see above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is written well and is easy to understand.\n2. The studied problem is very important.\n3. The results seem to outperform state-of-the-art.", "weaknesses": "1. I am curious about the comparison of the proposed method with the methods that can train with the real outliers in different benchmarks.\n2. In terms of the feature-shaping methods, I am curious how well the proposed method compared to the OOD detection using contrastive learning objectives to reshape the feature, such as CSI and SSD [2] on different benchmarks and mode architectures.\n\n[1] CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances\n\n[2] SSD: A Unified Framework for Self-Supervised Outlier Detection", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)