Unleashing Mask: Explore the Intrinsic Out-of-distribution Detection Capability
Authors: Jianing Zhu
Hengzhuang Li
Jiangchao Yao
Tongliang Liu
Jianliang Xu
Bo Han
Venue: Submitted to ICLR 2023 (2023)

Abstract: Out-of-distribution (OOD) detection is an important aspect for safely deploying machine learning models in real-world applications. Previous approaches either design better scoring functions or utilize the knowledge from unknown outliers to equip the well-trained models with the ability of OOD detection. However, few of them explore to excavate the intrinsic OOD detection capability of a given model. In this work, we discover the existence of an intermediate stage of a model trained on in-distribution data having higher OOD detection performance than that of its final stage across different settings, and further identify the critical attribution to be learning with atypical samples. Based on such empirical insights, we propose a new method, Unleashing Mask (UM), to reveal the once-covered detection capability of a given model. To be specific, we utilize the mask to figure out the memorized atypical samples and fine-tune the model to forget them. Extensive experiments have been conducted to characterize and verify the effectiveness of our method.

Under review as a conference paper at ICLR 2023
UNLEASHING MASK: EXPLORE THE INTRINSIC OUT- OF- OF-DISTRIBUTION DETECTION CAPABILITY
A Anonymous authors
Paper under double-bling review
ABSTRACT
nut-of-distribution (OOD) detection is an important aspect of safely deploying machine learning models in real-world applications. Previous approaches either design better scoring functions or utilize the knowledge of outliers to equip the well-trained models with the ability of OOD detection. However, few of them explore to excavate the intrinsic OOD detection capability of a given model. In this work, we discover the existence of an intermediate stage of a model trained on in in-distribution data having higher OOD detection performance than that of its final stage across different settings and further identify the critical attribution to be learning with atypical samples. Based on such empirical insights, we propose new method, Unleashing Mask (UM), that restores the OOD discriminative capabilities of the model. To be specific, we utilize the mask to figure out the memorized atypical samples and fine-tune the model to forget them. Extensive experiments have been conducted to characterize and verify the effectiveness of our method.
INTRODUCTION
-istridditiitttinnn (OOD) detection has drawn increasing attention when deploying machine learning
models into the open-world scenarios (Nguyen et al.) 2015 Lee al. 2018a). Since the test samples can naturally arise from a label-diffent for those safety-critical applications like autonomous driving and medical intelligence. Previous
distribution, identitying OOD inputs is important, especially
studies focus on designing series of scoring functions (Hendrycks & Gimpel 2017b) Liang et al.,
2018 [ee a 2018a Liu et al. 2020 Sun et al. 2021 | 2022) for OOD uncertainty estimation or
finetuning with auxiliary outlier data to better distinguish the OOD inputs (Hendrycks et al.) 2019c
Tack et al. 2020 Mohseni et al. 2020 Sehwag et al. 2021 I Wei et al.
2022 Ming et al. 2022).
Despite the promising results achieved by previous methods (Hendrycks & Gimpel 201 7a Hendrycks
ct al., 2019c [iu et al., 2020 Ming et al., 2022), little attention is paid to considering whether the well-trained given model is the most appropriate for OOD detection. In general, models deployed for various applications have different targets (c.g., multi-class classification) (Goodfellow et al. 2016)
instead of OOD detection (gguyen et al., 2015; Lee et al. 2018a). However, most representative score functions, e.g., MSP (Hendrycks et al. (019c)) ODIN(Liang et al. 2018), and Energy (Liu al. 2020), uniformly leverage the given models for OOD detection. Considering the target-oriented discrepancy, it arises a critical question: does the well-trained given model have the optimal OOD detection capability? If not, how can we find more appropriate model for OOD detection?
In this work, we start by revealing an important observation (as illustrated in Figure D.. i.e., there exists historical training stage where the model has a higher OOD detection performance than the final well-trained one. This is generally true across different OOD/ID datasets (Nezzer et al. 2011 Van Horn et al. 2018, Cimpoi et al. 2014), learning rate schedules (Loshchilov & Hutter 2017), and model structures (Huang et al., 2017 Zagoruyko & Komodakis] 2016). The empirical results of Figure reffect the inconsistency between gaining better OOD detection capability (Nguyen) et al. 2015) and pursuing better performance on ID data. We delve into the differences between the intermediate model and the final model by visualizing the misclassified examples. As shown in Figure [ one possible attribution for covering the detection capability should be memorizing the atypical samples (at the semantic level) that are hard to learn for the model. Seeking zero error on those samples makes the model more confident OOD data (see Figures [(b) ((c).

---

Under review as a conference paper at ICLR 2023
100
IDICIAR-111 CODISVHN)
Textures INsturllst
0.35
0.35
0OD(SVHN)
80
0.25
0.25
FPR95
60
0.15
6.15
40
20
0.05
0.05
0.00
-30
-15
0.00
Epochs
Score
Score
(a) Curves of FPR95 based on Energy
(b) Epoch 60
(c) At Epoch 100
Figure 1: (a) the curves of FPR95 (false positive rate of OOD examples when the true positive rate of in-distribution examples at 95%) based on the Energy score (Liu al. 2020 across three different OOD datasets during the training on the CIFAR- 10 dataset. (b) comparison between ID and OOD distribution at Epoch 60. (c) comparison between ID and OOD distribution at Epoch 100. All the experiments testing for OOD detection performance have been conducted multiple times. By backtracking the training phase, we can observe the existence of the model stage with better OOD) detection capability using the Energy score distinguish the OOD inputs. When zooming in the II) and OOD distributions at Epoch 60 and Epoch 100 respectively, it can be seen that, along with the training the later stage, the overlap between them grows. Figure 2 contains further exploration.
The above analysis inspires us to propose a new strategy, namely, Unleashing Mask (UM), to excavate the once-covered detection capability of well-trained given model by alleviating the memorization of those atypical samples (as illustrated in Figure 3 of ID data. In general, we aim to backtrack its previous stage with a better OOD detection capability. To achieve this target, there are two essential issues: (1) the model that is well-trained on ID data has already memorized some atypical samples; (2) how forget those memorized atypical samples considering the given model? Accordingly, proposed UM contains two parts utilizing different insights to address the above problems. First, as atypical samples are more sensitive the change of model parameters, the specific cutting rate to mine these samples with constructed discrepancy. Then, with the loss reference estimated by the mask, we conduct the constrained gradient ascent (i.e., Eq. 33 for model forgetting. It will encourage the model to finally stabilize around the optimal stage. To avoid the severe sacrifice of the original task performance on ID data, we further propose UM Adopts Pruning (UMAP) which performs the tuning on the introduced mask with the newly designed objective.
initialize mask with
For our proposed methods, we conduct extensive experiments characterize and understand the working mechanism (in SectionFannt Appendix F). The comprehensive results accordingly demon- strate their effectiveness. We have verified the effectiveness of UM with series of OOD detection benchmarks considering the two different ID datasets, i.e., CIFAR-10 10 and CIFAR-100. Under the various evaluation metrics, our UM, as well as UMAP, can indeed excavate the better OOD detection capability of given models and the averaged FPR95 can be reduced by significant margin. Finally, range of ablation studies and further discussions related to our proposed strategy are provided. We summarize main contributions follows,
Conceptually, explore the OOD detection performance via new perspective, i.e., back- tracking the model training phase without regularizing by any auxiliary outliers, different from most previous works that start with the well-trained model ID data.
Empirically, we reveal the potential detection capability of the well-trained model. We observe the general existence of intermediate stage where the model has more appropriate discriminative features that can be utilized for OOD detection.
Technically, we introduce new strategy, i.c., Unleashing Mask, to excavate the once-covered OOD detection capability of a given model. By introducing the mask, estimate the loss constraint for forgetting the atypical samples and empower the detection performance.
Experimentally, conduct extensive explorations to verify the general effectiveness improving the OOD detection performance of our methods. Using various ID and OOD benchmarks, provide comprehensive results across different setups and further discussion.
2

---

Under review as a conference paper at ICLR 2023
2
BACKGROUND
In this section, we briefly introduce the preliminaries and related work about OOD detection.
2.1 PRELIMINARIES
We consider multi-class classification as the training task, where X' C _2 denotes the input space and y = (l..... ......... C'} denotes the label space. Given the model deployed in the real world, the reliable classifier expected to figure out the OOD input, which can be formulated as a binary classification problem. Given P, the distribution over X x V, we consider Dm as the marginal distribution of P for X, namely, the distribution of ID data. At test time, the environment can present distribution Dou over X' of OOD data. In general, the OOD distribution Dou is defined as an irrelevant distribution of which the label set has no intersection with )' Yang et al. (202I) and therefore should not be predicted by the model. The decision can be made with the threshold A:
II
S(x) 2 A S(x) <
Dx(2; f) = OOO
Building upon the model f E H: X' + R trained on ID data with the logit outputs, the goal of decision is to utilize the scoring function S + R to distinguish the inputs of Din from that of
Doat by Szx). Typically, if the score value is larger than the threshold X, the associated input is
classified as ID and vice versa. We consider several representative scoring functions designed for
OOD detection, e.g., MSP (Hendrycks & Gimpel (017b). ODIN (Liang et al. [0188) and Energy (Liu
ct al.
2020 More detailed definitions and implementation can refer to Appendix A
To mitigate the issue of over-confident predictions for (Hendrycks & Gimpel| 2017b Liu al.
2020) some OOD data, recent works (Hendrycks et al.] [2019c] Tack et al.] (0200) utilize the auxiliary
unlabeled dataset to regularize the model behavior. Among them, one representative baseline is
outlier exposure (OE) (Hendrycks et al. 2019c). OE can further improve the detection performance
by making the model (()) finetuned from surrogate OOD distribution D Dow- and its corresponding
learning objective is defined as follows,
Lj = ED, [(EE((()) + XE [foR(
where À is the balancing parameter, fcE(.) is the Cross-Entropy (CE) loss, and for(.) is the Kullback-Leibler divergence to the uniform distribution, which can be written as (om(h(x)) = -E softmaxı (x)/CC where sof tmax(()) denotes the k-th element of a softmax output. The OE loss (oF(-) is designed for model regularization, making the model learn from surrogate OOD inputs return low-confident predictions. The general formulation of Eq 2
is also adopted in other related works for designing better tuning objectives that use different auxiliary outlier data.
Although previous works show promising results via designing scoring functions or regularizing models based on the model f trained on ID data, few of them investigated the original detection capability of the well-trained given model. In this work, we introduce the layer-wise mask m (Han) et al. 2016; Ramanujan et al. (020) to mine the atypical samples that memorized by the model. Accordingly, the decision can be written as D(1; m f), and the output masked model is m m (xx
2.2 RELATED WORK
Out-of-distributin Detection without auxiliary data. (Hendrycks & Gimpel| 2017a) formally shed light on outto--distriuution detection, proposing to use softmax prediction probability as bascline which is demonstrated to be unsuitable for OOD detection. Subsequent works keep focusing on designing post-hoe metrics to distinguish IID samples from OOD samples, among which ODIN (Liang et al.) 2018) introduces small perturbations into input images to facilitate the separation of softmax score, Mahalanobis distance-bases confidence score (Lee et al., 201 2018b) exploits the feature space by obtaining conditional Gaussian distributions, energy- score (Liu et al. 2020) aligns better with the probability density. Besides designing score functions, many other works pay attention to various aspects to enhance the OOD detection such that LogitNorm (Wei et al.) (022) produces confidence scores by training with a constant vector norm on the logits, and DICE (Sun & Li 2022) reduces the variance of the output distribution by leveraging the sparsification of the model.
3

---

Under review as a conference paper at ICLR 2023
100
100
300
0-4
Loss Value
0-2
FPRSS
100
0.5
Trainig Texiing AL
Epochs
Epochs
Epochs
Epocha
(a) Sraining/Testing Loss (b) Training/Testie Acc.
(c) ODIN Score
(d) Energy Score
airptane
(e) Correctly classified samples at Epoch 60
(I) Mis-classified samples learned after Epoch 60
Figure 2: We train CIFAR-11 10 for the original multi-class classification and check the details as follows: (a) training/testing loss on ID data; (b) training/testing accuracy on ID data; (c) curves of FPR95 based on ODIN score; (d) curves of FPR95 based on Energy score; (e) visualization of correct classified samples at Epoch 60; (f) visualization of misclassified samples at Epoch 60. We investigate the once-covered OOD detection capability by checking the model behavior during its training phase. We take closer look at the corresponding training and testing performance with the OOD detection capability indicated by two different scores. Through comparison, we find that achieving a reasonably small loss value (at round Epoch 60) on ID data is enough for OOD detection. However, continually optimizing on those atypical samples can impair the detection performance.
Detection with auxiliary data. Another promising direction towards OOD detection involves the auxiliary outliers for model regularization. On one hand, some works generate virtual outliers such that Lee et al. (2018a) uses generative adversarial networks to generate boundary samples, VOS (Du et al. (2022a) regularizes the decision boundary by adaptively sampling virtual outliers from the low-likelihodd region. On the other hand, other works tend to exploit information from natural outliers, such that outlier exposure (OE) is introduced by Hendrycks et al.) (2019)), given that diverse data available in enormous quantities. (Yu & Aizawa 2019) train additional "head" and maximizes the discrepancy of decision boundaries of the two heads to detect OOD samples. Energy-bounded learning (Liu et al. 2020) fine-tunes the neural network widen the energy gap by adding an energy loss term to the objective. Some other works also highlight the sampling strategy, such that ATOM (Chen al., [021] greedily utilizes informative auxiliary data tighten the decision boundary for OOD detection, and POEM (Ming et al. (022) adopts Thompson sampling to contour the decision boundary precisely. The performance training with outliers is usually superior that
without outliers, shown in many recent works (Mohseni et al. 2020 Liu et al. 2020 Fort et al. 2021
Sun et al. 2021 Sehwag et al. 2021 Yang et al. 2021 Chen et al. 2021 Salchi el al. 2021).
PROPOSED METHOD: UNLEASHING MASK
In this section, we introduce our new method, i.e., Unleashing Mask (UM), to reveal the potential OOD detection capability of the well-trained model. First, we present and discuss the important observation that inspires our methods (Section (.I). Second, we provide the insights behind the two critical parts of our UM (Section 3.2h. Lastly, we describe the framework and the learning objective of UM that incorporates the previous component, as well as its variant, i.e., UMAP (Section 3.3
3.1 ONCE-COVERED OOD DETECTION CAPABILITY
First, present the phenomenon of the inconsistent trend between better OOD detection capability and smaller training eттоr during training. Empirically, as shown in Figure [ we plot the OOD detection performance during the model training after multiple runs of the experiments. Across three
4

---

In general, the models that are developed for the original classification tasks are always seeking better performance (i.e., higher testing accuracy and lower training loss) in practice. However, the inconsistent trend revealed before indicates that the intrinsic OOD detection capability maybe Under review as a conference paper at ICLR 2023
different OOD testing datasets, can observe the existence of a better detection capability using the index of FPR95 metric based on the Energy (Liu et al., 2020) or ODIN (Liang et al. 2018)) score. The generality has also been verified using different learning schedules and model structures in experimental sections. We further show the comparison of the ID/OOD distributions in Figures| ((b) [[e] To be specific, the statics of the two distributions indicate that the gap between the IID and OOD data gets narrow as their overlap grows along with the training. After Epoch 60, although the model becomes more confident on ID data which satisfies part of the calibration target (Hendrycks et al. 2019a), its predictions on the OOD data also become more confident which is unexpected. Without seeing any auxiliary outliers, motivates us to explore how the model achieves that.
We take closer look at the model behaviors in Figure 2 where we check its corresponding train- ing/testing loss and accuracy. We find that the training loss has reached a reasonably small value at Epoch 60 where its detection performance also achieves a satisfactory level. However, if we further minimize the training loss, the trend of the FPR95 curve shows almost opposite directions with both
training and testing loss or accuracy (see Figures ((a) 2(b) and 2 Accordingly, we extract
those samples that were learned by the model at this period. As shown in Figures 2 and 2(1)
the misclassified samples learned after Epoch 60 present much atypical semantic features. As deep neural networks tend to first learn the data with typical features (Arpit et al.] (017)). we attribute the
inconsistent trend memorizing those atypical data the later stage. In Appendix [ we provide detailed discussion between it with the concept of conventional overfitting (Goodfellow et al. 2016).
3.2 UNLEASHING THE INTRINSIC DETECTION POWER
once-covered during the training. It gives us a chance to unleash the potential detection power only considering the ID data in training. To this end, we have two important issues that need to address: (1) the model that is well-trained on ID data may have already memorized some atypical samples which can not be figured out; (2) how to forget those atypical samples considering the given model?
Mining the atypical samples with constructed discrepancy. According to Figures ((a) and 2(b) both training accuracy and loss provide limited information that can differentiate the typical and atypical data. Inspired by the learning dynamics (Goodfellow et al. 2016 Arpit et al., 2017) of deep neural networks and the pathway conjecture (Barbam et al. 2022) for inference, we try to manually construct the parameter discrepancy to mine the atypical samples from well-trained model. To be specific, we introduce novel layer-wise mask to achieve the goal. The masks are applied to all layers, which is consistent with the mask generation in the conventional pruning pipeline (Han et al., (016)) In Figure 3( we provide empirical evidence to show that we can figure out the atypical samples via enlarging the mask rate. Utilizing the masked output for loss computation, the atypical samples can be better differentiated. We also provide more discussion about the intuition in Appendix B
Forgetting the atypical samples with gradient ascent. As the training loss achieves zero at the final stage of the given model, we need extra optimization signals forget those memorized atypical samples. Considering the previous consistent trend before the potential optimal stage (e.g., before Epoch 60 in Figure ((a). the optimization signal also needs to control the model update not be too greedy to drop the discriminative features for OOD detection. Starting with the given model, we can employ the gradient ascent (Sorg et al. (2010) [shida et al. (0220) to forget the targeted samples, while the tuning phase should also prevent further updates if the model can achieve the expected stage.
3.3 METHOD REALIZATION
Based on previous insights, we present our overall framework as well as the learning objective of the proposed Unleashing Mask for OOD detection. Lastly, we discuss its compatibility with either the fundamental scoring functions or the outlier exposure approaches utilizing auxiliary outlier data.
framework consists of two critical components for uncovering the intrinsic OOD detection capability: (1) the initialized mask with a specific pruning rate for constructing the output discrepancy with the original model; (2) the fine-tuning procedure for
Framework. As illustrated in Figure 3(a)
5

---

Under review as a conference paper at ICLR 2023
1.75
bird automabil aabbbabbe
Fadaar nil Toeyeiig
Training
Orggial Modil
Thinirg Bats
1.00
Controt
Sapple
Unlkmizzgg Mark
dlgg
0.25
horsa
0.90
0.92
0.24
3.90
0.98
1.00
0.92
0.95
0.98
(a) Illustration of Unleashing Mask (UM)
(b) Mask Rate w.r.t. Loss
(c) Mis classiasd data
Figure 3: (a) a brief illustration of the proposed Unleashing Mask (UM); (b) the mask rate w.r.t. loss value using the masked outputs; (c) examples of misclassified samples after masking the original well-trained model. As for our framework, given well-trained model, initialize an extra mask for mining the atypical samples that are sensitive to the changes in model parameters. Then we fine-tune the original model or adopt pruning with the estimated forgetting threshold, i.e., the loss value estimated by the UM. The final model can serve as the base of various score functions to utilize the discriminative features and also as the new initialization of fine-tuning with the auxiliary outliers.
alleviating the memorization of atypical samples. The overall workflow starts with obtaining the loss value of misclassifying those atypical samples and then conducts tuning with the model to forget.
Objective for forgetting, Based on our framework, we introduce the forgetting objective as,
min Lum min |fCE(f) - (mm @ f*) Eca(ms f").
(3)
m=E[0,1]"
where m& our proposed layer-wise wise mask with the pruning rate 6, lcg is the CE loss, fcz is the averaged CE loss over the ID training data, | indicates the computation for absolute value and my @ f denotes the masked output of the fixed pretrained model that used estimate the loss constraint for the learning objective of forgetting, which would be constant value during the whole finetuning process. Concretely, the well-trained model will start to optimize itself again if memorized the atypical samples and achieved almost zero loss value. We provide positive gradient signal when the current loss value is lower than the estimated one and vice versa. The model is expected to finally stabilize around the stage that can forget those atypical samples.
Unleashing Mask Adotts Pruning (UMAP). Considering the potential negative effect on the original task performance when conducting tuning for forgetting, further propose variant of UM Adopts Pruning, i.e., UMAP, to conduct tuning based on the masked output (e.g., replace fcE(f) to (mp( f*) in Eq using the different mask my with its pruning rate p follows,
min min LUMAP =.. (0, 1).
( O f*) Fce(ms C f*)] + (m 3 f*),
(4)
Different from the objective of UM (i.e., Eq that minimizes the loss value over the model parameter, the objective of UMAP minimizes the loss over the mask to achieve the target of forgetting those
atypical samples. UMAP provides an extra mask restore the detection capacity but doesn't affect
the model parameter for the inference on original tasks, indicating that UMAP is a more practical choice in real-world applications (as empirically verified in our experiments like Table D We summarize the algorithms of UM (in Algorithm and UMAP (in Algorithm 2 in Appendix
Compatible to other methods. As we explore the original OOD detection capability of the well- trained model, it is orthogonal and compatible with those promising methods that equip the given model with better detection ability. To be specific, through our proposed methods, we reveal the once- covered OOD detection capability via tuning the original model towards its intermediate training stage. The discriminative feature learned at that stage can be utilized by different scoring functions (Huang et al. 202 Sun & 2022, Wei et al. (0022), like ODIN (Liang et al.) 2018)) adopted in Figure 2(c) For those methods (Hendrycks et al. 20119| Liu et al. 2020 Ming et al. 2022) utilizing the auxiliary
outliers to regularize the model. our fine-tume tuned model obtained by UM and UMAP can also serve as
their starting point or adjustment. As our strategy does not require any auxiliary outlier data to be
involved in training, adjusting the model using ID data during its developing phase is practical.

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTS
In this section, we present the performance comparison of the proposed method in the OOD detection scenario. Specifically, we verify the effectiveness of our UM and UMAP with two mainstreams of OOD detection approaches: (i) fundamental scoring function methods; (ii) outlier exposure methods involving auxiliary samples. To better understand and characterize our proposed method, we further conduct extensive explorations on the ablation study and provide the corresponding discussion on each sub-aspect considered in our work. More details and additional results can also refer to Appendix
4.1 EXPERIMENTAL SETUPS
Datasets.
Following the common benchmarks used in previous work (Liu et al. 2020; Ming et al. 2022), we adopt CIFAR- 10, CIF 11FAR-100 (Krizhevsky 2009) as ID datasets. We use series of different image datasets as the OOD datasets, namely Textures (Cimpoi et al. 2014), P P Places 365 (Van Homn et al. 2018). We also use the other ID dataset as OOD dataset when training on a specific II) dataset, given that none of them shares any same classes (Yang al. (02T). e.g. we treat CTFAR-100 as the OOD dataset when training on CIFAR- 10 in our experiments for comparison.
(Zhou et al. 2017), SUN (Xiao al. 2010), LSUN (Yu et al., 2015), and
Training details. We conduct all major experiments DenseNet-101 (Huang al. 2017 with training epochs fixed to 100. We also include experiment results on other types of models in the Appendix [ The models are trained using stochastic gradient descent (Kiefer & Wolfowitz 1952) with Nesterov momentum (Duchi et al.) 2011). We adopt Cosine Annealing (Loshchilov & Hutter 2017) to schedule the learning rate which begins at 0.1. We set the momentum and weight decay to be 0.9 and 10 respectively throughout all experiments. The size of the mini-batch is 64 for both ID samples (when training and testing) and OOD samples (when testing). More details and further discussion about choosing the mask ratio in experiments can be referred to at the end of Appendix
Evaluation metrics. We employ the following three common metrics evaluate the performance of OOD detection: (i) Area Under the Receiver Operating Characteristic curve (AUROC) (Davis) Goadrich 2006) can be interpreted as the probability for a positive sample to have a higher discriminating score than a negative sample (Fawcett 2006); (ii) Area Under the Reccll curve (AUPR) (Manning & Schütze 1999) is an ideal metric to adjust the extreme difference between positive and negative base rates; (ini Faise Positive Rate (FPR) at 95% True Positive Rate (TPR) (Liang et al., 2018)) indicates the probability for a negative sample to be misclassified as positive when the true positive rate is at 95%. We also include in-distribution testing accuracy (ID-ACC) to reflect the preservation level of the performance for the original classification task on ID data.
OOD detection baselines. We compare the proposed method with several competitive baselines in the two directions. Specifically, we adopt Maximum Softmax Probability (MSP) (Hendrycks & Gimpel| 2017a), ODIN (Liang et al. 2018), Mahalanobis score (Lee et al.) 2018b), and Energy score (Liu et al. (0200) as scoring function baselines; We adopt OE (Hendrycks et al.) 2019b), Energy- bounded learning (Liu et al. 2020), and POEM (Ming et al. (022) baselines with outliers. For all scoring function methods, we assume the accessibility of well-trained models. For all methods involving outliers, we constrain all major experiments to fine-tuning scenario, which is more practical in real cases. Different from training a dual-task model at the very beginning, equipping deployed models with OOD Detection ability is much more common circumstance, considering the millions of existing and running deep learning systems. We leave more details in Appendix A
4.2 PERFORMANCE COMPARISON
In this part, we present the performance comparison with some representative baseline methods to demonstrate the effectiveness of our UM and UMAP Our proposed UM is designed for excavating the potential OOD detection capability of the given model. Here we consider several scoring functions to compare the detection performance, and also some outlier exposure methods to further regularize the given model and boost the OOD detection ability. In each category. we choose one with the best detection performance adopt UM/UMAP and check the detection results with the ID-ACC.
7

---

Under review as a conference paper at ICLR 2023
Table 1: Main Results (%). Comparison with competitive OOD detection baselines.
Din
Method
AUROC
AUPRI
FPR95]
ID-ACCT
w./.co Daux
MSP(Hendrycks & Gimpel, 72
89.90 + 0.30
91 91
+ 0.43
00.08 = 0.76
94,01 = 0.08
0.56
0.58
42.31
1.38
94,01 0.08
Mahalamonss Lee mh 2018b
75.10
1.04
72.32
1: 1.92
61.35
1. 1.25
94,01
0.08
Energy
92.07
0.22
92.72
0.39
42.00
1.31
94.01
Energy+UM [ours]
93.73 0.36
91.27
0 0.60
33.29
1.70
92.80 = 0.17
CIFAR-10
Energy+UMAP (ours)
93.97 : 0.11
94.38 =: 0.06
30.71
1.94
0.08 94.01
Hendrycks et 2019b
97.07 0.01
97.31
0.05
13.80
0.28
92.59
0.32
Energy (w. Daur Li
2020
94.58 + 0.64
94.69 + 0.65
18.79 + 2.31
80.91
3.13
Ming et al. 2052
94.37 0.07
91.51 0.06
18.50
0.33
77.24 =
OE+UNT (ours)
97.60 1: 0,03
97.87 + 0.02
11.22
0,16
93.66
0.12
OE-UMAP (ours)
97.48 0.01
97.74 1. 0.00
12.21
0.09
0.08 94,01
MSP(Henddycks & Gimpel 7017a]
74.06 0.69
76.37 +: 0.73
83.14
0.87
74.86
0.21
ODIN Cral. 2018
76.18 0.11
76.49 + 0.20
78.93
0.31
74,86
0.21
Mahalanooist Lee el sl
2018b
63.00
64.31 + 0.91
78.79
0.30
74.86
0.21
Energy (Liu
76,29 0.24
77.06 0.55
78.46
0.06
74.86
0.21
EnergyFUM (olns)
76.22 0.42
76.39 + 1.03
0.55 74.05
64.55 + 0.24
CIFAR-100
Energy+UMAP (ours)
75.57
75.66
1.46 72.21
74.86 0.21
OE Hendrycks al. 2014b
90.55 1: 0.87
90.34 +: 0.94
34.73 + 3.85
73.39 1 0.30
Energy LW. Zaue 1al et 2020
88.92 +: 0.57
89.13 J: 0.56
37.90
+: 2.59
57.85 2.65
POEM Ming al.
88.95 0.54
88.94 + 0.31
38. 1.30
58 1.92
OE+UNI (ouns)
91.01=0.11
91.13 = 0.24
31.71
= 0.81
75.15 0.18
OE+UMAP (ours)
0.16
90.99
33,62 0.26
74.76
Table 2: Fine-grained Results (%). Comparison on different OOD benchmark datasets.
OOD dataset Textures
dataset
Method
CIFAR-100
Places365
FPR95
AUROC
FPR95
AUROC1
FPR95
AUROC1
MSP
66.43 1: 1.25
87.73
0.02
65,20 1: 1.33
88.06
0.61
61.34 1 0.60
89.63
0.15
ODIN
35.31
0.85
87.75
0.37
53.11
+ 181
87.13
015=200 2.01
13.77
+ 0:20
91.70 = 0.30
Mahulamobis
0.96
64.52
0.773
20.04
0.78
86.21
64.00
1.21
Energy
51.65
1.24
89.01
1.18
57.09
+ 3.52
87.51
2 1.13
38.62
+ 1.61
93.03
2 0.20
UM (ours)
54,62
1,16
88.30
0.30
41.61
3.67
91.31
0.01
30.85
1 0.58
94,27
0.16
CIFAR-10
SUN
LSUN
iNaturalist
Method
FPR954
AUROC
EPR95,
AUROCT
BPR95]
AUROCT
MSP
60.27 :: 0.66
90.00
0.24
36.43 E: 1.94
96.17 :: 0.32
67.53 1: 1.64
88.01 :: 0.82
0.62
5.16
0.09
0.91
Mahalanobis
81.56 = 1.51
66.11
1.57
69.18
+ 3.55
66.11 = 4.57
80.76
+ 2.18
71.77
1.12
Energy
36.73
1.72
93.63
0.34
6.25
1 0.43
98.77
0.07
59.11
1 1.18
89.71
0.06
Energy+UM (ours)
27.88
0.73
94.83
0.11
2.91
1011:000 0.53
99.22
= 0.11
46.27
1 2.74
92.75 0.80
In Table we summarize the results of different OOD test sets using different methods. Note that, here the evaluation results are obtained by averaging several OOD test datasets across multiple independent trials. For the scoring- -aaed methods, our UM can further improve the overall detection performance by alleviating the memorization of atypical ID data, when the ID-ACC keeps comparable with the baseline, For the complex CIFAR-11 -000 dataset, our UMAP can be adopted as a practical way to empower the detection performance and simultaneously avoid affecting the original performance on ID data. As for those methods of the second category (i.e., involving auxiliary outlier Dauz sampling from ImageNet), since consider a practical workflow, i.e., fine-tuning, the given model, OE achieves the best performance on the task. Due to the special optimization characteristic, Energy (w.Da and POEM focus more on the energy loss on differentiating OOD data while performing not well on the preservation of ID-ACC. Without sacrificing much performance on ID data, OE with our UM can still achieve better detection performance. In Table the fine-grained detection performance on each OOD testing set demonstrates the general effectiveness of UM. We have comprehensively verified the significant improvement (up to 18% reduced on averaged FPR95) in OOD detection of our methods across different setups in Appendix F the complete results can refer to Tables to 20 More fine-grained results of the experiments on CIFAR- 100 provided Table 16
and 19 In addition, we also provide similar results using another model structure in Table 18
4.3
ABLATION AND FURTHER DISCUSSION
In this part, conduct various explorations provide thorough understanding of our presented Unleashing Mask from different perspectives. To be specific, we first present the general existence of the once-covered intrinsic OOD detection capability across different settings. Second, we present
8

---

Under review as a conference paper at ICLR 2023
010 has
Mock
Bate
Manoor cop
Pax
090
(a) Ir Schedule
(b) FPR95 Curves
(c) w./.o. Mask
(d) Masking Ratios (c) Pruning w. Mask
Figure 4: Ablation studies. (a)-(b) exploring the existence of potential OOD detection capability with different learning rate schedules; (c) comparison of loss value using original output with masked output (the x-axis represents the index of samples within a mini-batch); (d) effects of using different masking ratios in UM; (e) comparison of using original pruning with our proposed UMAP.
the exploration about the mask, which helps to characterize its effect on figuring the atypical samples. Third, we provide further exploration on excavating the detection power via pruning.
General existence of once-covered OOD detection capability. In Figure 4(a) and Figure 4(b) we explore 4 learning rate schedules to demonstrate the general existence of once-covered OOD detection capability. To be specific, the OOD performance (indicated by FPR95) evaluated along with the training in every 5 epoch, in which the model takes CTFAR CTFAR-10 0 as IID data and SVHN as OOD data. As shown by the curves, a middle stage exists with better OOD performance than that of the final stage across different schedules. We empirically verify the existence of this phenomenon without schedule specificity. More explorations on the other ID dataset and model structure reveal similar results in Figures 5 6 and The detailed information and discussion can refer to Appendix F
Effects of the mask on mining atypical samples. Following our previous illustration of Figure ( we scrutinize the change of training loss on a random batch of the training set Figure 4(c) The results further explain why the loss value estimated by the UM can be used to force the model to forget atypical samples. It can be seen from Figure 44c that the loss is proportionally increased by randomly knocking out 2.5% weights. In this case, the estimated loss is more influenced by those who have a higher initial loss and are what we termed as atypical samples. By controlling the training loss to the estimated value, the model is encouraged to backtrack to middle training stage where samples with high loss value have little influence on the forgetting process of the gradient ascent.
Exploration on revealing detection capability with model pruning. Although the large constrain on training loss can help reveal the model's OOD performance, the ID-ACC is undermined under such cireumstances. Generally speaking, the proposed UM forces the model forget the atypical samples and may result in lower test performance. To mitigate this issue, we further adopt pruning as a countermeasure to learn a mask instead of tuning the model parameters directly. In Figure 4(c) we experiment with various prune rates p and demonstrate that we can achieve the same or better OOD performance also by pruning. Specifically, our UMAP can achieve lower FPR95 than pure pruning with the original objective. The prune rate can be selected from a wide range (e.g. P E [0.3, 0.9) to guarantee a fast convergence and effectiveness. Since pruning doesn't change the well-trained model parameters, it can preserve the performance of the original task. We also provide additional empirical results and corresponding discussion about the effectiveness of our UMAP in Appendix
5 CONCLUSION
In this work, we explore the intrinsic OOD detection capability of a well-trained model. Without involving any auxiliary outliers in training, reveal the inconsistent trend between minimizing original training loss and gaining OOD detection capability. We further attribute it to the memorization behavior of atypical samples. To excavate the once-covered capability, we propose new method, namely, Unleashing Mask (UM). Through UM, we construct model-level discrepancy that figures out the memorized atypical samples and utilizes the constrained gradient ascent to encourage forgetting. It better utilizes the given model for OOD detection via backtracking or sub-structure pruning. We hope our work could provide new insights for revisiting the model development in OOD detection.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
For the experimental setups, we have provided the details in Section 4.1 and Appendixes A and E
We will also provide the anonymous repository about our source codes in the discussion phase reviewing purposes to ensure the reproducibility of our experimental results.
REFERENCES
Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron C. Courville, Yoshua Bengio, and Simon Lacoste- Julien. A closer look memorization in deep networks. In ICML,, 2017.
Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Daniel Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, et al. Pathways: Asynchronous distributed dataflow for ml. In MLSys, 2022.
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias-variane trade-off. In Proceedings of the National Academy of Sciences, 2019.
Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. Atom: Robustifying out-of- distribution detection using outlier mining. In ECML PKDD, 2021.
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In CVPR, 2014.
Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In ICML, 2006.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Xuefeng Du, Zhaoning Wang, Mu Cai, and Sharon Li. Towards unknown-aware learning with virtual outlier synthesis. In ICLR, 2022a.
Xuefeng Du Zhaoning Wang. Mu Cai, and Yixuan Li. VOS: learning what you don't know by virtual outlier synthesis. In ICLR, 2022b.
John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. In JMLR, 2011.
Tom Fawcett. An introduction roc analysis. In Patlern Recognition Letters, 2006.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distrribtion detection. In NeurlPS, 2021.
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Training pruned neural networks. In ICLR, 2019.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning. MIT Press, 2016.
Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, Yingbo Zhou, Chetan Ramaiah, Fangxiang Feng, Ruifan Li, Xiaojie Wang, Dimitris Athanasakis, John Shawe-Taylor, Maxim Milakov, John Park, Radu lonescu, Marius Popescu, Cristian Grozea, James Bergstra, Jingjing Xie, Lukasz Romaszko, Bing Xu, Zhang Chuang, and Yoshua Bengio. Challenges in representation learning: A report on three machine learning contests. In Minho Lec, Akira Hirose, Zeng-Guang Hou, and Rhee Man Kil (eds.), NewrIPS, Springer Berlin Heidelberg, 2013.
Song Han, Huizi Mao, and William J Dally. Deep compression: with pruning, trained quantization and huffman coding. In ICLR, 2016.
Compressing deep neural networks
10

---

Under review as a conference paper at ICLR 2023
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: Data mining, inference, and prediction, second edition. Springer New York, NY, 2009.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017a.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017b.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In ICML, 2019a.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019b.
Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019c.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.
Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. In NeurlPS, 2021.
Takashi Ishida, Ikko Yamane, Tomoya Sakai, Gang Niu, and Masashi Sugiyama. Do we need zero training loss after achieving zero training error? In ICML, 2020.
Kiefer and J. Wolfowitz. Stochastic estimation of the maximum of a regression function. In Ann. Math. Statist, 1952.
Alex Krizhevsky. Learning multiple layers of features from tiny images. In arXiv, 2009.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-caliiraaeed classifiers for detecting out-of-distribution samples. In ICLR, 2018a.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurlPS, 2018b.
Shiyu Liang, Yixuan Li. and R. Srikant. Enhancing the reliability of out-of-distribution image detection neural networks. In ICLR, 2018.
Ziqian Lin, Sreya Dutta Roy, and Yixuan Li. Mood: Multi-level out-of-distribution detection. In CVPR, 2021.
Weitang Liu, Xiaoyun Wang, John D Owens, and Yixuan Li. Energy-based out-of-distribution detection. In NeurlPS, 2020.
Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In ICLR, 2017.
Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Process- ing. The MIT Press, Cambridge, Massachusetts, 1999.
Yifei Ming, Ying Fan, and Yixuan Li. Poem: Out-of-diisributiin detection with posterior sampling. In ICML, 2022.
Sina Mohseni, Mandar Pitale, J. B. S. Yadawa, and Zhangyang Wang. Self-supervised learning for generalizable out-of-distiibutioo detection. In AAAI, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning, NeurIPS Workshop, 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.

---

Under review as a conference paper at ICLR 2023
Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari. What's 's hidden in a randomly weighted neural network? In CVPR, 2020.
Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Rohban, and Mohammad Sabokrou. A unified survey on anomaly, novelty, open-set, and out-of-distribution detection: Solutions and future challenges. In arXiv, 2021.
Vikash Sehwag, Mung Chiang, and Prateek Mittal. SSD: A unified framework for self-supervised outlier detection, In ICLR, 2021.
Jonathan Sorg, Richard Lewis, and Satinder Singh. Reward design via online gradient ascent. In NeurlPS, 2010.
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. In J. Mach. Learn. Res., 2014.
Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In ECCV, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. In NeurlPS, 2021.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In ICML, 2022.
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. CSI: novelty detection via contrastive learning on distributionally shifted instances. In NeurIPS, 2020.
Amirhossein Tavanaei. Embedded encoder-decoder convolutional networks towards explainable AI. In arXiv, 2020.
William R Thompson. On the likelihood that unknown probability exceeds another in view of the evidence of two sample. In Biometrika, 1933.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In CVPR, 2018.
Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. In ICML, 2022.
Jianxiong Xiao, James Hays, Krista A. Fhinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In IEEE Computer Society Conference Computer Vision and Pattern Recognition, 2010.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized dit-of-distribution detection: survey. In arXiv, 2021.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. In arXiv, 2015.
Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distribution detection by maximum classifier discrepancy. In ICCV, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Bolei Zhou, Agata Lapedriza, Antonio Torralba, and Aude Oliva. Places: image database for deep scene understanding, In Journal of Vision, 2017.
12

---

Under review as a conference paper at ICLR 2023
APPENDIX
REPRODUCIBILITY STATEMENT
We will provide the anonymous repository about source codes in the discussion phase for reviewing purposes to ensure the reproducibility of our experimental results. Below we summarize some critical aspects to facilitate the reproducible results:
Datasets. The datasets we used are all publicly accessible, which is introduced in Section 4.1 For methods involving auxiliary outliers, we strictly follow previous works (Sun et al., 2021
Du et al. 2022b) avoid overlap between the auxiliary dataset (ImageNet-IK) (Deng et al. 2009) and any other OOD datasets.
Assumption. We set our experiments post-hoc scenario where well-trained model is available, and some parts of training samples are also available for subsequent fine-tuning.
Open source. The code repository will be available in an anonymous repository for reviewing purposes. We provide a backbone of our experiments well as several auxiliary components, such as score estimation.
Environment. All experiments conducted multiple runs NVIDIA Tesla V100-SXM2- 32GB GPUs with Python 3.6 and PyTorch 1.8.
A DETAILED INFORMATION ABOUT THE USED BASELINES AND METRICS
In this section, we provide the details about the baselines for the scoring functions and finetuning with auxiliary outliers, and the corresponding hyper-parameters that are considered in our work.
Maximum Softmax Probability (MSP). (Hendrycks & Gimpel 2017a) proposes to use maximum softmax probability to discriminate ID and OOD samples. The score is defined as follows,
Susp(2; (2; max Pvy cl2;
softmax(f(z))
where f represents the given well-trained model and e is one of the classes y = il,... CJ. The larger softmax score indicates the larger probability for sample be ID data, reflecting the model's confidence on the sample.
ODIN.
(Liang al., 2018) designed the ODIN score, leveraging the temperature scaling and tiny perturbations to widen the gap between the distributions of ID and OOD samples. The ODIN score is defined as follows,
SODIN (r; f) max P(y c|ē: f) max softmax f(i)
(6) where represents the perturbed samples (controled by c), 7 represents the temperature. For fair comparison, we adopt the suggested hyperparameters (Liang al. 2018): 1.4 x 10-3 T 1.0 x 1
Mahalanobis. Lee al. 2018b) introduces a Mahalanobis distance-based confidence score, exploiting the feature space of the neural networks by inspecting the class conditional Gaussian distributions. The Mahalanobis distance score defined as follows,
SMahalanobis (x; f) = max (f((x) - -a.)) L (f(x)) - Po)
c
where fec represents the estimated mean of multivariate Gaussian distribution class c, represents the estimated tied covariance of the C' class-conditional Gaussian distributions.
Energy. (Luu et al.) 2020) proposes to use the Energy of the predicted logits to distinguish the ID and OOD samples. The Energy score defined as follows,
Senergy (z; = -7 log
f(x)./T
(8)
13

---

Under review as a conference paper at ICLR 2023
where T represents the temperature parameter. As theoretically illustrated in Liu al. (2020) lower Energy score indicates higher probability for a sample to be ID. Following (Liu et al. 2020). we fix the T to 1.0 throughout all experiments.
Outlier Exposure (OE). (Hendrycks et al. 2019b) initiates promising approach towards OOD
detections by involving outliers to force apart the distributions ID and OOD samples. In the experiments, use the cross-entropy from (zom)) the uniform distribution as the Loe (Lee et al. 2018a),
C = Ep, (c)((()) + XE, log
(+), ED (f(x))
(9)
Energy (w. Daux). In addition to using the Energy as post-hoc score to distinguish ID and OOD) samples, (Liu et al. 2020) proposes an Energy-bounder objective further separate the two distributions. The OE objective is as follows,
Cof Eng (0ax(0, Fnergy (x. J) min))² Eng, (iros.(maan (max((), Mout S (2r, l)))
(10)
We keep the thresholds same (Liu et al. 2020): minn -25.0, Mout -7.0.
POEM.
(Ming et al. 2022) explores the Thompson sampling strategy Thompson 1933 to make the most use of outliers to learn tight decision boundary. Though given the POEM's nature to be orthogonal to other OE methods, use the Energy(w. Daux) as the backbone, which is the same as Eq.( in Liu al. 2020). The details of Thompson sampling can refer to MMig et al. (2022).
Detailed formulations FPR and TPR. Suppose we have binary classification task (to predict an image to be ID or OOD sample in this paper). There two possible outputs: positive result (the model predicts an image to an ID sample); negative result (the model predicts an image to be an OOD sample). Since we have two possible labels and two possible outputs, form confusion matrix with all possible outputs as follows.
Table 3: Confusion Matrix.
Truth: ID
Truth: OOD
Predict: ID
True Positive (TP)
False Positive (FP)
Predict: OOD
False Negative (FN)
True Negative (TN)
Therefore, the false positive rate (FPR) calculated as
FP
FPR FP+TN
(11)
The true positive rate (TPR) is calculated as:
TP TPR TP+FN
(12)
B ADDITIONAL EXPLANATION TOWARDS MINING THE ATYPICAL SAMPLES
First, for identifying those atypical samples using a layer-wiss mask with the well-pre- rell-pre-trained model, the core intuition behind is constructing the parameter-leva discrepancy mine the atypical samples. It is inspired by and based on the evidence drawn from previous literature about learning behaviors (Arpit et al. 2017 Goodfellow et al. 2016) deep neural networks (DNNs) and sparse representation (Frankle & Carbin 2019 Goodfellow et al. 2013 Barham et al.) 2022). To be specific, the atypical samples tend to be learned by the DNNs later than those typical samples (Arit et al. 2017). and are relatively more sensitive to the changes of the model parameter the model does generalize well on that. By the layer-wise mask, the constructed discrepancy can make the model misclassify the atypical samples and estimate loss constraint for the forgetting objective, visualized in Figure 3(c
14

---

Under review as a conference paper at ICLR 2023
Second, introducing the layer-wist mask has several advantages for achieving the staged target of mining atypical samples in our proposed method, while we would also admit that the layer-wise mask is not irreplaccable option maybe not optimal. On the one hand, considering that the model has been trained to approach the zero error on training data, utilizing the layer-wise mask is an integrated strategy to 1) figure out the atypical samples and 2) obtain the loss value computed by the masked output that misclassifies them. The loss constraint is later used in the forgetting objective to fine- tue-ttne the model. On the other hand, the layer-wise mask is also compatible with the proposed UMAP to generate a flexible mask for restoring the detection capability of the original model.
Third, we also adopt the unit/weight mask and visualize the misclassified samples in Figure 12 We think they can also be used to mine the atypical samples and can be extended or improved to be more flexible choice. Further investigating the specific effect of different methods that construct the parameter-level discrepancy would be an interesting sub-topic in future work. For the value of CE loss, although the atypical samples tend to have high CE loss value, they are already memorized and correctly classified as indicated by the zero training error. Only using the high CE error can not provide the loss estimation when the model does not correctly classify those samples.
C CONCEPTUAL AND EMPIRICAL COMPARISON WITH OVERFITTING
First of all, we would refer to the concept of the conventional overfitting (Goodfellow et al. 2016 Belkin et al., 2019), i.c., the model "overfit" the training data but fail to generalize and perform well on the test data that is unseen during training. The common empirical reflection of overfitting is that the training error is decreasing while the test error is increasing at the same time, which enlarges the generalization gap of the model. It has been empirically confirmed not the case in our observation as observed in Figure [(a] and 2(b) To be specific, for the original classification task, there is no conventional overfitting observed the test performance is still improved the later training stage, which is a general pursuit of the model development phase on the original tasks.
Then, when we consider the OOD detection performance of the well well-pretrained model, our unique observation is about the inconsistency between gaining better OOD detection capability and pursuing better performance on the original classification task for the di-triibbuutonn (ID) data. It worth noting that here the training task is not the binary classification of OOD detection, but the classification task on ID data. It is out of the rigorous concept of the conventional overfitting and has received limited focus and discussion in the previous literature about OOD detection to the best of our knowledge, Considering the practical scenario that exists target-level discrepancy, our revealed observation may encourage us to revisit the detection capability of the well-trained model.
Third, through empirical observation, those strategies designed for preventing the conventional overfitting may need to change the target to the OOD detection based on the important observation. In our experiments, for all the baseline models including that used in Figure D. we have adopted those strategies (Srivastava et al., 2014 Hastie al. 2009) (e.g., drop-out, weight decay) to reduce overfitting. is found to be not enough to restore the OOD detection performance. For another shared issue, on the CIFAR-106 dataset, our UM restore the OOD detection capability of the well-trained model with a significant sacrifice on "ID-ACC". Using those strategies for reducing overfitting in the model development phase maybe not be acceptable to the users that it achieves such a lower performance on the original task. In contrast, our proposed UMAP can be a more practical and flexible way to restore detection performance.
We conduct the extra comparisons between our UM and UMAP with those methods for reducing overfitting. The results are summarized in the following Tables A 3 6 7nd According to our extra experiments, most conventional methods proposed to prevent conventional overfitting show limited benefits on gaining better OOD detection performance. Based on our important observation, the effective criterion, i.c., carly stopping, also need to change its validation target to be the OOD data. However, most of them suffer from higher sacrifice on the performance of the original task and maybe not compatible and practical in the current general setting, i.e., starting from a well-trained model.
Given the concept discrepancy aforementioned, one conclusive message is that "memorization of the atypical samples" are not "memorization in overfitting". Those atypical samples are empirically beneficial in improving the performance on the original classification task as shown in Figure 2 However, this part of knowledge is not very necessary and even harmful to the OOD detection task as
15

---

Under review as a conference paper at ICLR 2023
Table 4: Comparison among overfitting methods and ODIN with DenseNet-101 (%). f indicates higher values are better, and + indicates lower values are better.
Din
Method
AUROCT
AUPRT
[PR95
ID-ACC1
Baseline
91.67
91.89
40.74
93.67
Early Stopping ACC
92.13
92.46
38.86
93.69
Early Stopping OOD
92.95
93.26
35.23
93.18
Weight Decay 0.1
86.64
86.67
60.07
88.53
Weight Decay 0.01
90.76
91.25
41.20
92.07
CIFAR-10
Weight Decay 0.001
88.93
88.25
48.95
94,26
Drop Rate 0.3
91.14
92.21
46.58
90.05
Drop Rate
84.95
62.52
82.55
Drop Rate 0.5
83.75
85.17
62.17
75.31
UM (ours)
92.45
93.06
37.13
92.76
UMAP (ours)
91.92
92.88
37.69
93.69
Table 5: Comparison among overfitting methods and Energy with DenseNet-101 (%). 1 indicates higher values better, and indicates lower values are better.
Din
Method
AUROCT
AUPRI
FPR95
ID-ACCT
Baseline
92.72
93.48
38.30
93.67
Early Stopping w. ACC
92.75
93.54
37.84
93.69
Early Stopping w. OOD
93.04
93.78
36.56
93.18
Weight Decay 0.1
86.78
88.04
65.08
88.53
Weight Decay 0.01
90.86
91.77
47.64
92.07
CIFAR-10
Weight Decay 0.001
90.68
90.90
47.38
94.26
Drop Rate 0.3
90.52
91.79
51.23
90.05
Drop Rate 0.4
84.29
86.43
68.17
82.55
Drop Rate 0.5
83.29
85.14
68.17
75.31
UM (ours)
93.58
94.14
33.66
92.76
UMAP (ours)
93.17
93.87
36.11
93.69
Table 6: Comparison among overfitting methods and ODIN with WRN-40-4 (%). values are better, and & indicates lower values are better.
indicates higher
Din
Method
AUROCT
AUPR
FPR95
ID-ACC1
Baseline
86.24
85.90
60.13
93.86
Early Stopping ACC
83.80
83.30
65.13
93.99
Early Stopping OOD
89.92
90.94
49.87
91.47
Weight Decay
84.38
84.75
65.75
89.88
Weight Decay 0.01
88.08
88.45
55.16
93.16
CIFAR-10
Weight Decay 0.001
86.34
86.38
57.42
94.91
Drop Rate 0.3
87.53
87.25
56.12
94.22
Drop Rate 0.4
88.24
88.41
54.62
94.20
Drop Rate
89.13
53.07
93.91
UM (ours)
89.61
91.13
50.97
92.68
UMAP (ours)
90.43
91.73
46.96
93.86
the detection performance of the model is drop significantly. Based on the training and test curves in our observation, the memorization in overfitting is expected happen later than the final stage in which the test performance would drop. Since we have already used some strategies to prevent overfitting, it does not exist. Intuitively, the "atypical samples" identified in our work are relative to the OOD detection task. The memorization of "atypical samples" indicates that the model may not be able to draw the general information of the ID distribution through further learning on those atypical samples through the original classification task Since mainly provide the empirical observation and understanding of the proposed algorithm in this work, further analysis from other views or theoretically would be interesting and major part of future work.
16

---

Under review as a conference paper at ICLR 2023
Table 7: Comparison among overfitting methods and Energy with WRN-40-4 (%). values are better, and & indicates lower values are better.
indicates higher
Din
Method
AUROCT
AUPR
[PR95
ID-ACC1
Baseline
87.69
88.16
58.47
93.86
Early Stopping ACC
88.07
88.65
67.61
93.99
Early Stopping OOD
90.92
91.94
46.63
91.47
Weight Decay 0.1
86.97
88.51
63.54
89.88
Weight Decay 0.01
89.77
89.82
50.23
93.16
CIFAR-10
Weight Decay 0.001
89.25
89.84
50.95
93.91
Drop Rate 0.3
89.74
90.07
52.16
93.22
Drop Rate
89.94
90.53
51.13
94.20
Drop Rate 0.5
90.09
91.04
52.76
93.91
UM (ours)
91.74
92.67
40.40
92.68
UMAP (ours)
88.84
89.31
50.23
93.86
D DETAILED REALIZATION OF THE PROPOSED ALGORITHMS
In this section, we provide the detailed realization of our proposed Unleashing Mask (UM) (i.e., in Algorithm D and Unleashing Mask Adopt Pruning (UMAP) (i.e., in Algorithm 2
To estimate the loss constrain Ç (i.e.fcu.(m, (m, O f*) in Eq with the fixed given model f*) for forgetting, need randomly knock out parts of weights according to the given mask ratio 8. To be specific, we sample score from a Gaussian distribution for every weight. Then we initialize a unit matrix for every layer of the model concerning the size of the layer. We formulate the mask ms according to the sampled scores. Find the threshold for each layer that is smaller than the score of the given mask ratio in that layer (termed as quantile). Then set all the ones, whose corresponding scores are more significant than the layers' thresholds, to zeros. In our algorithms, the fine-tuning epochs k the epochs we fine-tune after we get the well-trained model.
We dot-multiply every layer's 's weights with the formulated binary matrix as if we delete some parts of the weights. We input a batch of training samples the masked model and treat the mean value of the outputs' cross-entropy loss as the loss constraint. After all of these have been done, we begin to fine-tune the model's weights with the loss constraint applied to the cross-entropy loss.
For UMAP, the only difference from UM is that, instead of fine-tuning the weights, we generate popup score for every weight, and force the gradients to pass through the scores. In every iteration, we need to formulate binary mask according to the given prune rate This is just what we do when estimating the loss constraint. For more details, it can refer to (Ramanujan et al. 2020). In Table S summarize the overall comparison results of UM and UMAP show their effectiveness.
E ADDITIONAL SETUPS OF THE EXPERIMENTS
In this section, we describe more details about the experimental setups for our exploration.
Model setups. For DenseNet- 101, we fix the growth rate and reduce rate to 12 and 0.5 respectively with the bottleneck block included in the backbone. We also explore the proposed UM on Net (Zagorukko & Komodakis 2016) with 40 depth and 4 widen factor, which is termed as WRN-40-4. The batch size for both ID and OOD testing samples is 64, and the batch size of auxiliary samples is 2000. The X in Eq. is 0.5 to keep the OE loss comparable to the CE loss. As for the strategy of sampling outliers, we randomly retrieve 50000 samples from ImageNet- 1k (Deng et al. 2009) for OE and Energy (w. D and 50000 samples using Thompson sampling (Ming et al. 2022) for POEM.
Learning rate schedules. We use 4 different learning rate schedules to demonstrate the existence of the once-covered OOD detection capability. For cosine annealing, we follow the common setups in Loshchilov & Hutter (2017); for linear schedule. the learning rate remains the same in the first one-third epochs, decreases linearly to the tenth of the initial rate in the middle one-thin third epochs, and decrease linearly to 1% of the initial rate in the last one-third epochs; for the multiple decay
17

---

Under review as a conference paper at ICLR 2023
Algorithm I Unleashing Mask (UM)
Input: well-trained model: 0, mask ratio: E [0, 1], fine-tuning epochs of UM: k, training samples:
D
Output: fine-tuned model 0;
1: Initialize popup for every weight
2: for w E 0 do
3: Sw (H, o²)
4: end for
5: Generate mask by the popup scores
6: for I Olayers do
7: mi = quantile(s/,0)
8: end for
9: Estimate loss constrain
10: Ç C EENDY (ECE(Z,, ms @ ())
Unleashing Mask: fine-tuning
12: for E (1,..., k) do
13: ((++1) = g(t) n allCe (z,0) (1+c))
00
14: end for
Algorithm 2 Unleashing Mask Adopt Pruning (UMAP)
Input: well-trained model 0, mask ratio: E [0, 1], fine-tuning epochs of UM: k, training samples:
D prune rate: p;
Output: learnt binary mask mp:
1: Initialize popup score for every weight
2: for E 9 do
3: Sw ~ NN,,
4: end for
5: Generate mask by the popup scores
6: for I Onayers do
7: m' =s quantile(s",d)
8: end for
9: Estimate loss constrain
10: < Er~DY (ÉCE(Z, ms O 0))
11: Unleashing Mask Adopt Pruning: fine-tuning
12: g(1) ~N
13: for (I....,h) k) do
14: for E 6 mayer do
mi gl(()
quantile(s"(1))
16: end for
g(++1)
(t)) 001 (x..ce (.
00) -00--+++))))))) <1+
09
18: end for
schedule, the learning rate decreases 10% of the initial rate (0.01) every 10% epochs (10 epochs); for the multiple step schedule, the learning rate decreases to 10% of the current rate every 30 epochs. All those learning rate schedules for our experiments intuitively illustrated in Figure 4(a)
F ADDITIONAL EXPERIMENT RESULTS AND ABLATION STUDY
In this section, we provide more experiment results. We first show the fine- mne-grained results on CIFAR- 10 and CIFAR-100 100, then conduct the experiments under different model structure (i.e., WRN-40-4), and finally apply an additional ablation study on the proposed UM and UMAP. The mean and variance of all metries (ID-ACC, AUROC, AUPR, FPR95) are reported based multiple independent trials.
18

---

Under review as a conference paper at ICLR 2023
Table 8: Completed Results (%). Comparison with competitive OOD detection baselines. indicates higher values are better, and & indicates lower values are better.
Dmn
Method
AUROCT
AUPRE
FPR95]
ID-ACC1
w/w.o Das
HeadHryds & Gimpel 2017a
89,90 0.30
91.48 + 0.43
60.08
0.76
94,01 0.08
LiiNg et al 2018
91.46 106600.55 0.56
91.67
4 0.58
42.31 2: 1.38
94,01 : 0.08
Mahulannoss
2018b
75.10 1.04
72.32
61.35
1.25
0.08
Energy Liu dl at 2040
92.07 = 0.22
92.72 + 0.39
42.69
1.31
94.01 + 0.08
Energy FUM (W)))
9
94.27 + 0.60
33.29
1.70
92.80 0.47
Energy+UMAP (ours)
93,97 = 0.11
94.38
= 0.06
30.71
1.94
94,01 0.08
CIFAR-10
OB (Tenddyyykk et al. 2019h
97.07 0.01
97.31 1: 0.05
13.80
1 0.28
92.59 1. 0.32
Energy [w: Pan La
94.58 0.64
94.69 + 0.65
2.31
80.91 3.13
POEM al. RUM
91.37 = 0.07
91.51 0.06
18.50
+ 0.33
77.24 = 2.22
OE+UM (ouss)
97.60 0.03
97.87 1: 0.02
11.22
93.66 0.12
Energy+UM (ours)
93.02.10.42
92.36 1 0.38
24.41
1 1.65
71.97 0.92
POEM-UM (ours)
93.04 0.02
92.90
0.02
23.52
0.16
67.41 + 0.27
OE+UMAP (ours)
97.48 0.01
97.74 0.00
12.21
93.44 0.21
Energy+UMAP (ours)
95.63 = 1.15
95.92 1.17
17.51
85.12 = 4.22
POEM+UMAP (ours)
2.
94.15
8.70
76.62 17.95
MSidHendrykks Gimpel
2017a
0.69
75.37
83.14 0.87
74.86
0.21
ODIN el al. 2018
76.18 0.14
76.49 1 0.20
78.93 0.31
74.86 1: 0.21
Mahulannbis ce
2018b
63.90
1.91
64.31
78.79
74.86 0.21
Energy Liu of a A00
76.29 1 0.24
77.06 + 0.55
78.46 0.06
74.86 0.21
Energ) + U.M (ours)
76.22 12
76.39 + 1.03
0.55
61.55 0.24
Energy+UMAP (ours)
75.37 0.59
75,66 = 0.07
1.46 72.21
0.21 74.86
CIFAR-100
OB HHeedryyyk al 20196
90.35 = U.87
90.34 = 0.99
34.73
3.85
73,59 + U.30
Energy Pad (all
88,92 0.57
89.13
37.90
2.65
PMing alr ZUAE
88.95 00 0.54
88.94 + 0.31
38. 10
1.30
56.18 + 1.92
OE+US (ours)
91.01=
91.13 0.24
34.71 0.81
75.15 0.18
Energy+UM (ours)
90.39 0.40
90.14 0.45
32.65
- 3.13
71.95 0.23
POEM+UM (ours)
91.18 0.35
91.45 1: 0.27
30.78 1.76
70.17 0.01
OE+UMAP (aurs)
91.10 0.16
90.99 + 0.23
13.62
74.76
0.11
Energy+UMAP (ours)
90.52 = 0.26
90.46
0.50
32.17 0.300
72.76 + 0.18
POEM-UMAP (ours)
0.29
31.02
0.04
Empirical verification on typical/atypical data. In the following Tables 9 further conduct the experiments to identify the negative effect of learning on those atypical samples by comparing with a counterpart that learning only with the typical samples. The results confirm that the degeneration on detection performance is more likely to come from learning atypical samples.
TO
12 and we
In Table 9. we provide the main results for verification on typical/atypical samples. Intuitively, we intend to separate the training dataset into a typical set and an atypical set and train respectively on these two sets to see whether it is learning atypical samples that causes OOD performance to decrease during the latter part of the training phase. We force training samples through the model (DenseNet-101 of the 60th epoch and get the CE loss for separation. We provide the ACC of the generated sets on the model of the 60th epoch (ACC in the tables). The extremely low ACCs of the atypical sets show that the model of the 60th epoch can hardly predict the samples, which meets our definition of the atypical sample. We then fine-tune the model of the 60th epoch with the generated dataset and report the OOD performance. The results show learning from only those atypical data fail to gain better detection performance than its counterpart, i.e., learning from only those typical data. Learning on those atypical samples fails to draw the suitable features for the OOD detection task, though still can improve the original task performance. The experiments provide a conceptual verification of our conjecture which links our observation and the proposed method.
Results of fine-tuning for less epochs. UM adopts finetuning on the proposed objective for forgetting has shown the advantages of being cost-effective compared with train-from froo-sscrtch. For the tuning epochs, we show in FiguresPande 10 that fine-tuning using UM can converge within about 20 epochs, indicating that we can apply our UM/UMAP for far less than 100 epochs (compared with (rain-from-ccratch) to restore the better detection performance of the original well-trained model. It is intuitively reasonable that finetuning with the newly designed objective would benefit from the tell-trained model, allowing a faster convergence since the two phases consider the same task with the same training data. As for the major experiments conducted in our work, finetuning adopts 100 epochs for better exploring and understanding its learning dynamics for research purposes, this configuration is indicated in the training details of Section 4.1
19

---

Under review as a conference paper at ICLR 2023
Table 9: Fine-tuning on typical/atypical samples with different model structures (%) indicates higher values are better, and 4 indicates lower values are better.
Din
Dataset Size
Structure
Atypical/Typical
AUROCT
AUPRT
FPR95
Atypical Typical
81.45
82.40
62.10
DenseNet-101
82.86
84.38
60.01
CIFAR-10
200
Atypical
85.13
86.57
66.41
WRN-40-4
Typical
86.26
86,89
59.93
Atypical
71.96
73.16
85.57
1enseNet-101
Typical
74.79
75.83
80.97
CIFAR-100
1000
Atypical Typical
66.64 71.95
67.41
86.92
WRN-40-4
72.02
80.00
Table 10: Fine-tuning on typical/atypical CIFAR-10 samples with DenseNet-101 (%). 1 indicates higher values better, and indicates lower values are better.
Din
Dataset Size
Atypical/Typical
ACC
AUROC1
AUPR
FPR95
Atypical
3.50
81.45
82.40
62.10
200
Typical
100.00
82.86
83.48
60.01
Atypical
11.14
85.90
86.01
55.10
CIFAR-10
350
Typical
100.00
85.90
86.16
52.81
Atypical
16.80
84.94
85.33
59.27
500
Typical 100.00
85.53
86.10
58.74
Fine-tuning on typical/atypical CIFAR-10 10 samples with WRN-40-4 (%). Table 11:
indicates higher
values are better, and J indicates lower values are better.
Din
Dataset Size
Atypical/Typical
ACC
AUROC1
AUPR
FPR95_
Atypical
3.50
85.13
86.57
66.41
200
Typical
100.00
86.26
86.89
59.93
Atypical
11.14
82.92
84.24
68.57
CIFAR-10
350
Typical
100.00
85.82
87.84
65.54
Atypical
16.80
82.88
83. 83.22
66.75
500
Typical
100.00
87.38
87.93
52.27
Table 12: Fine-tuning on typical/atypical CIFAR- 100 samples with DenseNet-101 (%). f indicates higher values are better, and & indicates lower values are better.
DNn
Dataset Size
Atypical/Typical
ACC
AUROCT
AUPR
FPR95
Atypical
1.00
72.69
73.28
80.71
500
Typical
100.00
74.07
75.20
80.19
Atypical
2.88
69.74
71.15
85.46
CIFAR-100
800
Typical
100.00
72.49
73.17
81.97
Atypical
3.50
71.96
73.16
85.57
1000
Typical 100.00
74.79
75.83
80.97
We also provide extra comparison to show the relative efficiency of our proposed UM/UMAP in the following Table [4] and Table The results show that UM and UMAP can efficiently restore detection performance compared with the baseline. is intuitively reasonable that fine-tuning would
20

---

Under review as a conference paper at ICLR 2023
Table 13:
Fine-tuning on typical/atypical CIFAR-100 samples with WRN-4004 (%) higher values are better, and 4 indicates lower values are better.
indicates
Dan
Dataset Size
Atypical/Typical
ACC
AUROCT
AUPR
FPR95
Atypical
1.00
66.03
66.17
89.56
500
Typical
100.00
68.60
69.93
86.53
Atypical
2.88
67.59
68.66
85.61
CIFAR-100
800
Typical
100.00
70.25
68.95
79.66
Atypical
3.50
66.64
67.4 67.41
86.92
1000
Typical
100.00
71.95
72.02
80.00
benefit from the well-trained model, allowing faster convergence as the two phases consider the same task and training data. Considering the significance of OOD awareness those safety- critical areas, it is worthwhile to further excavate the OOD detection capability of the deployed well-trained model using our UM and UMAP.
Table 14: Fine-tuning for 20 epochs with DenseNet-101 (%). 1 indicates higher values are better, and 4 indicates lower values are better.
Din
Epoch
Method
AUROCT
AUPR
FPR9S]
ID-ACCT
MSP
89.90
91.48
60.08
94.01
ODIN
91.46
91.67
42.31
94.01
100
Energy
92.07
92.72
42.69
94.01
Energy UM
93.73
94.27
33.29
92.80
Energy UMAP
93.97
94.38
30.71
94.01
CIFAR-10
MSP UM
90.31
91.99
53.61
91.70
ODIN +UM
94.08
94.67
31.01
91.70
Energy +
93.60
94.32
33.03
91.70
20
MSP + UMAP
88.70
90.39
57.69
94.01
ODIN UMAP
92.88
93.33
35.19
94.01
Energy UMAP
92.88
93.39
35.60
94.01
Table 15: Fine-tuning for 20 epochs with WRN-40-4 (%). 1 indicates higher values are better, and indicates lower values are better.
Din
Epoch
Method
AUROCT
AUPR
FPR9S]
ID-ACCT
MSP
87.12
87.84
68.29
93.86
ODIN
83.29
82.74
65.68
93.86
100
Energy
87.69
88.16
58.47
93.86
Energy UM
91.74
92.67
40.40
92.68
Energy UMAP
88.84
89.31
50.23
93.86
CIFAR-10
MSP UM
89.86
91.32
51.62
91.96
ODIN +UM
91.97
92.58
41.78
91.96
Energy + UM
92.95
93.64
36.21
91.96
20
MSP + UMAP
88.77
90.61
61.60
93.86
ODIN UMAP
90.85
91.89
45.70
93.86
Energy UMAP
91.66
92.49
42.94
93.86
Fine-grained results on OOD data. In order to further figure out the effectiveness of the pro- posed UM and UMAP different OOD datasets, further report the fine-grained results of our experiments on CIFAR- 10 and CIFAR-100 with OOD datasets (CIFAR- 10/CIFOR- textures, Places365, SUN, LSUN, iNaturalist). The results on the 6 OOD datasets show the general effective- ness the proposed UM as well as UMAP. In Table I6] OE + UM can outperform all the OOD baselines, and further improve the OOD performance though the original detection performance
21

---

Under review as a conference paper at ICLR 2023
Table 16: Fine-grained Results (%) of DenseNet-101 on CIFAR-10. Comparison on different OOD benchmark datasets respectively. indicates higher values are better, and y indicates lower values are better.
OOD datnset Textures
CIFAR-100 AUROCT
Places365
FPR05]
EPR95
AUROC1
FPR95]
Energy UMAP
54.95 4: 2.61
87.72 2: 1.05
33.50
1.32
92.67 d: 0.21
32.80 4: 1.14
93.57 # 1.12
OB
59:29 :: 1.30
88.51 : 0.22
2.89
1: 0.30
99. 16
0.05
1114411.11
97.50 0.19
Energy (w. Douz)
79.88 4: 2.47
74.99 4: 2.40
4.27
2: 0.57
96.80
0.19
14.22
4 3.99
97.07 2: 0.72
POEM
52.30 2:00::111 1.57
72.74
2: 1.12
1.91
0.411
99. 30
0.10
11.25
2:770
96.67
2:0.5
OE + UM (ours)
56.74 1: 1.47
89.53
0.18
1,42 0.115
99.49
0.04
7,77
1 0.69
98.15 : 0.08
Energy (wc Dew.) UM (ours)
$4.52
0.01
70.09
# 0.47
8.30 0.88
97.76
0.05
20.21
1.30
96.06
0.31
POEM- UM (ours)
$4.87
1.56
68.97
# 0.39
4.73
2: 0.52
96.8
0.13
19.83
0.34
98.35 : 0.09
OF + UMAP tours)
08.05
1.41
89.14
0.14
1.86
0.07
90.35
0.00
8.21
+ 0.12
98.07 + 0.03
Energy (w. D UMAP (ours)
15.18
4.96
80.53
+ 4.40
2.24
1.34
90:25
0.29
9.30
2.12
97.90 + 0.40
POEM UMAP (ours)
79:33
4.14
76.89
+ 4.86
2.10
1.37
90.34
0.30
9.94
6.92
98.01 = 1.11
CIFAR-10
Method
FPR95L
SUN
AUROCC
PPR95]
LSUN
AUROC1
FPRO5]
INaturalist
AUROCT
Energy
0:01
47.22
14.03
8.38 +
5,90
96.60
Energy (w: Dous)
10:30
97.77
12.80 4.67
96:08
POEM
8.30 +: 2.42
9.09 +:
97 25:25: 0.58
3.78
08.00
0.17
UM (ours)
5,51 + 0.44
0.17 98.55
3,51 0.43
V8.93
0.09
2.87
0.09
Energy (w. Deu=+ UM (ours)
16.13 1.80
96.84
0.30
23.27 2.40
92.11 0.91
11:20 2.35
97.54
0.42
POEM- UM (ouns)
16.16
0.57
97.00
0.08
25.09 0.15
93.38 1. 0.27
9.30 1.60
08.05
0.21
+ UMAP (ours)
6.16 0.02
0.01
4.53
0.16
96 86 0.06
3.40 0.74
08.96
Energy (w. D + UMAP (ours)
0.67 +: L.50
98.40
0.32
23.50
5.61
94.78
2.04
3.77 2.14
98.93
0.46
POEM: UMAP (ours)
7.00 4: 5.80
98.46
2: 0.96
21.17 1 12.84
91.74 3.67
3.63
2.78
98.99 1: 0.54
Table 17: Fine- seeeeggraned Results (%) of DenseNet-101 on CIFAR-100. Comparison on different OOD benchmark datasets respectively. indicates higher values are hetter, and indicates lower values better.
OOD dataset Textures
ID datnset
Method
CIFAR-10
Places365
FPR95L
AUROC
FPR95,
AUROC1
FPR95
AUROCT
MSP ODIN
$3.53 0:33
15.11
0.22
86,90 =: 0.18
71.45 + 00400
55.85
0.48
70.54 + 00.42
85.29
0.17
13.31 + 0.24
86.45 =: 1.27
71.91 2: 0.27
$4.30
2:
0.64
73.58
:: 0.51
Mahalanobis
18. 25 0.01
49.60
1.51
33,06 + 3,76
90.19
1.21
96
20
0.49
53.60 + 1.55
Energy
82.16 0.59
75.31
0.21
00.20
+000-0000 0.30
68.98 + 0.34
02
+ 0.97
73.78
+ 0.66
Energy+UM (ours)
89. 0.07
66.12
0.93
86.99 2: 1.22
65.39
1.44
77.30
2.08
76.06 1 1.05
90.97 0.46
0.25
0.11
40.19
2.01
Energy Pas))
96. 0.06
37.52
9.026
0.26
90.29
1.14
POEM
96. 19.19:1
0.16
55.82
7.60 1.40
97
07.89-
0.06
32.67
90.94 0.08
(ours)
0.25
0.13
Energy 1 UM (oxrs)
0.60
POEM UM (ouns)
23
(ours)
0.23
UMAP (ours)
POEM-
(ouns)
95.69
0.17
61.62
26.06
CIFAR-100
Method
FPR95L
SUN
AUROOC
FPR95
LSUN
AUROCT
EPR95L
INaturatis
AUROCC
MSP
88. 75 4: 0:23
73
0:25
67.83
1.37
82.94 0.32
85.00
05.00-+00 0.75
76.62 0.25
OPIN
R8. 40
+ 0.00
60.64
0.61
34.80
+ 2.56
93.92 + 0.75
81 11.67
+ 2.77
78.36 + 1.57
Mahalanabis
95 13
+ 0.37
54.37
1.35
80.31
+ 4.83
43.
16:96
93.63
1.19
49.00
1.5.
Energy
97 17
1: 0.92
68.04 : 01.83
35.09-
3.17
93 48 1 0.87
85. 70
2: 14
75.82 +: 1.72
Knergy+UM (surs)
81.96 2: 2.26
71.47 : 1.8M
22.54
:: 5.93
94,98 1: 1.75
74.2%
3.72
80.72 -: 3.75
OB
44.47
9.10
90.70 :: 2.01
5.75 1.18
98
0.13
25.51
4.12
94.446 =: 0.88
Energy 17...)
32 68
5:600
13.633
1.48
55.75
+ 4.31
87.96
1033
17:34
2- 4.54
16.50 : 0.81
POFM
30
1.
94.26
(.500
46.68 + 3.50
90.30
2.17
16.50
+- 2.09
96.663 + 0.23
OF + UM (ours)
23
0:20
90.28
+ 0.033
5.80 1.80-0.33 0.33
63
0.03
26.72
1.95
94.51 + 0..4
Energy (v. Pam3+ UM (ouns)
28.
15
85. 18 0.69
37
4.81
91.88
0.53
10.83
2:2.0
09.00 1 0.69
POEM UM (ouns)
21.34
1,07
95.76
1: 0.22
33.74
6.22
94 43
18
8.85 4. 0.23
97.93
: 0,03
+ UMAP
2.02
91.37
5.17 0.71
98.64
0.12
23.33
95.08
0.22
Energy (v.
UMAP (ours)
29. 65
93.38
35,94 0.75
92.06
0.39
13.96
2.48
96.87
0.15
POEM UMAP (ours)
23. 73.73.4
0.71
95.25
0.06
33.09 3 5.94
93.57 0.88
9.70 1. 1.09
97.77 1 0.24
already well. By equipping with our proposed UM and UMAP, the baselines can outperform their counterparts on most of the OOD datasets. For instance, the FPR95 can decrease from 1.91 to 1.42. In Table 17 we also take a closer check about results on CIFAR- -00 with 6 OOD datasets. Our proposed method can almost improve all competitive baselines (either the scoring functions or the finetuning with auxiliary outliers) on the OOD datasets. In both w. Dauz and w.o. Dauz scenarios, Unleashing Mask can significantly excavate the intrinsic OOD detection capability of the model. In addition to unleashing the excellent OOD performance, UMAP can also maintain the high ID-ACC by learning binary mask instead of tuning the well-trained original parameters directly.
22

---

Under review as a conference paper at ICLR 2023
Table 18: Results (%) of WRN-40044. Comparison with competitive OOD detection baselines. We respectively train WRN-40-4 on CIFAR-10 10 and CIFAR-100, For those methods involving outliers, we retrieve 5000 samples from ImageNet-Ik, 1 indicates higher values are better, and + indicates lower values are better.
Din.
Method
AUROCT
AUPR
FPR95.
ID-ACC
Heendryks & Gimpel. 2017a]
87.12 0.25
87.84 + 0.30
68.29
0.96
93,86 + 0.19
ODIN et al. 2018
83.29 + 0.72
82.74 + 0.79
65.68 + 0.77
93,86 + 0.19
Mahalanobisi
2018b
77.57 0.28
76.11 0.10
6
93,86 = 0.19
CIFAR-10
Energy (Luu et 2020
87.69 1 0.54
88,16 1 0.69
58.47
1. 1.94
93,86 1 0.19
Energy+UM (ours)
91.74 0.43
92.67 + 0.52
40.40
1.32
92.68 + 0.23
Energy+UMAP (ours)
88.84 1.02
89.31 = 1.44
50.23 =:: 2.25
93.86 = 0.19
MSP(Ienndrykks & Gimpel 2017a
72.34 1 0.63
72.69 1 0.44
85.40
0.59
75.01 1 0.07
Liang al. 2018
68.78 0.67
66.92 = 0.72
85.28
0.64
75.01 = 0.07
MahalanobisiLee et 2018b
68.20 0.99
68.30 = 1.15
76.46
2.02
75.01 = 0.07
CIFAR-100
Energy (Liu ct al. 2020
74.00 + 0.41
73.02 = 0.47
81.37
0.08
75.01 = 0.07
EnergyFUM (outs)
76.07 0.04
76.94
1 0.06
74.29
1.66 68.20
59.08 2.75
Energy+UMMPP (ours)
77.35 0.78
77.43 + 0.91
0.06
75.01 + 0.07
Table 19: Fine- gen--grained Results (%) of WRN-40 on CIFAR-10. Comparison on different OOD benchmark datasets, indicates higher values are better, and & indicates lower values are better.
OOD datuset Textures
ID dataset
Method
CIFAR-100
Places365
FPR95L
AUROC
FPR951
AUROCT
FPR95
AUROCT
MSP
70.96 + 0.70
86.08 = 0.08
68.81 + +.1.1.4. 29
86.53
0.83
68.31 + 0.25
86.71 + 0.13
ODIN
64.97
81.36
66.86
1.16
0.93
Mahalanobis
79.81 + 0.55
70.33 = 0.21
22.56 1 0.08
94.07 = 0.04
85.09
+ 0.59
67.90
1 0.37
Energy
61.09 1 0.58
86.66
0.04
64,29
1.72
85.56
0.53
55.32
0.13
88:29
0.26
Energy+UP (ours)
57.21
1.41
87.56
46.49
+ 1.03
80.74
0.45
4.46
92.51
0.97
Energy+UMA. cours)
65.45 + 1.10
81.65
0.95
59.11
+ 1.61
85.27
1.71
48.16 + 1.89
90.43 2: 0.17
Method
LSUN
iNaturalist
EPR95L
AUROC
FPR95]
AUROCT
EPR95]
AUROCT
68.62 +
52.97 +
92.41
76.05
0.36
ODIN
65.47 + 0.788
83. 79 = 1.10
31.89
3.41
94.31 = 0.89
79.28
0.18
79.80 + 0.35
Mahalanobis
82.92 1: 0.28
70.52
0.47
64.31
1: 0.57
67.75
0.55
81.50
1 2.91
74.97 1 2.91
Energy
54.88
24.99
95.98
0.37
75.80
0.85
0.22
Emergy+ (ours)
38.92 1. 3.46
92.98
0.95
8,38
0.77
98.18
0.16
66.02
6.70
85.22
3.42
Energy+ UMMP (ours)
45.94 + 2.64
91.27
0.56
14.10
97.46
0.14
74.09
0.15
81.13 + 1.12
Experiment on different model structure. Following 4.2 we additionally conduct critical experi- ments on the WRN-40-4 (Lin et al. (022)) backbone to demonstrate the effectiveness of the proposed UM and UMAP. In Figure 7 we can find during the model training phase on ID data, there also exists the once-covered OOD detection capability can be explored in later development. In Table [8 we show the comparison of multiple OOD detection baselines, evaluating the OOD performance on the OOD datasets mentioned in Section 4.1 The results again demonstrate that proposed method indeed excavates the intrinsic detection capability and improves the performance.
As for the fine- finengrained results of WRN-40-4, report results on 6 OOD datasets respectively. When trained on CIFAR-10, 10, UM can outstrip all the scoring function baselines on OOD datasets except Textures which Mahalanobis performs better while UMAP still has excellent OOD performance ranking only second to UM. When trained on CIFAR-100, UM and UMAP can also outperform the baselines on most OOD datasets. The fine-grained results of WRN-40-4 further demonstrate the effectiveness of the proposed UM/UMAP on other architectures.
Additional results about the general existence of once-covered OOD detection capability. In Section we display the once-covered OOD detection capability on C1FAR-10 using SVHN as the OOD dataset. Here, we additionally verify the previously observed trend during training when training DenseNet-101 on CIFAR-100 using iNaturalist as OOD dataset. In Figure we trace the three evaluation metrics during training on CIFAR-100 using 4 different learning rate schedules. Consistent with the original experiment, for all the three metrics that exists a middle stage where the model has the better OOD detection capability (For FPR95, it is smaller (better) in the middle stage; for AUROC and AUPR, they
still use iNaturalist as the OOD dataset. It can be seen
23

---

Under review as a conference paper at ICLR 2023
Table 20: Fine-grained Results (%) of WRN-40-4 on CIFAR-100. Comparison on different OOD
benchmark datasets. 1 indicates higher values are better, and + indicates lower values are better.
OOD dalaset
Method
Places365
FPR95_
AUROCT
EPR95
AUROCT
FPR95L
AUROC
ODIN
83.70
0.30
74.32 + 0.03
81.57 + 1.74
71.67
0.28
88.07 + 0.11
64.83 + 1.36
Mahalanobis
96 96.89
0.11
68.78
= 0.67
31,02
2,04
91.85 = 0.91
93.31 = 0.11
61.28 = 1.62
Emergy
Energy+UM (ours)
89.23
1.51
63.85
1.73
78.90
0.07
72.58
1.33
80.46
1.99
70.49 + 1.01
+nergy+UMMr MAP (ours)
91.11
0.72
60.77
+ 0.96
66.91
4.44444449 4.49
75.82 5.33
82.59
0.27
71.92 = 3.58
CHEAR-100
Method
SUN
LSUN
INaturalist
FPR95,
AUROC1
FPR95]
AUROCT
FPR95
AUROC
MSP
88 88 0.83
63.22
0.85
78.56
0.66
79:10
0.33
86. 86.72
0.29
T3. 0.56
ODIN
91.00
0.10
59.06
1.81
70.14
1.144-2..44 2.42
84.03
0.86
87.86
1.13
64.52
1.51
Mahulanobis
94.22 0.01
60.00
1.56
40.81
3.07
87. 25
74.98
2.85
Eaergy
88.35 0.52
64.01
0.76
59.81
87
0.53
88
0.78
67.81
0.91
Energy+ +LM (ours)
84.04
0.09
67.19
0.14
33.87
1 1.21
92.29
0.02
76.91
+ 6.07
79.28 1 4.17
UUAP (ours)
80.53
1.31
72.68
4.50
27.79
2.19
93.39
0.57
58.53
4.61
85.65
0.83
80
Mullip Step
90.0
75
AUROC
70
AUPR
83
65
80.0
Unear
Multipi Decay
Muliple Decay
77.5
Multipic Stop
Muliple Stop
20
40
60
80
100
20
40
60
80
100
20
40
60
80
100
Epochs
Epochs
E
Epochs
(a) FPR95 Curves
(b) AUROC Curves
(c) AUPR Curves
Figure 5: Ablation studies on three metries with 4 different learning rate schedules. The model is (a) change of FPR95
DenseNet-101 trained on CIFAR- 100 with iNaturalist as the OOD dataset. throughout the pruning phase when training on CIFAR- 100: (b) change of AUROC throughout the pruning phase when training CIFAR-100; (c) change of AUPR throughout the pruning phase when training on CIFAR-100. It demonstrates better middle stage exists according to the three metrics.
higher (better) in the middle stage). Besides that. also look into the change of OOD performance on other architecture (c.g., WRN-40-4) in Figure and Figure 7 In Figure we display the curves of three metrics of WRN-40-4 when trained on CIFAR-10 with SVHN and Textures OOD) datasets. The trend that the OOD performance first goes better and then converges to worse OOD performance can be reflected. In Figure 7 continually provide curves of the three metries of WRN-40-4 during training on CIFAR-I0 100 with iNaturalist, Places365, and SUN as OOD datasets. A clear better middle stage can still be excavated in this scenario.
UMAP: adopting pruning on UM. We conduct various experiments to see whether pruning has an impact Unleashing Mask itself. To be specific, we expect the pruning to learn a mask on the given model while not impairing the excellent OOD performance that UM brings. In Figure g presents that pruning from wide range (e.g. p E [O.3, 0.9) can well maintain the effectiveness of UM while possessing a terrific convergence trend. For simplicity, we use prune to indicate the original pruning approach and UMAP indicate UM with pruning on the mask with our newly designed forgetting objective in Figure 8 In Figure [(a] the solid lines represent the proposed UMAP and the dashed lines represent only pruning the well-trained model at prune rates 0.2, 0.5, and 0.8. While the model's 's OOD performance can't be improved (not better than the baseline) through only pruning, using our proposed forgetting objective for the loss constrain can significantly bring out better OOD performance at wide range of mask rates (e.g. p E [0.5, 0.8)). In Figure 8(b) intuitively reflect the effect the estimated loss constraint by the initialized mask which redirects the gradients when the loss reaches the value, while the loss will just approach 0 when pruning only. In Fig)re(((e)
24

---

Under review as a conference paper at ICLR 2023
SWANN Tecturus
100
SWHN
100
Tentures
90
90
80
FPR95
60
AUROC
AUPR
70-
60
40
50
60
40
SVHN
20
Epochs
Epochs
Epochs
(a) FPR95 Curves
(b) AUROC Curves
(c) AUPR Curves
Figure 6: Ablation studies on three metrics of WRN-40-4 with CIFAR-10 10 ID dataset, SVHN,
and Textures as OOD datasets. (a) change of FPR95 throughout the pruning phase when training on CIFAR-10: 10: (b) change of AUROC throughout the pruning phase when training on CIFAR-10; (c) change of AUPR throughout the pruning phase when training on CIFAR-10. It demonstrates a better middle stage exists according the three metries.
90
Hhetur efist
95
80
90
AUROC
AUPR
85
70
70
60
75
50
70
0
20
40
60
80
100
20
40
60
80
100
0
20
40
60
80
100
Epochs
Epachs
Epachs
(a) FPR95 Curves
(b) AUROC Curves
(c) AUPR Curves
Figure 7: Ablation studies on three metrics of WRN-40-4 with CIFAR-100 as ID dataset, iNaturalist, Places365, and SUN as OOD datasets. (a) change of FPR95 throughout the pruning phase when training on CIFAR-100 100; (b) change AUROC throughout the pruning phase when training on CIFAR- (c) change of AUPR throughout the pruning phase when training CIFAR-100. It demonstrates a better middle stage exists according to the three metrics.
can see that ID-ACC for both UMAP and Prune can converge to approximately the same high level (92 94), though can simply remove the learned mask to recover the original ID-ACC.
The effectiveness of UM. In Figure 9 we present the FPR95, AUROC, and AUPR curves during training to show the comparison of the original training and our proposed UM on ID data. We observe that training using UM can consistently outperform than the vanilla model training, either for the final stage or the middle stage with the best OOD detection performance indicated by the FPR95 curve. In Figure I0 we also adopt different mask rates for the initialized loss constraint estimation for forgetting the atypical samples. The results show that a wide range of mask ratios (i.e., from 96% to 99%) to estimate the loss constraint used in Eq 3 can gain better OOD detection performance than the baseline. shows the mask ratio would be robust to hypper-aaaaeeer selection under certain value. The principle intuition behind this is our revealed important observation indicated in Figures ((a) 2bb and 2(c) With the guidance of the general mechanism, empirically choosing the Syyer--paramet
using the validation set is supportable and valuable for excavating better OOD
detection capability of the model as conducted by previous literature (Hendrycks et al.
2019b Liu
al. 2020 Sun et al. 2021).
In our experiments, we empirically determine the value our proposed UM and UMAP by examining the training loss on the masked output. For CIFAR-10 ID datasets, the value of mask rate is 97.5%
25

---

Under review as a conference paper at ICLR 2023
UMAP 0.2
1.6
UMAP 0.2
90
UMAF0 0.5
UMA 0.5
90
UMAP
UMA 0.5
Testing Accuracy C
80
Prune
Training Loss
1.2
Prune
80
F2R9S
75
60
0.8
0.6
UMAP 0.2
50
83
UMAP 0.5
0.4
UMAP 6.8
40
60
Prone 0.2
Prune
30
0.0
55
Epochs
Epochs
Epochs
(a) FPR95 Curves
(b) Train Loss Curves
(c) Test Acc Curves
Figure 8: Ablation studies Prune Rate of UMAP. (a) change of OOD performance throughout the pruning phase; (b) training loss converges to estimated loss constraint properly; (c) though ID-ACC not taken into consideration for UMAP, it still raise high after training for 100 epochs.
UM (Hask Radoo 0.9951
95
Non UM
90
85
80
80
FPR85
AUROC
AUPR
75
70
40
60
55
50
Non
20
40
60
80
100
0
20
40
60
100
20
40
60
80
100
Epochs
Epochs
Epochs
(a) FPR95 Curves
(b) AUROC Curves
(c) AUPR Curves
Figure 9: Ablation studies to reflect the effectiveness of UM. The mask ratio UM is 99.5%. (a) change of FPR95 throughout the training phase on CIFAR-10; (b) change of AUROC throughout the training phase on CIFAR-10; (c) change AUPR throughout the training phase on CIFAR-IO.
and the estimated loss constraint for forgetting is 0.10 for tuning until the convergence; For 100AR-100, the value of mask rate is 97% and the estimated loss constraint for forgetting is 1.20 for our tuning until the convergence.
To choose the parameters of the estimated loss constraint, we use the TinyImageNet (Tavanaei 2020) dataset the validation set, which is not seen during training and is considered in evaluation of OOD detection performance. Since the core intuition behind our method is to restore the OOD detection performance starting from the well gelittr model stage, forgetting a relatively small portion (empirically found around 97% mask ratio) of atypical samples can be beneficial. To find the optimal parameter for tuning, more advanced searching techniques like AutoMI. or validation design based on the important observation in our work may be further employed in the future.
Fine-grained comparison of the model weights. We display the weights of the original model, pruned model, and the UMAP model respectively in Figure 11 The histograms show that the adopted pruning algorithm tends to choose weights far from 0 for the first convolution layer, shown in Figure [T(a) However, for almost all layers (from the 2nd to the 98th), the pruning chooses weights with no respect to the value of weights, shown in Figure [1bb For the "head" of the model (the fully connected layer), the pruning algorithm itself still keeps behavior the first layer, while UMAP forces the prune algorithm to choose weights near 0, shown in Figure TI(c) indicating that forgetting learned atypical samples doesn't necessarily correspond larger weights or smaller weights
26

---

Under review as a conference paper at ICLR 2023
70
0.98
0365
Rado 0800
Mace Redo 0375
FPR95
Masi Rado 0.75
Mask Raão 0372
0970
Rato 8
30
Mask Rado 0.60
Mosi Rado 0.55
Baseline
Epochs
(a) FPR95 Curves
96
Most Rado 0.850
94
Mosi Rado 0965
Mask Reão 0960
AUROC
92
80
Mask Rase 0958
Mask Radoo 0.770
Mavc Ratio D3955
88
Mask Relio 03862
Ratio 0.962
86
20
40
60
80
100
Epochs
AUROC Curves
96
Mask Relo 0390
AUPR
Mask Ratto 0.755
Mask Rade 0972
Mosil Rado 0.70
Masi Rell 038
90
Ratto 0.955
88
Baseline
Epochs
(c) AUPR Curves
Figure 10: Ablation studies of WRN-40-4 on various Mask Ratios. The mask rate is from 95.8% to 99.0%. (a) change of FPR95 throughout the training on CIFAR-10: 10; (b) change of AUROC throughout the training on CIFAR-10; (c) change of AUPR throughout the training on CIFAR-10.
30
Original
500
Original
Original
UMAP
UMAP
UMAP
400
200
20
Density
Density
Density
300
150
15
200
10
-0.2
1.0
0.5
Weight
Weight
Weight
(a) First Layer
(b) 50th Layer
(c) Last Layer (FC)
Figure 11: Histograms of different layers for the original model, pruned model, and UMAP model. The model is DenseNet-101 with a prune rate of 50%. (a) the histogram of the first convolution layer; (b) the histogram of the 50th convolution layer, (c) the histogram of the last (fully connected) layer.
27

---

Under review as a conference paper at ICLR 2023
0.90
0.92
0.94
plane automobile
dog
frog
ship
truck
(a) Layer-wise wise masking scores
0.92
0.94
plane automobile bird
deer
dog
frog
horse
ship
truck
(b) Layer-wise masking weights
0
0.92
0.94
plane automobile bird
cat
deer
dog
frag
horse
ship
truck
(c) Model-wise masking scores
0.90
0.92
0.94
plane automobile bird
dog
frog
ship
truck
(d) Model-wipg wise masking weights
Figure 12: Examples of misclassified samples after masking the original well-trained model. The scores are estimated according uniform distribution. (a) layer-wise masking scores; (b) layer-wise masking weights directly; (c) model-wise masking scores; (d) model-wise masking weights directly.
28


Reviews:
[{"review_id": "Zb72inDiKc", "reviewer": "Reviewer_3ovB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In summary, this paper first observes that there exists a historical training stage where the model has a higher OOD detection performance than the final well-trained one. After that, this paper aims to backtrack the model to that stage. To achieve this, this paper proposes a method called UM to forget those \"atypical samples\" that are sensitive to the changes in model parameters.", "summary_of_the_review": "Overall, I think this paper gives an exciting observation and the proposed method sounds reasonable. However, with certain important discussions lacking in the current version, while I tend to accept this paper, I still expect the authors to answer my questions well in the rebuttal phase.", "strength_and_weaknesses": "Strength:\n1. I find the observation that there exists a historical training stage where the model has a higher OOD detection performance than the final well-trained one very interesting and intuitive. I am pretty supervised that no one ever made such an observation.\n2. I believe that throughout the main paper and the appendix, a quite comprehensive set of experiments has been conducted.\n\nWeaknesses (or confusions that may need further clarification):\n1. I prefer if the authors could discuss the relationship between this paper and the concept of overfitting. Specifically, from my perspective, the main inspiration of this paper from its own observation is that after a certain number of epochs of training, the model starts to memorize rather than learn knowledge. However, this concept is quite similar to the idea of overfitting. Hence, my two questions will be: (1) Does this mean that the large number of methods that are designed to alleviate overfitting can also be used to facilitate OOD detection, and has the authors tried this? (2) If (1) is not the case, could the authors possibly explain more on what is the difference between the memorization of \"atypical samples\" here and the memorization in overfitting?\n\n2. I may also prefer if the authors could discuss more on the relationship between their observation and their proposed method. Specifically, when the authors simply say \"We further attribute it to the memorization behavior on atypical samples.\", I feel confused about this. In other words, why the reason behind the observation that a historical training stage can have a better OOD detection result than the final stage is necessarily the memorization behavior on atypical samples? I hope to see a little bit more discussion on this.\n\n3. Personally, I think the description of UMAP may be too brief in the main paper. I understand that there is a page limit there for the main paper, but I still have the feeling that UMAP may need to be discussed slightly more since a large number of experiments are based on UMAP, which leaves me a feeling that it is quite important but less explained.\n\n4. Two minor comments are: (1) Is it a typo in equation 3 that the loss aims to minimize over the mask instead of the model parameter? (2) I think that especially in Table 1, the author should not only bold their \"94.01 ± 0.08\" and \"74.86 ± 0.21\", but also the same results of the previous methods. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XEgs-O0GkeU", "reviewer": "Reviewer_miMq", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is motivated by the observation that a fully-converged model does not necessarily give the best OOD detection performance. The overfitting is likely due to the memorization of the atypical samples in the training data. The author proposes the Unleashing Mask used in the training stage to identify these atypical samples and let the network forgets them. The experimental results show that the proposed methods are effective in both post hoc methods and OE-based methods. \n", "summary_of_the_review": "This paper reveals an important observation that well-converged networks do not necessarily lead to a good OOD detection performance. The cause is likely the over-memorization of the atypical samples. The paper proposes an algorithm to identify and forget them, which makes sense. Overall this paper is a good study of the OOD detection problem. So I vote for acceptance. ", "strength_and_weaknesses": "\nStrength:\n\n1. The paper is well-written and easy to follow in most sections.\n\n2. The motivation is clear and the proposed methods make sense in addressing the related problems. \n\n3. The related works are extensive and well-organized\n\n4. The experimental section is comprehensive and covers the majority of the necessary empirical study.  \n\n\nWeakness (not necessary, but questions/suggestions): \n\n1. Why can the atypical samples be mined by the layer-wise mask? I was lost in reading section 3.2. More explanations and details can be elaborated on there. \n\n2. Are the masks applied at all layers or just some typical layers? \n\n3. How does Equation 3 stabilize around the stage that can forget those atypical samples? Seems like the absolute value term makes sure the masked loss is close to the original loss. The second term is minimizing the masked loss. Where is the \"forgetting\" mechanism happening? \n\n4. Why use the layer-wised mask? Is it the optimal choice for identifying the atypical samples? Can we use a unit/weight mask or use high CE error to identify atypical samples? \n\n5. Can we use other strategies like the early stopping criterion to prevent overfitting? The comparison would be interesting to see. \n\n6. Another interesting experiment to do is to separate the training dataset into two types (typical samples / atypical samples) and train two models respectively on them. It would be good to show that atypical samples are indeed the cause of OOD performance getting worse. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "V4v9UJrFyG", "reviewer": "Reviewer_PAPe", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes Unleashing Mask (UM) with the goal of maintaining a model's capability to distinguish between out-of-distribution (OOD) and in-distribution (ID) training examples. \n\nThe paper posits that as classification models train and the training loss decreases, there is a point where simple measures such as ODIN effectively distinguish between ID and OOD, but then as training loss trends to zero, the model is forced to learn atypical and difficult, in-domain class examples which degrades. While this atypical example memorization does improve model accuracy at the primary task, it also harms the model's performance for OOD. \n\nTo address this, the UM method specifies a binary layer-wise mask that serves to force forgetting of atypical examples.  ", "summary_of_the_review": "The paper presents the idea of using forgetting of atypical examples via UM to preserve a models' ability to distinguish OOD.\nWhile experiments support the overall proposal, it seems there are some additional probing or analysis that could be performed to more deeply try to understand whether the improvement is coming from the model or from some other factors. The paper has some issues with clarity and would benefit from a heavy edit.  ", "strength_and_weaknesses": "The paper presents experiments training on CIFAR10 and CIFAR100, then measuring classification accuracy as well as OOD false positive rates. The proposed method appears to work well across many OOD sets. However, I do have a few questions. \n\nWhile I can understand the basic idea linking the memorization of atypical ID examples to a decrease in OOD discrimination ability as a motivation for UM, I am not sure that this is solidly supported by the evidence of the paper. \n\nFurther, it isn't clear to me exactly how the parameters of the key equation (3) are selected. According to the paper: \"For\nCIFAR-10 as ID datasets, the value of mask rate is 97.5% and the estimated loss constraint for\nforgetting is 0.1 for our tuning until the convergence; For CIFAR-100, the value of mask rate is\n97% and the estimated loss constraint for forgetting is 1.2 for our tuning until the convergence.\" \nWould it be possible to run a baseline with parameters selected to effectively turn off masking (i.e., mask rate 100%)?\n\nWould it be possible to run CIFAR100 for OOD and add to Table 2? \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DY27CwtEqn", "reviewer": "Reviewer_Ga94", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is to restore the out-of-distribution sample detection ability given a well-trained model by fine-tuning with the proposed method Unleashing Mask (UM). Based on empirical observations, authors show that a model is trained to learn how to classify in-distribution samples while distinguishing out-of-distribution samples in early phase. Then, at epoch 60 in the authors' example, the model keeps learning in-distribution samples while losing its ability to distinguish out-of-distribution samples. To recover the model's ability to identify out-of-distribution samples, authors suggest UM that identifies atypical samples and helps the model forget them.", "summary_of_the_review": "Based on the weaknesses and overall quality assessment, I would recommend reject of this paper. I welcome the enlightenment of the authors on my ignorance and misunderstanding if there is any. ", "strength_and_weaknesses": "Strength: Adaptability to previous robust loss functions.\n\nDetailed information on baselines in appendix A is helpful for readers who needs baseline details in a page.\n\nWeaknesses\n1. Expensive: Because UM denoises a pretrained model via fine-tuning about 100 (I believe so based on several empirical results in the table although not specified for fine-tuning; authors need to clarify it) epochs, I don't think it worths such expensive fine-tuning process.\n2. Weak logical support: In addition to Weaknesses 1, if we can access to already deployed model and its training data, it worths applying UM or UMAP to a model when it's at best condition (at epoch 60 according to the authors) and help it learn only in-distribution data. Why should we start as fine-tuning?\n3. Labor-intensive: mask rate should be empirically found by UM users. Besides, we need to choose first the OOD baseline, second UM or UMAP to obtain the best result because no case consistently performs the best. \n4. Lack of empirical support: We need to evaluate the impact of UM or UMAP, but results such as Table 1 and 2 lacks their results (I understand the authors' intention to emphasize the extendability of UM and UMAP.) \n5. No theoretical analysis: To support the authors' claim on UM, even a typical theoretical analysis is required which is absent.\n  \nQuestions\n1. Why UM model itself doesn't have (i.e., FC + UM or FC + UMAP)?\n2. What is fine tuning epochs $k$ and loss constraint $zeta$?\n3. Why only Fig 4(c) has 200 epochs? Besides, is 200 fine-tuning stage as a whole?\n4. What are OOD recall before/after fine-tuning by UM/UMAP?\n5. Some acronym words such as FPR95 needs explanation for readers without sufficient expertise or knowledge in this field.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject