Watermarking for Out-of-distribution Detection
Anonymous Author(s)
AffiliationAddress email
Abstract
Out-of-distribution (OOD) detection aims to identify OOD data based on representations extracted from well-trained deep models.
However, existing methods largelyignore the reprogramming property of deep models and thus may not fully unleashtheir intrinsic strength: without modifying parameters of a well-trained deep model,we can reprogram this model for a new purpose via data-level manipulation (e.g.,adding a specific feature perturbation).
This property motivates us to reprogram aclassification model to excel at OOD detection (a new task), and thus we propose ageneral methodology named watermarking in this paper.
Specifically, we learn aunified pattern that is superimposed onto features of original data, and the model’sdetection capability is largely boosted after watermarking.
Extensive experimentsverify the effectiveness of watermarking, demonstrating the significance of thereprogramming property of deep models in OOD detection.
Introduction
Deep learning systems in the open world often encounter out-of-distribution (OOD) inputs whoselabel spaces are disjoint with that of training data, known as in-distribution (ID) data.
For safetycritical applications, deep models should make reliable predictions for ID data, meanwhile detecting
OOD data without prediction [2].
This leads to the OOD detection problem [25, 33, 37, 45] whichhas attracted intensive attention in reliable machine learning.
Identifying OOD data remains non-trivial since deep models can be overconfident with them [35].
Asa promising technique, the classification-based OOD detection [52] relies on various scoring functionsderived by the well-trained classification models, taking those inputs with small scores as OODcases.
In general, the scoring functions can be defined by logit outputs [15, 29], gradients [19] andembedding features [25, 40].
Without interfering with original models or requiring extra computation,they exploit the inherent capability of models learned from only ID sample.
In general, theseadvantages can be critical in reality, where the cost of re-training is prohibitively high and theacquisition of real OOD data is largely limited [52].
Although promising progress has been achieved, previous works largely ignore the reprogrammingproperty [9] of deep models: a well-trained model can be repurposed for a new task by a propertransformation of original inputs (e.g., a universal feature perturbation), without modifying any modelparameter.
For example, a model pre-trained on ImageNet [7] dataset can be reprogrammed forbiomedical images [44] and an acoustic model can excel at time series analysis [51].
The aboveexamples have shown the significance of the reprogramming property for deep models, motivating usthe first attempt to investigate if it can also help to address OOD detection, i.e., can we reprogramwell-trained deep models for OOD detection (a new task)?
To this end, we propose a novel method, named watermarking , to reprogram the well-trainedclassification model for effective OOD detection.
In our context, “reprogramming” is reflected by awatermark with its features on the shape of original inputs, which is defined as a static pattern that can be added for the test-time inputs (cf., Figure 1).
After watermarking, the pre-defined scoringstrategy is expected to be enhanced, with an improved gap of OOD scores between ID data and OODdata during the test (cf., Figure 2).
Actually, it is non-trivial to find the proper watermarks since we do not know much about thetest-time OOD inputs in advance.
To address theissue, we propose a learning framework for effective watermarking.
The insight is to make themodel produce low scores when it perceives thewatermark, except that the pattern of ID inputsis also observed.
In this case, the model willhave a relatively high score for a watermarked ID input, while the score is low for an OOD case(cf., Figure 2).
The reason is that the watermarkis not trained to perceive OOD pattern and onlythe watermark can be observed.
In our realization, we adopt several representative scoring strategies, devising specified learningobjectives and proposing a reliable optimization algorithm.
To understand our mechanism, Figure 1depicts the watermark learned on CIFAR- 10 [23] dataset, with the free energy scoring [29] (anadvanced classification-based method).
As we can see, the centre area of the learned watermarklargely preserves the original input pattern, containing the semantic message that guides the detectionprimitively.
By contrast, the edge area of the original input is superimposed by the specific patternof the watermark, which may encode the knowledge once hidden by the model in boosting OODdetection.
Overall, watermarking can preserve the meaningful pattern of original inputs in detectionwith further enhancement learned from the trained model.
Therefore, the detection capability of theoriginal model is largely improved after watermarking.
Figure 2 demonstrates the effect of our learned watermark, which is an example with the free-energyscoring.
After watermarking, the scoring distributions are much concentrated, and the gap between
ID (i.e., CIFAR- 10 ) and OOD (i.e., SVHN [34] and Texture [5] datasets) cases is enlarged notably.
We conduct extensive experiments for a wide range of OOD evaluation benchmarks in Section 6 and
Appendix C, and the results verify the effectiveness of our proposal.
The success of watermarking takes roots in the following aspects: (1) a well-trained model onclassification has the potential to be reprogrammed for OOD detection since they are two closelyrelated tasks; (2) reprogramming has been widely studied, ranging from image classification to timeseries analysis [7, 44], making our proposal general across various domains; and (3) OOD detectionsuffers from the lack of knowledge about the real-world OOD distributions.
Fortunately, with onlydata-level manipulation in low dimensions, watermarking can largely mitigate this issue of limiteddata.
Overall, this data-level manipulation is orthogonal to existing methods, and thus provides a newroad in OOD detection and can inspire more ways to design OOD detection methods in the future.
Related Works
To begin with, we briefly review the related works in OOD detection and model reprogramming.
Please refer to Appendix A for the detailed discussion.
OOD Detection discerns ID and OOD data by their gaps regarding specified metrics/scores, andexisting methods can mainly be attributed into three categories [52], namely, the classification-basedmethods, the density-based methods, and the distance-based methods.
Specifically, the classificationbased methods [15, 19, 29, 40] use representations extracted from the well-trained models in OODscoring; and the distance-based methods [3, 18, 36, 54] measure the distance of inputs from classcenters in the embedding space.
Moreover, the density-based methods estimate input density withprobabilistic models [25, 37, 41], identifying those OOD data with small likelihood values.
Sincedistance-based and density-based methods may suffer from complexity in computation [25] anddifficulty in optimization [56], recently, more researchers focus on developing classification-basedmethods and have made big progress on benchmark datasets [19, 29].
D
en s i t y
D
en s i t y
Free Energy Score
[Figure]
Energy Score iSUN
Places
Texture SVHN
LSUN-C
LSUN
D
en s i t y (b)
w/ watermarking
D
en s i t y (a) w/o watermarking (a) before watermarking
D
en s i t y
D
en s i t y
Free Energy Score
Free Energy Score iSUN
Places365 Texture SVHN LSUN-C LSUN 𝜎 𝜎 𝜎 Free Energy Score
D
en s i t y (b)
w/ watermarking
D
en s i t y (a)
w/o watermarking
[Figure]
Model Reprogramming repurposes the well-trained models for new tasks with only data-levelmanipulation [9], indicating that deep models are competent for different jobs without changingmodel parameters.
In previous works, the data-level manipulation typically refers to a static paddingpattern (different from our proposal) learned for the target task, which is added to the test-time data.
The effectiveness of the model reprogramming is verified across image classification [9, 44] andtime-series analysis [13, 51].
In this paper, we use the reprogramming property of deep models foreffective OOD detection, which has been overseen previously.
Preliminary
Let X ⊂ R d be the input space and Y = { 1 , . .
, c } be the label space.
We consider the ID distribution
D ID X , Y defined over X × Y , the training sample S n = { ( x i , y i ) } ni =1 of size n independently drawnfrom D ID X , Y , and a classification model f : X → R c (with logit outputs) well-trained on S n .
Based on the model f ( · ) , the goal of the classification-based OOD detection is to design a detectionmodel g : X → { 0 , 1 } that can distinguish test-time inputs given by the ID distribution
D ID X fromthat of the OOD distribution D OOD X .
In general, D OOD X is defined as an irrelevant distribution of whichthe label set has no intersection with Y , and thus should not be predicted by f ( · ) .
Overall, with 0 the
OOD case and 1 the ID case, a detection model g ( · ) is defined as
[Equation]
where τ ∈ R is a threshold and s : X → R is the scoring function defined by f ( · ) , typically withits fixed parameters.
Here, we focus on two representative methods in the classification-based OODdetection, namely, the softmax scoring and the free energy scoring .
Softmax Scoring Function [15] uses the maximum softmax prediction in OOD detection, of whichthe scoring function s SM ( · ) is given by
[Equation]
where softmax k ( · ) denotes the k -th element of the softmax outputs.
In general, with a large (small)s SM ( x ; f ) , the detection model will take the input x as an ID (OOD) case.
However, previousworks have found that the softmax prediction may be overconfident on OOD inputs [29], which isdetrimental for its detection capability.
Free Energy Scoring Function [29] adopts the free energy function for scoring, defined by the logitoutputs with a negative logsumexp operation, namely,
[Equation]
where T > 0 is the temperature parameter.
It aligns with the density of inputs to some extent, andthus is less susceptible to the overconfidence issue mentioned above [29].
Watermarking Strategy
This section introduces the key concepts of watermarking for classification-based OOD detection.
Definition.
A watermark w ∈ R d is a unified pattern with the exact shape as original inputs.
Itis added to test-time inputs statically, and we refer w + x a watermarked input for ∀ x ∈ X .
Inexpectation, regarding the specified scoring function s ( · ) , our watermarking should make the modelexcel at OOD detection for watermarked data.
Learning Strategy.
Given the scoring function s ( · ) , it is challenging to devise the exact watermarkpattern by predefined rules.
Therefore, for the proper watermarks in OOD detection, we need todevise learning objectives regarding watermarks, which consider both ID and OOD cases.
We generally have no information about the OOD distribution D OOD X , Y , while we hope the model toproduce low scores for unseen OOD inputs after watermarking.
For this challenge, we adopt thefollowing training strategy—the model should produce high scores if both the watermark and ID dataare observed, while the score should be low if the model perceives only the watermark.
From the lensof our model, the scores should remain low if a watermarked OOD input is given since the learningprocedure does not see any OOD data during training, and only the watermark can be observed.
Benefits of Watermarking.
Watermarking directly reprograms the model to make an adaptationto our specified task for scoring, such that the detection capability of the original model is largelyimproved.
By contrast, previous methods typically adapt to their specified tasks by only the thresholdτ as in Eq.
However, it requires the trade-off between false positive (ID) and false negative(OOD) rates when densities of scoring are non-separable (cf., Figure 2(a)).
Further, watermarking enjoys the benefits of previous classification-based methods in that we do notmodify the original training procedure in classification, making our proposal easy to be deployed inreal-world systems.
Although the watermark also should be learned, the parameter space is in lowdimension, and the learning procedure could be conducted post-hoc after the systems are deployed.
Comparison with Existing Works .
In OOD detection, this paper is a first attempt in using thereprogramming property of deep models, leading to an effective learning framework named watermarking.
At first glance, our methodology is seemingly similar to ODIN [28], which also conductsdata-level perturbation for OOD detection.
However, their instance-specified perturbation relies onextra backward-forward iterations during the test, which is not required in our method.
Further, ODINis designed for the softmax scoring, but our proposal is much general in OOD detection.
Realization of Watermarking Strategy
In this section, we discuss our learning framework of watermarking in detail.
Learning Objectives.
As mentioned above, we need to consider the ID and OOD situations separately,with the associated objective functions denoted by ℓ IN ( · ) and ℓ OUT ( · ) .
For the ID case, the ID trainingsample is required, where we make the high scores for their watermarked counterparts.
By contrast,since we typically lack knowledge about the test-time OOD data, only the watermark is used here,and we expect the model to produce the score as low as possible when perceiving the watermark.
Further, since only the watermark is adopted for training in the OOD case, the learned watermark ispretty sensitive regarding the detection model, i.e., the model may return different predictions whenfacing small perturbations.
Thus, it hardly generalizes to real-world OOD cases, and the watermarked
OOD inputs may not guarantee the low scores.
To this end, the watermark is further perturbed duringtraining.
Here we adopt the Gaussian noise, leading to the perturbed watermark of the form, [Equation] [Equation] where ϵ is the (i.i.d.)
Gaussian noise of d -dimension with the 0 mean and the standard deviationσ 1 I d .
In summary, the overall risk is written as,
[Equation]
with β ≥
the trade-off parameter, L IN n ( w ) the risk for ID data, and L OUT n ( w ) the risk for OOD data.
Optimization.
Data-level optimization remains difficult in deep learning, of which the results mayget stuck at suboptimal points [47].
A common approach is to use the signum of first-order gradients,guiding the updating rule of the current watermark via
[Equation]
where sign ( · ) denotes the signum function and α > 0 is the step size.
Further, for generality and insensibility, we prefer the solution that lies in the neighbourhood havinguniformly low loss, i.e., with a smooth loss landscape [21].
Therefore, we adopt the sharpness-awareminimization (SAM) [10], an effective optimization framework in the seek of both the low loss valueand the smooth loss landscape.
Specifically, given the original risk L n ( w ) , the SAM problem is:
[Equation]
where ρ ≥ 0 is a constraint.
For efficiency, the SAM makes the first-order Taylor expansion w.r.t.
κaround 0 , obtaining the approximated solution of the form 1 :
[Equation]
[Equation]
where 1 /p + 1 /q = 1 and we set p = q = 2 for simplicity.
Therefore, the final form of the SAM iswritten as L n ( w + κ ) , with corresponding updating rule of [Equation]
[Equation]
yielding an efficient optimization algorithm that induces the effective watermark.
The Overall Algorithm.
In summary, we describe the overall learning framework.
To begin with, thewatermark is initialized by the (i.i.d.)
Gaussian noise with the 0 mean and a small standard deviationσ 2 I d , and the learning procedure consists of three stages for each updating step:
• Negative sampling: a set of noise data given by Eq. (4) is sampled, assuming be of the size
m as that of the mini-batch regarding the ID sample;
• Risk calculating: the risk for ID and OOD cases are computed, and the overall risk is givenby their sum with a trade-off parameter β as in Eq.
• Watermark updating: the first-order gradient guides the pixel-level update of the watermark,
using the signum of gradients and the SAM to make a reliable update as in Eq.
The learned watermark is added to test-time inputs for OOD detection, and the detection model withthe pre-defined scoring function is then deployed.
Appendix B summarizes our learning frameworkof watermarking.
Moreover, two specifications of watermarking are discussed in the following.
Two Realizations.
Here, we focus on two representative methods in OOD detection, namely, thesoftmax scoring and the free energy scoring.
For other representative methods in OOD detection,please refer to Appendix C for their descriptions and the experiments.
Softmax Scoring-based Watermarking.
Following [16], we set ℓ INSM ( · ) to be the cross entropy loss andℓ OUT SM ( · ) to be the cross entropy regarding the uniform distribution, namely,
[Equation]
INSM
[Equation]
softmax
[Equation]
and ℓ
[Equation]
x
[Equation]
f ) = −
[Equation]
specifying the learning objectives in Eq.
(16) for the softmax scoring-based watermarking.
Free-Energy Scoring-based Watermarking.
[29] use a set of learning objectives for model re-trainingwith free energy scoring.
However, their logsumexp operation originating from the free energyfunction is difficult for optimization, posing notorious computing issues [31].
To this end, we dropthe log operation and make the overall risk always positive by the following learning objectives:
[Equation]
realizing the learning objectives in Eq.
(16) for the free-energy scoring.
Experiments
In this section, we conduct extensive experiments for watermarking in OOD detection.
Specifically,we demonstrate the effectiveness of our method on a wide range of OOD evaluation benchmarks;we conduct ablation analysis for the important hyper-parameters in our learning framework; and weprovide further experiments for an improved interpretation of our proposal.
Baselines results are achieved by our re-run of the publicly available codes.
All the methods arerealized by Pytorch 1 . 81 with CUDA 11 .
1 , where we use several machines equipped with GeForce
RTX 3090 GPUs and AMD Ryzen Threadripper 3960 X Processors.
ID and OOD Datasets .
Following previous works [29, 32], we use CIFAR- 10 and CIFAR- 100 [23]datasets as two ID datasets, with data pre-processing including the horizontal random flip and thenormalization.
In Appendix C, we also conduct experiments on ImageNet [38] dataset, which hasa larger semantic space than that of the CIFAR benchmarks.
Furthermore, for the OOD datasets,we adopt several commonly-used benchmarks, including Textures [5], SVHN [34], Places 365 [57],
LSUN [53], and iSUN [50].
Evaluation Metrics .
The performance in OOD detection is measured via three representative metrics,which are all threshold-independent [6]: (1) the false positive rate of OOD sample when true positiverate of ID data is at 95% (FPR 95 ); (2) the area under the receiver operating characteristic
curve(AUROC), interpreted as the probability that an ID input has a greater score than an OOD one; and(3) the area under the precision-recall curve (AUPR), which further adjusts for different base rates.
Configuration .
Following previous works [29], we employ WideResNet [55] (WRN-40-2) as thebackbone model.
Specifically, the models are trained for 200 epochs via the stochastic gradientdescent, with the batch size 64 , the momentum 0 . 9 , and the initial learning rate 0 . 1 .
Further, thelearning rate is divided by 10 after 100 and 150 epochs.
Hyper-parameters .
By default, our method is executed for 50 epochs.
The initial learning rateα = 0 .
01 divided by 10 after 25 epochs.
σ 2 is fixed to be 0 .
001 and T is set to be 1 following [29].
Please refer to the Appendix C for the hyper-parameter setups and the selection policy regarding thestandard deviation σ 1 , the perturbation constraint ρ , and the trade-off parameter β .
Performance Evaluation .
We depict the learned watermarks in Figure 3.
As we can see, the centreareas maintain the pattern of original inputs, which are helpful in OOD detection primitively.
Bycontrast, the edge areas of the watermarks distort the original features, superimposed with the patternthat may further boost the capability of the original models in OOD detection.
Then, we demonstrate the improvement of watermarking on CIFAR- 10 and CIFAR- 100 datasets.
For the results of the softmax scoring in Table 1, our watermarking reduces the average FPR 95 by2 .
99 ∼ 12 . 84 , boosts the average AUROC by 3 .
09 ∼ 3 .
69 and the average AUPR by 1 .
10 ∼ 1 . 18 .
Moreover, for each of the considered test-time OOD dataset, there also exist stable improvementsafter watermarking, except for the comparable results regarding Places 365 on CIFAR- 100 dataset.
For the results of the free energy scoring in Table 2, the improvement via watermarking is alsosubstantial, with 4 .
54 ∼ 13 . 98 , 3 .
43 ∼ 4 . 65 , and 1 . 13 ∼ 1 .
37 better performance regarding theaverage FPR 95 , AUROC, and AUPR.
Overall, Table 1 and Table 2 not only justify the effectiveness ofwatermarking and also demonstrate the generality of the proposed watermarking.
Further, comparingbetween CIFAR- 10 and CIFAR- 100 datasets, the improvements of watermarking on CIFAR- 10 ismuch greater than that of CIFAR- 100 , aligning with the previous observations [20] that a largesemantic space can exaggerate the challenge in effective OOD detection.
[Figure]
[Table] [Table]
Figure 4 illustrates the scoring distributions before (a) and after (b) watermarking on CIFAR- 10 ,where we take the free energy scoring as an example.
Due to the space limit, we only considertwo test-time OOD datasets, namely, Texture and SVHN.
As we can see, after watermarking, thedistribution gap between the ID (i
[Table]
e., CIFAR- 10 ) and the OOD (i.e., Texture and SVHN) cases isenlarged, and thus the detection capability is improved.
Ablation Study.
To further interpret our learning framework, we compare the performance of theresultant watermarks regarding various setups of hyper-parameters, focusing on the standard deviationσ 1 in the Gaussian noise and the perturbation constraint ρ in the SAM, which are both critical for ourproposal.
As a case study, we conduct experiments regarding the softmax scoring on CIFAR- 10 .
Fordetailed results about the ablation study, please refer to Appendix C.
Table 3 lists the results with various values of σ 1 , ranging from 0 .
to 1 . 00 .
Note that σ 1 is thestandard deviation of the Gaussian noise added to the watermark, with different values indicatingvarious degrees of the perturbation.
As we can see, a mild perturbation of the watermark (e.g.,σ 1 = 0 . 4 ) can truly lead to improved results in detection, with 10 . 05 , 1 . 60 , and 0 .
34 improvementsregarding the average FPR 95 , AUROC, and AUPR.
It indicates that the learned watermark is sensitivewhen facing small perturbations, if the Gaussian noise is not applied during training.
However, someextreme values (e.g., σ 1 = 1 ) may overwhelm the watermark pattern and thus be detrimental.
We alsoobserve a trade-off on various test-time OOD datasets for the watermarks learned from different σ 1 . In Figure 5, test-time OOD datasets such as Places 365 and LSUN-C prefer small values of σ 1 , while [Figure] Texture
SVHN LSUN [Figure] C
[Figure] [Figure] [Table] ( ↑ ) indicates smaller (larger) values are preferred.
[Table]
iSUN and LSUN-R much prefer that of the large ones.
It may indicate that the Gaussian perturbationis not optimal for the stability purpose, requiring our study in the future.
In Table 4, we summarize the experimental results given by various values of ρ for the SAM.
Overall,a large value of ρ indicates that a wide range in the solution’s neighbours should be smooth, and thusthe stability of the result is expected to be improved.
However, such a solution may consume too muchcapacity of the watermark, misleading the learning procedure to some unsatisfactory results.
On theother side, our watermarking truly benefits from the SAM with a mild choice of the hyper-parameter.
Specifically, comparing with the results without the SAM (i.e., ρ = 0 . 00 ), the detection capability ofthe watermark with a suitable ρ (i.e., ρ = 3 . 5 ) is largely improved, with 11 . 26 , 1 . 09 , and 0 .
26 betterresults regarding the average FPR 95 , AUROC, and AUPR.
Impact on test accuracy . Now, we study the impact of watermarking on the classification accuracy in Table 5, comparing with the results before watermarking. As we can see, watermarking has a negativeimpact on the test accuracy, dropping from 94 .
84% to 91 .
85% and 93 .
49% on CIFAR- 10 ; andfrom 75 .
96% to 72 .
03% and 74 .
08% on CIFAR- 100 .
Further, after watermarking, the classificationaccuracy with the free energy scoring is much better than that of the softmax scoring, with only1 .
35% to 1 .
88% decrease in classification accuracy.
Therefore, we suggest using the free energyscoring in watermarking as a default setup, which leads to better detection capability and largelypreserves the original capability in classification.
Transferability of watermarking .
Further, we explore the transferability of the watermarks learnedwith different scoring functions, e.g., we study the effect of the watermark learned with the softmaxscoring when it is deployed with the free energy scoring.
The results are summarized in Table 6.
Note that, a “learn” with FE and “score” with SM indicates that the watermark is learned with thesoftmax scoring and tested regarding the free energy scoring.
As we can see, the watermarks learnedwith the free energy scoring can be reused for the softmax scoring (even with better results), whilethe reverse typically leads a deterioration.
It may indicate that our learning objectives in Eq.
(11) aregeneral and effective for both the softmax scoring and the free energy scoring.
However, we do notobserve any transferability between different datasets.
For example, for the softmax scoring, when
[Table]
[Table] [Table] the watermark learned on CIFAR-10 is adopted for CIFAR-100, there is a drop of performance with51 .
93 (from 42 .
86 to 94 . 79 ), 35 .
(from 92 .
91 to 57 . 62 ) and 12 .
42 to 86 .
42 ) regarding theaverage FPR 95 , AUROC, and AUPR. D en s i t y [Figure]
en [Figure] i t y [Figure]
iSUN
Places
Texture
SVHN
LSUN-C
LSUN-R 𝜎 𝜎 𝜎
[Figure]
i t y
[Figure]
D
en s i t y
[Figure]
[Figure]
D
en s i t y
[Figure]
en s i t y
[Figure]
iSUN
Places
Texture
SVHN
LSUN-C
LSUN-R 𝜎 𝜎 𝜎
[Figure]
i t
[Figure]
Different areas in the watermark.
After watermarking, the edge area of the image is overwhelmed by the watermark’s pattern, while thecentre part is not much affected.
However, it doesnot mean that the centre area is not important.
Instead, under the premise of maintaining the original features, the centre area also encodes usefulinformation in OOD detection.
Table 7 is a verification of this conclusion on CIFAR- 10 datasetwith the free energy scoring, where we employvarious masks in only preserving the watermark’sfeatures with their absolute values larger (smaller)than a threshold χ 1 ( χ 2 ).
As we can see, even ifonly a small portion of the watermark is masked(e.g.
, χ 1 = 0 .
10 or χ 2 = 10 . 0 ), there is a large drop in performance, even lower than the resultswithout any watermarking.
It indicates that both areas of the watermark contribute, and the overallwatermark works as a whole for effective OOD detection.
Conclusion
This paper demonstrates that the model’s inherent capability in OOD detection can be largelyimproved with only data-level manipulation, where we propose a general and effective methodologynamed watermarking.
Overall, we learn a discriminative pattern that could be superimposed ontooriginal inputs, such that the OOD scores become much separable between ID and OOD situations.
Our results indicate a promising direction in OOD detection that warrants our exploration in the future.
Further investigation should focus on the stability in optimization and general learning objectivesregarding advanced OOD detection methods.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g., type
of GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] We use only standard datasets
(e) Did you discuss whether the data you are using/curating
contains personally identifiable information or offensive content
No] We use only standard datasets
If you used crowdsourcing or conducted research with human subjects
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[-The experimental results on CIFAR- 
-]
spent on participant compensation?
[N/A]