Watermarking for Out-of-distribution Detection
Anonymous Author(s)
AffiliationAddress email
Abstract
Out-of-distribution (OOD) detection aims to identify OOD data based on representations extracted from well-trained deep models.
However, existing methods largelyignore the reprogramming property of deep models and thus may not fully unleashtheir intrinsic strength: without modifying parameters of a well-trained deep model,we can reprogram this model for a new purpose via data-level manipulation (e.g.,adding a specific feature perturbation).
This property motivates us to reprogram aclassification model to excel at OOD detection (a new task), and thus we propose ageneral methodology named watermarking in this paper.
Specifically, we learn aunified pattern that is superimposed onto features of original data, and the model’sdetection capability is largely boosted after watermarking.
Extensive experimentsverify the effectiveness of watermarking, demonstrating the significance of thereprogramming property of deep models in OOD detection.
Introduction
Deep learning systems in the open world often encounter out-of-distribution (OOD) inputs whoselabel spaces are disjoint with that of training data, known as in-distribution (ID) data.
For safetycritical applications, deep models should make reliable predictions for ID data, meanwhile detecting
OOD data without prediction [2].
This leads to the OOD detection problem [28, 36, 40, 49] whichhas attracted intensive attention in reliable machine learning.
Identifying OOD data remains non-trivial since deep models can be overconfident with them [38].
Asa promising technique, the classification-based OOD detection [57] relies on various scoring functionsderived by the well-trained classification models, taking those inputs with small scores as OODcases.
In general, the scoring functions can be defined by logit outputs [18, 32], gradients [22] andembedding features [28, 43].
Without interfering with original models or requiring extra computation,they exploit the inherent capability of models learned from only ID sample.
In general, theseadvantages can be critical in reality, where the cost of re-training is prohibitively high and theacquisition of real OOD data is largely limited [57].
Although promising progress has been achieved, previous works largely ignore the reprogrammingproperty [10] of deep models: a well-trained model can be repurposed for a new task by a propertransformation of original inputs (e.g., a universal feature perturbation), without modifying any modelparameter.
For example, a model pre-trained on ImageNet [7] dataset can be reprogrammed forbiomedical images [48].
In classification-based OOD detection, it indicates the possibility of makingthe target models adapt for effective detection given fixed methods and scoring strategies, motivatingus the first attempt to investigate if the reprogramming property of deep models can help to address OOD detection, i.e., can we reprogram well-trained deep models for OOD detection (a new task)?
To this end, we propose a novel method, named watermarking , to reprogram the classification modelfor effective OOD detection.
In our context, “reprogramming” is reflected by a watermark with itsfeatures on the shape of original inputs, which is defined as a static pattern that can be added for the test-time inputs (cf., Figure 1).
After watermarking, the pre-defined scoring strategy is expected to beenhanced, with an improved gap of OOD scores between ID and OOD data (cf., Figure 2).
It is non-trivial to find the proper watermarksdue to our lack of knowledge about unseen inadvance.
To address the issue, we propose alearning framework for effective watermarking.
The insight is to make the model produce highscores for watermarked ID inputs meanwhileregularize the watermark such that the modelwill return low confidence without perceiving ID patterns.
In this case, the model will have arelatively high score for a watermarked ID input,while the score is low for an OOD case (cf., Figure 2).
The reason is that the target modelencounters a watermarked OOD input, withoutseeing any ID pattern.
In our realization, we adopt several representative scoring strategies, devising specified learningobjectives and proposing a reliable optimization algorithm.
To understand our mechanism, Figure 1depicts the watermark learned on CIFAR- 10 [26] dataset, with the free energy scoring [32] (anadvanced classification-based method).
As we can see, the centre area of the learned watermarklargely preserves the original input pattern, containing the semantic message that guides the detectionprimitively.
By contrast, the edge area of the original input is superimposed by the specific patternof the watermark, which may encode the knowledge once hidden by the model in boosting OODdetection.
Overall, watermarking can preserve the meaningful pattern of original inputs in detectionwith further enhancement learned from the trained model.
Therefore, the detection capability of theoriginal model is largely improved after watermarking.
Figure 2 demonstrates the effect of our learned watermark, which is an example with the free-energyscoring.
After watermarking, the scoring distributions are much concentrated, and the gap between
ID (i.e., CIFAR- 10 ) and OOD (i.e., SVHN [37] and Texture [5] datasets) cases is enlarged notably.
We conduct extensive experiments for a wide range of OOD evaluation benchmarks in Section 6 and
Appendix C, and the results verify the effectiveness of our proposal.
The success of watermarking takes roots in the following aspects: (1) a well-trained model onclassification has the potential to be reprogrammed for OOD detection since they are two closelyrelated tasks; (2) reprogramming has been widely studied, ranging from image classification to timeseries analysis [7, 48], making our proposal general across various domains; and (3) OOD detectionsuffers from the lack of knowledge about the real-world OOD distributions.
Fortunately, with onlydata-level manipulation in low dimensions, watermarking can largely mitigate this issue of limiteddata.
Overall, this data-level manipulation is orthogonal to existing methods, and thus provides a newroad in OOD detection and can inspire more ways to design OOD detection methods in the future.
Related Works
To begin with, we briefly review the related works in OOD detection and model reprogramming.
Please refer to Appendix A for the detailed discussion.
OOD Detection discerns ID and OOD data by their gaps regarding specified metrics/scores, andexisting methods can mainly be attributed into three categories [57], namely, the classification-basedmethods, the density-based methods, and the distance-based methods.
Specifically, the classificationbased methods [18, 22, 32, 43] use representations extracted from the well-trained models in OODscoring; and the distance-based methods [3, 21, 39, 59] measure the distance of inputs from classcenters in the embedding space.
Moreover, the density-based methods estimate input density withprobabilistic models [28, 40, 44], identifying those OOD data with small likelihood values.
Sincedistance-based and density-based methods may suffer from complexity in computation [28] anddifficulty in optimization [61], recently, more researchers focus on developing classification-basedmethods and have made big progress on benchmark datasets
D
en s i t y
D
en s i t y
Free Energy Score
[Figure]
Energy Score iSUN
Places
Texture SVHN
LSUN-C
LSUN
Energy
D
en s i t y (b)
w/ watermarking
D
en s i t y (a) w/o watermarking (a) before watermarking
D
en s i t y
D
en s i t y
Free Energy Score
Free Energy Score iSUN
Places365 Texture SVHN LSUN-C LSUN 𝜎 𝜎 𝜎 Free Energy Score
D
en s i t y (b)
w/ watermarking
Free Energy Score
D
en s i t y (a)
w/o watermarking
[Figure]
Model Reprogramming repurposes the well-trained models for new tasks with only data-levelmanipulation [10], indicating that deep models are competent for different jobs without changingmodel parameters.
In previous works, the data-level manipulation typically refers to a static paddingpattern (different from our proposal) learned for the target task, which is added to the test-time data.
The effectiveness of the model reprogramming is verified across image classification [10, 48] andtime-series analysis [14, 56].
In this paper, we use the reprogramming property of deep models foreffective OOD detection, which has been overseen previously.
Preliminary
Let X ⊂ R d be the input space and Y = { 1 , . .
, c } be the label space.
We consider the ID distribution
D ID X , Y defined over X × Y , the training sample S n = { ( x i , y i ) } ni =1 of size n independently drawnfrom D ID X , Y , and a classification model f : X → R c (with logit outputs) well-trained on S n .
Based on the model f ( · ) , the goal of the classification-based OOD detection is to design a detectionmodel g : X → { 0 , 1 } that can distinguish test-time inputs given by the ID distribution
D ID X fromthat of the OOD distribution D OOD X .
In general, D OOD X is defined as an irrelevant distribution of whichthe label set has no intersection with Y , and thus should not be predicted by f ( · ) .
Overall, with 0 the
OOD case and 1 the ID case, a detection model g ( · ) is defined as
[Equation]
where τ ∈ R is a threshold and s : X → R is the scoring function defined by f ( · ) , typically withits fixed parameters.
Here, we focus on two representative methods in the classification-based OODdetection, namely, the softmax scoring and the free energy scoring .
Softmax Scoring Function [18] uses the maximum softmax prediction in OOD detection, of whichthe scoring function s SM ( · ) is given by
[Equation]
where softmax k ( · ) denotes the k -th element of the softmax outputs.
In general, with a large (small)s SM ( x ; f ) , the detection model will take the input x as an ID (OOD) case.
Free Energy Scoring Function [32] adopts the free energy function for scoring, defined by the logitoutputs with a negative logsumexp operation, namely,
[Equation]
where T > 0 is the temperature parameter fixed to 1 in our paper.
It aligns with the density of inputsto some extent, and thus is less susceptible to the overconfidence issue mentioned above [32].
Watermarking Strategy
This section introduces the key concepts of watermarking for classification-based OOD detection.
Definition.
A watermark w ∈ R d is a unified pattern with the exact shape as original inputs.
Itis added to test-time inputs statically, and we refer w + x a watermarked input for ∀ x ∈ X .
Inexpectation, regarding the specified scoring function s ( · ) , our watermarking should make the modelexcel at OOD detection for watermarked data.
Learning Strategy.
Given the scoring function s ( · ) , it is challenging to devise the exact watermarkpattern by predefined rules.
Therefore, for the proper watermarks in OOD detection, we need todevise learning objectives regarding watermarks, which consider both ID and OOD cases.
We generally have no information about the OOD distribution D OOD X , Y , while we still want the modelexcels in discerning ID and OOD cases from scoring.
For this challenge, we make the model producehigh scores if watermarked ID data are observed; meanwhile, we regular the watermark such thatthe model will return low scores when ID patterns do not exist.
From the lens of our model, thescores should remain low if a watermarked OOD input is given since the watermark is not trained toperceive OOD data, of which the patterns are very different from the ID cases.
Benefits of Watermarking.
Watermarking directly reprograms the model to make an adaptationto our specified task for scoring, such that the detection capability of the original model is largelyimproved.
By contrast, previous methods typically adapt to their specified tasks by only the thresholdτ as in Eq.
However, it requires the trade-off between false positive (ID) and false negative(OOD) rates when densities of scoring are non-separable (cf., Figure 2(a)).
Further, watermarking enjoys the benefits of previous classification-based methods in that we do notmodify the original training procedure in classification, making our proposal easy to be deployed inreal-world systems.
Although the watermark also should be learned, the parameter space is in lowdimension, and the learning procedure could be conducted post-hoc after the systems are deployed.
Comparison with Existing Works .
In OOD detection, this paper is a first attempt in using thereprogramming property of deep models, leading to an effective learning framework named watermarking.
At first glance, our methodology is seemingly similar to ODIN [31], which also conductsdata-level perturbation for OOD detection.
However, their instance-specified perturbation relies onextra backward-forward iterations during the test, which is not required in our method.
Further, ODINis designed for the softmax scoring, but our proposal is much general in OOD detection.
Realization of Watermarking Strategy
In this section, we discuss our learning framework of watermarking in detail.
Learning Objectives.
As mentioned above, we need to consider the ID and OOD situations separately,with the associated objective functions denoted by ℓ ID ( · ) and ℓ OOD ( · ) .
For the ID case, the ID trainingsample is required, where we make the high scores for their watermarked counterparts.
By contrast,since we typically lack knowledge about the test-time OOD data, only the watermark is used here,and we expect the model to produce the score as low as possible when perceiving the watermark.
Further, since only the watermark is adopted for training in the OOD case, the learned watermark ispretty sensitive regarding the detection model, i.e., the model may return different predictions whenfacing small perturbations.
Thus, it hardly generalizes to real-world OOD cases, and the watermarked
OOD inputs may not guarantee the low scores.
To this end, the watermark is further perturbedduring training.
Here we adopt the Gaussian noise, leading to the perturbed watermark of the formϵ + w with ϵ ∼ N ( 0 , σ 1 I d ) the (i.i.d.)
Gaussian noise of d -dimension (the mean 0 and the standarddeviation σ 1 I d ).
In summary, the overall risk is written as,
[Equation]
with β ≥
the trade-off parameter, L ID n ( w ) the risk for ID data, and L OOD n ( w ) the risk for OOD data.
Optimization.
To find the proper watermark, we use the first-order gradient update to iterativelyupdate watermark’s elements.
However, data-level optimization remains difficult in deep learning, ofwhich the results may get stuck at suboptimal points [51].
A common approach is to use the signumof first-order gradients, guiding the updating rule of the current watermark via
[Equation]
where sign ( · ) denotes the signum function and α > 0 is the step size.
Further, for generality and insensibility, we prefer the solution that lies in the neighbourhood havinguniformly low loss, i.e., with a smooth loss landscape [24].
Therefore, we adopt the sharpness-awareminimization (SAM) [11], an effective optimization framework in the seek of both the low loss valueand the smooth loss landscape.
Specifically, given the original risk L n ( w ) , the SAM problem is:
[Equation]
where ρ ≥ 0 is a constraint.
For efficiency, the SAM makes the first-order Taylor expansion w.r.t.
κaround 0 , obtaining the approximated solution of the form 1 :
[Equation]
where 1 /p + 1 /q = 1 and we set p = q = 2 for simplicity.
Therefore, the final form of the SAM iswritten as L n ( w + κ ) , with corresponding updating rule of
[Equation]
yielding an efficient optimization algorithm that induces the effective watermark.
The Overall Algorithm.
In summary, we describe the overall learning framework.
To begin with, thewatermark is initialized by the (i.i.d.)
Gaussian noise with the 0 mean and a small standard deviationσ 2 I d , and the learning procedure consists of three stages for each updating step:
• Negative sampling: a set of noise data ϵ is sampled, assuming be of the size m as that of the
mini-batch regarding the ID sample;
• Risk calculating: the risk for ID and OOD cases are computed, and the overall risk is given by their sum with a trade-off parameter β as in Eq.
• Watermark updating: the first-order gradient guides the pixel-level update of the watermark,
using the signum of gradients and the SAM to make a reliable update as in Eq.
The learned watermark is added to test-time inputs for OOD detection, and the detection model withthe pre-defined scoring function is then deployed.
Appendix B summarizes our learning frameworkof watermarking.
Moreover, two specifications of watermarking are discussed in the following.
Two Realizations.
Here, we focus on two representative methods in OOD detection, namely, thesoftmax scoring and the free energy scoring.
For other representative methods in OOD detection,please refer to Appendix C for their descriptions and the experiments.
Softmax Scoring-based Watermarking.
Following [19], we set ℓ IDSM ( · ) to be the cross entropy loss andℓ OODSM ( · ) to be the cross entropy regarding the uniform distribution, namely,ℓ
[Equation]
OODSM
[Equation]
specifying the learning objectives in Eq.
(16) for the softmax scoring-based watermarking.
Free-Energy Scoring-based Watermarking.
[32] use a set of learning objectives for model re-trainingwith free energy scoring.
However, their logsumexp operation originating from the free energyfunction is difficult for optimization, posing notorious computing issues [34].
To this end, we dropthe log operation and make the overall risk always positive by the following learning objectives:
[Equation]
realizing the learning objectives in Eq.
(16) for the free-energy scoring. [Figure]
Experiments
In this section, we conduct extensive experiments for watermarking in OOD detection.
Specifically,we demonstrate the effectiveness of our method on a wide range of OOD evaluation benchmarks;we conduct ablation analysis for the important hyper-parameters in our learning framework; and weprovide further experiments for an improved interpretation of our proposal.
Baselines results are achieved by our re-run of the publicly available codes.
All the methods arerealized by Pytorch 1 . 81 with CUDA 11 .
1 , where we use several machines equipped with GeForce
RTX 3090 GPUs and AMD Ryzen Threadripper 3960 X Processors.
ID and OOD Datasets .
We use CIFAR- 10 , CIFAR- 100 [26], and ImageNet [41] datasets as three IDdatasets, with data pre-processing including horizontal flip and normalization.
Furthermore, for the
OOD datasets, we adopt several commonly-used benchmarks, including Textures [5], SVHN [37],
Places 365 [62], LSUN [58], and iSUN [55].
Evaluation Metrics .
The performance in OOD detection is measured via three representative metrics,which are all threshold-independent [6]: (1) the false positive rate of OOD sample when true positiverate of ID data is at 95% (FPR 95 ); (2) the area under the receiver operating characteristic
curve(AUROC), interpreted as the probability that an ID input has a greater score than an OOD one; and(3) the area under the precision-recall curve (AUPR), which further adjusts for different base rates.
Configuration .
Following previous works [32], we employ WideResNet [60] (WRN-40-2) as thebackbone model.
For the CIFAR benchmarks, the models are trained for 200 epochs via the stochasticgradient descent, with the batch size 64 , the momentum 0 . 9 , and the initial learning rate 0 . 1 . Thelearning rate is divided by 10 after 100 and 150 epochs. For the ImageNet Benchmark, the model istrained for 120 epochs via the stochastic gradient descent, with the batch size 32 , the momentum 0 .
9and the initial learning rate 0 . The learning rate is divided by 10 after 60 and 90 epochs.
Hyper-parameters .
For the hyper-parameter setups in our experiments, we choose the proper σ 1from the candidate parameter set { 0 . 0 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 . 0 , 1 . 2 , 1 . 4 , 1 . 6 , 1 . 8 , 2 . 0 } , and the properρ from { 0 . 0 , 0 . 02 , 0 . 05 , 0 . 07 , 0 . 1 , 0 . 2 , 0 . 5 , 0 . 7 , 1 . 0 , 2 . 0 , 5 . 0 } .
For the softmax scoring, β is chosenfrom { 0 . 0 , 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 , 2 . 5 , 3 . 0 , 3 . 5 , 4 . 0 , 4 . 5 , 5 . 0 } ; and for the free energy scoring, β is chosenfrom { 0 . 0 , 0 .
08 , 0 . 1 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 . 0 } by random search (cf., Appendix C.7).
We adopt validation OOD datasets that are separated from the original iSUN, Places365, Texture,
SVHN, and LSUN datasets, and we choose the proper hyper-parameters from the candidates.
On CIFAR benchmarks, our method is executed for 50 epochs.
The initial learning rate α = 0 .
01 dividedby 10 after 25 epochs.
For the softmax scoring, we set σ 1 = 0 . 4 , ρ = 1 . 0 , β = 3 .
5 in CIFAR- 10 andσ 1 = 1 . 0 , ρ = 0 . β scoring we ρ method is [Table] [Table]
D en s i t y [Figure] D
en s i t y [Figure] before watermarking and (b) is the scoring distributions after watermarking. CIFAR Benchmarks .
We depict the learned watermarks in Figure 3.
As we can see, the centre areasmaintain the pattern of original inputs, which are helpful in OOD detection primitively.
By contrast,the edge areas of the watermarks distort the original features, superimposed with the pattern that mayfurther boost the capability of the original models in OOD detection.
Then, we demonstrate the improvement of watermarking on CIFAR- 10 and CIFAR- 100 datasets.
For the results of the softmax scoring in Table 1, our watermarking reduces the average FPR 95 by2 .
99 ∼ 12 . 84 , boosts the average AUROC by 3 .
09 ∼ 3 .
69 and the average AUPR by 1 .
10 ∼ 1 . 18 .
Moreover, for each of the considered test-time OOD dataset, there also exist stable improvementsafter watermarking, except for the comparable results regarding Places 365 on CIFAR- 100 dataset.
For the results of the free energy scoring in Table 2, the improvement via watermarking is alsosubstantial, with 4 .
54 ∼ 13 . 98 , 3 .
43 ∼ 4 . 65 , and 1 . 13 ∼ 1 .
37 better performance regarding theaverage FPR 95 , AUROC, and AUPR.
Overall, Table 1 and Table 2 not only justify the effectiveness ofwatermarking and also demonstrate the generality of the proposed watermarking.
Further, comparingbetween CIFAR- 10 and CIFAR- 100 datasets, the improvements of watermarking on CIFAR- 10 ismuch greater than that of CIFAR- 100 , aligning with the previous observations [23] that a largesemantic space can exaggerate the challenge in effective OOD detection.
Figure 4 illustrates the scoring distributions before (a) and after (b) watermarking on CIFAR- 10 ,where we take the free energy scoring as an example.
Due to the space limit, we only considertwo test-time OOD datasets, namely, Texture and SVHN.
As we can see, after watermarking, thedistribution gap between the ID (i.e., CIFAR- 10 ) and the OOD (i.e., Texture and SVHN) cases isenlarged, and thus the detection capability is improved.
[Table] [Table]
ImageNet Benchmark. Huang and Li [23] show that many advanced methods developed on the CIFAR benchmark can hardly work for the ImageNet dataset due to its large semantic space with 1 kclasses. In order to verify the power of watermarking in large semantic space, we conduct experimentswith ImageNet being the ID dataset, and the results regarding the softmax scoring and the free-energyscoring are summarized in Table 3 and Table 4.
For the softmax scoring, we decrease the average
FPR 95 from 54 . 93 to 40 . 50 after watermarking; for the free-energy scoring, we reduce the average FPR 95 from 52 .
73 to 43 . 23 .
It demonstrates that our watermarking still works well in the case of thelarge semantic space, with considerable improvements in OOD detection power.
[Table]
Near OOD Detection.
The above experiments focuson the far OOD detection setups where ID and OODdata are different regarding semantics and stylish.
Now, we further demonstrate the power of our watermarking strategy in a near OOD situation [53], covering a challenging situation where ID and OOD dataare stylistically similar (i.e., near-OOD data).
Besides the common learning setup in Section 5, we further consider the use of shifting augmentations [46],which are data augmentations that are harmful to thestandard contrastive learning but can be used to construct near-OOD data similar to ID data.
We considertwo representative shifting augmentations: ”perm”(permute evenly partitioned data) and “rotate” (rotate90 degrees of original data).
The shifting-augmented
ID data are taken as OOD data fed into ℓ OOD ( · ) alongwith random Gaussian noise.
The experimental results are summarized in Table 5,where we take CIFAR- 10 as ID data and CIFAR- 100 as OOD data.
Here, the common watermarklearning setup (common) can already lead to improved performance in near-OOD detection compared to the cases without watermarking (w/o watermark).
Moreover, watermarking with shiftingaugmentations (perm and rotate) can further boost the detection power of the models, leading to atmost 8 .
60 and 4 . 70 improvements in FPR95 for the softmax and the free energy scoring. Effect of Hyper-parameters. To further interpret our learning framework, we compare the performance of the resultant watermarks regarding various setups of hyper-parameters, focusing on thestandard deviation σ 1 in the Gaussian noise and the perturbation constraint ρ in the SAM, which areboth critical for our proposal.
As a case study, we conduct experiments regarding the softmax scoringon CIFAR- 10 .
For detailed results about the ablation study, please refer to Appendix C.
Table 6 lists the results with various values of σ 1 , ranging from 0 .
to 1 . 00 .
Note that σ 1 is thestandard deviation of the Gaussian noise added to the watermark, with different values indicatingvarious degrees of the perturbation.
As we can see, a mild perturbation of the watermark (e.g.,σ 1 = 0 . 4 ) can truly lead to improved results in detection, with 10 . 05 , 1 . 60 , and 0 .
34 improvementsregarding the average FPR 95 , AUROC, and AUPR.
It indicates that the learned watermark is sensitivewhen facing small perturbations, if the Gaussian noise is not applied during training.
However, someextreme values (e.g., σ 1 = 1 ) may overwhelm the watermark pattern and thus be detrimental.
[Table]
[Table]
In Table
[Table]
we summarize the experimental results given by various values of ρ for the SAM.
Overall,a large value of ρ indicates that a wide range in the solution’s neighbours should be smooth, and thus
[Table]
the stability of the result is expected to be improved.
However, such a solution may consume too muchcapacity of the watermark, misleading the learning procedure to some unsatisfactory results.
On theother side, our watermarking truly benefits from the SAM with a mild choice of the hyper-parameter.
Specifically, comparing with the results without the SAM (i.e., ρ = 0 . 00 ), the detection capability ofthe watermark with a suitable ρ (i.e., ρ = 3 . 5 ) is largely improved, with 11 .
26 , 1 . 09 , and 0 .
26 betterresults regarding the average FPR 95 , AUROC, and AUPR.
[Table]
Transferability of watermarking .
Further, we explore the transferability of the watermarking strategy learned with different scoring functions, e.g., westudy the effect of the watermark learned with thesoftmax scoring when it is deployed with the freeenergy scoring.
The results are summarized in Table 8.
Note that, a “learn” with FE and “score” with SM indicates that the watermark is learned with the [Table] free-energy scoring and tested regarding the softmax
[Table]
scoring. As we can see, the watermarks learned withthe free energy scoring can be reused for the softmax [Table] scoring (even with better results), while the reversetypically leads a deterioration.
It may indicate thatour learning objectives in Eq.
(10) are general andeffective for both the softmax scoring and the free energy scoring.
However, we do not observe any transferability between different datasets.
For example,for the softmax scoring, when the watermark learnedon CIFAR-10 is adopted for CIFAR-100, there is adrop of performance with 51 .
93 (from 42 .
86 to 94 . 79 ), 35 .
(from 92 .
91 to 57 . 62 ) and 12 .
42to 86 .
42 ) regarding the average FPR 95 , AUROC, and AUPR.
Conclusion
This paper demonstrates that the model’s inherent capability in OOD detection can be largelyimproved with only data-level manipulation, where we propose a general and effective methodologynamed watermarking.
Overall, we learn a discriminative pattern that could be superimposed ontooriginal inputs, such that the OOD scores become much separable between ID and OOD situations.
Our results indicate a promising direction in OOD detection that warrants our exploration in the future.
Further investigation should focus on the stability in optimization and general learning objectivesregarding advanced OOD detection methods.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[No](d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g., type
of GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] We use only standard datasets
(e) Did you discuss whether the data you are using
/curating contains personally identifiableinformation or offensive content?
[No] We use only standard datasets
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Additional Related Works
We review the recent studies in OOD detection, model reprogramming, and backdoor attack.
A.1 OOD Detection
Following [57], we attribute existing works into three categories, namely, the classification-basedmethods, the density-based methods, and the distance-based methods.
In general, these methods aimto maximize the gap between ID and OOD data regarding specified metrics in identifying OOD cases.
The classification-based methods use the representations extracted from the well-trained classification models in OOD scoring.
For example, [18, 31, 32, 35, 45, 50] employ logit outputs frommodels in estimating the confidence of ID cases; [28, 43] adopt Mahalanobis distance and Gram
Matrix to exploit models’ detection capability from embedding features; [22, 31] further demonstratethe importance of gradient information, either perturbing inputs with its gradients or directly usingthe gradient norm in scoring.
The classification-based methods are easy to be deployed withoutmodifying the models [57], and thus it is the main focus in this paper.
The distance-based methods measure the distance regarding the embedding space, taking those datafar away from the class prototypes as the OOD cases.
Representative works adopt the Mahalanobisdistance [22, 28, 39], the cosine similarity [3, 59], and the Euclidean distance [21].
Our methods canalso be used in the distance-based methods.
However, extra computation, such as calculating theprecision matrix [28] and background splitting [39], may lead difficulty in devising proper learningobjectives, out of the scope of our paper.
The density-based methods explicitly estimate the density of ID samples with various probabilisticmodels, identifying those OOD data based on the likelihood [28], the likelihood ratio [40, 44], and thelikelihood regret [54].
Typically, the input density is modelled by the mixture of Gaussian models [28]and the flow-based methods [25, 36].
Although the density-based methods can directly characterizethe properties of ID density, these methods are difficult to be trained and may make overconfidentpredictions, as demonstrated in previous works [35, 36].
Recent works also focus on the causes of key challenges in OOD detection, from the lens of the
BatchNorm statistics [45], the density estimation [35], and the spurious correlation [43].
Improvedmethods, related to specified model architectures [8, 52], data perturbation [1, 4, 31], data augmentation [16, 20, 47], and outlier exposure [19], are also well-studied.
However, these methods typicallyoverlook the reprogramming property of deep models, which remain orthogonal to our proposal.
A.2 Reprogramming Property
The seminal work [10] introduces adversarial reprogramming as an attack method in the literatureof adversarial learning [12], adversarially reprogramming the target model to perform a new taskwithout changing the original model.
The term “attack” lies in the fact that, by reprogramming, anattacker can easily steal public machine learning services, abusing their computational resourcesfor tasks that violate their original purposes.
Overall, [10] claim the reprogramming property ofdeep models—without modifying parameters of a well-trained model, we can reprogram it for anew purpose with only data-level manipulation.
The reprogramming property of deep learning ispreliminarily verified for various tasks, and its further applications are not limited to adversariallearning.
Actually, advanced works often take reprogramming as an effective transfer learningtechnique in cases with limited data and resources.
In the context of the image classification, [10]reprogram a model trained on ImageNet in solving vision-based counting tasks; and [48] furtherconsider reprogramming a black-box system for biomedical image classification, which suffersfrom label scarcity issue.
In the context of the natural language processing, well-trained models arereprogrammed for time-series classification [56] and sentiment analysis [14], where data scarcityissues frequently occur.
In this paper, we also employ the reprogramming property of deep models for transfer learning.
However, instead of reprogramming across different datasets, we reprogram our original classificationmodel for the task in OOD detection, considering the situation with the same (ID) dataset before/afterreprogramming.
Further, to preserve the benefits of previous classification-based detection methods,
[Table]
[Table]
[Table]
we adopt the perturbing pattern (i.e., the watermark) on the same shape as the original inputs, insteadof reshaping original inputs and adding padding features as previous works [10, 56].
A.3 Backdoor Attack
The model reprogramming is related to the backdoor attack , which also change models’ behaviourduring the test.
Overall, when a backdoor is embedded during training and the trigger is activatedduring the test, model predictions will be modified to the attacker-specified labels deliberately [13].
Nowadays, the data poisoning [13, 29, 33, 42] is among the best to realize the backdoor attackregarding deep models—a portion of the training sample is modified with the attacker-specifiedpattern (i.e., pre-defined trigger) and the attacker-specified labels.
The target models are trained onthese poisoned data, and the resultant models will suffer from the backdoor attack when the trigger isactivated.
Please refer to [30] for a comprehensive survey.
However, the backdoor attack and the model reprogramming exploit different aspects of deep models.
In general, the backdoor attack utilises the excessive learning ability in memorizing noise features [30],while the reprogramming property states that the well-trained models can be reprogrammed for newtasks without modifying the original models.
B The Overall Algorithm
The overall learning framework is summarized in Algorithm 1, optimizing in a stochastic mannerwith num_step iterations.
The watermark is initialized by the Gaussian noise with the 0 mean anda small standard deviation σ 2 I d (Step 2 ), and the learning procedure consists of three stages foreach update: (1) a set of Gaussian noise data is sampled, assuming be of the size m as that of themini-batch regarding the ID sample (Step 4 ); (2) the risk for ID and OOD cases are computed and theoverall risk is given by their sum with a trade-off parameter β
(Step 5 ); (3) the first-order gradientguides the pixel-level update of the watermark, using the signum of gradients and the SAM to make areliable update (Step 7 ).
After watermark training, the learned watermark is added to test-time inputsfor OOD detection and the detection model with the pre-defined scoring function is deployed.
C Further Experiments
This section conducts further experiments about our proposal.
C.1 Impact on Test Accuracy
To begin with, we study the impact of watermarking on the classification accuracy in Table 9,comparing with the results without watermarking.
As we can see, watermarking has a negative impacton the test accuracy, dropping from 94 .
84% to 91 .
85% and 93 .
49% on CIFAR- 10 ; and from 75 .
[Table]
[Table]
to 72 .
03% and 74 .
08% on CIFAR- 100 .
Further, after watermarking, the classification accuracywith the free energy scoring is much better than that of the softmax scoring, with only 1 . 35% to1 .
88% decrease in classification accuracy.
Therefore, we suggest using the free energy scoring inwatermarking as a default setup, which leads to better detection capability and largely preserves theoriginal capability in classification.
C.2 Other Scoring Strategies with Watermarking
Note that watermarking is orthogonal to much of the existing methods and the watermarking strategycan boost many other advanced OOD detection methods.
To further verify the generality andthe effectiveness of our proposal, we utilize watermarking for three representative OOD detectionmethods, namely, OE [19], MaxLogit [17], ODIN [31], and ReAct [45].
For OE, Hendrycks et al.
[19] state that the target model can benefit from fine-tuning with extra OODtraining data, in the context of the softmax scoring.
In general, OE requires to re-train the targetmodel, which will be prohibitively expensive for many real-world applications.
However, since themodel has seen some kinds of OOD data during training, it typically reveals superior results thanmany other advanced detection methods.
We are interested in whether our watermarking can improve the detection capability of the modelsthat have been fine-tuned with OE.
Typically, we assume that the training-time OOD data aredifferent from that of the test time.
Therefore, following previous works [19, 32], we adopt thetiny-ImageNet [27] as the training-time OOD data for model fine-tuning.
The learning objectivesregarding the model parameters in OE is similar to Eq.
(9) and we follow the default hyper-parametersetups as in [19].
We learn the watermarks for the fixed OE-trained models with the softmax scoring,and the results on CIFAR benchmarks are summarized in Table 10.
Overall, the experimental resultssuggest that our watermarking can still benefit OE in effective OOD detection, which may requireour in-depth exploration in the future.
MaxLogit, ODIN, and ReAct can be viewed as the improved versions of the softmax scoring in
Eq.
Specifically, MaxLogit takes the maximal logit outputs in OOD scoring, which is betterthan softmax scoring when facing large-class setting; ODIN clamps embedding features from thesecond-last layer of model outputs, which can attenuate the overconfidence issue caused by theout-sized activation of abnormal hidden units; and ReAct observes that temperature scaling andadversarial feature perturbation can improve model capability in discerning OOD data from ID cases.
For MaxLogit, we directly use the learning objective of the softmax scoring-based watermarking,
[Equation]
[Table]
[Table]
which directly use the logit outputs (instead of softmax outputs) in discerning ID and OOD data.
The scoring function for ODIN is
[Equation]
where ˜ x = x − ξ sign ( −∇ x log softmax y f ( x )) is the perturbed data point and ξ is the perturbationmagnitude.
For the watermark training, the learning objectives are of the form:
[Equation]
following the same definition of ˜ x as in Eq.
For ReAct, we assume the feature extractor defined by the second-last of model outputs by f FEA andthe above classifier by f CLA , i.e., f ( x ) =
f CLA ( f FEA ( x )) .
Then, the rectified model output is
[Equation]
(14) truncating values of hidden units from the second-last layer that are above τ .
The correspondinglearning objectives in watermark training can be written asℓ IDSM ( x , y ; f )
=
− log softmax y f ReAct ( x ) and ℓ OODSM ( x
[Equation]
f ) = − (cid:88)
[Equation]
(15) which is similar to that of the softmax scoring in Eq.
Tables 11 to 13 summarize the experimental results on CIFAR- 10 and CIFAR- 100 datasets, where Tis fixed to 1000 , ξ is set to 0 .
0014 , and τ is chosen such that 10% of the activation values are clamped,following the setups of the original papers.
The performance improvements are illustrious regardingthe performance of MaxLogit, ReAct, and ODIN with and without watermarking, largely confirmingthe fact that our proposed watermarking is orthogonal to existing works.
Further, since we directlyuse the training strategy of softmax scoring-based watermarking for MaxLogit, our results in Table 11demonstrate
that watermarking can also benefit from improved choices of scoring strategies.
C.3 Other Learning Strategies with Watermarking
Also, we consider the situation where a set of OOD data are available for watermark training, wherewe adopt the tiny-ImageNet [27] dataset as training-time OOD dataset.
Here, we replace the Gaussiannoise in the OOD learning objective ℓ OOD to be the randomly selected sample from the tiny-ImageNetdataset.
Then, the learning objective with training-time OOD data is of the form
[Equation]
n ℓ ID ( x i + w , y i ; f )
+ β
[Equation]
n ℓ OOD ( o j + w ; f ) , (16)
[Table]
[Table]
[Table]
[Table]
where o j denotes the randomly selected sample from the tiny-ImageNet dataset.
{+The experimental results on CIFAR- 
+}
and CIFAR- 100 datasets are summarized in Table 14 and Table 15.
Unfortunately,for our current realization, we observe that watermark training with extra OOD data fails to induce alarge performance improvement in OOD detection.
Even worse, in some cases, learning with extra
OOD data can impair the power of the resultant watermarks.
We conjecture that the inductive biasintroduced by the training-time OOD data may deviate from the test-time cases, severely misleadingthe resultant watermarks in showing results even lower than that of the simple Gaussian noise.
We also list the detection performance with “perm” and “rotate” in Section 6, demonstrating theeffectiveness of the resultant watermarks on far OOD cases.
Here, the training objective is:
[Equation]
where ˜ x is an augmentation of the original x with either “perm” or “rotate”.
The results aresummarized from Tables 16 to 19.
As we can see, the results with "perm" and "rotate" is comparablewith (even better than) the original learning setup with only the Gaussian noise (common).
[Table]
[Table]
[Table]
C.4 Comparison with State-of-the-art Methods
For the concreteness of our discussion, this section compares our proposal with state-of-the-artmethods in OOD detection.
In particular, we compare with softmax scoring (Softmax) [18], freeenergy scoring (Energy) [32], ReAct [45], ODIN [31], Mahalanobis [28], GradNorm [22], and
OE [19].
The experimental results on CIFAR benchmarks are summarized in Table 20.
Theaverage performance on iSUN, Places 365 , Texture, SVHN, and LSUN is reported.
As we can see,watermarking can boost the performance of various scoring methods in OOD detection, achieving thebest detection performance compared with all other advanced methods.
C.5 Experiments with Mean and Standard Deviation
This section further verifies the results from Table 1 to Table 4 with five individual trails (randomseeds).
In Table 21, Table 22, and Table 23, we summarize the average results and the standarddeviation for the softmax scoring and the free energy scoring, respectively.
In Figure 5, we also depictthe learned watermarks for each trial on CIFAR- 10 and CIFAR- 100 .
Overall, we observe that thelearned watermarks preserve some similar pattern (e.g., the shape of areas with large values) giventhe same dataset and the same scoring strategy, and the improvement of watermarking is stable acrossdifferent datasets and scoring methods.
C.6 Experiments with different models
We demonstrate the effectiveness of our watermarking across various model architectures, including
ResNet-18 [15], WRN-40-2, and ViT-B/16 [9].
We conduct experiments on the ImageNet benchmarkand summarized the results in Table 24.
As we can see, in both the softmax and free energy scoringcases, our watermarking can improve the detection performance across various models.
However, the
[Table]
[Table]
improvements after watermarking on the large-scale models (i.e., ViT-B/16) are not as remarkableas that of the small models (e.g., ResNet-18).
It is because that the large-scale models themselvescan already excel at OOD detection (better results without watermarking than that of ResNet-18 and
WRN-40-2), so there does not remain a large space for their further improvements.
C.7 Hyper-parameter Setups
For the hyper-parameter setups in our experiments, we use random search to choose the proper σ 1from the candidate parameter set { 0 . 0 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 . 0 , 1 . 2 , 1 . 4 , 1 . 6 , 1 . 8 , 2 . 0 } , and the properρ from { 0 . 0 , 0 . 02 , 0 . 05 , 0 . 07 , 0 . 1 , 0 . 2 , 0 . 5 , 0 .
For softmax scoring, β is chosen from{ 0 . 0 , 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 , 2 . 5 , 3 . 0 , 3 . 5 , 4 . 0 , 4 . 5 , 5 . 0 } ; and for free energy scoring, β is chosen from{ 0 . 0 , 0 . 02 , 0 . 04 , 0 . 06 , 0 .
The model performance is tested on a set ofvalidation OOD datasets that are separated from the original iSUN, Places 365 , Texture, SVHN, and
LSUN datasets.
Accordingly, for softmax scoring, we set σ 1 = 0 .
4 , ρ = 1 . 0 , β = 3 . 5 in CIFAR- 10and
σ 1 = 1 . 0 , ρ = 0 . 2 , β = 2 . 5 in CIFAR- 100 ; for free energy scoring, we set σ 1 = 0 .
6 , ρ = 0 .
7 ,β = 0 .
1 and σ 1 = 1 . 0 , ρ = 0 .
05 , β = 1 .
We adopt the random search with many trials by thefollowing three steps.
Step 1: we randomly select a hyperparameter (e.g., β ) and fix the values ofall other hyperparameters to be their current optimal values.
Step 2: we choose the best β from thecandidate set.
Step 3: do Steps 1-2 again.
We repeat Steps
[Table]
[Table]
Further
[Table]
from Table 25 to
[Table]
we list the performance of watermarking with different hyper-parametersettings for reference, where we fix the values of all other hyper-parameters (except for the consideredone) to be their optimal values.
Finally, we list the results on CIFAR benchmarks regarding the freeenergy scoring with different values of T in Table 37.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Figure]
D
en s i t y
D
en s i t y
Free Energy Score
[Table]
iSUN
Places
Texture SVHN
LSUN-C
LSUN-R 𝜎 𝜎
D
en s i t y (b)
w/ watermarking
Free Energy Score
D
en s i t y (a) w/o watermarking (a)
D
en s i t y
D
en s i t y
[Table]
Free Energy Score iSUN
Places
Texture SVHN
LSUN-C
LSUN-R 𝜎
D
en s i t y (b)
w/ watermarking
Free Energy Score
D
en s i t y (a)
w/o watermarking
[Figure]
C.8 Different Areas in the Watermark
[Table]
After watermarking, the edge area of the image is overwhelmed by the watermark’s pattern,while the centre part is not much affected.
However, it does not mean that the centre area is notimportant.
Instead, under the premise of maintaining the original features, the centre area alsoencodes useful information in OOD detection.
Table 38 is a verification of this conclusion on
CIFAR- 10 dataset with the free energy scoring,where we employ various masks in only preserving the watermark’s features with their absolute values larger (smaller) than a threshold χ 1( χ 2 ).
As we can see, even if only a small portionof the watermark is masked (e.g., χ 1 = 0 .
10or χ 2 = 10 . 0 ), there is a large drop in performance, even lower than the results without anywatermarking.
It indicates that both areas of thewatermark contribute, and the overall watermarkworks as a whole for effective OOD detection.