Watermarking for Out-of-distribution Detection
Abstract
Out-of-distribution (OOD) detection aims to identify OOD data based on representations extracted from well-trained deep models.
However, existing methods largely ignore the reprogramming property of deep models and thus may not fully unleash their intrinsic strength: without modifying parameters of a well-trained deep model, we can reprogram this model for a new purpose via data-level manipulation (e.g., adding a specific feature perturbation to the data).
This property motivates us to reprogram a classification model to excel at OOD detection (a new task), and thus we propose a general methodology named watermarking in this paper.
Specifically, we learn a unified pattern that is superimposed onto features of original data, and the model’s detection capability is largely boosted after watermarking.
Extensive experiments verify the effectiveness of watermarking, demonstrating the significance of the reprogramming property of deep models in OOD detection.
The code is publicly available at: github.com/qizhouwang/watermarking.
Introduction
Deep learning systems in an open world often encounter out-of-distribution (OOD) inputs whose label spaces are disjoint with that of training data, known as in-distribution (ID) data.
For safetycritical applications, deep models should make reliable predictions for ID data, meanwhile detecting
OOD data and avoiding making predictions for the detected ones.
This leads to the OOD detection task [28, 38, 41, 50], which has attracted intensive attention in the real world.
Identifying OOD data remains non-trivial since deep models can be overconfident with them [40].
As a promising technique, the classification-based OOD detection [58] relies on various scoring functions derived by classification models well trained with ID data (i.e., well-trained models), taking those inputs with small scores as OOD cases.
In general, the scoring functions can be defined by logit outputs [17, 33], gradients [21], and embedding features [28, 44].
Without interfering with the well-trained models or requiring extra computation, they exploit the inherent capability of models learned from only ID data.
In general, these advantages can be critical in reality, where the cost of re-training is prohibitively high and the acquisition of true OOD data is very difficult [58].
Although promising progress has been achieved, previous works largely ignore the reprogramming property [9] of deep models: a well-trained model can be repurposed for a new task by a proper transformation of original inputs (e.g., a universal feature perturbation), without modifying any model parameter.
For example, a model pre-trained on ImageNet [6] dataset can be reprogrammed for classifying biomedical images [49].
This property indicates the possibility of making a well-trained model adapt for effective OOD detection, motivating us to make the first attempt to investigate if the reprogramming property of deep models can help to address OOD detection, i.e., can we reprogram well-trained deep models for OOD detection (a new task)?
In this paper, we propose a novel method, watermarking , to reprogram a well-trained model by adding a watermark to original inputs, making the model can help detect OOD data well.
The watermark has the same shape with original inputs, which is a static pattern that can be added for test-time inputs (cf., Figure 1).
The pre-defined scoring strategy (e.g., the free energy scoring [33]) is expected to be enhanced, with an enlarged gap of OOD scores between the watermarked ID and OOD data (cf., Figure 2).
It is non-trivial to find the proper watermark due to our lack of knowledge about unseen OOD data in advance.
To address the issue, we propose a learning framework for effective watermarking.
The insight is to make a well-trained model produce high scores for watermarked ID inputs meanwhile regularize the watermark such that the model will return low confidence without perceiving ID pattern.
In this case, the model will have a relatively high score for a watermarked ID input, while the score remains low for OOD data (cf., Figure 2).
The reason is that the model encounters a watermarked input but not seeing any ID pattern.
In our realization, we adopt several representative scoring strategies, devising specified learning objectives and proposing a reliable optimization algorithm to learn an effective watermark.
To understand our watermarking, Figure 1 depicts the watermark learned on CIFAR- 10 [26] dataset, with the free energy scoring [33].
As we can see, the centre area of the learned watermark largely preserves the original input pattern, containing the semantic message that guides the detection primitively.
By contrast, the edge area of the original input is superimposed by the specific pattern of the watermark, which may encode the knowledge once hidden by the model in boosting OOD detection.
Overall, watermarking can preserve the meaningful pattern of original inputs in detection, with the improved detection capability that is learned from the trained model and ID data.
Figure 2 demonstrates the effect of our learned watermark, which is an example with the free energy scoring.
After watermarking, the scoring distributions are much concentrated, and the gap between
ID (i.e., CIFAR- 10 ) and OOD (i.e., SVHN [39] and Texture [4] datasets) data is enlarged notably.
We conduct extensive experiments for a wide range of OOD evaluation benchmarks , and the results verify the effectiveness of our proposal.
The success of watermarking takes roots in the following aspects: (1) a well-trained model on classification has the potential to be reprogrammed for OOD detection since they are two related tasks; (2) reprogramming has been widely studied, ranging from image classification to time series analysis [6, 49], making our proposal general across various domains; and (3) OOD detection suffers from the lack of knowledge about the real-world OOD distributions.
Fortunately, with only data-level manipulation in low dimensions, watermarking can largely mitigate this issue of limited data.
Overall, this data-level manipulation is orthogonal to existing methods, and thus provides a new road in OOD detection and can inspire more ways to design OOD detection methods in the future.
Related Works
To begin with, we briefly review the related works in OOD detection and model reprogramming.
Please refer to Appendix A for the detailed discussion.
D
en s i t y
D
en s i t y
Free Energy Score
Free Energy Score iSUN
Places
Texture SVHN
LSUN-C
LSUN
D
en s i t y (b)
w/ watermarking
D
en s i t y (a) w/o watermarking (a) before watermarking
D
en s i t y
D
en s i t y
Free Energy Score
Free Energy Score iSUN
Places365 Texture SVHN LSUN-C LSUN 𝜎 𝜎 𝜎 Free Energy Score
D
en s i t y (b)
w/ watermarking
D
en s i t y (a)
w/o watermarking
[Figure]
OOD Detection discerns ID and OOD data by their gaps regarding the specified metrics/scores, and existing methods can be roughly divided into three categories [58], the classification-based methods, the density-based methods, and the distance-based methods.
Specifically, the classificationbased methods [17, 21, 33, 44] use representations extracted from the well-trained models in OOD scoring; and the distance-based methods [2, 20, 60] measure the distance of inputs from class centers in the embedding space.
Moreover, the density-based methods estimate input density with probabilistic models [28, 41, 45], identifying those OOD data with small likelihood values.
Distancebased and density-based methods may suffer from complexity in computation [28] and difficulty in optimization [62].
Therefore, more researchers focus on developing classification-based methods and have made big progress on benchmark datasets recently [21, 33].
Model Reprogramming repurposes well-trained models for new tasks with only data-level manipulation [9], indicating that deep models are competent for different jobs without changing any model parameter.
In previous works, the data-level manipulation typically refers to a static padding pattern (different from our proposal) learned for the target task, which is added to the test-time data.
The effectiveness of the model reprogramming is verified across image classification [9, 49] and time-series analysis [13, 57].
In this paper, we use the reprogramming property of deep models for effective OOD detection, which has been overseen previously.
Preliminary
Let X ⊂ R d be the input space and Y = { 1 , . .
, c } be the label space.
We consider the ID distribution
D ID X , Y defined over X × Y , the training sample S n = { ( x i , y i ) } ni =1 of size n independently drawn from D ID X , Y , and a classification model f : X → R c (with logit outputs) well-trained on S n .
Based on the model f ( · ) , the goal of the classification-based OOD detection is to design a detection model g : X → { 0 , 1 } that can distinguish test-time inputs with the ID distribution D ID X from those with the OOD distribution D OOD X .
In general, D OOD X is defined as an irrelevant distribution of which the label set has no intersection with Y , and thus should not be predicted by f ( · ) .
Overall, withdenoting the OOD case and 1 the ID case, the detection model g ( · ) is defined as
[Equation]
where τ ∈ R is a threshold and s : X → R is the scoring function defined by f ( · ) whose parameters are fixed.
Here, we focus on two representative methods in the classification-based OOD detection, namely, the softmax scoring and the free energy scoring .
Softmax Scoring Function [17] uses the maximum softmax prediction in OOD detection, of which the scoring function s SM ( · ) is given by
[Equation]
where softmax k ( · ) denotes the k -th element of the softmax outputs.
In general, with a large (small) s SM ( x ; f ) , the detection model will take the input x as an ID (OOD) case.
Free Energy Scoring Function [33] adopts the free energy function for scoring, defined by the logit outputs with the logsumexp operation, namely,
[Equation]
where T > 0 is the temperature parameter, fixed to 1 [33].
It aligns with the density of inputs to some extent, and thus is less susceptible to the overconfidence issue than the softmax scoring [33].
Watermarking Strategy
This section introduces the key concepts of watermarking for classification-based OOD detection.
Definition.
A watermark w ∈ R d is a unified pattern with the exact shape as original inputs.
It is added to test-time inputs statically, and we refer w + x a watermarked input for ∀ x ∈ X .
In expectation, regarding the specified scoring function s ( · ) , our watermarking should make the model excel at OOD detection for watermarked data.
Learning Strategy.
Given the scoring function s ( · ) , it is challenging to devise the exact watermark pattern by predefined rules.
Therefore, for the proper watermarks in OOD detection, we need to devise learning objectives with respect to watermarks, which consider both ID and OOD data.
We generally have no information about the OOD distribution D OOD X , Y , while we still want the model excels in discerning ID and OOD data from scoring.
For this challenge, we make the model produce high scores if watermarked ID data are observed; meanwhile, we regularize the watermark such that the model will return low scores when ID patterns do not exist.
From the lens of our model, the scores should remain low if a watermarked OOD input is given since the watermark is not trained to perceive OOD data, of which the patterns are very different from the ID data.
Benefits of Watermarking.
Watermarking directly reprograms the model to make an adaptation to our specified task of scoring, such that the detection capability of the original model is largely improved.
By contrast, previous methods typically adapt to their specified tasks by only the threshold τ as in Eq.
However, it requires the trade-off between false positive (ID) and false negative (OOD) rates when densities of scoring are non-separable (cf., Figure 2(a)).
Further, watermarking enjoys the benefits of previous classification-based methods in that we do not modify the original training procedure in classification, making our proposal easy to be deployed in real-world systems.
Although the watermark also should be learned, the parameter space is in low dimension, and the learning procedure could be conducted post-hoc after the systems are deployed.
Comparison with Existing Works .
In OOD detection, this paper is a first attempt in using the reprogramming property of deep models, leading to an effective learning framework named watermarking.
At first glance, our methodology is seemingly similar to ODIN [32], which also conducts data-level perturbation for OOD detection.
However, their instance-specified perturbation relies on extra backward-forward iterations during the test, which is not required in our method.
Further, ODIN is designed for the softmax scoring, but our proposal is much general in OOD detection.
Realizations of Watermarking Strategy
In this section, we discuss our learning framework of watermarking in detail.
Learning Objectives.
As mentioned above, we need to consider the ID and OOD situations separately, with the associated loss functions denoted by ℓ ID ( · ) and ℓ OOD ( · ) .
For the ID case, the ID training data are required, where we make the high scores for their watermarked counterparts.
By contrast, since we typically lack knowledge about the test-time OOD data, only the watermark is used here, and we expect the model to produce the score as low as possible when only perceiving the watermark.
Further, since only the watermark is adopted for training in the OOD case, the learned watermark is pretty sensitive regarding the detection model, i.e., the model may return different predictions when facing small perturbations.
Thus, the watermarked OOD inputs may not guarantee the low scores.
To this end, the watermark is further perturbed during training.
Here we adopt the Gaussian noise, leading to the perturbed watermark of the form ϵ + w with ϵ ∼ N ( 0 , σ 1 I d ) the independent and identically distributed (i.i.d.)
Gaussian noise of d -dimension (the mean 0 and the standard deviation σ 1 I d ).
Then, the overall risk can be written as,
[Equation]
with β ≥
the trade-off parameter, L ID n ( w ) the risk for ID data, and L OOD n ( w ) the risk for OOD data.
Optimization.
To find the proper watermark, we use the first-order gradient update to iteratively update watermark’s elements.
However, data-level optimization remains difficult in deep learning, of which the results may get stuck at suboptimal points [52].
A common approach is to use the signum of first-order gradients, guiding the updating rule of the current watermark via
[Equation]
where sign ( · ) denotes the signum function and α > 0 is the step size [35].
Further, for generality and insensibility, we prefer the solution that lies in the neighbourhood having uniformly low loss, i.e., with a smooth loss landscape [24].
Therefore, we adopt the sharpness-aware minimization (SAM) [10], an effective optimization framework in the seek of both the low loss value and the smooth loss landscape.
Specifically, given the original risk L n ( w ) , the SAM problem is:
[Equation]
where ρ ≥ 0 is a constraint.
For efficiency, the SAM makes the first-order Taylor expansion w.r.t.
κ around 0 , obtaining the approximated solution of the form 3 :
[Equation]
where 1 /p + 1 /q = 1 and we set p = q = 2 for simplicity.
Therefore, the estimation form of the SAM is written as L n ( w + κ ) , with corresponding updating rule of
[Equation]
yielding an efficient optimization algorithm that induces the effective watermark.
The Overall Algorithm.
In summary, we describe the overall learning framework.
To begin with, the watermark is initialized by the i.i.d.
Gaussian noise with the 0 mean and a small standard deviation σ 2 I d , and the learning procedure consists of three stages for each updating step:
• Negative sampling: a set of noise data ϵ is sampled, assuming be of the size m as that of the mini-batch regarding the ID sample; • Risk calculating: the risk for ID and OOD data are computed, and the overall risk is given by their sum with a trade-off parameter β as in Eq.
(4); • Watermark updating: the first-order gradient guides the pixel-level update of the watermark
, using the signum of gradients and the SAM to make a reliable update as in Eq.
The learned watermark is added to test-time inputs for OOD detection, and the detection model with the pre-defined scoring function is then deployed.
Appendix B summarizes our learning framework of watermarking.
Moreover, two specifications of watermarking are discussed in the following
Two Realizations.
Here, we focus on two representative methods in OOD detection, namely, the softmax scoring and the free energy scoring.
For other representative methods in OOD detection, please refer to Appendix C for their descriptions and the experiments.
Softmax Scoring-based Watermarking.
Following [18], we set ℓ IDSM ( · ) to be the cross entropy loss and ℓ OODSM ( · ) to be the cross entropy regarding the uniform distribution, namely, [Equation] log softmax [Equation]
f
[Equation]
x
[Equation]
and [Equation] OODSM ( x [Equation]
− (cid:88)
[Equation]
specifying the learning objectives in Eq.
(4) for the softmax scoring-based watermarking.
Free-Energy Scoring-based Watermarking.
[33] use a set of learning objectives for model re-training with free energy scoring.
However, their logsumexp operation originating from the free energy function is difficult for optimization, posing notorious computing issues [36].
To this end, we drop the log operation and make the overall risk always positive by the following learning objectives:
[Equation]
realizing the learning objectives in Eq.
(4) for the free energy scoring.
Experiments
In this section, we conduct extensive experiments for watermarking in OOD detection.
Specifically, we demonstrate the effectiveness of our method on a wide range of OOD evaluation benchmarks; we conduct experiments for the important hyper-parameters in our learning framework; and we provide further experiments for an improved interpretation of our proposal.
Baselines results are achieved by our re-run of the publicly available codes.
The source code of our proposal is released at github.com/qizhouwang/watermarking.
All the methods are realized by Pytorch 1 . 81 with CUDA 11 . 1 , where we use several machines equipped with GeForce RTX
GPUs and AMD Ryzen Threadripper 3960 X Processors.
ID and OOD Datasets .
We use CIFAR- 10 , CIFAR- 100 [26], and ImageNet [42] datasets as three ID datasets, with data pre-processing including horizontal flip and normalization.
Furthermore, for the
OOD datasets, we adopt several commonly-used benchmarks, including Textures [4], SVHN [39],
Places 365 [63], LSUN [59], and iSUN [56].
Referring to Appendix C.7 for hyper-parameter settings.
Evaluation Metrics .
The performance in OOD detection is measured via three representative metrics, which are all threshold-independent [5]: (1) the false positive rate of OOD sample when true positive rate of ID data is at 95% (FPR 95 ); (2) the area under the receiver operating characteristic curve (AUROC), interpreted as the probability that an ID input has a greater score than an OOD one; and (3) the area under the precision-recall curve (AUPR), which further adjusts for different base rates.
Configuration .
Following previous works [33], we employ WideResNet [61] (WRN-40-2) as the backbone model.
For the CIFAR benchmarks, the models are trained for 200 epochs via the stochastic gradient descent, with the batch size 64 , the momentum 0 . 9 , and the initial learning rate 0 . 1 .
The learning rate is divided by 10 after 100 and 150 epochs.
For the ImageNet Benchmark, the model is trained for 120 epochs via the stochastic gradient descent, with the batch size 32 , the momentum 0 .and
the initial learning rate 0 .
The learning rate is divided by 10 after 60 and 90 epochs.
CIFAR Benchmarks .
We depict the learned watermarks in Figure 3.
As we can see, the centre areas maintain the pattern of original inputs, which are helpful in OOD detection primitively.
By contrast, the edge areas of the watermarks distort the original features, superimposed with the pattern that may further boost the capability of the original models in OOD detection.
Then, we demonstrate the improvement of watermarking on CIFAR- 10 and CIFAR- 100 datasets.
For the results of the softmax scoring in Table 1, our watermarking reduces the average FPR 95 by 2 .
99 ∼ 12 . 84 , boosts the average AUROC by 3 .
09 ∼ 3 .
69 and the average AUPR by 1 .
10 ∼ 1 . 18 .
Moreover, for each of the considered test-time OOD dataset, there also exist stable improvements after watermarking, except for the comparable results regarding Places 365 on CIFAR- 100 dataset.
For the results of the free energy scoring in Table 2, the improvement via watermarking is also substantial, with 4 .
54 ∼ 13 . 98 , 3 .
43 ∼ 4 . 65 , and 1 . 13 ∼ 1 .
37 better performance regarding the
[Figure] [Table] [Table] average FPR 95 , AUROC
[Table] and AUPR. Overall, Table 1 and Table 2 not only justify the effectiveness of watermarking and also demonstrate the generality of the proposed watermarking.
Further, comparing between CIFAR- 10 and CIFAR- 100 datasets, the improvements of watermarking on CIFAR- 10 is much greater than that of CIFAR- 100 , aligning with the previous observations [22] that a large semantic space can exaggerate the challenge in effective OOD detection.
Figure 4 illustrates the scoring distributions before (a) and after (b) watermarking on CIFAR- 10 , where we take the free energy scoring as an example.
Due to the space limit, we only consider two test-time OOD datasets, namely, Texture and SVHN.
As we can see, after watermarking, the distribution gap between the ID (i.e., CIFAR- 10 ) and the OOD (i.e., Texture and SVHN) data is enlarged, and thus the detection capability is improved.
ImageNet Benchmark.
Huang and Li [22] show that many advanced methods developed on the
CIFAR benchmarks can hardly work for the ImageNet dataset due to its large semantic space with 1 k classes.
In order to verify the power of watermarking in large semantic space, we conduct experiments with ImageNet being the ID dataset, and the results regarding the softmax scoring and the free energy scoring are summarized in Table 3 and Table 4.
For the softmax scoring, we decrease the average
FPR 95 from 54 .
93 to 40 .
50 after watermarking; for the free energy scoring, we reduce the average
FPR 95 from 52 .
73 to 43 . 23 .
It demonstrates that our watermarking still works well in the case of the large semantic space, with considerable improvements in OOD detection power. D en s i t y D
en s i t y
Free Energy Score
Free Energy Score Free Energy Score Free Energy Score D
en s i t y
D
en s i t
y iSUN
Places
Texture
SVHN
LSUN-C
LSUN-R 𝜎 𝜎 𝜎 (a) w/o watermarking
D
en s i t y
D
en s i t y
Free Energy Score
Free Energy Score
Free Energy Score
Free Energy Score
D
en s i t y
D
en s i t y iSUN
Places365 Texture
SVHN LSUN-C LSUN-R 𝜎 𝜎 𝜎 (b) w/ watermarking
[Figure]
[Table]
[Table]
[Table]
Near OOD Detection.
The above experiments focus on the far OOD detection setups where ID and OOD data are different regarding semantics and styles. Now
[Table] we further demonstrate the power of our watermarking strategy in a near OOD situation [54], covering a challenging situation where ID and OOD data have similar styles (i [Table] e., near OOD data).
Except for the common learning setup in Section 5, we further consider the use of shifting augmentations [47], which are data augmentations that are harmful to the standard contrastive learning but can be used to construct near OOD data similar to ID data.
We consider two representative shifting augmentations: “permute” (permute evenly partitioned data) and “rotate” (rotate 90 degrees of original data).
The shifting-augmented
ID data are taken as OOD data fed into ℓ OOD ( · ) along with random Gaussian noise.
The near OOD experiments are summarized in Table 5, where we take CIFAR- 10 as ID data and CIFAR- 100 as OOD data.
Here, the common learning setup (common) already leads to improved performance compared to the cases without watermarking (w/o watermark).
Moreover, watermarking with shifting augmentations (permute and rotate) can further boost the detection power of the models, leading to at most 8 . 60 and 4 .
70 improvements in FPR95 for the softmax and the free energy scoring.
Effect of Hyper-parameters.
To further interpret our learning framework, we compare the performance of the learned watermarks regarding various setups of hyper-parameters, focusing on the standard deviation σ 1 in the Gaussian noise and the perturbation constraint ρ in the SAM, which are both critical.
As a case study, we conduct experiments regarding the softmax scoring on CIFAR- 10 .
For detailed results about the ablation study, please refer to Appendix C.
Table 6 lists the results with various values of σ 1 , ranging from 0 .
to 2 . 00 .
Note that σ 1 is the standard deviation of the Gaussian noise added to the watermark, with different values indicating [Table] [Table] various degrees of the perturbation.
As we can see, a mild perturbation of the watermark (e.g., σ 1 = 0 . 40 ) can truly lead to improved results in detection, with 10 . 05 , 1 . 60 , and 0 .
34 improvements regarding the average FPR 95 , AUROC, and AUPR.
It indicates that the learned watermark is sensitive when facing small perturbations, if the Gaussian noise is not applied during training.
However, some extreme values (e.g., σ 1 = 2 . 00 ) may overwhelm the watermark pattern and thus be detrimental.
In Table 7, we summarize the experimental results given by various values of ρ for the SAM.
Overall, a large value of ρ indicates that a wide range in the solution’s neighbours should be smooth, and thus the stability of the result is expected to be improved.
However, such a solution may consume too much capacity of the watermark, misleading the learning procedure to some unsatisfactory results.
On the other side, our watermarking truly benefits from the SAM with a mild choice of the hyper-parameter.
Specifically, comparing with the results without the SAM (i.e., ρ = 0 . 00 ), the detection capability of the watermark with a suitable ρ (i.e., ρ = 1 . 0 ) is largely improved, with 3 . 92 , 0 . 52 , and 0 .
10 better results regarding the average FPR 95 , AUROC, and AUPR.
[Table]
Transferability of watermarking .
Further, we explore the transferability of the watermarking strategy learned with different scoring functions, e.g., we study the effect of the watermark learned with the softmax scoring when it is deployed with the free energy scoring.
The results are summarized in Table 8.
Note that, a “learn” with FE and “score” with
SM indicates that the watermark is learned with the free energy scoring and tested regarding the softmax scoring.
As we can see, the watermarks learned with the free energy scoring can be reused for the softmax scoring (even with better results), while the reverse leads a deterioration.
It indicates that our learning objectives in Eq.
(10) are general and effective for both the softmax scoring and the free energy scoring.
However, we do not observe any transferability between different datasets.
For example, for the softmax scoring, when the watermark learned on CIFARis adopted for CIFAR-100, there is a drop of performance with 51 .
93 (from 42 .
86 to 94 . 79 ), 35 .
(from 92 .
91 to 57 . 62 ) and 12 .
42 to 86 .
42 ) regarding the average FPR 95 , AUROC, and AUPR.
Further studies may be required here.
Conclusion
This paper demonstrates that the model’s inherent capability in OOD detection can be largely improved with only data-level manipulation, where we propose a general and effective methodology named watermarking.
Overall, we learn a discriminative pattern that could be superimposed onto original inputs, such that the OOD scores become much separable between ID and OOD data.
Our results indicate a promising direction in OOD detection that warrants our exploration in the future.
Further investigation should focus on the stability in optimization and general learning objectives regarding advanced OOD detection methods.
Acknowledgments and Disclosure of Funding
{+QZW, YGZ and BH were supported by the RGC Early Career Scheme No. 22200720, NSFC Young Scientists Fund No. 62006202, Guangdong Basic and Applied Basic Research Foundation No. 2022A1515011652, RGC Research Matching Grant Scheme No. RMGS2022_11_
 and No. RMGS2022_13_06, and HKBU CSD Departmental Incentive Grant.+}
CG was supported by NSF of China No. 61973162, NSF of Jiangsu Province No. BZ2021013, NSF for Distinguished Young Scholar of Jiangsu Province No.
BK and the Fundamental Research Funds for the Central Universities
No. 30920032202 and No. 30921013114.
TL was partially supported by Australian Research Council Projects DP180103424, DE-190101473, IC-190100031, DP-220102121, and FT-220100318. Checklist For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[No] We use only standard datasets (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[No] We use only standard datasets 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Additional Related Works
We review the recent studies in OOD detection, model reprogramming, and backdoor attack.
A.1 OOD Detection
Following [58], we attribute existing works into three categories, namely, the classification-based methods, the density-based methods, and the distance-based methods.
In general, these methods aim to maximize the gap between ID and OOD data regarding specified metrics in identifying OOD data.
The classification-based methods use the representations extracted from the well-trained classification models in OOD scoring.
For example, [17, 32, 33, 37, 46, 51] employ logit outputs from models in estimating the confidence of ID data; [28, 44] adopt Mahalanobis distance and Gram
Matrix to exploit models’ detection capability from embedding features; [21, 32] further demonstrate the importance of gradient information, either perturbing inputs with its gradients or directly using the gradient norm in scoring.
The classification-based methods are easy to be deployed without modifying the models [58], and thus it is the main focus in this paper.
The distance-based methods measure the distance regarding the embedding space, taking those data far away from the class prototypes as the OOD data.
Representative works adopt the Mahalanobis distance [21, 28], the cosine similarity [2, 60], and the Euclidean distance [20].
Our methods can also be used in the distance-based methods.
However, extra computation, such as calculating the precision matrix [28], may lead difficulty in devising proper learning objectives, out of the scope of our paper.
The density-based methods explicitly estimate the density of ID samples with various probabilistic models, identifying those OOD data based on the likelihood [28], the likelihood ratio [31, 41, 45], and the likelihood regret [55].
Typically, the input density is modelled by the mixture of Gaussian models [28] and the flow-based methods [25, 38].
Although the density-based methods can directly characterize the properties of ID density, these methods are difficult to be trained and may make overconfident predictions, as demonstrated in previous works [37, 38].
Recent works also focus on the causes of challenges in OOD detection, from the lens of the BatchNorm statistics [46], the density estimation [37], and the spurious correlation [44].
Improved methods, related to specified model architectures [7, 53], data perturbation [1, 3, 32], data augmentation [15, 19, 48], and outlier exposure [18, 23], are also well-studied.
However, these methods typically overlook the reprogramming property of deep models, which remain orthogonal to our proposal.
A.2 Reprogramming Property
The seminal work [9] introduces adversarial reprogramming as an attack method in adversarial learning [11], adversarially reprogramming the target model to perform a new task without changing the original model.
The term “attack” lies in the fact that, by reprogramming, an attacker can easily steal public machine learning services, abusing their computational resources for tasks that violate their original purposes.
Overall, [9] claim the reprogramming property of deep models—without modifying parameters of a well-trained model, we can reprogram it for a new purpose with only data-level manipulation.
The reprogramming property of deep learning is preliminarily verified for various tasks, and its further applications are not limited to adversarial learning.
Actually, advanced works often take reprogramming as an effective transfer learning technique in the cases with limited data and computing resources.
In the context of the image classification, [9] reprogram a model trained on ImageNet in solving vision-based counting tasks; and [49] further consider reprogramming a black-box system for biomedical image classification, which suffers from label scarcity issue.
In the context of the natural language processing, well-trained models are reprogrammed for time-series classification [57] and sentiment analysis [13], where data scarcity issues frequently occur.
In this paper, we also employ the reprogramming property of deep models for transfer learning.
However, instead of reprogramming across different datasets, we reprogram our original classification model for the task in OOD detection, considering the situation with the same (ID) dataset before/after reprogramming.
Further, to preserve the benefits of previous classification-based detection methods, we adopt the perturbing pattern (i.e., the watermark) on the same shape as the original inputs, instead of reshaping original inputs and adding padding features as previous works [9, 57].
[Table] [Table]
[Table]
A.3 Backdoor Attack
Model reprogramming is also related to the backdoor attack , which also change models’ behaviour during the test.
Overall, when a backdoor is embedded during training and the trigger is activated during the test, model predictions will be modified to the attacker-specified labels deliberately [12].
Nowadays, data poisoning [12, 30, 34, 43] is among the best to realize the backdoor attack for deep models—a portion of the training sample is modified with the attacker-specified pattern (i.e., pre-defined trigger) and the attacker-specified labels.
The target models are trained on these poisoned data, and the resultant models will suffer from the backdoor attack when the trigger is activated.
Please refer to [29] for a comprehensive survey.
However, the backdoor attack and the model reprogramming exploit different aspects of deep models.
In general, the backdoor attack utilises the excessive learning ability in memorizing noise features [29], while the reprogramming property states that the well-trained models can be reprogrammed for new tasks without modifying the original models.
B The Overall Algorithm
The overall learning framework is summarized in Algorithm 1, optimizing in a stochastic manner with num_step iterations.
The watermark is initialized by the Gaussian noise with the 0 mean and a small standard deviation σ 2 I d (Step 2 ), and the learning procedure consists of three stages for each update: (1) a set of Gaussian noise data is sampled, assuming be of the size m as that of the mini-batch regarding the ID sample (Step 4 ); (2) the risk for ID and OOD data are computed and the overall risk is given by their sum with a trade-off parameter β
(Step 5 ); (3) the first-order gradient guides the pixel-level update of the watermark, using the signum of gradients and the SAM to make a reliable update (Step 7 ).
After watermark training, the learned watermark is added to test-time inputs for OOD detection and the detection model with the pre-defined scoring function is deployed.
C Further Experiments
This section conducts further experiments about our proposal.
C.1 Impact on Test Accuracy
To begin with, we study the impact of watermarking on the classification accuracy in Table 9, comparing with the results without watermarking.
As we can see, watermarking has a negative impact on the test accuracy, dropping from 94 .
84% to 91 .
85% and 93 .
49% on CIFAR- 10 ; and from 75 .
96% to 72 .
03% and 74 .
08% on CIFAR- 100 .
Further, after watermarking, the classification accuracy with the free energy scoring is much better than that of the softmax scoring, with only 1 . 35% to [Table]
[Table]
1 . 88% decrease in classification accuracy.
Therefore, we suggest using the free energy scoring in watermarking as a default setup [Table] which leads to better detection capability and largely preserves the original capability in classification [Table]
C.2 Other Scoring Strategies with Watermarking
Note that watermarking is orthogonal to much of the existing methods and the watermarking strategy can boost many other advanced OOD detection methods.
To further verify the generality and the effectiveness of our proposal, we utilize watermarking for three representative OOD detection methods, namely, OE [18], MaxLogit [16], ODIN [32] [Table] and ReAct [46].
For OE [Table] Hendrycks et al.
[18] state that the target model can benefit from fine-tuning with extra OOD training data.
In general, OE requires to re-train the target model, which will be prohibitively expensive for many real-world applications.
However, since the model has seen some kinds of OOD data during training, it typically reveals superior results than many other advanced detection methods.
We are interested in whether our watermarking can improve the detection capability of the models that have been fine-tuned with OE.
Typically, we assume that the training-time OOD data are different from that of the test time.
Therefore, following previous works [18, 33], we adopt the tiny-ImageNet [27] as the training-time OOD data for model fine-tuning.
The learning objectives regarding the model parameters in OE is similar to Eq.
(9) and we follow the default hyper-parameter setups as in [18].
We learn the watermarks for the fixed OE-trained models with the softmax scoring, and the results on CIFAR benchmarks are summarized in Table 10.
Overall, the experimental results suggest that our watermarking can still benefit OE in effective OOD detection.
MaxLogit, ODIN, and ReAct can be viewed as the improved versions of the softmax scoring.
Specifically, MaxLogit takes the maximal logit outputs in OOD scoring, which is better than softmax scoring when facing large-class setting; ODIN clamps embedding features from the second-last layer of model outputs, which can attenuate the overconfidence issue caused by the out-sized activation of abnormal hidden units; and ReAct observes that temperature scaling and adversarial feature perturbation can improve model capability in discerning OOD data from ID data.
For MaxLogit, we directly use the learning objective of the softmax scoring-based watermarking, with the corresponding scoring function of the form:
[Equation]
which directly use the logit outputs (instead of softmax outputs) in discerning ID and OOD data.
Moreover, for ODIN, the associated scoring function is given by
[Equation]
[Table] [Table] where ˜ x = x − ξ sign ( −∇ x log softmax y f ( x )) is the perturbed data point and ξ is the perturbation magnitude.
For the watermark training, the learning objectives are of the form: ℓ IDSM ( x , y ; f )
= − log softmax y f
( ˜ x ) and ℓ OODSM ( x ; f ) = − (cid:88)
[Equation]
following the same definition of ˜ x as in Eq.
For ReAct, we assume the feature extractor defined by the second-last of model outputs by f FEA and the above classifier by f CLA , i.e., f ( x ) =
f CLA ( f FEA ( x )) .
Then, the rectified model output is
[Equation]
truncating values of hidden units from the second-last layer that are above τ .
The corresponding learning objectives in watermark training can be written as
[Equation]
which is similar to that of the softmax scoring in Eq.
[Equation]
Tables 11 to 13 summarize the experimental results on CIFAR- 10 and CIFAR- 100 datasets, where T is fixed to 1000 , ξ is set to 0 .
0014 , and τ is chosen such that 10% of the activation values are clamped, following the setups of the original papers.
The performance improvements are illustrious regarding the performance of MaxLogit, ReAct, and ODIN with and without watermarking, largely confirming the fact that our proposed watermarking is orthogonal to existing works.
Further, since we directly use the training strategy of softmax scoring-based watermarking for MaxLogit, our results in Tabledemonstrate that watermarking can also benefit from improved choices of scoring strategies.
C.3 Other Learning Strategies with Watermarking
Also, we consider the situation where a set of OOD data are available for watermark training, where we adopt the tiny-ImageNet [27] dataset as training-time OOD dataset.
Here, we replace the Gaussian noise in the OOD learning objective ℓ OOD to be the randomly selected sample from the tiny-ImageNet dataset.
Then, the learning objective with training-time OOD data is of the form
[Equation]
where o j denotes the randomly selected sample from the tiny-ImageNet dataset.
The experimental results on CIFAR-
and CIFAR- 100 datasets are summarized in Table 14 and Table 15.
Unfortunately,
[Table]
[Table] [Table] [Table] for our current realization, we observe that watermark training with extra OOD data fails to induce a large performance improvement in OOD detection.
Even worse, in some cases, learning with extra
OOD data can impair the power of the resultant watermarks.
We conjecture that the inductive bias introduced by the training-time OOD data may deviate from the test-time data, severely misleading the resultant watermarks in showing results even lower than that of the simple Gaussian noise.
We also list the detection performance with “perm” and “rotate” in Section 6, demonstrating the effectiveness of the resultant watermarks on far OOD cases.
Here, the training objective is:
[Equation]
where ˜ x is an augmentation of the original x with either “perm” or “rotate”.
The results are summarized from Tables 16 to 19.
As we can see, the results with "perm" and "rotate" is comparable with (even better than) the original learning setup with only the Gaussian noise (common).
[Table]
[Table] [Table]
C.4 Comparison with State-of-the-art Methods
For the concreteness of our discussion, this section compares our proposal with state-of-the-art methods in OOD detection.
In particular, we compare with softmax scoring (Softmax) [17], free energy scoring (Energy) [33], ReAct [46], ODIN [32], Mahalanobis [28], GradNorm [21], and
OE [18].
The experimental results on CIFAR benchmarks are summarized in Table 20.
The average performance on iSUN, Places 365 , Texture, SVHN, and LSUN is reported.
As we can see, watermarking can boost the performance of various scoring methods in OOD detection, achieving the best detection performance compared with all other advanced methods.
C.5 Experiments with Mean and Standard Deviation
This section further verifies the results from Table 1 to Table 4 with five individual trails (random seeds).
In Table 21, Table 22, and Table 23, we summarize the average results and the standard deviation for the softmax scoring and the free energy scoring, respectively.
In Figure 5, we also depict the learned watermarks for each trial on CIFAR- 10 and CIFAR- 100 .
Overall, we observe that the learned watermarks preserve some similar pattern (e.g., the shape of areas with large values) given the same dataset and the same scoring strategy, and the improvement of watermarking is stable across different datasets and scoring methods.
C.6 Experiments with different models
We demonstrate the effectiveness of our watermarking across various model architectures, including
ResNet-18 [14], WRN-40-2, and ViT-B/16 [8].
We conduct experiments on the ImageNet benchmark and summarized the results in Table 24.
As we can see, in both the softmax and free energy scoring cases, our watermarking can improve the detection performance across various models.
However, the
[Table] [Table]
improvements after watermarking on the large-scale models (i [Table] e., ViT-B/16) are not as remarkable as that of the small models (e.g., ResNet-18).
It is because that the large-scale models themselves can already excel at OOD detection (better results without watermarking than that of ResNet-18 and
WRN-40-2), so there may not remain a large space for their further improvements.
C.7 Hyper-parameter Setups
For the hyper-parameter setups in our experiments, we use random search to choose the proper σ 1 from the candidate parameter set { 0 . 0 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 . 0 , 1 . 2 , 1 . 4 , 1 . 6 , 1 . 8 , 2 . 0 } , and the proper ρ from { 0 . 0 , 0 . 02 , 0 . 05 , 0 . 07 , 0 . 1 , 0 . 2 , 0 . 5 , 0 .
For softmax scoring, β is chosen from { 0 . 0 , 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 , 2 . 5 , 3 . 0 , 3 . 5 , 4 . 0 , 4 . 5 , 5 . 0 } . For free energy scoring, β is chosen from { 0 . 0 , 0 . 02 , 0 . 04 , 0 . 06 , 0 .
08 , 0 . 1 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 . 0 } , and T 1 , T 2 are chosen from { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 . 0 } .
model performance is tested on validation OOD datasets that are separated from iSUN, Places 365 , Texture, SVHN, and LSUN datasets.
We adopt the random search with many trials by the following three steps.
Step 1: we randomly select a hyperparameter (e.g., β ) and fix the values of all other hyperparameters to be their current optimal values. Step 2: we choose the best β from the candidate set. Step 3: do Steps 1-2 again. We repeat Steps 1 and 2 for 50 times in our experiments.
Further, from Tables 25 to 36, we list the performance of watermarking with different hyper-parameter settings for reference, where we fix
[Table]
[Table]
the values of all other hyper-parameters (except for the considered one) to be their optimal values.
Finally, we list the results on CIFAR benchmarks regarding the free energy scoring with different values of T in Table 37.
In the end, we summarize our choices of hyper-parameters, which we adopt in Section 6 for the related experiments.
On CIFAR benchmarks, our method is executed for 50 epochs.
The initial learning rate α = 0 .
01 divided by 10 after 25 epochs.
For the softmax scoring, we set σ 1 = 0 .
4 , ρ = 1 . 0 , β = 3 . 5 in CIFAR- 10 and σ 1 = 1 . 0 , ρ = 0 .
2 , β = 2 . 5 in CIFAR- 100 ; for the free energy scoring, we set σ 1 = 0 .
6 , ρ = 0 .
7 , β = 0 .
1 , T 1 = 0 . 2 , T 2 = 0 . 7 in CIFAR- 10 and σ 1 = 1 . 0 , ρ = 0 .
05 , β = 1 . 2 , T 1 = 0 . 9 , T 2 = 0 . 1 in CIFAR- 100 .
On the ImageNet benchmark, our method is executed for 10 epochs and the initial learning rate α = 0 .
01 is divided by 10 after 5 epochs.
We set ρ = 0 .
5 , σ 1 = 0 .
2 , β = 1 . 5 for the softmax scoring and ρ = 0 .
05 , σ 1 = 0 .
4 , β = 0 .
1 T 1 = T 2 = 0 .for
the free energy scoring.
Further, we fix σ 2 = 0 .
001 and T = 1 .
C.8 Different Areas in the Watermark
After watermarking, the edge area of the image is overwhelmed by the watermark’s pattern, while the centre part is not much affected.
However, it does not mean that the centre area is not important.
Instead, under the premise of maintaining the original features, the centre area also encodes useful information in OOD detection.
Table 38 is a verification of this conclusion on CIFAR- 10 dataset with the free energy scoring, where we employ various masks in only preserving the watermark’s features with their absolute values larger (smaller) than a threshold χ 1 ( χ 2 ).
As we can see, even if only a small portion of the watermark is masked (e.g., χ 1 = 0 .
10 or χ 2 = 10 . 0 ), there is a large drop in performance, even lower than the results without any watermarking.
It indicates that both areas of the watermark contribute, and the overall watermark works as a whole for effective OOD detection. [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Figure]
[Table]
[Figure]
D
en s i t y
D
en s i t y
Free Energy Score
[Figure]
D
en s i t y
[Figure]
Free Energy Score
D
en s i t y (a) w/o watermarking (a)
D
en s i t y
[Figure]
i t y
Free
[Figure]
Score
[Figure]
[Table]
[Figure]
Free
[Figure]
D
en s i t y (a) w/o watermarking (
[Figure]
[Table]