Paper: Safe and Robust Watermark Injection with a Single OoD Image (ICLR-2024)

Reviewer comment (Experiments):
**About the *Model Extraction* attack**: It is mentioned in section 4 that *the behavior of a victim model is cloned by fine-tuning it with the queried image-prediction pairs*, then what's the difference between the fine-tuning attack and model extraction? As retraining-based extraction attacks are generally considered in previous works [1-2], the robustness against such an attack should be further demonstrated.

Author response:
Thank you very much for your comment. We add the results for re-training-based model extraction in Table 8 in the supplementary. For this method, a large standard accuracy degradation (more than 10%) is observed, but the watermark is still extracted by the suspect model with high OoDWSR and p-value close to 0. We have included a detailed analysis of the results in the supplementary.

Reviewer comment (Experiments):
**Possible OOD detection**: As the verification process is conducted with the augmentation of a single OOD image, the adversary may simply reject the query of OOD samples to reject the ownership verification process, as illustrated in [2]. Since OOD detection is entirely possible in the realistic scenarios proposed by the submission, the adaptive defenses should be discussed in further detail.

Author response:
Thank you very much for your suggestion.  Following [2], we add one experiment using energy score [3] for OoD detection in Table 9 in the supplementary.  We compute the mean energy score for ID samples and our verification samples, and also AUROC and AUPR for ID and OoD classification according to [3]. AUROC and AUPR are two frequently used evaluation metrics for OoD detection. The higher these two metrics, the better the OoD detection performance. For the watermarked model pre-trained on CIFAR-100 and GTSRB, the energy scores for ID and verification samples are very close, and the AUROC and AUPR are close to random guesses. For CIFAR-10, despite the differences of energy score, AUPR for OoD detection is only 20\%, which indicates that it will be very hard for the adversaries to filter out verification samples from ID ones, if verification samples are mixed with other samples while querying the suspect model. Therefore, the query of our verification samples cannot be found out by the IP infringers. A possible reason is that the distribution of our generated verification samples lies closely to the ID ones, since previous work [4] shows that the generated samples from the single OoD image can also be used to train a classifier yielding reasonable performance on the main prediction task.

Reviewer comment (Experiments):
As the submission highlights the use of a single OOD image for watermark injection, the author may provide more comparison between the "pachify" of a single image and sampling several OOD images as the verification dataset, as to enhance the persuasiveness.

Author response:
Thank you very much for your suggestions. We investigate the effect of different OoD images in Table 5, and the combination of different OoD images in Table 10 in the supplementary. We also add the results of OoD detection in Table 9 in the supplementary as explained in the last question.

Reviewer comment (Experiments):
More adaptive defenses such as OOD detection and Neural Cleanse [3] should be considered in addition to the traditional watermark removal attacks.

Author response:
Thank you very much for your suggestions. We investigate the effect of different OoD images in Table 5, and the combination of different OoD images in Table 10 in the supplementary. We also add the results of OoD detection in Table 9 in the supplementary as explained in the last question.

Reviewer comment (Experiments):
Relying on a single OoD image as a secret key might introduce vulnerabilities. If an adversary gains access to this single image, the watermarking scheme could be compromised. Using multiple OoD images or a combination of techniques might enhance security?

Author response:
Thank you very much for your suggestion. We add the evaluation for the effect of a combination of different source OoD images in Table 10 in the supplementary. Although the watermark is not more robust against fine-tuning conducted by the IP infringer, we agree with your point that using multiple OoD images will make it harder for the IP infringers to get access to the source OoD image and guess the composition of our validation set, which can further enhance the security of the watermark.

Reviewer comment (Experiments):
It would be interesting to see how the watermarking technique performs when models are transferred across different tasks or domains. The watermark might lose its effectiveness in such scenarios.

Author response:
An interesting scenario you have mentioned! We think re-training-based model extraction attacks using data from different domains as the auxiliary dataset might be similar to the scenario you have mentioned. It retrains a student model using the auxiliary data and the label queried from the victim (teacher) model. We add the results and corresponding analysis for this method in Table 8 in the supplementary. The results show that our method is still robust against this extraction attack using data from different domains.

Reviewer comment (Experiments):
How many verification samples are needed for each successful verification?

Author response:
We investigate the effects of different numbers of verification samples in Figure 9 in the supplementary. We generate 45000 surrogate samples for watermark injection, but we do not need to use all the surrogate data for verification. We report the OoDWSR w.r.t. different numbers of samples in the verification dataset in Fig. 9. According to the figure, only 450 verification samples are enough for accurate verification.

Reviewer comment (Experiments):
In experiments the authors tried many different triggers. Which trigger should an IP protector choose in practice?

Author response:
Thank you very much for your comment. Trigger patterns in this paper are derived from prior arts, of which no trigger could be claimed to be successfully injected on all datasets with 100\% WSR. Whether these triggers can be injected successfully is related to the pattern instead of our proposed method. In practice, we can search for a valid pattern from a large enough candidate set that covers as many existing trigger patterns as possible. As we have done in our experiments, we found proper triggers for all evaluated datasets.

Reviewer comment (Novelty):
The authors do not give a comprehensive discussion of previous work on this topic.

Author response:
Thank you very much for your comment. We discuss the DNN watermarking in section 2.1, including parameter-based watermarking, backdoor-based watermarking, and a special kind of backdoor-based watermarking without i.i.d data, which is closely related to our problem setting. We also discuss watermark removal attack in section 2.2.

Reviewer comment (Experiments):
The experimental justification of this work is not sufficient, only compared to the basic backdoor-based strategy.

Author response:
Thank you very much for your comment. We want to argue that the problem setting for our proposed method and existing backdoor-based strategies are different. The existing backdoor-based strategy requires training data or in-distribution (ID) data to maintain the model utility, while our paper explores a more realistic scenario where access to ID data can be prohibitive due to data safety and confidentiality. 

Despite the different settings, we also compare with traditional methods to further verify the robustness of our method. The traditional backdoor-based method uses in-distribution (ID) data to inject watermarks, we compare our method with the traditional backdoor-based method in Table 3. According to the results, after RT-AL, the watermark success rate drops to 4.13% and 3.42%, respectively for ID poison (basic backdoor), while drops to 57.52% and 24.19% for OoD poison (proposed method), which verifies that our proposed method is also much more robust against watermark removal attacks.

Reviewer comment (Writing):
On page 2. ‘…, which fills in the gap of IP protection of deep models without training data’. Most of the white-box watermarking methods do not need the training data, you should make it clear that you target the backdoor-based methods.

Author response:
Thank you very much for your comment. We have clarified this point in our revised manuscript.

Reviewer comment (Writing):
On page 5. ‘If the protected model shares a similar parameter distribution with the pre-trained model, the injected watermark could be easily erased by fine-tuning using clean i.i.d. data or adding random noise to parameters’. What is the pre-trained model? Does it mean that fine-tuning with the i.i.d data can introduce more perturbations on the watermarks than fine-tuning with OoD data? What’s the intuition behind such a claim?

Author response:
Clarification for pre-trained model: The pre-trained model is a high-performance large model that requires curating a massive amount of training data from different sources, powerful computational resources, and often great efforts from human experts. Our goal is to inject watermarks to these pre-trained models for IP protection.

Clarification for fine-tuning: First we want to clarify that there are two kinds of fine-tuning in our paper. One kind of fine-tuning is conducted by the model protector, which uses our generated OoD verification samples to inject watermarks. Another kind of fine-tuning is conducted by the IP infringer, which is one kind of watermark removal attack (the detailed introduction can be referred to section 2.2). For fine-tuning attack, a clean set of i.i.d or OoD data will be used to wash out the backdoor-based watermark injected by the poisoned dataset. “Fine-tuning” in your list claim is the second one. We would like to clarify by rephrasing the claim in your question: if the parameter perturbation of the protected model is small compared to the pre-trained one, the watermark will not be resilient to random noise or clean data fine-tuning. This claim is obtained from [1-2].

We do not have the conclusion that fine-tuning with the i.i.d data can introduce more perturbations on the watermarks than fine-tuning with OoD data. Fine-tuning using the OoD data will usually sacrifice the model utility (standard accuracy) no matter for model protector or IP infringer. We propose a novel method in our paper to deal with this problem, and we assume an unfair scenario for the protector where the removal attack uses a part of in-distribution data to show the robustness of our proposed watermarks through extensive empirical results.

We hope the above clarifications can solve your concerns about this claim.

[1] Hong S, Carlini N, Kurakin A. Handcrafted backdoors in deep neural networks[J]. Advances in Neural Information Processing Systems, 2022, 35: 8068-8080.

[2] Garg S, Kumar A, Goel V, et al. Can adversarial weight perturbations inject neural backdoors[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020: 2029-2032.

Reviewer comment (Evaluation):
On page 5. The statement of ‘the anomalies of the parameter distribution could be easily detected by an IP infringer’ may be conflict with the experimental results of Ref. [1]. The authors of Ref. [1] conduct experiments to check whether the distributions of model weights or model representations could be used as indicators to detect backdoors or anomalies in a DNN model.

Author response:
Thank you very much for your comment. The statement in page 5 does not conflict with [3], since we have a precondition for this statement as stated in the original submission. **If the perturbation is too large**, the anomalies of the parameter distribution could be easily detected by an IP infringer [4]. Thus, we have to control the scale of the perturbation to avoid being detected. Both [3] and our proposed method show no obvious statistical difference from the original model. Thus, the backdoor is hard to be detected.

[3] Programmable neural network trojan for pre-trained feature extractor.

[4] Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. Tbt: Targeted neural network attack with bit trojan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.13198–13207, 2020.

Reviewer comment (Experiments):
Experimental settings are not clear enough: 1) How many patches are used for training? 2) Comparison experiments are not well organized. There are many previous methods with similar strategies, e.g., BadNets, 10-invisible, smooth, Trojan Square, and Trojan watermark, and it's essential to compare with them.

Author response:
1)We investigate the effects of different numbers of verification samples in Figure 9 in the supplementary. We generate 45000 surrogate samples for watermark injection, but we do not need to use all the surrogate data for verification. We report the OoDWSR w.r.t. different numbers of samples in the verification dataset in Fig. 9. According to the figure, only 450 verification samples are enough for accurate verification.

2)6 different backdoor patterns are adopted in our experiments, including BadNets with grid, l0-invisible, smooth, Trojan Square, Trojan Square, and Trojan watermark. The watermark injection results can be found in Figure 2 in section 4.1 and Figure 5 in the supplementary.

Reviewer comment (Evaluation):
In Table 2, we observe that the OoDWSR may increase after fine-tuning and pruning. Could you give some explanations? It seems the IDWSR may also increase after such two attacks. Why? It seems to be contradicted by your previous claim.

Author response:
A possible reason for the phenomenon you have mentioned is that for some of the cases, with limited clean data used for fine-tuning or pruning, the shortcut for the backdoor is well preserved, while the path in the DNN to preserve the model utility is easily corrupted (a significant ACC degradation). These results will not contradict our precious claim, since the results in Table 2 are all with weight perturbation, in this way, the backdoor is more resilient to fine-tuning or pruning attacks. The results in Table 6 show that the backdoor-based watermark without weight perturbation can be quite vulnerable to attacks, which is consistent with our previous claim.

Reviewer comment (Reproducibility):
How did you implement the model extraction attack? According to the experimental results in Table 4, we still have a high OoDWSR after model extraction. Could you explain why the model extraction methods using i.i.d data can also extract the watermarks that are generated using the OoD data?

Author response:
Thank you very much for your comment. We give our explanation as follows:

1)**The model extraction results shown in Table 4 are based on fine-tuning.**

**Implement**: We use knockoff as an example of the model extraction attack, which queries the model to get the predictions of an auxiliary dataset, and then clones the behavior of a victim model by fine-tuning the model with queried image-prediction pairs. 

**Explanation**: The model extraction method will query the watermarked model for the label of the training samples, although they may get the correct label, this method will just function like clean data fine-tuning, the watermark is robust to fine-tuning as shown in our previous experiments.

2)**We also add one re-training-based model extraction in Table 8 in the supplementary.** A detailed analysis can also be found there.

**Implementation**: The extracted models are re-trained from scratch. For this method, a large standard accuracy degradation (more than 10%) is observed, but the watermark is still extracted by the suspect model with high OoDWSR and p-value close to 0.

**Explanation**: The backdoor information hidden in the soft label queried by the IP infringers can also embed the watermark in the extracted model. The extracted model will behave more similarly to the victim model as its decision boundary gradually approaches that of the victim model. In this way, the watermark is still extracted by the suspect model.