Paper: DiffusionShield: A Watermark for Data Copyright Protection against Generative Diffusion Models (ICLR-2024)

Reviewer comment (Presentation):
The paper carries out extensive experiments but could improve in terms of layout. A considerable amount of experimental data is placed in the appendix, with minimal mention in the main text. As a potential user of this model, I would be particularly concerned about the issue of watermark rates. Specifically, it's worth questioning whether the proposed method remains effective when the training dataset contains only a limited number of watermarked images. Appendix F's Figure 11 briefly touches upon this issue, but this aspect should be more explicitly mentioned in the main text.

Author response:
**Response**: Thanks for your suggestion about the paper organization. We updated the main text of Section 4.2 to incorporate the discussion on the influence of watermark rate. In the main text, we point out that, although the low watermark rate might hurt the performance, this problem is mitigated in the multi-user case. Even though the watermark rate for each user is low and they encode different messages and do not share the pattern uniformity, our method can still perform well. Then we refer the readers to read more details about the experiment in Appendix F. The updated part is marked blue in the revision for your convenience.

Reviewer comment (Experiments):
The paper employs the HiDDeN model for deep learning-based watermarking, which is no longer state-of-the-art (SOTA). Nevertheless, HiDDeN outperforms the proposed method in certain experiments, such as under the 'Uncond.' condition in Table 1 for CIFAR10 and in the distortion method of Gaussian noise in Table 4. Therefore, the authors should utilize a state-of-the-art deep learning-based watermarking model to further substantiate the superiority of their method.

Author response:
**Response**: Thanks for your suggestion regarding the inclusion of new baselines. In response, we have expanded our comparative analysis to include three additional baselines: IGA[1], MBRS[2], and CIN[3]. The results are reported in Table 15 below. We can see that the performance of IGA is very similar to HiDDeN and DFD. Although IGA’s bit accuracy is comparable to our DiffusionShield, it demands a significantly higher budget—nearly 10 times the Linf and LPIPS values of our approach. As for MBRS and CIN, despite having budgets similar to DiffusionShield, their bit accuracies are worse than ours, especially on CIFAR100, where MBRS achieves only 87.68% and CIN is 51.13% in bit accuracy. In contrast, DiffusionShield maintains the bit accuracy of 99.80% with a comparable budget. This is because of the higher pattern uniformity of DiffusionShield. In summary, the performances of the new baselines are similar to the previous baseline methods. They either compromise the budget for bit accuracy close to ours, or fail to be reproduced well in the generated images. 

[*Table 15*]
|Dataset|budget|IGA|MBRS|CIN|ours|
|-|-|-|-|-|-|
|CIFAR10|$L_{\infty}$|52/255|16/255|8/255|4/255|
||$L_{2}$|3.38|0.36|0.42|0.72|
||LPIPS|0.08910|0.00182|0.00185|0.00720|
||Bit Acc.(Cond.)|99.63%|99.97%|99.97%|100.00%|
||Uniformity|0.063|0.518|0.599|0.964|
|CIFAR100|$L_{\infty}$|66/255|19/255|9/255|4/255|
||$L_{2}$|5.31|0.43|0.44|0.72|
||LPIPS|0.08830|0.00129|0.00105|0.00134|
||Bit Acc.(Cond.)|97.25%|87.68%|51.13%|99.80%|
||Uniformity|0.162|0.394|0.527|0.915|

[1] Zhang, Honglei, et al. Robust data hiding using inverse gradient attention. 

[2] Jia et al. Mbrs: Enhancing robustness of dnn-based watermarking by mini-batch of real and simulated jpeg compression, ACM MM '21

[3] Ma et al. Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanisms, ACM MM '22

Reviewer comment (Experiments):
In the Implementation Details section, the authors build training dataset by "designating one random class of images as watermarked images, while treating other classes as unprotected images." While using images from the same class might be an ideal and simpler scenario that could further promote pattern uniformity, in real-world applications, images could come from various classes. The authors need to elaborate more on the effectiveness of their method under such circumstances.

Author response:
**Response**: Thanks for your suggestions on the implementation details. In response to your concerns, we have added one set of experiments using CIFAR-100. In the following Table 16, we have considered that the protected images are from various classes (5, 10 or 15 classes randomly selected in CIFAR-100) and added watermarks to those protected classes. The bit accuracy of the watermark on the generated images from the protected class are shown in the table. It reveals that the watermarks are predominantly detectable in the images across various classes, with limited exceptions noted in scenarios where the watermark budget is minimal (1/255) and only 5 classes are watermarked.

[Table 16]
|Budget|5 class|10 class|15 class|
|-|-|-|-|
|8/255|100.000%|99.995%|99.994%|
|4/255|99.998%|99.980%|99.993%|
|2/255|99.748%|99.973%|99.969%|
|1/255|65.525%|98.824%|99.707%|

Reviewer comment (Experiments):
For the first weakness, I understand that due to the constraints of the main text's length, it might be challenging for the authors to include all experimental details. However, I would still suggest at least mentioning the experiment of watermark rates in the main body of the paper. This is merely a suggestion on my part; even if the authors choose not to adopt it, it won't lead me to lower my rating of the paper.

Author response:
**Response**: Thanks for your suggestion about the paper organization. We updated the main text of Section 4.2 to incorporate the discussion on the influence of watermark rate. In the main text, we point out that, although the low watermark rate might hurt the performance, this problem is mitigated in the multi-user case. Even though the watermark rate for each user is low and they encode different messages and do not share the pattern uniformity, our method can still perform well. Then we refer the readers to read more details about the experiment in Appendix F. The updated part is marked blue in the revision for your convenience.

Reviewer comment (Experiments):
The second point of weakness is a significant concern for me. I hope the authors can provide direct experiments to alleviate my doubts. The state-of-the-art (SOTA) model that I'm currently aware of is "Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanisms."

Author response:
**Response**: Thanks for your suggestion regarding the inclusion of new baselines. In response, we have expanded our comparative analysis to include three additional baselines: IGA[1], MBRS[2], and CIN[3]. The results are reported in Table 15 below. We can see that the performance of IGA is very similar to HiDDeN and DFD. Although IGA’s bit accuracy is comparable to our DiffusionShield, it demands a significantly higher budget—nearly 10 times the Linf and LPIPS values of our approach. As for MBRS and CIN, despite having budgets similar to DiffusionShield, their bit accuracies are worse than ours, especially on CIFAR100, where MBRS achieves only 87.68% and CIN is 51.13% in bit accuracy. In contrast, DiffusionShield maintains the bit accuracy of 99.80% with a comparable budget. This is because of the higher pattern uniformity of DiffusionShield. In summary, the performances of the new baselines are similar to the previous baseline methods. They either compromise the budget for bit accuracy close to ours, or fail to be reproduced well in the generated images. 

[*Table 15*]
|Dataset|budget|IGA|MBRS|CIN|ours|
|-|-|-|-|-|-|
|CIFAR10|$L_{\infty}$|52/255|16/255|8/255|4/255|
||$L_{2}$|3.38|0.36|0.42|0.72|
||LPIPS|0.08910|0.00182|0.00185|0.00720|
||Bit Acc.(Cond.)|99.63%|99.97%|99.97%|100.00%|
||Uniformity|0.063|0.518|0.599|0.964|
|CIFAR100|$L_{\infty}$|66/255|19/255|9/255|4/255|
||$L_{2}$|5.31|0.43|0.44|0.72|
||LPIPS|0.08830|0.00129|0.00105|0.00134|
||Bit Acc.(Cond.)|97.25%|87.68%|51.13%|99.80%|
||Uniformity|0.162|0.394|0.527|0.915|

[1] Zhang, Honglei, et al. Robust data hiding using inverse gradient attention. 

[2] Jia et al. Mbrs: Enhancing robustness of dnn-based watermarking by mini-batch of real and simulated jpeg compression, ACM MM '21

[3] Ma et al. Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanisms, ACM MM '22

Reviewer comment (Experiments):
For the third weakness, I believe the authors need to conduct more comprehensive experiments to further substantiate the effectiveness of their method across multiple real-world scenarios.

Author response:
**Response**: Thanks for your suggestions on the implementation details. In response to your concerns, we have added one set of experiments using CIFAR-100. In the following Table 16, we have considered that the protected images are from various classes (5, 10 or 15 classes randomly selected in CIFAR-100) and added watermarks to those protected classes. The bit accuracy of the watermark on the generated images from the protected class are shown in the table. It reveals that the watermarks are predominantly detectable in the images across various classes, with limited exceptions noted in scenarios where the watermark budget is minimal (1/255) and only 5 classes are watermarked.

[Table 16]
|Budget|5 class|10 class|15 class|
|-|-|-|-|
|8/255|100.000%|99.995%|99.994%|
|4/255|99.998%|99.980%|99.993%|
|2/255|99.748%|99.973%|99.969%|
|1/255|65.525%|98.824%|99.707%|

Reviewer comment (Experiments):
This method mostly describes adding a watermark to the original images instead of the latent space, which is commonly used by existing GDMs such as Stable Diffusion (SD). Although there is a section describing fine-tuning an SD and adding watermarks to the latent space, there is much less visualization and analysis on it. The author can provide more examples and analysis of adding a watermark to latent space and how it will affect the VAE encoding/decoding process.

Author response:
Thanks for your insightful query regarding the influence of watermark in latent space. To address this, we compared the budget and visualized the change of latent space in updated Appendix J. As shown in the below Table 13, in latent space, the budget of baselines including both DFD and HiDDeN are much larger than our method. Even though the baseline methods have such large budgets, we know that, from Table 3 in Section 4.4, their performance of protection is still worse than ours. (For your convenience, we also repeat Table 3 below. ) Also, we visualize the change of latent space after watermarking in Appendix J and find that the change of DFD and HiDDeN is much more obvious than ours. Regarding the VAE, after we decode from the watermarked latent space, the budget is still small and the watermark is imperceptible in pixel space. This means that our watermark has a much smaller influence on the decoding process compared with baseline methods.

[*Table 13*]
||DFD|HiDDeN|ours|
|-|-|-|-|
|latent space|191.3678|202.1912|42.1265|
|pixel space|88.9103|81.2732|31.2287|

Reviewer comment (Experiments):
There is less analysis on how DiffusionShield can protect against the removal attack — if the malicious actor took the watermarked data and performed some augmentations on the data, will the watermark still be able to be detected?

Author response:
Thanks for your question. In Table 4 in Section 4.6, we have conducted experiments to demonstrate the robustness of our method against various forms of image corruption including Gaussian noise, low-pass filtering, grayscale, and JPEG compression. In order to further clarify your concern about the robustness, we have expanded the scope of our experiments to include image resizing and a deep-learning-based watermark removal method [1]. These tests are conducted using the CIFAR-10 dataset. For the resizing experiments, we altered image sizes from 32x32 to 64x64 (termed "large" in the table), or from 32x32 to 16x16 (termed "small" in the table). During detection, we resize all the data back to 32x32 before inputting them to the detector. In order to improve the robustness, we have incorporated the corresponding disturbances as an augmentation in the training of the watermark. All the results related to robustness are put together in the following Table 14. From the results we can see that DiffusionShield consistently achieves a high bit accuracy and outperforms the two baseline methods in all scenarios except under the Gaussian Noise corruption. In summary, the results from these experiments indicate that our DiffuisonShield method is able to maintain robustness against multiple disturbances.

[*Table 14*]
||DFD|HiDDeN|Ours|
|-|-|-|-|
|No corruption|93.57%|98.93%|99.99%|
|Resize (Small)|92.38%|83.13%|99.30%|
|Resize (Large)|93.20%|79.69%|99.99%|
|Watermark Removal [1]|91.11%|82.20%|99.95%|
|Gaussian Noise|68.63%|83.59%|81.93%|
|Low-pass filter|88.94%|81.05%|99.86%|
|Greyscale|50.82%|97.81%|99.81%|
|JPEG Comp.|62.52%|74.845|94.45%|

[1] Invisible Image Watermarks Are Provably Removable Using Generative AI. Zhao et. al., 2023

Reviewer comment (Experiments):
If the malicious actor took the watermarked data and performed some augmentations on the data, will the watermark still be able to be detected?

Author response:
Thanks for your question. In Table 4 in Section 4.6, we have conducted experiments to demonstrate the robustness of our method against various forms of image corruption including Gaussian noise, low-pass filtering, grayscale, and JPEG compression. In order to further clarify your concern about the robustness, we have expanded the scope of our experiments to include image resizing and a deep-learning-based watermark removal method [1]. These tests are conducted using the CIFAR-10 dataset. For the resizing experiments, we altered image sizes from 32x32 to 64x64 (termed "large" in the table), or from 32x32 to 16x16 (termed "small" in the table). During detection, we resize all the data back to 32x32 before inputting them to the detector. In order to improve the robustness, we have incorporated the corresponding disturbances as an augmentation in the training of the watermark. All the results related to robustness are put together in the following Table 14. From the results we can see that DiffusionShield consistently achieves a high bit accuracy and outperforms the two baseline methods in all scenarios except under the Gaussian Noise corruption. In summary, the results from these experiments indicate that our DiffuisonShield method is able to maintain robustness against multiple disturbances.

[*Table 14*]
||DFD|HiDDeN|Ours|
|-|-|-|-|
|No corruption|93.57%|98.93%|99.99%|
|Resize (Small)|92.38%|83.13%|99.30%|
|Resize (Large)|93.20%|79.69%|99.99%|
|Watermark Removal [1]|91.11%|82.20%|99.95%|
|Gaussian Noise|68.63%|83.59%|81.93%|
|Low-pass filter|88.94%|81.05%|99.86%|
|Greyscale|50.82%|97.81%|99.81%|
|JPEG Comp.|62.52%|74.845|94.45%|

[1] Invisible Image Watermarks Are Provably Removable Using Generative AI. Zhao et. al., 2023

Reviewer comment (Experiments):
The setting of the experiments is not practical and should defend against possible attacks. At least, the author should consider JPEG compression and Resize attack which is very common in the real scenario. If the generated images are resized to other shape, does this method still work?

Author response:
Thanks for your suggestions about the robustness analysis. In Table 4 of Section 4.6, we have conducted experiments to demonstrate our robustness against attack including JPEG compression, Gaussian noise, low-pass filter and grayscale. To further clarify your concern about the robustness, we add more experiments about resize and a deep-learning-based watermark removal method [1]. In the following Table 12, we merge all the results including Table 4 for comparison. These tests are conducted using the CIFAR-10 dataset. For the resizing experiments, we altered image sizes from 32x32 to 64x64 (termed "large" in the table), or from 32x32 to 16x16 (termed "small" in the table). During detection, we resize all the data back to 32x32 before inputting them to the detector. In order to improve the robustness, we have incorporated the corresponding disturbances as an augmentation in the training of the watermark. Notably, DiffusionSheild maintains bit accuracy above 94% and outperforms the two baseline methods in all scenarios except under the Gaussian Noise corruption. In comparison, DFD suffers from a significant reduction in Gaussian noise, greyscale and JPEG compression, and HiDDeN shows a poor performance under low-pass filter, resize, watermark removal attack, and JPEG Compression. In summary, the results from these experiments indicate that our DiffuisonShield method is able to maintain robustness against multiple disturbances.

[1] Invisible Image Watermarks Are Provably Removable Using Generative AI. Zhao et. al., 2023

[*Table 12*]
||DFD|HiDDeN|Ours|
|-|-|-|-|
|No corruption|93.57%|98.93%|99.99%|
|Resize (Small)|92.38%|83.13%|99.30%|
|Resize (Large)|93.20%|79.69%|99.99%|
|Watermark Removal [1]|91.11%|82.20%|99.95%|
|Gaussian Noise|68.63%|83.59%|81.93%|
|Low-pass filter|88.94%|81.05%|99.86%|
|Greyscale|50.82%|97.81%|99.81%|
|JPEG Comp.|62.52%|74.85%|94.45%|

Reviewer comment (Experiments):
The setting of the experiments should introduce high resolution images ([1] is a good example). It may not make sense to validate with low resolution images because GDM usually generates higher resolution images.

Author response:
Thanks for your suggestion. The experiments of high-resolution images are reasonable and necessary and we do have ImageNet in Table 1 in Section 4.2 and pokemon-blip-captions in Table 3 in Section 4.4. ImageNet is in shape 256x256, and pokemon-blip-captions is in 512x512. These datasets have similar resolution as the dataset in the suggested paper [1], and they are also similar to most of Diffusion Model papers, like [2] [3], whose experiments are mainly conducted with CIFAR-10 in 32x32, ImageNet in 64x64 and LSUN in 256x256, and Latent Diffusion papers like [4], whose experiments are mainly conducted using image with size 512x512 and 256x256. We hope this can clarify your concern about the image resolution.

[1] Guo et al. Practical deep dispersed watermarking with synchronization and fusion, ACM MM'23

[2] Ho et al. Denoising Diffusion Probabilistic Models, NeurIPS’ 20

[3] Nichol et al. Improved Denoising Diffusion Probabilistic Models, ICML’ 21

[4] Rombach et al. High-Resolution Image Synthesis with Latent Diffusion Models, CVPR’ 22

Reviewer comment (Theory):
It seems that each 4x4 basic block can represent 2 bit (if B=4), so how to embed 128 bit in CIFAR-10? For me, the message should be (28/4) x (28/4) x 2 bit = 98 bit, right?

Author response:
Thanks for your question. The resolution of CIFAR10 and CIFAR100 [1] is 32x32. Thus, the message length for CIFAR should be (32/4)x(32/4)x2bit=128bit. We updated this calculation process in the Appendix C.2 for readers’ convenience. 

[1] Krizhevsky et al. Learning multiple layers of features from tiny images. (2009): 7.

Reviewer comment (Evaluation):
The paper does not include an examination of the method's false positive rate and AUROC.

Author response:
**Response**: Thank you for the suggestion to include FPR and AUROC in our watermark detection accuracy assessment. In the following Table 10, we show the TPR, FPR and AUROC of our DiffusionShield with CIFAR-10 dataset. The threshold for the TPR and FPR is 90%, which means that, if more than or equal to 90% of the detected message match the message encoded to the watermarks on the training data, then the image will be considered as an image generated by a model which was trained on the protected data. From the table we can see that our DiffusionShield has a perfect performance in terms of TPR and FPR, which demonstrates DiffusionShield's powerful effectiveness in safeguarding image copyrights.

[*Table 10*]
|budget=8/255|TPR|FPR|AUROC|
|-|-|-|-|
|CIFAR-10|100%|0%|1|
|CIFAR-100|100%|0%|1|
|STL-10|100%|0%|0.99|
|ImageNet|100%|0%|1|

Reviewer comment (Evaluation):
The paper does not offer an analysis of the robustness of their watermark against recent attacks known to compromise imperceptible watermarks.

Author response:
**Response**: Thank you for your suggestions on the analysis of the robustness of watermarks. In Table 11 we demonstrate the bit accuracy and AUROC of our DiffusionShield together with two baselines methods under the context of watermark removal attempts by [2]. The results show that DiffusionShield is able to survive and achieve a high bit accuracy and AUROC in the experiments on CIFAR-10 and Pokemon. In comparison, the performance of HiDDeN is much worse. Although generally DFD performs well in defending against the removal method, we note that it requires a much higher budget compared to our method, and larger budgets are more difficult to remove.
Regarding the experiments in Table 4, the standard deviation of the Gaussian Noise is 0.1 and its effect on the images can be visualized in Figure 13 in Appendix J. From the figure we can see that the Gaussian noise considered in Table 4 has a great influence on the quality of images. Even with such an influence on the images, DiffusionShield can still achieve bit accuracy higher than 80%, and AUROC=1, demonstrating the robustness of DiffusionShield against Gaussian Noise.

[*Table 11*]
||DFD||HiDDeN||Ours (8/255)||
|-|-|-|-|-|-|-|
||Bit Acc.|AUROC|Bit Acc.|AUROC|Bit Acc.|AUROC|
|CIFAR-10|91.11%|1.000|82.20%|0.972|99.95%|1.000|
|ImageNet|79.43%|1.000|48.53%|0.192|54.30%|0.500|
|pokemon|77.83%|0.958|60.73%|0.970|86.67%|1.000|

[1] Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks. Saberi et. al., 2023
[2] Invisible Image Watermarks Are Provably Removable Using Generative AI. Zhao et. al., 2023

Reviewer comment (Experiments):
Employing the same watermarking pattern for every image simplifies the learning process for potential attackers, making it easier to potentially break the watermark.

Author response:
**Response**: Thank you for your helpful observation and question. Although our watermarks share the same patterns for each image, given that the budgets for the watermarks are relatively low, it is still hard for the potential attackers to detect the existence of the watermarks. Even in hidden representation, it is still hard to detect the difference of watermarked images. In Figure 15 in Appendix J, we illustrate and compare the feature representations of a set of unaltered images from one class with their corresponding watermarked versions, along with images from a different class.  The feature extractor is from a Resnet18 Model pre-trained by Contrastive Learning. From the figure we can see that the watermark of DiffusionShield makes almost no difference to the features of images, demonstrating that it is not easy for the potential attacker to discover the watermark in feature space and search for a way to break it. Besides, based on our study in watermark removal attack, even though the offender knows there is watermark, it is hard to remove them, which provides a robust protection for the copyright of the artworks.

Reviewer comment (Evaluation):
The class-conditional evaluation lacks practicality, as real-world offenders typically do not assign specific class labels to the protected training data used for training diffusion models. In the case of unconditional generation, no definitive ground truth criteria are established for identifying which generated samples should be considered as protected. Currently, the authors define these protected images as those filtered using a classifier, which we will refer to as C. This approach raises concerns as the authors have not sufficiently demonstrated the quality of classifier C, and it may not accurately gauge the impact of protected training data on image generation. A more effective approach to assigning ground truth labels to generated images might involve utilizing an influence function to quantify the influence of protected training data on image generation.

Author response:
**Response**: Thanks for your question. We want to clarify that both the class-condition and the classifier in the unconditional case are only used for experimental evaluation. We use them to test the performance of our watermark but do not rely on the classifier or class-condition in real-world scenarios. Below we discuss two viable examples of using the watermark to reveal the infringement in practice:

- First example: Assuming that the unconditional model by data offender generates watermarked and unwatermarked images and the offender releases them. To find the infringement, the data owner does not need to first find the images that are similar to their artworks before watermark detection. They can just use the watermark detector to detect from all the images (both watermark-protected and unwatermarked). Then the watermarked images can be effectively selected. The data owner can choose to engage some human labor to further make sure that the watermarked images are copied from their owned artworks like similar art style and similar object design. Both the watermark and the copied similar things can work as the evidence to reveal and accuse the infringement.
- Second example: The data owner notices some generated images are very similar to their artwork. They doubt that the images come from generated models trained with their protected watermarked protected images. They use the watermark detector to detect the watermarks in the suspect images. If the watermark is detected, this provides the evidence to reveal and accuse the infringement and then protect their copyright.

Besides, we also demonstrate the effectiveness of our classifier in evaluation to show the correctness of our experimental evaluation. We show TPR (True Positive Rate) and FPR (False Positive Rate) of the classifiers in the following Table 9. Here, TPR reflects the classifier's accuracy in correctly labeling generated images from protected classes as protected. Conversely, FPR indicates the proportion of generated images that, although not from protected classes, were mistakenly identified as protected by the classifier. We can see that generally the FPR is very low, meaning that the bit accuracy of unconditionally generated images is mainly measured on the images from the protected classes, thus the results related to the unconditional model are valid. 

In summary, in practical applications, we do not need such classifiers and we only use them for experimental evaluation. The demonstrated ability of classifiers shows that our experimental results are valid. We hope this can clarify your concern about practicality.

[*Table 9*]
|Dataset|TPR|FPR|
|-|-|-|
|CIFAR-10|93.500%|0.667%|
|CIFAR-100|93.000%|0.061%|
|STL-10|89.600%|1.356%|

Reviewer comment (Evaluation):
It is imperative to conduct an analysis of the method's false positive rate, specifically to determine if the method assigns high bit accuracy to samples that should not be considered as protected data. This assessment could be performed using metrics like AUROC.

Author response:
**Response**: Thank you for the suggestion to include FPR and AUROC in our watermark detection accuracy assessment. In the following Table 10, we show the TPR, FPR and AUROC of our DiffusionShield with CIFAR-10 dataset. The threshold for the TPR and FPR is 90%, which means that, if more than or equal to 90% of the detected message match the message encoded to the watermarks on the training data, then the image will be considered as an image generated by a model which was trained on the protected data. From the table we can see that our DiffusionShield has a perfect performance in terms of TPR and FPR, which demonstrates DiffusionShield's powerful effectiveness in safeguarding image copyrights.

[*Table 10*]
|budget=8/255|TPR|FPR|AUROC|
|-|-|-|-|
|CIFAR-10|100%|0%|1|
|CIFAR-100|100%|0%|1|
|STL-10|100%|0%|0.99|
|ImageNet|100%|0%|1|

Reviewer comment (Evaluation):
Regarding Table 4, the method exhibits a relatively low resistance to Gaussian noise. It is essential for the authors to specify the standard deviation (std) of the Gaussian noise applied in this context. Recent research has demonstrated the vulnerability of imperceptible watermarks to attacks involving significant Gaussian noise addition and subsequent denoising using diffusion models. Given that DiffusionShield relies on an imperceptible watermark, the authors should provide a thorough analysis of the watermark's robustness against such attacks, including assessments of bit accuracy and AUROC.

Author response:
**Response**: Thank you for your suggestions on the analysis of the robustness of watermarks. In Table 11 we demonstrate the bit accuracy and AUROC of our DiffusionShield together with two baselines methods under the context of watermark removal attempts by [2]. The results show that DiffusionShield is able to survive and achieve a high bit accuracy and AUROC in the experiments on CIFAR-10 and Pokemon. In comparison, the performance of HiDDeN is much worse. Although generally DFD performs well in defending against the removal method, we note that it requires a much higher budget compared to our method, and larger budgets are more difficult to remove.
Regarding the experiments in Table 4, the standard deviation of the Gaussian Noise is 0.1 and its effect on the images can be visualized in Figure 13 in Appendix J. From the figure we can see that the Gaussian noise considered in Table 4 has a great influence on the quality of images. Even with such an influence on the images, DiffusionShield can still achieve bit accuracy higher than 80%, and AUROC=1, demonstrating the robustness of DiffusionShield against Gaussian Noise.

[*Table 11*]
||DFD||HiDDeN||Ours (8/255)||
|-|-|-|-|-|-|-|
||Bit Acc.|AUROC|Bit Acc.|AUROC|Bit Acc.|AUROC|
|CIFAR-10|91.11%|1.000|82.20%|0.972|99.95%|1.000|
|ImageNet|79.43%|1.000|48.53%|0.192|54.30%|0.500|
|pokemon|77.83%|0.958|60.73%|0.970|86.67%|1.000|

[1] Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks. Saberi et. al., 2023
[2] Invisible Image Watermarks Are Provably Removable Using Generative AI. Zhao et. al., 2023