Hidden in the Noise: Two-Stage Robust Watermarking for Images

Authors: Kasra Arabi, Benjamin Feuer, R. Teal Witter, Chinmay Hegde, Niv Cohen
Primary area: generative models
Keywords: Watermarking

Abstract:
As the quality of image generators continues to improve, deepfakes become a topic of considerable societal debate. Image watermarking allows responsible model owners to detect and label their AI-generated content, which can mitigate the harm. Yet, current state-of-the-art methods in image watermarking remain vulnerable to forgery and removal attacks. 
This vulnerability occurs in part because watermarks distort the distribution of generated images, unintentionally revealing information about the watermarking techniques.

In this work, we first demonstrate a distortion-free watermarking method for images, based on a diffusion model's initial noise.
However, detecting the watermark requires comparing the initial noise reconstructed for an image to all previously used initial noises. 
To mitigate these issues, we propose a two-stage watermarking framework for efficient detection. During generation, we augment the initial noise with generated Fourier patterns to embed information about the group of initial noises we used. For detection, we (i) retrieve the relevant group of noises, and (ii) search within the given group for an initial noise that might match our image. This watermarking approach achieves state-of-the-art robustness to forgery and removal against a large battery of attacks. The project code is available at https://github.com/Kasraarabi/Hidden-in-the-Noise.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6, 5, 6

Reviews and discussion:
<thread>
creation date:
1730366170957

thread_type:
official_review

<thread_object>
creator: reviewer_hapf

<content>
summary:
The paper presents a novel two-stage watermarking framework, named WIND, for images generated by diffusion models. It leverages the initial noise used in the diffusion process as a distortion-free watermarking method. The approach aims to enhance the robustness of watermark detection against both removal and forgery attacks by embedding group identifiers through Fourier patterns, thereby improving the efficiency of watermark retrieval.

strengths:
The paper presents a novel two-stage watermarking framework, named WIND, for images generated by diffusion models. It leverages the initial noise used in the diffusion process as a distortion-free watermarking method. The approach aims to enhance the robustness of watermark detection against removal and forgery attacks by embedding group identifiers through Fourier patterns, thereby improving the efficiency of watermark retrieval.

weaknesses:
(-)  Some parts of the paper are overclaimed. For example, in Lines # 78-84, these contents are already addressed in Tree-ring [1], yet the author seems to want to claim it as their contribution. The introduced method is built upon Tree-ring [1] and [2]. However, the introduction does not discuss these works [1-2].  

(-)  The problem of the previous method and the motivation of this work are not clearly delivered. The relationship between the third and fourth paragraphs is confusing and lacks logic. 

(-)  The effectiveness of the proposed method may be limited to specific types of diffusion models, and its performance on other diffusion models remains untested.

(-)  Although the paper suggests methods to reduce runtime, the need for searching through a large number of initial noises could still pose challenges in real-time applications, particularly on resource-constrained devices.

Some typos:

For example, in lines 88 and 137, add ( ) for the citations.

[1] Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust. In NeurIPS 2023.

[2] Steganalysis on Digital Watermarking: Is Your Defense Truly Impervious? In NeurIPS 2024.

questions:
How does the performance of the WIND framework compare with existing state-of-the-art watermarking techniques when applied to diverse generative models beyond diffusion models?

What specific measures have been taken to ensure that the watermarking process does not compromise the visual quality of the generated images?

Could the authors elaborate on the potential implications of attackers successfully inverting the model and how that might affect the watermarking effectiveness?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730511238697

thread_type:
official_review

<thread_object>
creator: reviewer_jngf

<content>
summary:
The paper introduces the WIND framework for embedding watermarks in AI-generated images. WIND leverages the initial noise of the diffusion process as a distortion-free watermark, embedding group-specific Fourier patterns in the noise to enable efficient watermark verification. The proposed two-stage approach aims to reduce search complexity, improve robustness against various attacks, and optimize verification accuracy.

strengths:
1. The two-stage approach employed in WIND is similar to a multi-level indexing mechanism. Initially, group-specific Fourier patterns narrow down the search scope (Tier-1), and subsequently, an exact matching process with the original noise ensures precise verification (Tier-2). This hierarchical structure not only enhances efficiency but also serves as a novel adaptation of multi-level indexing in watermarking, supporting scalable, high-speed identification in large datasets.
2. The framework demonstrates robustness against several attacks, including forgery, removal, and distortive transformations. The two-tiered verification and Fourier embedding make WIND resilient to various manipulations, ensuring watermark persistence even under adversarial conditions.

weaknesses:
1. While WIND presents a promising adaptation of existing noise-based watermarking techniques, it does not fundamentally diverge from the concepts established in Tree-Ring and RingID. The addition of a two-stage detection process is a novel enhancement but builds on the same theoretical foundation of Fourier-based watermarking in noise. The primary innovation lies in the grouping and search-space reduction, which increases efficiency and robustness but does not introduce a fundamentally new approach to noise-based watermarking.
2. The multi-stage approach, while innovative, introduces complexity that may hinder ease of adoption. Managing group identifiers and applying Fourier-based transformations within the initial noise may pose challenges for large-scale implementations. Additionally, as the number of watermarked images grows, the demand for managing extensive initial noise records could impact computational scalability.

questions:
Please see Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730598254910

thread_type:
official_review

<thread_object>
creator: reviewer_xgaj

<content>
summary:
The paper introduces a watermarking method using initial noise in diffusion processes as a robust, distortion-free way to mark AI-generated images. The authors propose a two-stage framework called WIND, which employs initial noise and Fourier patterns to facilitate efficient watermark detection and withstand forgery and removal attacks.

strengths:
1. The methodology is easy to follow.

2. Evaluates several settings.

3. The two-stage approach efficiently narrows down the search space by using Fourier pattern group identifiers, significantly improving detection runtime and scalability

weaknesses:
1. The idea of picking the initial noise for diffusion is not new. And this method changes initial noise a lot compared to Tree-Ring watermark.

2. The evaluation of robustness if not comprehensive.

3. Potential vulnerability to more advanced attacks

questions:
1. The organization of the paper appears somewhat disorganized. For example, Section 5.2 and the Algorithms section should be relocated to precede Section 5, which discusses the experiments. This adjustment would provide better coherence, as watermarking non-synthetic images is an integral part of the methodology.

2. The evaluation of different watermarking methods is somewhat limited. The focus of this work is primarily on watermarks that involve selecting initial noise for the diffusion process, while other watermarking techniques, such as HiDDeN, are excluded. The authors should provide an explanation for this choice to give context to the scope of their evaluation.

3. While the paper makes significant claims regarding robustness, the robustness evaluation itself is not comprehensive. Beyond image transformations and regeneration attacks, there are numerous adaptive attacks where the adversary possesses greater capability. Examples include transfer-based, query-based, and white-box attacks such as WEvade. These types of evaluations should be considered to strengthen the robustness analysis.

4. How is threshold $\tau$ determined in the experiments?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730683155782

thread_type:
official_review

<thread_object>
creator: reviewer_7vcu

<content>
summary:
1. This paper introduces a novel distortion free diffusion model based watermarking method for images. The method is distortion free as it employs initial random noise that is already used by the model versus previous methods/models that distort the distribution of generated images which negatively affects the robustness of those models. 
2. The method tackles both watermark removal attacks and forgery attacks.
3. To make the watermark detection process more efficient, during generation grouped noises are augmented with distinct fourier patterns/identifiers. Therefore during detection, first the group identifier is recovered and then the exact match within it is found making the detection much faster versus comparing the noise to all previously used initial noises.

strengths:
originality and significance: 
- The method presented by the paper is able to tackle both forgery and removal attacks unlike previous works.
- Shows that initial noise used by diffusion models can be a watermark and improves robustness.

quality and clarity: The paper is generally well written - maintains a good flow of ideas, very few typos and explains concepts and background where necessary

weaknesses:
1. Qualitative results only cover Watermark Detection Accuracy in Table 1 but it is important to add other detection metrics such as TP/AUC etc for a stronger evaluation and to be more convincing. 
- Also, no ablation on perturbation strenghts vs detection accuracy

2. Section 5.2 Watermarking non-synthetic images: Issues with this section
- Quantative Evaluation - Only evaluates FID, but does not touch upon more image similarity and quality metrics of watermarked images like CLIP score, SSIM and PSNR. 
- Qualitative: Lacks side by side comparison of watermarked images results from different models
3. Experiments: Details of how the experiments were performed are lacking. Ex: how was the detection threshold (picked?
4. Supplementary (nit, optional): Would be good to see side by side comparisons of competing methods on runtime or atleast approximate numbers that demonstrate the factor by which this method is slower.

questions:
Please see the weaknesses section above.

Questions:
1. In Section 5.2 Watermarking non-synthetic images: It does not seem like the image quality is actually close to the non-watermarked images - In images which contain text in Fig 5 and Fig 9, the watermarked images are not able to retain the text seen in the original non-watermarked image (ex: last row of images with traffic text directions and text on the aircraft). Are there ways in which this problem can be resolved/handled?
2. It seems like this method relies on the (approximate) invertibility property of DDIM. Is so, please mention this in section 2.2. Fig 1 mentions it but it would be nice to also add it in 2.2 for clarity if this is the case.   
3. Is there any other diffusion model besides SD v2 that this method was tried with?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730712108844

thread_type:
official_review

<thread_object>
creator: reviewer_qcsj

<content>
summary:
This paper presents an innovative watermarking method that uses the initial noise in diffusion models as a distortion-free watermark, showcasing its potential application in addressing societal challenges such as deepfakes. The two-stage watermarking framework proposed by the authors incorporates Fourier patterns during the generation process to enhance the robustness of the initial noise and effectively retrieves relevant noise groups during detection.
Despite the significant contributions, this paper has several shortcomings. Firstly, the structure and logical flow of the paper could be improved, particularly in the abstract section. Additionally, some technical terms and concepts should be explained in more detail to aid reader comprehension. Furthermore, the depth of the results analysis could be enhanced, suggesting an exploration of the potential reasons for changes in results and their impact on the overall robustness of the method.

strengths:
Strengths：
1.	The two-stage framework proposed in this paper addresses the vulnerabilities of current watermarking techniques, effectively demonstrating the combination of deep learning and traditional watermarking methods. This innovation offers a fresh perspective in the watermarking field.
2.	The research provides specific solutions, emphasizing the robustness of the method against various attacks. This contribution not only enhances the practical applicability of the paper but also lays a valuable foundation for future research in the area.

weaknesses:
Weakness:
1. The research motivation is unclear. In the Abstract section, the structure could be further optimized. There is a lack of smooth logical connection between the two paragraphs of the abstract. The first paragraph primarily discusses deepfakes and the limitations of current watermarking methods, but there is no clear transition when moving to the new framework in the second paragraph. This makes it difficult for readers to understand why the new method is necessary and how it relates to the issues discussed earlier.
2. The interpretation of the experimental results is insufficient. Descriptions of the tables and figures are very clear, but a deeper analysis of these results could enhance the paper. For example, in Table 2, while the changes in similarity are displayed, it would be beneficial to explore the potential reasons behind these changes and how they impact the overall robustness of the method.
3. In the experimental section, while images generated by your framework are presented, there is a lack of quantitative analysis of image quality. 
4. There is a lack of the performance across a broader range of inference steps beyond the 50-step setting currently used. Testing various inference step counts (e.g., 20, 50, 100, and 200 steps) can help determine how the step count affects model performance in terms of robustness.

questions:
See weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731134741119

thread_type:
official_review

<thread_object>
creator: reviewer_yvzd

<content>
summary:
This paper investigates reverse vulnerabilities in diffusion watermarking methods to forgery and removal attacks. Specifically, the authors propose WIND, a two-stage watermarking framework that first embeds a unique group identifier within the initial noise of a diffusion model using Fourier patterns and then searches the reconstructed noise within the noise group to identify the match. Empirical results show that WIND can achieve better detection performance and robustness against various attacks.

strengths:
(1) The paper is organized by some theoretical analysis and empirical evaluations. Furthermore, the motivations and methodology are clearly stated overall.

(2) The work leverages the inherent initial noise of diffusion models as a watermark without external watermarking processes that might degrade image quality. It is innovative that grouped noise patterns are used with Fourier-based group identifiers, which is effective and robust. 

(3) The search efficiency is heavily relied on the number of watermarks N and noise group M, which is partially solved by proposing a faster detection variant.

(4) The effectiveness of the proposed watermarking generalized to inpainting task.

weaknesses:
(1) The evaluations of the proposed approach are limited, where only one diffusion model is investigated. Moreover, some experimental settings are missing.

(2) Proof of Theorem 4.1 seems not entirely convincing. Although it is stated as a mathematical result, the actual evidence is more empirical rather than rigorous. For example, the paper’s results show low false positives, but the proof does not provide a probability bound for these occurrences.

More details can be seen in Questions.

questions:
(1) The authors only evaluate the method on Stable Diffusion v2. The generalization is unknown to other diffusion models, such as Stable Diffusion XL, consistency models and transformer-based diffusion architectures. Moreover, it is unclear how detection would perform at higher generation resolutions. And what is the prompt template for the generation? 

(2) Can WIND be adapted to GAN-based generative models when DDIM inversion is replaced by other reconstruction approaches?

(3) The inversion step could affect both reconstruction speed and performance. The trade-off between the inversion step, watermarking overhead, and detection accuracy is not discussed.

(4) It could be better to include group index embedding in Algorithm 1.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732225966596

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Response (1/2)

comment:
We thank all the reviewers for their valuable feedback. Our work introduces WIND, a two-stage watermarking method for diffusion models, using initial noise as a distortion-free watermark that is robust to removal and forgery attempts. We appreciate that the reviewers found our techniques innovative and novel (YVZd, hApF, jnGF), acknowledged the effectiveness and strong results of our method (YVZd, QcSJ, 7vcU, XGAj, jnGF,hApF), recognized our paper as clear and well written (7vcU, YVZd), and even foundational for future research in the area (QcSJ). Following your suggestions, we highlight further improvements:

`(A)` We extend the evaluation of our method's applicability to images generated by other models. We find that WIND can also supply strong robustness to images not generated by DDIMs while maintaining image quality.

`(B)` We validate the robustness of our method in additional settings, including new attack types and parameter regimes.

`(C)` Additional qualitative results on the image quality produced by our method and additional quantitative results.

Answers to individual reviewer concerns are detailed below. We would be very happy to keep the discussion going, addressing any points that remain unclear, or any new suggestions. Thanks again for your suggestions!

**`(A)` Applicability to image generated by other models**

Many reviewers asked about the applicability of methods to other types of models. While our technique relies on the inversion to approximate the initial noise, we can still use our inpainting capability (please see Sec. 4.3) to watermark images generated from other models.
  

We report below the robustness of this watermark using a set of 100 noises:

  
| Clean | Rotate | JPEG | C&S | Blur | Noise | Bright | Avg $\uparrow$ |
|-------|--------|------|------|------|-------|--------|-------|
| 1.000 | 1.000 | 1.000| 0.880| 1.000| 0.950 | 0.950 | 0.969 |

And the image quality it gives:

| Method | FID $\downarrow$ |
|-------------|------------------|
| DwtDctSvd | 25.01 |
| RivaGAN | 24.51 |
| Tree-Ring | 25.93 |
| RingID | 26.13 |
| WIND | **24.33** |

  

This allows our method to watermark images generated by other model types, as well as natural images. 
  

**`(B)` New attack types and new parameter regimes**


We extend our results to explore new attack types and other parameter regimes (all these experiments have been done with $N$ = 10000 and $M$ = 2048), including:
  
  
`1.` We explore a different number of inference steps:
  

| Steps | Clean | Rotate | JPEG | C&S | Blur | Noise | Bright | Avg $\uparrow$ |
|-------|-------|--------|-------|-------|-------|-------|--------|--------|
| 20 | **1.000** | 0.780 | **1.000** | 0.880 | 0.920 | **1.000** | 0.960 | 0.934 |
| 50 | **1.000** | **0.930** | **1.000** | **0.940** | **1.000** | 0.980 | 0.980 | 0.976 |
| 100 | **1.000** | **0.930** | **1.000** | **0.940** | **1.000** | **1.000** | 0.990 | **0.980** |
| 200 | **1.000** | 0.850 | **1.000** | **0.940** | **1.000** | **1.000** | **1.000** | 0.970 |

`2.`  We report the success rate of additional attacks; including transfer-based, query-based, and white-box methods. Specifically, we employ the WeVade white-box attack [1], the transfer attack described in [3], a black-box attack utilizing NES queries [4], and a random search approach discussed in [2], adopted to attempt watermark removal. The success rates of these attacks are detailed in the table below:
   
| WeVade | Random Search | Transfer Attack | NES Query |
|--------|---------------|-----------------|-----------|
| 1% | 2% | 3% | 2% |

`3.`  We evaluate against a stronger (more iteration) regeneration attack, by iteratively using the attack we reported in the paper [5]. 

  
| Iteration | Cosine Similarity | Detection Rate |
|-----------|-------------------|----------------|
| 10 | 0.493 | 100% |
| 20 | 0.342 | 100% |
| 30 | 0.243 | 100% |
| 40 | 0.170 | 100% |
| 50 | 0.121 | 100% |

**`(C)` Additional quantitative and qualitative results on the image quality produced by our method**


`1.`  We add quantitative results for the preservation of image quality by our method. 
To further assess the effect of WIND watermark on image quality we report the CLIP score [6] before and after watermarking. ($N$ = 10000 and $M$ = 2048)

| CLIP Before Watermark | CLIP After Watermark |
|------------------------|----------------------|
| 0.366 | 0.360 |

Results indicate that adding the watermark has a negligible effect on the CLIP score.

Additional qualitative comparisons can be found in Fig. 12,13,14,15,16,17 in the revised manuscript.

`2.`  We evaluate the AUC and True Positive (TPR@1\%FPR) rate:
     

| AUC | TP@1% |
|-------|---------|
| 0.971 | 1.000 |



All the above results, as well as answers to specific reviewer concerns, are also added to the revised version of the manuscript.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732508423385

thread_type:
official_comment

<thread_object>
creator: area_chair_muzq

<content>
comment:
Hi Reviewers,

We are approaching the deadline for author-reviewer discussion phase. Authors has already provided their rebuttal. In case you haven't checked them, please look at them ASAP. Thanks a million for your help!
</content>
</thread_object>

</thread>
<thread>
creation date:
1733255442328

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Paper Revisions

comment:
We thank all reviewers for their time and effort invested in reviewing our paper. Below, we provide a summary of the main updates we made:

- Revised the introduction to give a more comprehensive overview of prior works and their relation to our method
- Added results demonstrating our robustness to additional attacks
- Extended the evaluation of our watermarking technique when applied to images not generated by a diffusion model
- Incorporated additional metrics to assess image quality

Once again, we sincerely thank the area chair and all reviewers!
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper is working on image watermarking. Authors proposed a distortion-free watermarking method for images based on a diffusion model's initial noise. Authors proposed a two-stage watermarking framework for efficient detection. Authors showed that proposed achieved SOTA robustness to forgery and removal against a large set of attacks. 

This paper was reviewed by 6 reviewers and got mixed scores as five 6, one 5.

Strength and weaknesses given by reviewers before rebuttals are as follows (notes that different reviewers has different perspectives of the paper, so conflicts in strength and weaknesses might happen):

Strength: 1) paper is well written; 2) proposed method is novel, effective and robust; 3) lays a valuable foundation for future research;

Weaknesses: 1) evaluation is limited; 2) Proof of Theorem 4.1 seems not entirely convincing; 3) research motivation is unclear; 4) The interpretation of the experimental results is insufficient; 5) lack of quantitative analysis of image quality; 6) lack of the performance across a broader range of inference steps; 7) important to add other detection metrics such as TP/AUC etc for a stronger evaluation and to be more convincing; 8) Details of how the experiments were performed are lacking; 9) The idea of picking the initial noise for diffusion is not new; 10) The evaluation of robustness if not comprehensive; 11) Potential vulnerability to more advanced attacks; 12) does not introduce a fundamentally new approach to noise-based watermarking; 12) The multi-stage approach, while innovative, introduces complexity that may hinder ease of adoption; 13) Some parts of the paper are overclaimed; 14) The problem of the previous method and the motivation of this work are not clearly delivered; 15) The effectiveness of the proposed method may be limited to specific types of diffusion models, and its performance on other diffusion models remains untested; 16) large number of initial noises could still pose challenges in real-time applications; 

During author-reviewer discussion phase:

Reviewer YVZd (rating 6) mentioned that their concerns on theoretical proof, training and evaluation efficiency, generalization to other models are addressed. but higher-resolution application or how the proposed method performs on consistency models or transformer-based diffusion models is unknown. However, the reviewer increased the rating to 6.

Reviewer QcSJ (rating 6) didn't reply during rebuttal.

Reviewer 7vcU (rating 6) raised the score to 6, but still has concerns on image quality. 

Reviewer XGAj (rating 6) suggested most of their concerns are partially addressed. but also mentioned while they find the evaluation of different watermarking methods not bad, it remains somewhat limited. however they increased rating to 6.

Reviewer jnGF (rating 5) didn't reply in the discussion.

Reviewer hApF (rating 6) suggested that the current version is much clearer, and they acknowledge the novelty and insights of this work and increase rating to 6. 

Reviewers didn't give any comments during the reviewer-AC discussion phase.

Given 5 reviewers are positive about this paper and the only review with rating 5 didn't involve in the discussion and AC checked their concerns were addressed. So AC decided to accept this paper.