Tree-Rings Watermarks: Invisible Fingerprints for Diffusion Images
Authors: Yuxin Wen
John Kirchenbauer
Jonas Geiping
Tom Goldstein
Venue: NeurIPS 2023 poster (2023)
Keywords: Diffusion Model
Watermark
Privacy and Security

Abstract: Watermarking the outputs of generative models is a crucial technique for tracing copyright and preventing potential harm from AI-generated content. In this paper, we introduce a novel technique called Tree-Ring Watermarking that robustly fingerprints diffusion model outputs.  Unlike existing methods that perform post-hoc modifications to images after sampling, Tree-Ring Watermarking subtly influences the entire sampling process, resulting in a model fingerprint that is invisible to humans. The watermark embeds a pattern into the initial noise vector used for sampling. These patterns are structured in Fourier space so that they are invariant to convolutions, crops, dilations, flips, and rotations.  After image generation, the watermark signal is detected by inverting the diffusion process to retrieve the noise vector, which is then checked for the embedded signal.  We demonstrate that this technique can be easily applied to arbitrary diffusion models, including text-conditioned Stable Diffusion, as a plug-in with negligible loss in FID. Our watermark is semantically hidden in the image space and is far more robust than watermarking alternatives that are currently deployed.

Treg-Rings Watermarks: Invisible Fingerprints for Diffusion Images
Yuxin Wen, John Kirchenbauer, Jonas Geiping, Tom Goldstein University of Maryland (ywen, jkirchen, jgeiping, tomg) @umndedu
Abstract
Watermarking the outputs of generative models is a crucial technique for tracing copyright and preventing potential harm from Al-generated content. In this paper, we introduce novel technique called Tree-Riie Ring Watermarking that robustly fin- gerprints diffusion model outputs. Unlike existing methods that perform post-hoc modifications to images after sampling, Tree-Ring Watermarking subtly influences the entire sampling process, resulting in model fingerprint that is invisible humans. The watermark embeds pattern into the initial noise vector used for sampling- These patterns structured in Fourier space so that they invari- ant to convolutions, crops, dilations, flips, and rotations. After image generation, the watermark signal is detected by inverting the diffusion process to retrieve the noise vector, which is then checked for the embedded signal. We demonstrate that this technique can be easily applied to arbitrary diffusion models, includ- ing text-conditioned Stable Diffusion, as a plug-in in with negligible loss in FID. Our watermark is semantically hidden in the image space and far more robust than watermarking alternatives that are currently deployed. Code is available at https: / thub. www./om////////e Somm/YxxinWenRieee wwwckkkktteereeeeeeeeeee watermark.
Introduction
The development of diffusion models has led surge in image generation quality. Modern text- to-image diffusion models, like Stable Diffusion and Midjourney, are capable of generating wide variety of novel images in an innumerable number of styles. These systems are general-purposs image generation tools, able to generate new art just as well photo-realistic depictions of fake events for malicious purposes.
The potential abuse of text-to-image models motivates the development of watermarks for their outputs. A watermarked image generated image containing signal that is invisible humans and yet marks the image as machine-generated Watermarks document the use of image generation systems, enabling social media, news organizations, and the diffusion platforms themselves to mitigate harms or cooperate with law enforcement by identifying the origin of an image [Bender al., 2021, Grinbaum and Adomaitis, 2022].
Research and applications of watermarking for digital content have a long history, with many approaches being considered over the last decade [O Ruanaidh and Pun, 1997, Langelaar al., 2000]. However, so far research has always conceptualized the watermark as minimal modification imprinted onto existing image [Solachidis and Pitas, 2001, Chang et al., 2005, Liu et al., 2019, Fei et al., 2022]. For example, the watermark currently deployed in Stable Diffusion [Cox et al., 2007], works by modifying a specific Fourier frequency in the generated image.
The watermarking approach we propose in this work is conceptually different: This is the first watermark that is truly invisible, as no post-hoc modifications made to the image. Instead,
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Generation
Watermarking Fourier Space
Watermarked K
Watermarked Image
FFT
IFFT
DDIM
Teddy
Predefined Key:
Detection
Attack
Distance tc
Inverted Fourier Space
Strong Perturbation
DDIM Inversion
Predict
< T
(unpty
Figure 1: Pipeline for Tree-Ring Watermarking. A di diffusion model generation is watermarked and later detected through sung-paterms in the Fourier space of the initial noise vector.
the distribution of generated images is imperceptibly modified and an image is drawn from this modified distribution. This way, the actual sample carries watermark in the classical additive sense, however an algorithmic analysis of the image can detect the watermark with high accuracy. From a more practical perspective, the watermark materializes in minor changes in the potential layouts of generated scenes, that cannot be distinguished from other random samples by human inspection.
This new approach to watermarking, which call Tree-Ring Watermarking based the patterns im- printed into the Fourier space of the noise vector the diffusion model, can be easily incorporated into existing diffusion model APIs and is invisible per-sample basis. Most importantly, Tree-Ring Wa- termarking is far more robust than existing methods against large battery of common image transfor- mations, such crops, color jitter, dilation, flips, rotations, noise. Tree-Ring Watermarking requires no additional training or finetuning to implement, and the watermark can only be detected by parties in control of the image generation model. We validate the watermark in number of tests, measuring negligible impact on image quality scores, high robustness to transformations. the low false-positive rate in detection, and usability for arbitrary diffusion models both with and without text conditioning.
2 Related Work
Diffusion Models Diffusion Models, arising out of the score-based generative models of the formalism of Song and Ermon [2019, 2020], are the currently strongest models for image generation [Ho et al., 2020, Dhariwal and Nichol, 2021]. Diffusion models are capable of sampling new images inference time by iteratively processing an initial noise map. The most prominent sampling algorithm in deployment is DDIM sampling [Nichol and Dhariwal, 2021 1] without additional noise, which can generate high-quality images in fewer steps than traditional DDPM sampling. Diffusion models are further accelerated for practical usage by optimizing images only in the latent space of pre-trained VAE, such as latent diffusion [Rombach et al., 2022].
Watermarking Digital Content Strategies to imprint watermarks onto digital content, especially images, have long tradition in computer vision. Approaches such Boland [1996], Cox al. [1996], O' Ruanaidh and Pun [1997] describe traditional watermark casting strategies based on imprinting a watermark in suitable frequency decomposition of the image, constructed through DCT, DWT, Nourier-Mellin, or complex wavelet transformations. These frequency transformations all share the beneficial property that simple image manipulations, such as translations, rotations, and resizing are easily understandable and watermarks can be constructed with robustness to these transformations mind. A fair evaluation of watermarking approaches appears Pitas [1998], Kutter and Petitcolas [1999], which highlight the importance of measurement of false-pon aalse-oositive rates

---

for each strategy and ROC-curves under attack through various image manipulations. Work continues on imprinting watermarks, with strategies based on SVD decompositions [Chang et al., 2005], Radon transformations [Seo et al., 2004] and based on multiple decompositions [AI-Haj, 2007].
Fingerprinting and Watermarking Generative Models The development of modern deep neural networks opened up new possibilities for "deep" watermarking. Hayes and Danezis [2017] and Zhu et al. [2018] propose strategies to learn watermarking end-to-end, where both the watermark encoder and the watermark decoder are learned models, optimized via adversarial objectives maximize transmission and robustness [Zhang et al., 2019]. Notably these approaches still work like a traditional watermark in that the encoder imprints post-hoc signal onto given image- however the type of imprint is now learned. We refer to Wan et al. [2022] for an overview. A recent improvement is two-stage processes like Yu al. [2022], where the trained encoder is used to imprint the watermark onto the training data for a generative model. This leads to a trained generative model where the watermark encoder is "baked in" to the model, making it easier generate watermarked data. The Stable Signature of Fernandez et al. [2023], applies this idea to latent diffusion models by finetuning the latent decoder based on a pre-trained watermark encoder. Zhao et al. [2023] similarly train on watermarked data for unconditional diffusion models.
Existing image watermarking approaches first learn watermark signal and then learn to either embed it into generated data or the generating model. This pipeline stands in contrast to watermarking approaches for language models such as Kirchenbauer et al. [2023]. There, no training is necessary to generate watermarked data and the output distribution of the generative model is altered to encode a watermark into generated data in a distributional sense. In the same vein, we propose an approach to alter the output distribution of diffusion models to effectively watermark their outputs. As discussed, this has a number of advantages, in comparison to related work we especially highlight that no training is necessary, that the watermark works with existing models, and that this is the first watermark that does not rely on minor modification of generated images. In this sense, this the first watermark that is really "invisible", see Appendix Figure 6.
We note in passing that watermarking the output of generative models is not be confused with the task of watermarking the weights of whole models, such as in Uchida et al. [2017]. Zhang al. [2018], Bansal al. [2022], who are concerned with identifying and fingerprinting models for intellectual property reasons.
2.1 Diffusion Models and Diffusion Inversion
We first introduce the basic notation for diffusion models and DDIM sampling [Ho et al., 2020, Song and Ermon, 2020, Dhariwal and Nichol, 2021]. A forward diffusion process consists of T steps of the noise process a predefined amount of Gaussian noise vector to a real data point zo E q(x), where ((x) is the real data distribution, specifically:
[( 1) -Nx;; /1 B, 3r1), for i E 1..,,
where Br E (0, 1) is the scheduled variance at step t. The closed-form for this sampling is
Tt = varzo + vI- - āgE,
(I)
where, a; II 31).
For the reverse diffusion process, DDIM [Song and Ermon, 2020] is an efficient deterministic sampling strategy, mapping from a Gaussian vector FT ~ NO, 1) to an image zo E q(x). For each denoising step, a learned noise-predictor cø estimates the noise ca (2x) added zo. According to Equation (1), we can derive the estimation of In as:
It /1 (xco(
Voi
Then, we add the estimated noise to èo to find Ft-1:
Iz- va + /1 11-aar-1e9la
We denote such recursively denoising process from FT to xo Do Do(xT).
3

---

However, given the learned model (0(z)), it is also possible to move in the opposite direction' Starting from an image Fo. Dhariwal and Nichol [2021] describes an inverse process that retrieves which maps to an image in close to zo through DDIM, where 2o
an initial noise vector 'D (2T,00) zg. This inverse process depends on the assumption that Iz-1 It R at+1 Zt- Therefore, from Ft - I++1, we follow:
F+ v0 0(-00(1
We denote the whole inversion process from a starting real image To to 2T = (zO))
In this work, re-purpose DDIM inversion Dj for watermark detection. Given generated image To with a starting noise Ir, we apply DDIM inversion to find &r. We empirically find DDIM's inversion performance to be quite strong, and žr IT- While may not be surprising that inversion is accurate for unconditional diffusion models, inversion also succeeds well-enough for conditional diffusion models, even when the conditioning is not provided. This property of inversion will be exploited heavily by our watermark below.
Method
In this section, we provide a detailed description of each layer of Tree-Ring Watermarking.
3.1 Overview of Tree-Ring Watermarking
Diffusion models convert an array of Gaussian noise into a clean image. Tree-Ring Watermarking chooses the initial noise array that its Fourier transform contains carefully constructed pattern near its center. This pattern is called the "key." This initial noise vector is then converted into an image using the standard diffusion pipeline with modifications. To detect the watermark in image, the diffusion model is inverted using the process descrihed in Section 2.1 to retrieve the original noise array used for generation. This array is then checked to see whether the key is present.
Rather than imprint the key into the Gaussian array directly, which might cause noticeable patterns in the resulting image, we imprint the key into the Fourier transform of the starting noise vector. We choose a binary mask M, and sample the key k. CIMI, As such, the initial noise vector Zr E R. can be described in Fourier space as
K (N 1)
if E M otherwise.
F(IT):
For reasons described below, we choose M as a circular mask with radius r centered on the low- frequency modes.
At detection time, given an image z/1 the model owner can obtain an approximated initial noise vector 4' through the DDIM inversion process: x² = (16)) The final metric is calculated as the L1 distance between the inverted noise vector and the key in the Fourier space of the watermarked area M, i.e.
ddetection distance
Σ k -((t) IEM
M
and the watermark is detected if this falls below tuned threshold The process described above straightforward. However, its success depends strongly on the construction the "key" pattern, which we discuss below.
3.2 Constructing Tree-Ring Key
We watermark images by placing "key" pattern into the Fourier space of the original Gaussian noise array. Our patterns can exploit several classical properties of the Fourier transform for periodic signals that we informally state here.
'oo reduce confusion we will always describe the generative diffusion process that goes from Py to 2p as the "reverse process". We use "inverse process" to denote the estimation of the noise vector zr from the final output zo.

---

A rotation in pixel space corresponds to rotation in Fourier space.
translation in pixel space multiplies all Fourier coefficients by a constant complex number.
dilation/compprssionn in pixel space corresponds to compression/dilation in Fourier space.
Color jitter in pixel space (adding constant to all pixels in channel) corresponds to changing the magnitude of the zero-frequency Fourier mode.
A number of classical watermarking strategies rely on watermarking in Fourier space and exploit similar invariances [Pitas, 1998, Solachidis and Pitas, 2001]. Our watermark departs from classical methods by applying Fourier watermark to random noise array before diffusion takes place. Curiously, we will observe below that the invariant properties above are preserved in IT even when image manipulations are done in pixel space of Zig.
In addition to exploiting the invariances above, the chosen key should also be statistically similar to Gaussian noise. Note that the Fourier transform of a Gaussian noise array is also distributed as Gaussian noise. For this reason, choosing a highly non-Gaussian key may cause distribution shift that impacts the diffusion model.
We consider three different types of keys, with the respective benefits of each pattern being demon- strated in subsequent experimental sections. We believe there are numerous other interesting and practical types that can be explored in future work.
Tree-Ringe : We choose the mask to be a circular region to preserve invariance to rotations in image space. The key is chosen to be array of zeros, which creates invariance to shifts, crops, and dilations. This key is invariant to manipulations, but at the cost of departing severely from the Gaussian distribution. It also prevents multiple keys from being used distinguish between models.
Tree- Rond We draw the a fixed key k- from a Gaussian distribution. The key has the same iid Gaussian nature the original Fourier modes of the noise array, and so we anticipate this strategy will have the least impact on generation quality. This method also offers the flexibility for the model owner possess multiple keys. However, it is not invariant to make image manipulations.
Tree- Ring We introduce pattern comprised of multiple rings, and constant value along each ring. This makes the watermark invariant to rotations. We choose the constant ring values from Gaussian distribution. This provides some invariance to multiple types of image transforms, while also ensuring that the overall distribution is only minimally shifted from an isotropic Gaussian.
3.3 Deriving P-values for Watermark Detection
A key desideratum for reliable watermark detector is that it provide an interpretable P-value that communicates to the user how likely is that the observed watermark could have occurred in natural image by random chance. In addition to making detection results interpretable, P-values can be used to set the threshold of detection, i.e., the watermark is "detected" when p is below a chosen threshold a. By doing so, one can explicitly control the false positive rate a, making false accusations statistically unlikely.
To this end, we construct a statistical test for the presence of the watermark that produces rigorous P-value. The forward diffusion process is designed to map images onto Gaussian noise, and so assume null hypothesis in which the entries in the array xty obtained for natural image Gaussian. We find that this assumption holds quite well in practice, see Figure 7.
For any test image 2/6 we compute the approximate initial vector x'r and then set y F((T)) We then define the following null hypothesis
Hg y is drawn from Gaussian distribution (0, o
(4)
Here, o² is an unknown variance, which we estimate for each image? using the formula o² M Ziem 1ya12. To test this hypothesis, define the score
1
1 0
Σ JEM
k
"Our statistical test is only sensitive to Tree- Tree-Ringaau and pathological case that zz 0 for watermarked images resulting in overly conservative/large P P-values.
Tree- Free-Rinzaon. results in the
5

---

No Watermark
Watermarked
Attacked
"Anime art of day in Shenandoah National Park"
P-value = 0.27
3.7 3.73e-60
7.41e-16
"Synthwave style artwork a person is kayaking in Acadia National Park
0.15
1.38e-19
1.51e-8
"An astrenaut riding horse in Zion National Park"
0.91
9.91e-51
2.90c-05
"A painting of Yosemite National Park in Van Gogh style"
0.41
1.22e-35
9.46e-07
Figure The qualitative results show three types of images: non watermarked, Tree- Diee-Ringxing watermarked, and attacked watermarked images. A P-value is provided below each image, which corresponds to the probability of the detected watermark structure occurring by random chance. From top to bottom, the watermarked images are attacked by color jitter with brightness factor of 6, Gaussian blur with an 8 8 filter size, Gaussian noise with 0.1, and 180 rotation, respectively.
When Ho is true, the distribution of is exactly a noncentral x² distribution [Patnaik, 1949], with |M| degrees of freedom and non-centrality parameter À = 32 Σ, 1k
We declare an image to be watermarked if the value of 17 is too small to occur by random chance.
The probability of observing value as small as rj given by the cumulative distribution function T xa of the noncentral x2 distribution:
p Pr Pr ( S 7/ (o) = (2))
2 is a standard statistical function [Glasserman, 2003], available in scipy and many other statistics libraries.
We show qualitative examples of the proposed watermarking scheme and accompanying P-values in Figure 2. For each prompt, we show the generated image with and without the watermark. and

---

also a watermarked image subjected to a transformation. For each image, we report P-value. As expected, these values are large for watermarrrrrked images, and small (enabling rejection of the null hypothesis) when the watermark is present. Transformations reduce the watermark strength as reflected the increased P-value.
Experiments
We perform experiments two common diffusion models to measure the efficacy and reliability of the Tree-Ring Watermarking technique across diverse attack scenarios. Furthermore, we carry out ablation studies to provide an in-depth exploration of this technique.
4.1 Experimental Setting
We employ Stable Diffusion-v2 [Rombach et al., 2022], an open-source, state-of-the-art latent text-to- image diffusion model, along with a 256 x 256 ImageNet diffusion model [Dhariwal and Nichol, 2021]. In the main experiment, we 50 inference steps for generation and detection for both models. For Stable Diffusion, we use the default guidance scale of 7.5, and we use an empty prompt for DDIM inversion, emulating that the image prompt would be unknown detection time. The watermark radius r we use is 10. Later, we conduct more ablation studies on these important hyperparameters. All experiments are conducted on single NVIDIA RTX A4000.
In our comparative analysis, we consider four baselines: two training-fre free methods, DwtDet and DwtDetSvd [Cox al., 2007]; pre-trained GAN-based watermarking model, RivaGAN [Zhang et al., 2019, Goodfellow et al., 2014]; and a pre-trained watermarking VAE decoder, used exclusively for Stable Diffusion model experiments, named Stable Signature [Fernandez al., 2023]. However, these baseline methods are designed for steganography, which conceals a target bit-string within an image. To ensure fair comparison with our exclusively watermarking method, we employ the distance between the decoded bit-string and the target bit-string (Bit Accuracy) as the measurement metric. The approach of Cox al. [2007] is currently deployed as a watermark mechanism in Stable Diffusion".
4.2 Benchmarking Watermark Accuracy and Image Quality
To benchmark the effectiveness of the watermark, primarily report the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, and the True Positive Rate when the False Positive Rate is at 1 %, denoted as TPR @ FPR. To demonstrate the generation quality of the watermarked images, we assess the Frechet Inception Distance (FID) [Heusel et al., 2017] 7] for both models. Additionally, for the Stable Diffusion model, we also evaluate the CLIP score [Radford et al., 2021] between the generated image and the prompt, as measured by OpenCLIP-ViT/C [Cherti create 1, 000 watermarked and 1, 000 unwatermarked
al., 2022]. For AUC and TPR @ %FPR, images for each run. For FID, we generate 5, 000 images for Stable Diffusion and 10, 000 images for the ImageNet Model. The FID of Stable Diffusion evaluated on the MS 1-S-COOO-201 training dataset [Lin et al., 2014], and the FID of the ImageNet Model is gauged on the ImageNet- 1k training dataset [Deng al., 2009]. All reported metries are averaged across runs using different random seeds following this protocol.
In Table 1, present the main experimental results for Stable Diffusion and the ImageNet model. In the clean setting, all baselines except DwtDet and all Tree-Ring Watermarking variants are strongly detectable. Tree-RingRoud and Tree-Ringe Ringgiig show negligible impact on the FID and no impact on the CLIP score.
4.3 Benchmarking Watermark Robustness
To benchmark the robustness of our watermark, we focus documenting its performance under 6 prevalent data augmentations utilized as attacks. These include 75" rotation, 25% JPEG compression, 75% random cropping and scaling, Gaussian blur with an 8 8 filter size, Gaussian noise with a 0.1, and color jitter with a brightness factor Additionally, we conduct ablation studies to
wtt.aa/wwwwooooo//aaaaacctttttta py4L69
7

---

Table 1: Main Results. T@1%F represents TPR IIPP. We evaluate watermark accuracy in both benign and adversarial settings. Adversarial here refers average performance over a battery of image manipulations. extended version with additional details and standard error estimates can be found in Appendix Table 3.
AUC/T@ %%
AUCTT11%F
Model
Method
FID +
CLIP Score
(Clean)
(Adversarial)
DwtDet
0.974/ 0.624
0.574/0.099
25. 25.10,er 10,09
0.362, ,000
Stable Diff.
DwtDetSvd
1.000 / 1.000
0.702 / 0.262
25.01.08
0.359 .000
FID 25.29
RivaGAN
0.999 0.999
0.854/ 0.448
24.51.17
0.361.000
CLIP Score
Tree-Ringzen
0.999 / 0.999
0.963 / 0.715
26.56,07
0.356,000
= 0.363
Free-RingRand
1.000 1.000
0.918 / 0.702
25.47,05
0.363,00 .001
Tree-RingRings
1.000 1.000
0.975 / 0.694
25.93.13
0.364.00m
DwtDet
0.899 0.244
0.536 0.037
17.77.01
DwiDetSvd
1.000 1.000
0.713 / 0.187
18.55.02
ImageNet
RivaGAN
1.000 1.000
0.882 0.509
18.70,02
FID 17.73
Tree-Ringzoor
0.999 1.000
0.921 0.476
18.78.00
Tree-RingRand
0.999 1.000
0.940 / 0.585
18.68.09
Free-RingRings
0.999 0.999
0.966 0.603
17.68,
Table 2: AUC under each Attack for Stable Diffusion, showing the effectiveness of Tree-Ringgwa over number of augmentations. Cr. & Sc. refers to random cropping and rescaling. Additional results for the ImageNet model be found in Appendix Table 4.
Method
Clean
Rotation
JPEG
Cr. & Sc.
Blurring
Noise
Color Jitter
Avg
DwtDet
0.974
0.596
0.492
0.640
0.503
0.293
0.519
0.574
DwtDetSvd
1.00
0.431
0.753
0.511
0.979
0.706
0.517
0.702
RivaGin
0.999
0.173
0.981
0.999
0.974
0.888
0.963
0.854
Stable Sig.
1.00
0.658
0.989
1.00
0.565
0.731
0.976
0.880
T-R 0.999
0.994
0.984
0.999
0.977
0.877
0.907
0.963
T-RRand 1.00
0.486
0.999
0.971
0.999
0.972
0.994
0.918
T-RRings 1.00
0.935
0.999
0.961
0.999
0.944
0.983
0.975
investigate the impact of varying intensities these attacks. We report both AUC and TPR@1 1PP in the average case where average the metries over the clean setting and all attacks. In all ablation studies, we report the average case.
In Table 1, the baseline methods fail in the presence of adversaries. On the contrary, our methods demonstrate higher reliability adversarial settings. Among them, Ting-Ring performs the best under adversarial conditions, result of our careful watermark pattern design.
Further, show the AUC for each attack setting in Table 2. Notably, Tree-Ringzen demonstrates high robustness against most perturbations, except for Gaussian noise and color jitter. Similarly, Tree-Ringoond is robust most scenarios but performs poorly when faced with rotation, expected. Overall, Tree-RingRing delivers the best average performance while offering the model owner the flexibility of multiple different random keys. is worth noting that the baseline method RivaGan also demonstrates strong robustness in most scenarios, but is important to highlight that our method training- free and really "invisible".
4.4 Ablation Experiments
In this section, undertake exhaustive ablation studies with the Ring pattern several key hyperparameters to demonstrate the efficacy of Tree-Ring Watermarking. Except for the ablation on attacks, the reported numbers represent averages over all attack scenarios and clean images.
Number Steps Used for Generation and Detection. A key unknown variable for the model owner the detection time is the actual number of inference steps used during the generation time.

---

1.0
ALC
28.96
1,0
WE Netermon
AUC/TPRa IMTPRT
NUCCTRI MUCTPRIl 16FPR
0.6
27.52
0.6
0.4
26.79
0.4
0.2-
26.07
0.2
ABC
0.0
25.35
0.0
2
2
8
10
16
18
Radius
Guidance Scale
(a) Ablation on Watermark Radii
(b) Ablation Guidance Scales
Figure 4: Ablation on Watermark Radii and Guidance Scales,
This factor could potentially impact the precision of the DDIM inversion approximation of the initial noise vector. To scrutinize this, we systematically vary the number of steps for both the generation and detection time. Due the computational demands of sampling with high number inference steps, we employ a total of 400 images for each run.
In Figure 3,
compare AUC
across all step combinations. Surpris- ingly, even with significant differ- ence between the generation-tin time and detection-time #steps, the decrease in AUC is minimal when the model owner reasonable number of inference steps for detection without knowledge of the true generation-tin time steps. This indicates that the DDIM inversion maintains its robustness in approximating the initial noise vec- tor, and is effective for watermark de- tection irrespective of the exact num- ber of steps employed. Interestingly, we notice trend where the detection power appears to be slightly stronger with fewer inference steps at detection time or a larger number of inference steps at generation time. This is an advantageous scenario as the model owner now does not actually need to carry while concurrently, the model owner (or the user) is free to choose the number of generation steps that achieve the best quality [Rombach al., 2022].
860
-0.808
0,969 0.968
0.950
400
-0.801 0.919
0.962
0.967 0.966 0.965 0.964
0.925
Generaaioon Time Steps
200
-0.799 0.919
0.962
0.968 0.968 0.967 0.965 0.965 0.964
0.900
100
-0.800 0.918
0.961
0.969 0.969 0.967 0.965 0.964 0.963
0.875
10 25 50
-0.806 0.921 961 0. 966 0. 966 0. 964 0,963 0.963 0.963
0.850
-0.810 0.925 0. 0.961 965 D. 964 0. 962 0. 960 0.962 0.962
-0.785 0.909 0.948 0.952 0.951 0. 948 0. 949 0.948 0.947
0.800
25
50
100
200
400
800
Detecti on-Time # Steps
Figure 3: Ablation on Number of Generation Steps versus Detec- tion Steps. Detection succeeds independent of the number of DDIM used to generate data.
large number of steps for DDIM inversion,
Watermark radii. The radius of injected watermarking patterns is another critical hyperparameter affecting robustness and generation quality. The corresponding results are shown in Figure 4(a). As watermarking radius increases, the watermark's robustness improves. Nevertheless, there is trade-off with generation quality. We overall confirm a radius of 16 to provide reasonably low FID while maintaining strong detection power.
Guidance scales. Guidance scale is hyperparameter that controls the significance of the text condition. Higher guidance scales mean the generation more strictly adheres the text guidance, whereas lower guidance scales provide the model with greater creative freedom. Optimal guidance scales typically range between and 15 for the Stable Diffusion model we employ. We explore this factor from 2 to 18 in Figure 4(b) and highlight that the strength of the guidance is always unknown during detection time. Although a higher guidance scale does increase the error for DDIM inversion due to the lack of this troun guidance during detection, the watermark remains robust and reliable even at guidance scale of 18. This again beneficial for practical purposes, allowing the model owner to keep guidance scale tunable setting for their users.

---

1.0
1.0
AUC/PR/ 1 1KFP77
0.8
0.8
0.8
0.6
0.4
0.4
0.2
AUC
0.2
0.2
TPR@ 1XFPR
0.0
0.0
45
90
135 180 225 270 315
90
80
20
10
90
80
70
60
50
40
30
20
10
Rotation Degree
JPEG Compression Quality
SCCooping++Scaling
(a) Rotation
(b) JPEG Compression
(c) Cropping Scaling
1.0
1.0
1.0
AUC/TRR@ 196FPRT
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.0
10
15
20
25
30
35
40
0
0.05 0.1 0.15
0. 0.2.05500.2... 0.3 0.4
2
4
8
16
Kernel Size
Noise Std
Brightness Factor
(d) Gaussian Blurring
(c) Gaussian Noise
(f) Color Jitter
Figure 5: Ablation Different Perturbation Strengths. Attack strengths. Further, we test out the robustness of Tree-Ring Watermarking under each attack with various attack strengths. As shown in Figure 5, even with extreme perturbations like Gaussian blurring with kernel size 40, Tree-Ring Wutermarking can still be reliably detected.
5 Limitations and Future Work
Tree-Ring Watermarking requires the model owner to use DDIM during inference. Today, DDIM is still likely the most popular sampling method due to its economical quality. However, the proposed watermark will need to adapted to other sampling schemes should DDIM fall out of favor. Further, the proposed watermark is by design only verifiable by the model owner because model parameters are needed to perform the inversion process. This has advantages against adversaries, who cannot perform white-box attack the watermark an ensemble of manipulations broke the watermark. However, also restricts third parties from detecting the watermark without relying on API. Finally, it currently not yet clear how large the capacity for multiple keys k* would be. Would it be possible to assign a unique key to every user of the API?
of GPU resources and high
verify whether
The effectiveness of the proposed watermark is directly related to the accuracy of the inverse DDIM process. Future work that improves the accuracy of this inversion [Zhang et al., 2023], utilizes invertible diffusion models as described in Wallace et al. [2022], would also improve watermarking power further.
6 Conclusion
We propose new approach to watermarking generative diffusion models using minimal shifts of their output distribution. This leads to watermarks that truly invisible on per-sample basis. We describe how to optimally shift, so that the watermark remains detectable even under strong image manipulations that might be encountered in daily usage and handling of generated images.
7 Acknowledgements
This work was made possible by the ONR MURI program, the Office of Naval Research (N0001421 12557), and AFOSR MURI program. Commercial support was provided by Capital One Bank, the Amazon Research Award program, and Open Philanthropy. Further support was provided by the National Science Foundation (IIS-2212182, and by the NSF TRAILS Institute (2229885).
10

---

References
Ali Al-Haj Combined DWT-DCT Digital Image Watermarking. Journal of Computer Science, (9):740-746, September 2007. ISSN 15493636. doi: w..8844/jessp2207774....7 URLhttp:
www...hhecciub.... com/ messtrract/doijjcssset 2007 .0007.740..0 746.
Arpit Bansal, Ping-Yet Yeh Chiang, Michael Curry, Rajiv Jain, Curtis Wigington, Varun Manjunatha, John P. Dickerson, and Tom Goldstein. Certified Neural Network Watermarks with Randomized Smoothing. In Proceedings the 39th International Conference on Machine Learning, pages 1450- 1465. PMLR, June 2022. URL https: httppee:////rooeinngge/ mlr
html.
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Proceedings of the 2021 Faireess, Accountability, and Transparency, FAccT '21, pages 610-623, New York, NY, USA, March 2021. Association for Computing Machinery. ISBN 978- 1-500-833977
ACM Conference
doi: 10.1145/3442188 188. 3445922. 18544444222183445992. URL https: org/ 1............1.1 1144/344421188 44444/43332888.449492222
Francis Morgan Boland. Watermarking digital images for copyright protection. 1996. URL http://www. wwt.taa....aaaeeaeet.ttt.te http:at.....aaann/////////000/
Chin-Chen Chang, Piyu Tsai, and Chia-Chen Lin. SVD-based digital image watermarking
scheme. Puttern Recognition Letters, 26(10):157--1886 July 2005. ISSN 0167-8655. doi: 00.1016fi.patree2000..000 100000001..04,... URL https:///ww...scieeeeereee..... com//ccinnce///rtiicee.
pi1/801678655500144..
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning, ArXiv, abs/2212.0714 12.011.,,4 2022.
IJJ. Cox, J. Kilian, T. Leighton, and T. Shamoon. Secure spread spectrum watermarking for images, audio and video. Proceedings of 3rd IEEE International Conference on Image Processing, 3:243- 246, 1996. doi: 1.11099CII..1996..55000 560429/.
URL http::/ieeeepllorrt ieee. org//ocument/
Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. Digital Watermarking and Steganography. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2 edition, 2007.
ISBN 9780080555805.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
doi: 10.1 2WW.CCVPR.000.520000
Prafulla Dhariwal and Alex Nichol. Diffusion Models Beat GANs on Image Synthesis.
Stkri:2205.00233/c stat), June 2021. doi: 10.485500rxx..220550.555 210503333....... URL http::////aaii org/abs/2105. 05233.
Jianwei Fei, Zhihua Xia, Benedetta Tondi, and Mauro Barni. Supervised GAN Watermarking for Intellectual Property Protection. arxiv:2209 (3)200/ss34666s September 2022. doi: 10.48550/arXiv. 2209.03466. URL http:://taiiiiii www.oogg//.o.oooooooooo.oo 2209 03466.
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The Stable Signature: Rooting Watermarks in Latent Diffusion Models. 2303. /5435/cs), March 2023. doi: 10. 2...5500arX....33.... 15435. URL http: http///axxvvvv. wwwpwiv..rr//bbbabb /2033. 15435.
Paul Glasserman. Monte Carlo Methods in Financial Engineering, volume 53 of CettohhsstticMMMllli and Applied Probability. Springer, New York, NY, 2003. ISBN 978- 21617-1. doi: 10. 00.10077978800338--1117.7 URL http::/ttliii //lp::///ikk...////////// spriiksrr.nnnnreeeeeee com/10.
778-0-387-21617-1
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL https: httppts://pocccceiggggg neurips mourips.ce. cc/ pdf..
files/ http:///e//ooooooo......a.aaaaan

---

Alexei Grinbaum and Laurynas Adomaitis. The Ethical Need for Watermarks in Menerte-Generated Language. arxiv:220.003/18/c1 September 2022. doi: 11155500rrXxv.22200011..... URL
http://
httt://arxiv.orr http//axxvvv.ggg.. http://aawwv..rrr/////./..000 03118.
Jamie Hayes and George Danezis. Generating steganographic images via adversarial train- ing. In Advances in Neural Information Processing Systems, volume 30. Curran As- sociates, Inc., 2017. URL https://paper:. 20000000000011199333400878686463 22t-raaeee77244--aaarratc...... .htm..
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NIPS, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, volume 33, pages 6840-6851, Cur- ran Associates, Inc., 2020. URL https:://pppt ptttps://pooceedings.ooo neurips cccpapper/222//eeeeee 4ebb:ees@@arr@@00faaa000roaaa-b. Abstract. html.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A 10226/cs/] January 2023. doi: 10.48550/
Watermark for Large Language Models.
arXiv.2301.10226. URL http: arxiv htggabb//330 10226.
Martin Kutter and Fabien A. P. Petitcolas. Fair benchmark for image watermark- ing systems. In Security and Watermarking of Multimedia Contents, volume 3657, pages 226-239. SPIE, April 1999. doi: 10.1117/12.344672. wwww.spiedigitalliirrro Fair-benchmark for- image- wetereewwttmmmkiingg. systems/10 11w./11.1111111 11.17//12..4472.ffu111..
URL https:
org/ conference- pwo.eedings-----pieeeeeeeeeeee /ttp:///////50000////// sattermarriing-yyyteees/oooo..
G.C. Langelaar, I. Setyawan, and R.L. Lagendijk. Watermarking digital image and video data. A state-of-the-art overview. IEEE Signal Processing Magazine, 17(5):20-46, September 2000. ISSN 1558-0792. doi: 10.1 1997799977777.
Tsung-Y. Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014.
Junxiu Liu, Jiadong Huang, Yuling Luo, Lvchen Cao, Su Yang, Duqu Wei, and Ronglong Zhou, An Optimized Image Watermarking Method Based on HD and SVD in DWT Domain. IEEE Access, doi: 10.1 200.CCCSSSS2219991550000
2019. ISSN 2169-3536.
Alex Nichol and Prafulla Dhariwal. Improved Denoising Diffusson Probabilistic Models. arxiv:2102.096721cc,, stat), February 2021.
doi: 10.48550aaxxxw22102200777 URL http:
//arxiv. org/ 0bbs/2102.09677
J.J.K. O' "Ruanaidh and T. Pun. Rotation, scale and translation invariant digital image watermarking. In Proceedings of International Conference on Image Processing, volume 1, pages 536-539 vol.1, October 1997. doi: 10.1109/CIPP1199....M 6647968.
P. B. Patnaik. The Non-Central X2- and F-Distribution and their Applications. Biometrika, 36(1/2): 202-232, 1949. ISSN 0006-3444. doi: 10.2307/2332542. URL https: bostts://wwit h/wp:/jstorrrorro. org/ stable/
stable/2332542. 2332542..
I. Pitas. A method for watermark casting on digital image. IEEE Transactions on Circuits and Systems for Video Technology. 866):775-780, October 1998. ISSN 1558-2205. doi: 10.1109/76.7288
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. International Conference on Machine Learning. 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 10684-10695, 2022. URL https: //opeacccess.. thecvf. com/ http://otttttt/eeeee /ttpee///////OO222/em///////oaa/ Image_ Soage__Sntheeis..cmm Wi th Latent SittaatiDiffuuuooooooeeeeeee
BeetombachHHgghheeee NiihhResolution. 2et__l________222_______ html.
12

---

Jin S Seo, Jaap Haitsma, Ton Kalker, and Chang D Yoo. A robust image fingerprinting system using the Radon transform. Signal Processing: Image Communication, 19(4):325-339, April 2004. ISSN 0923-5965. doi: 10. im0.ee/ii.a....2003....... 12.000. URI.http:://wnt http:/wwwwwccincecrreettttt com/
V. Solachidis and L. Pitas. Circularly symmetric watermark embedding in 2-D DFT domain. IEEE Transactions Image Processing, 10111):14111153, November 2001. ISSN 1941-0042. doi: 10.1109/83.967401.
Yang Song and Stefano Ermon. Generative Modeling by Estimating Gradients of the Data Distribution. arXiv: /907. 05600 [cs, stat], October 2019. URL http: http:///rxiin wwt.abs.////oooo.ooooo...osse aasaabb///// 10wps/1900000000...
Yang Song and Stefano Ermon. Improved Techniques for Training Score-Based Generative Models. arXiv:2006.090) [cs, stat], June 2020. URL
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding Watermarks into Deep Neural Networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, pages 269-277, Bucharest Romania, June 2017. ACM. ISBN 978- 1-5503- 4701-3. doi: 10. 00.1145/3078971.30788877 URL htttps://l.. acm. wwtp////////oo///////t/otooottot ort///o//10..114///077555 1115//797711///777777711 3078974.
Bram Wallace, Akash Gokul, and Nikhil Naik. EDICT: Exact Diffusion Inversion via Coupled Transformations. arxiv:2211. /2/46//ss/ December 2022. doi: 10.485500arXxx,,2111.2224 URL
http://arxiv.ort org. wttp://..gga///b////////o /21. 12446.
Wenbo Wan, Jun Wang. Yunming Zhang, Jing Li, Hui Yu, and Jiande Sun. A comprehensive survey on
robust image watermarking, Neurocomputinng 488:226-247, June 2022. ISSN 0925-2312. doi: 10.
(01b6/.neuccm.2222200.0000 URL https: //ww://wwwwww//. www.seeiieeennnnccccceooeoeeceee http:///ee...oocccccccc///tttttt pa1/$0995233222000553.
Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution Training Data. arxiv:2007.08457/esss March URL
2022. doi: 11485500aXXiv.2007.004/4
088457.
Jaalong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Stoccklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermark- ing. In ACM Symposium Information, Computer and Communications Security. Association for Computing Machinery. Inc., May 2018. ISBN 978-1-4503-557666, URL https:/rresearch. ibm http.//////lcciioons/////////// htt.://.......eeea....o.o.oooooa ni
doi: 10.1 10.145531994944319555. proteetinggiinteleeecul-------pp
Jiaxin Zhang, Kamalika Das, and Sricharan Kumar. On the Robustness of Diffusion Inversion in Image Manipulation. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, April 2023. http:t:s:
Kevin Alex Zhang. Lei Xu, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Robust Invisible Video Watermarking with Attention. arxiv: arxiv: /9tp:///99.///css////o/ September 2019. doi: 10.48550/arXiv. 1909.01285. URL /ttp::/arxiv..rrt htwgg/abs/199/ 190999999.. 01285.
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, and Min Lin. A Recipe for March 2023. doi: 10.485500/rxiv.2330..
Watermarking Diffusion Models. 233iv 10137. URL http::tt/// /at:::///aivveeee org/ abs / /303 10137.
Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei. HiDDeN: Hiding Data with Deep Net- works. In Proceedings of the European Conference on Computer Vision (ECCV), pages 657- 672, 2018. URL https: htttts::///peaaccssssss thecvf. com/ content_ECCC 2aVV//201//haaaa httttmm/iireenn...ee/ html.
Zhu_HiDue HiDDeN Hii.DeNNHHinnngD.ttaaeoOo__OOOO
13

---

A
Appendix
(a) W/o Watermark
(b) DwtDet
(c) RivaGAN
(d) Tree-Ring (Ours)
Figure 6: Various watermarked generations with the same random seed are presented, showcasing the "invisible" nature of proposed watermark. A zoomed-in view with high contrast is provided in the bottom right comer. more high-resolution watermarked images, please refer to Appendix Figure 8.
14

---

Gaussian
0.4
x
(um-0.0051 c-0.9556)
0.3
Denstty
0.2
-2
Figure 7: Histogram of the array 2/p obtained for natural image, which is Gaussian.
Table 3: Main Results with Error Bars. T&1% %F represents TPR@1 %FPR. We evaluate watermark accuracy in both henign and adversarial settings. Adversarial here refers average performance battery of image manipulations.
AUC/T@ 1%F
AUC/T@
Model
Method
FID 4
CLIP Score f
(Clean)
(Adversarial)
DwtDet
0.974 001 0..244,0zz
0.574 o0s 0..92..0
25.10 .09
0.362, .000
Stable Diff.
DwtDetSvd
1.000 oo0o 11.000,00ur
0.702.000
0.262.u11
25.01,09
0.359, .000
FID 25.29
RivaGAN
0.999 000 0.999.990mmm .000
0.854 002
00.48..0m .006
24.51 .17
0.361 .000
CLIP Score
T-Rzeros
0.999 o00
0.999.09000 .000
0.963 001 / 0.7115, .21
26.56.07
0.356, 000
0.363
T- R-Rd
1.000 1.000,o00 .000
0.918 905 0.702..1r
25.47.05
0.363 001
T-RRings
1.000.000 1.000..om
0.975.00 001 0..94..o
25.93.
0.364.om
DwtDet
0.899 0.244, 203
0.536 016 0.037 .299
17.77.01
DwiDetSvd
1.000,000 1.000,,000
0.713 019 0.187
18.55.02
ImageNet
RivaGAN
1.000 ,000 1.000.000
0.882 .10 0.509 .099
18.70,02
FID 17.73
T-Rzaro
0.999 000 1.000.
0.921 000 00.76.omm
18.78.00
T-RRand
0.999 000 1.000,.00 .000
0.940 004 0.585.00 .006
18.68.09
T-R T-R
0.999 000 0.999.
0.966 oos 0.603.o0 .008
17.68 16
Table 4: AUC under cach Attack for the ImageNet model, showing the effectiveness of Tree-Ringaugs over a number of augmentations. Cr. & Sc. refers to random cropping and rescaling.
Method
Clean
Rotation
JPEG
Cr. & Sc.
Blurring
Noise
Color Jitter
Avg
DwtDet
0.899
0.478
0.522
0.433
0.512
0.365
0.538
0.536
Dd
1.000
0.669
0.568
0.614
0.947
0.656
0.535
0.713
RivaGan
1.000
0.321
0.978
0.999
0.988
0.962
0.924
0.882
T-R 0.999
0.953
0.806
0.997
0.999
0.938
0.775
0.921
T-Rgand 0.999
0.682
0.962
0.997
0.999
0.986
0.956
0.940
T-R T-R 0.999
0.975
0.940
0.994
0.999
0.979
0.861
0.966
Table 5: Additional experiments about watermark capacity for multiple keys. We employ Bonferroni correction for multiple hypotheses and set the threshold FPR 1e The identification accuracy is presented.
Method 50 users/keys
100 users/keys
500 users/keys
1000 users/keys
R-R
1.00
1.00
1.00
1.00
T- T-RRings
0.99
0.99
0.98
0.98
15

---

W/o Watermark
Ren
Free-RingRane
Tree- FreeRRingiigrr
Wille bestl
ATHA MEAJI
Paaset MAT
TOP
STOP
STTP
STPTOP
TOP
TO
Figure 8: More generated images with Tree-Rth Ring Watermarking with the first prompts in MS-COC0-2017 training dataset.
16

---

1.0
AUC 1PR PR
AUC/TPR@1% 1FRF1%FPR f
0.8
0.6
0.4
0.2
0.0
1
3
4
5
#Attacks
Figure 9: Results on number of random attacks applied at the same time.
(a) 75 Rotation
(b) 25% JPEG Compression
(c) 75% Random Cropping Scaling
(d) Gaussian Blurring with an 8 x (c) Gaussian Noise with a = 0.1 (f) Color Jitter with Brightness Factor of 6
8 Filter
Figure 10: Attacked images.
17


Reviews:
[{"review_id": "Raxefx1UR2", "reviewer": "Reviewer_9hBC", "summary": "This paper proposes watermarking the generated image from diffusion models by watermarking the initial noise and the reverse DDIM process is directly used as the watermark extraction. Experiments show the influence on the visual quality and robustness of the proposed method against various distortions.", "questions": "1) What is the influence on the editability of the watermarked noise?\n\n2) If there are two different tree rings embedded into two initial noises, will it affect the integrity of the proposed method? This may be related to the capacity mentioned in Limitations.\n\n", "limitations": "Mentioned in the manuscript.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1) This paper is easy to read.\n2) Watermarking the AIGC is an interesting problem.", "weaknesses": "1) The threat model shall be clarified. For example, who conducts the watermarking process and executes the verification stage? If the user’s ability/behavior (generation process) is unknown as mentioned in L236, why does the user like to use the tree-rings as his initial noise rather than a clean initial noise? \n\n2) As shown in Fig2, compared with other post-processing watermarking, the generated image by the proposed method is charged by a large margin, although the objective FID score is better.\n\n3) What is the influence on the editability of the watermarked noise? In other words, given different prompts, it would be better to show some visual examples of generated images by clean and tree-rings noise, respectively.\n\n4) Section 2.1 shall be reorganized. For example, in L100, a forward diffusion process is from clean data $X_0$ to noise.\n\n5) Some related work is missing. [1] focuses on watermarking the generative models, which also tries to watermark the internal noise.\n\n[1] Ong, Ding Sheng, et al. \"Protecting intellectual property of generative adversarial networks from ambiguity attacks.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.\n\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable feedback and the time you've dedicated to providing it. Below, we address specific points you raised:\n\n> The threat model shall be clarified.\n\nThanks for pointing out this. We will add the following threat model section in the future version.\n\n#### Threat Model\nThe goal of watermarking is to allow for image generation without quality degradation while enabling the model owner the ability to identify if a given image is generated from their model. Meanwhile, the watermarked image is subjected to a number of image manipulations and modifications that appear in everyday use. We formalize this as an adversary who tries to remove the watermark in the generated image to evade detection using common image manipulations (i.e., augmentations) but note that informally, we are also interested in watermark robustness across common usage. Ultimately, this setup leads to a threat model with two agents that act sequentially. \n\n- Model Owner (Generation Phase): Gene owns a generative diffusion model $\\epsilon_\\theta$ and allows images $x$ to be generated through an API containing the private watermarking algorithm $\\mathcal{T}$. The watermarking algorithm $\\mathcal{T}$ should have a negligible effect on the generated distribution so that quality is maintained and watermarking leaves no visible trace. For the conditional-based diffusion models, the API also allows conditioning $c$ during generation by users, multiple possible DDIM step settings, and guidance strength.\n-  Forger: Fiona generates an image $x$ through the API, then tries to evade the detection of $\\mathcal{T}$ by applying strong data augmentations that convert $x$ to $x'$. Later, Fiona uses $x'$ for a prohibited purpose and claims that $x'$ is her intellectual property.\n- Model Owner (Detection Phase): Given access to $\\epsilon_\\theta$ and $\\mathcal{T}$, Gene tries to determine if  $x'$ originated from $\\epsilon_\\theta$. Gene has no knowledge of the text used to condition the model or other hyperparameters like guidance strength and the number of generation steps.\n\n> As shown in Fig2, compared with other post-processing watermarking, the generated image by the proposed method is charged by a large margin, although the objective FID score is better.\n\nWe, of course, agree that the generated image, shown in Fig.2, is different from the watermarked version. This is a hallmark of our \"invisible\" approach. This difference occurs because the two images are generated with a different initial random $x_T$. Importantly, had the unwatermarked diffusion model been run again with a different random Gaussian seed $x_T$, it would *also* have produced a different image from the original. Likewise, the watermarked image is produced from a different initial $x_T$ than the unwatermarked image, causing a different image to be produced. This image is of the same quality as the original unwatermarked image, and in this sense, it is truly invisibly watermarked because the image is a direct generation from the diffusion model.\n\nWe think this is an interesting reformulation of the problem. Instead of a “classical watermark” that manipulates/watermarks the image in the pixel domain where changes result in visible image distortion, we propose a watermark that embeds itself in the initial noise $x_T$. After this vector passes through the image generation pipeline, the resultant image contains no visible distortions, even when a high watermark strength is used.\nThis is clearly a much more limited setup than general watermarking (as only diffusion images can be watermarked), but one that is uniquely suited to watermarking for generated data.\n\nWe hope to discuss this further, as we think this is a core part of the argument and departure from classical post-processing watermarks, so we do want to get our explanations here just right.\n\n\n\n> What is the influence on the editability of the watermarked noise?\n\nWe have shown some generated examples in Appendix Figure 6. In Figure 2 in the rebuttal PDF, we also provide more examples generated with more \"challenging\" prompts and their p-values derived from the formulation in the global response.\n\n> If there are two different tree rings embedded into two initial noises, will it affect the integrity of the proposed method? This may be related to the capacity mentioned in Limitations.\n\nWe've now formalized the detection process, please see our global response. In this framework, the detection produces rigorous p-values. Armed with this setup, the question of multiple keys, which we raised as a limitation in our current submission, can be simply reduced to the question of a multiple comparisons test. Standard methods, such as the Holm–Bonferroni approach, can now be applied to rigorously test against an arbitrary number of keys.\n\nThank you again for your thoughtful review. We made a significant effort to address your feedback including experiments, and we would appreciate it if you would consider raising your score in light of our response. Do you have any additional questions we can address?"}, {"review_id": "MpPnTNpF7D", "reviewer": "Reviewer_ZCXQ", "summary": "This paper proposes a method for watermarking images created by diffusion models, a popular class of generative models. Whereas traditional watermarking methods operate directly on images (e.g. in pixel space or Fourier/wavelet representations), the proposed watermark is embedded in a Fourier representation of the diffusion model’s initial noise distribution. As a result the watermark does not introduce perceptible artifacts, is robust to common image transformations, and can be detected with knowledge of the model weights via a noise inversion process. This method is a promising step towards allowing the operators of generative models to identify synthetic media produced by their systems.", "questions": "(These are more comments than questions, and aren't critical to my review/score of the paper)\n\nFrom Tables 1-4, it looks like RivaGAN is actually very competitive with the proposed method on robustness aside from the rotation transform, on which it is not trained. Given that RivaGAN’s attention mechanism supposedly encourages it to hide image modifications in textures, it is possible that a RivaGAN trained on rotation transformations might achieve robustness results on par with the proposed method while allowing for 32-bit watermark capacity. I don’t think this necessarily merits a mention in the paper, but it seems worth pointing out.\n\nOne class of adversaries against which the proposed method might fare substantially better than post-hoc methods like RivaGAN is “generative autoencoder” attacks that use off-the-shelf pretrained image models [2]. An experiment with this kind of transformation probably falls into the realm of future work, but could potentially present a very strong argument for the kind of distributional watermarking approach proposed in the paper over post-hoc image modification.\n\n[2] Zhao, Xuandong and Zhang, Kexun and Wang, Yu-Xiang and Li, Lei. “Generative Autoencoders as Watermark Attackers: Analyses of Vulnerabilities and Threats”. https://arxiv.org/abs/2306.01953\n", "limitations": "Yes", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed method is novel and operates by intervening in the sampling process (akin to recent language-model watermarks [1]) rather than “post-hoc” on generated images. This kind of “distributional” watermarking of generative models is increasingly emerging as an attractive alternative to traditional methods.\n\nThe proposed method appears straightforward to implement and does not require re-training the diffusion model.\n\nSolid experimental evaluation against reasonable watermarking baselines, including ablations on key generation & detection parameters.\n\nThe authors provide code for reproducing their experiments.\n\n\n[1] Kirchenbauer, John and Geiping, Jonas and Wen, Yuxin and Katz, Jonathan and Miers, Ian and Goldstein, Tom. “A Watermark for Large Language Models”. https://arxiv.org/abs/2301.10226", "weaknesses": "In line 157, the authors briefly mention a key result: “Curiously, we will observe below that the invariant properties above are preserved in xT even when image manipulations are done in pixel space of x0.” I think this could be discussed in more depth, as the success of the proposed approach essentially rests on this seemingly counterintuitive property. Do the authors have any theories as to why this property holds?\n\nWatermark capacity is barely touched on. It is common knowledge that watermark robustness, imperceptibility, and information-carrying capacity typically exist in a trade-off. The authors essentially propose a 1-bit watermark in the case of “Tree-Rings-Zeros,” while the actual capacity of the “Rand” and “Rings” variants is murkier (it is not clear how much two keys need to differ in order to be distinguishable, and how many simultaneous keys could operate in practice). While the authors’ main concern is differentiating between watermarked and un-watermarked content — for which capacity may not be essential — I think that at the very least, the difference in capacity between the proposed and baseline approaches should be mentioned. For instance, RivaGAN embeds 32-bit messages (which may be sufficient to “assign a unique key to every user of the API,” in the authors’ words). It is therefore reasonable to wonder whether RivaGAN might achieve better imperceptibility and robustness if trained with a lower capacity closer to that of the proposed method. I don’t think it is necessary for the authors to perform additional experiments along these lines, but a brief mention of the capacities of each method would better contextualize their existing results.\n\nRobustness of the proposed method to multiple simultaneous transformations looks low (Figure 7 in appendix) — it looks like even two simultaneous transformations can lower TPR@1%FPR to under 10%? It would be nice to see some discussion of this in the appendix (e.g. if any transformation combinations prove particularly effective at breaking the watermark without significantly degrading image quality). It would also be nice to see versions of Figure 7 for both baseline watermarks in the appendix.\n\nWhile model weight watermarking approaches are addressed briefly in the related work section (lines 94-97), the authors do not state why such approaches are not viable for the task at hand. Naively, it seems reasonable that a watermark embedded within the model weights during training could be at least as strong as one embedded in the diffusion noise at sampling. An extra sentence or two explaining why such approaches are not considered would be good (e.g. training-free methods are more desirable, existing model weight watermarking methods lack robustness or produce characteristic outputs for only a few \"trigger\" inputs, etc.).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable feedback and the time you've dedicated to providing it. Below, we address specific points you raised:\n\n> It is therefore reasonable to wonder whether RivaGAN might achieve better imperceptibility and robustness if trained with a lower capacity closer to that of the proposed method.\n\nWe fully acknowledge that exists the trade-off between the robustness and capacity of RivaGAN. This aspect is indeed worth investigating in future research. However, it's important to note that fine-tuning these hyperparameters would necessitate retraining the model for each desired capacity.\n\n> It would be nice to see some discussion of this in the appendix (e.g. if any transformation combinations prove particularly effective at breaking the watermark without significantly degrading image quality). It would also be nice to see versions of Figure 7 for both baseline watermarks in the appendix.\n\nThank you for bringing this to our attention. We plan to conduct thorough and extensive experiments in the upcoming version. In the meantime, we'd like to underscore that the perturbations we are applying have a significant impact. Consequently, even combining two attacks can drastically alter the image, making them less practical due to their detectability. Given this, it would be valuable for us to explore attacks like [2] as you mentioned in the question section, as they present an intriguing attack for future investigation.\n\n> While model weight watermarking approaches are addressed briefly in the related work section (lines 94-97), the authors do not state why such approaches are not viable for the task at hand. Naively, it seems reasonable that a watermark embedded within the model weights during training could be at least as strong as one embedded in the diffusion noise at sampling. An extra sentence or two explaining why such approaches are not considered would be good (e.g. training-free methods are more desirable, existing model weight watermarking methods lack robustness or produce characteristic outputs for only a few \"trigger\" inputs, etc.).\n\nWe have observed that the model weight watermarking techniques primarily rely on backdoored-style watermarks. For instance, when verified using trigger prompts, the model owner can determine if the model weights have been watermarked during training by incorporating specific trigger prompt-image pairs into the training data. These watermarking strategies don't apply to all outputs of the model; they pertain only to a subset generated using the designated trigger prompts.\n\n> From Tables 1-4, it looks like RivaGAN is actually very competitive with the proposed method on robustness aside from the rotation transform, on which it is not trained. Given that RivaGAN’s attention mechanism supposedly encourages it to hide image modifications in textures, it is possible that a RivaGAN trained on rotation transformations might achieve robustness results on par with the proposed method while allowing for 32-bit watermark capacity. I don’t think this necessarily merits a mention in the paper, but it seems worth pointing out.\n\nWe really appreciate this suggestion. We will include this information in the future version. It indeed holds intriguing potential for future verification.\n\nThank you for your feedback on this submission. Hope our response resolves your questions. Please let us know if you have other questions and comments that we can address."}, {"review_id": "ItjfrBeRa5", "reviewer": "Reviewer_KNue", "summary": "This paper proposes a watermarking scheme for image generative AI based on diffusion process. The watermark is embedded in the initial noise pattern before diffusion. Detection succeeds in reversing the diffusion to get an estimate of the initial noise pattern. ", "questions": "Q1. Why the detection score is the L1 distance? A watermarker would have use the cosine similarity.\n\nQ2. How do you explain the robustness to translation and crop/scaling? Is the crop random (ie. not a central crop)?\n\n", "limitations": "Some features are presented as advantages whereas these are limitations.\n\nL1. Line 192 \"However, these baselines methods are designed for steganography\". No, these are watermarking schemes. They offer multi-bit watermarking, whereas you only offer zero-bit watermarking.\n\nL2. The scheme is extremely sensitive to Gaussian noise. A jitter of 2 is ok, but a noise std = 0.1 (out of 255?) pulls down the watermark.\n\nL3. Line 285: \"Further, the proposed watermark is by design only verifiable by the model owner ... this has advantages\". I do not think so. For security reason, the model cannot be disclosed. Fernandez et al. does not have this drawback.    ", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "S1. The idea is simple and super original.\nS2. The watermarking technique seems to be very robust.", "weaknesses": "W1. Terminology. Being a watermarker for a long time, I am surprised by some atypical wordings. Mainly:\n- fingerprint / fingerprinting: Fingerprint usually denotes a passive forensics technique (like robust hash) whereas watermarking is an active technique (the content/model is modified). I found the title very confusing for instance.\n- to imprint: the correct term is 'to embed' in the watermarking literature.\n\nW2. Bold claims. I disagree with the following statements\n- Line 35: \"This is the first watermark that is truly invisible as no post-hoc are made to the image\". I strongly disagree. The difference between watermarked and non-watermarked generated images are super visible since their visual content are not the same (see Fig.~2). How can you pretend to be invisible. The thing is that nobody has access to the non-watermarked generated images therefore we do not care about the visibility. We only care about the generation quality (FID and CLIP score).\n- Line 47: \"the watermark can only be detected by parties in control of the image generation model\". This is a drawback, not an advantage.\n- Line 81: \"Stable Signature applies this idea\". If \"this idea\" relates to training a model with watermarked images (a la Yu et al. [2022]), then this statement is wrong.\n\nW3. No theoretical background.\n- Line 157: \"Curiously, we will observe that ...\" I understand that \"curiously\" as \"we do not know why but this happens to hold.\"\n- The threshold for a required FPR is *experimentally* set. First, over 1,000 unwatermarked images ONLY for a FPR=1%. A population size of 1,000 yields an accuracy above +/- 1% (95% confidence) in the estimation of a small probability. So, 1,000 is definitively not enough... and this is just FPR=1%. See https://en.wikipedia.org/wiki/Sample_size_determination\nBy the way, since the secret key is a Gaussian vector (even for the three-ring), there might be a way to set the threshold theoretically. At least, in watermarking literature, this is the case for Gaussian secret vector and cosine similarity thanks to the regularised beta incomplete function.\n- Some properties are outlined but they are not exactly put into practice:\nLine 149: Rotation - Done -> this explains the rings.\nLine 150: Translation - Not Done. The magnitude of the Fourier coefficients are invariant to translation but neither the embedding nor the detection works with magnitude. Therefore, the robustness to translation remains unexplained.\nLine 151: Dilation (ie. stretching) - Not Done. Same comment. I am surprised that Rand and Ring show some robustness since a stretching should desynchronise the watermark signal.\nLine 152: Color jitter - Done -> don't rely on the DC coefficient.\n\nW4: Flawed benchmark.\nFernandez et al. [2023] is cited but not included in the comparison. I do not see any reason. On the contrary, some old schemes called \"DwtDct\" and \"DwtDctSvd\" are included. The attached reference is the bible Cox et al. 2007. I do not remember that this book proposes these transformations. Dwt alone -> Yes. Dct alone -> Yes (But, it is well known that there are not robust to geometric attacks). However, DwtDct (ie. the concatenation of Dwt o Dct) -> Does not makes sense. DwtDctSvd  -> even worse. It would have been much better to compare to recent image watermarking like those cited in Lines 70-80 (especially HiddeN and followers).\n\nConclusion: the benchmark does not include key schemes but includes 2 non-sense techniques.     \n\nW5. I prefer A LOT the metric TPR@1%FPR because it makes sense from a practical point of view whereas AUC delivers poor information. For instance, Fig. 3 is hard to decipher: AUC = 0.948 is green lighted whereas such AUC corresponds to a TPR of 0.1 in some other plots. \nSame for Table 2.\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable feedback and the time you've dedicated to providing it. In the limited space below, we have made an effort to address the specific points you raised:\n\n>Some atypical wordings.\n\nThank you for pointing out this, we are happy update to our terminology.\n\n>Bold claim about invisibility of the watermark.\n\nWe do think that Fig.2. is a poignant example of our \"invisible\" approach. We, of course, agree that the generated image, shown in Fig.2., is different from the watermarked version. But we contend that this difference is the point of our paper. Instead of conceptualizing the watermark process as one where a \"non-watermarked generated image\" is hidden and modified, we argue that such a \"base\" image does not exist - the watermarked generation directly produces only watermarked images. These images are truly invisibly watermarked because they are direct generations from the diffusion model and contain no noticeable artifacts. Importantly, note that running the non-watermarked generator a second time with a different random seed $x_T$ would *also* result in a different image. The fact the different versions of the generator produce different images does not mean that there is any change in utility for the user.\n\nWe think this is an interesting reformulation of the problem. Instead of a “classical watermark” that manipulates/watermarks the image in the pixel domain where changes result in visible image distortion, we propose a watermark that embeds itself in the initial noise $x_T$. After this vector passes through the image generation pipeline, the resultant image contains no visible distortions, even when a high watermark strength is used.\nThis is clearly a much more limited setup than general watermarking (as only diffusion images can be watermarked), but one that is uniquely suited to watermarking for generated data.\n\nWe do think that this is a core contribution of our work, and we want to make sure that we get the description right for all readers. Please let us know what you think of this, we are glad to be able to discuss this. \n\n>set the threshold theoretically.\n\nWe now greatly increased the theoretical underpinning of the detection scheme and include a formalized way of calculating the P-value. More details can be found in the global response.\n\n>Some properties are outlined but they are not exactly put into practice.\n\nWe have added two more augmentations: PyTorch random affine transformation with an absolute fraction of 0.2 for both horizontal and vertical translations and random affine transformation with stretching factor of 20. The results are shown in the table below. We found that Tree-ring watermarks are robust under these augmentations except for *Tree-Ring$_{\\text{Rand}}$* under translation.\n\n|Method|Translation|Stretching|\n|:-:|:-:|:-:|\n|DwtDct|.537/.000|.691/.006|\n|DwtDctSvd|.463/.000|.357/.003|\n|RivaGan|.999/1.00|.996/.942|\n|*Tree-Ring$_{\\text{Zeros}}$*|.998/.976|.998/.983|\n|*Tree-Ring$_{\\text{Rand}}$*|.764/.089|.952/.638|\n|*Tree-Ring$_{\\text{Rings}}$*|.911/.138|.966/.599|\n\n>flawed benchmark.\n\nWe really do appreciate the work of Fernandez et al. [2023]. However, their [code](https://github.com/facebookresearch/stable_signature) was made available only a couple of weeks ago, well after our original submission. Nonetheless, we have now evaluated their approach in our experiments. The results can be found in the table below. We find the watermark proposed therein,  Stable Signatures, is a very robust method under many attacks, but not as robust to a number of augmentations, such as rotation, blurring, and noise. \n\n|Method|Clean|Rotation|JPEG|Cr.&Sc.|Blurring|Noise|Color Jitter|Translation|Stretching|Avg|\n|:-:|:-----:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Stable Signature|1.00|.658|.989|1.00|.565|.731|.976|1.00|.999|.880|\n|Tree-ring|1.00|.935|.999|.961|.999|.944|.983|.911|.966|.966|\n\n>Naming scheme of DWT-based watermarks.\n\nFor better or worse, we chose to follow the naming convention of the currently implemented stable diffusion watermark, which can be found here https://github.com/ShieldMnt/invisible-watermark, which is also followed by Fernandez et al. [2023]. We'd be very happy if we were provided with additional references or alternative naming schemes for these algorithms that you would consider more accurate, which we will happily include.\n\n>I prefer the metric TPR@1%FPR.\n\nThanks for the suggestion, we're happy to modify our draft.\n\n>Q1. A watermarker would have use the cosine similarity.\n\nWe have conducted some experiments with a cosine similarity metric but observed it to be less resilient compared to the L1 distance in our experiments. We intend to delve further into this matter in the future.\n\n>Q2. How do you explain the robustness to translation and crop/scaling? Is the crop random (ie. not a central crop)?\n\nYes, we used random crops. That being said, against strong translations (see above), our approach is not so robust. Thank you for bringing this up! We are fixing our formula to rely on either magnitude or correlation and are currently running experiments to verify that this improves robustness to non-central crops. We will post another update about this soon when the experiments finish. \n\n>The scheme is extremely sensitive to Gaussian noise. ... a noise std = 0.1 (out of 255?).\n\nThe std corresponds to 0.1 multiplied by 255, resulting in a value of 25.5 and as such *the approach is actually very robust to Gaussian Noise*. The magnitude of this perturbation is depicted in Appendix Figure 8 (e), showing that the perturbation is substantial. We're sorry for the confusion and will clarify the exact unit conversions in our draft.\n\nThank you again for your thoughtful review. We made a significant effort to address your feedback including experiments, and we would appreciate it if you would consider raising your score in light of our response, or commenting in response so we can update you on experiments that are still running?"}, {"review_id": "2SZ1xOXdH6", "reviewer": "Reviewer_CpUq", "summary": "This paper proposes a method to add the watermark to the images generated by the diffusion model during the generation process. It subtly inﬂuences the entire sampling process, resulting in a model fingerprint that is invisible to humans. In terms of the specific approach, the watermark embeds a pattern into the initial noise vector used for sampling. These patterns are structured in Fourier space so that they are invariant to convolutions, crops, dilations, ﬂips, and rotations. After image generation, the watermark signal is detected by inverting the diffusion process (DDIM Inversion) to retrieve the noise vector, which is then checked for the embedded signal.", "questions": "See Weaknesses", "limitations": "The negative impact of using this technique needs to be declared and discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Although not practical, I think the idea is interesting.\n- Good writing. This paper is well written and easy to follow.", "weaknesses": "- I consider the method proposed in this paper as image watermarking. My biggest concern is that the method is not practical. Image watermarking algorithms have been extensively studied, what are the advantages of this method over those methods? What's more, the method has some fatal flaws, such as the modified initial noise no longer satisfies the standard Gaussian noise, which will have an impact on the diversity and quality of the synthesis results, the necessity to use DDIM inversion to recover the watermark, etc.\n- Although three construction strategies are proposed in this paper, the resulting initialized noise does not satisfy the standard Gaussian distribution, which has an impact on the diversity and quality of the final synthesis results. However, there is no relevant experimental analysis in this paper. Please provide quantitative and qualitative experiments regarding the diversity and quality (CLIP Score, FID does not accurately reflect this, so user study is required) of the synthesis results.\n- The stable diffusion uses a classifier-free guidance in the synthesis process, and the guidance is generally set to 7.5. This paper uses empty text to recover the initial noise through DDIM inversion is not able to reconstruct accurately. The experiments in this paper also prove this, so there is a big vulnerability in the method presented in this paper.\n- For multiple key capacity, the experimental analysis needs to be provided. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable feedback and the time you've dedicated to providing it. Below, we address specific points you raised:\n\n> I consider the method proposed in this paper as image watermarking. My biggest concern is that the method is not practical. Image watermarking algorithms have been extensively studied, what are the advantages of this method over those methods? What's more, the method has some fatal flaws, such as the modified initial noise no longer satisfies the standard Gaussian noise, which will have an impact on the diversity and quality of the synthesis results, the necessity to use DDIM inversion to recover the watermark, etc.\n\nThank you for your feedback. Our watermarking approach is unique in that it directly applies to generative models. Instead of a classical image watermark, the watermark is embedded through a minimal change in the output distribution. Further, we feel the approach is practical and inexpensive because it can be applied at inference time without re-training large models. We show that this results in a watermark that is more robust than existing approaches that are currently deployed in Stable Diffusion. We'd be happy to compare to additional training-free image watermarks, which do you have in mind?\n\nWe further agree that our approach implies that we have to evaluate whether the distribution of generated images is negatively affected, which we do extensively in FID and CLIP-based metrics, measuring the diversity and quality of generated images. While we're happy to include additional metrics of image quality, such as a human study, as you mention as well, so far, we see no indication that the proposed watermark is \"fatally flawed,\" or that it introduces any noticeable image artifacts. We'd be glad if you could clarify your argument and we'd revise our study if there is a clear direction in which you think it can be improved.\n\nPlease see Figure 2 in the attached PDF, which contains typical examples of images generated with and without the watermark. We do not see any evidence of quality degradation in these or other examples. \n\nAs an aside, our method is based on DDIM inversion, but it is unclear to us why this would be considered a limitation. We're happy to discuss further.\n\n> Although three construction strategies are proposed in this paper, the resulting initialized noise does not satisfy the standard Gaussian distribution, which has an impact on the diversity and quality of the final synthesis results. However, there is no relevant experimental analysis in this paper. Please provide quantitative and qualitative experiments regarding the diversity and quality (CLIP Score, FID does not accurately reflect this, so user study is required) of the synthesis results.\n\nFID scores and CLIP scores are widely regarded as reasonable estimates of image diversity and quality, and we study the impact of the proposed watermark extensively under these metrics. As such our submission contains substantive experimental analysis. \nWhile we acknowledge your viewpoint that FID and CLIP Score may not provide a complete measure of the diversity in the generated images, arguing that there is no signal in experimental evaluation of image quality short of a human study is not a viewpoint shared by the broader community.\n\nWe agree that it a human study would be great to round out our evaluation, but obtaining IRB approval and completing human evaluations within the rebuttal period is not practical. We do intend to conduct more comprehensive human evaluations in our future work. In the meantime, we invite you to review some typical qualitative results presented in Appendix Figure 6 and Figure 2 of the rebuttal PDF.\n\n> The stable diffusion uses classifier-free guidance in the synthesis process, and the guidance is generally set to 7.5. This paper uses empty text to recover the initial noise through DDIM inversion is not able to reconstruct accurately. The experiments in this paper also prove this, so there is a big vulnerability in the method presented in this paper.\n\nWe indeed use only empty text to recover the initial noise. However, we find that this is actually sufficient to recover the original noise closely enough, so that detection is possible. 7.5 is indeed the guidance scale that we use in all experiments in this submission. We additionally ablate larger guidance strengths in Fig. 4b, finding that even larger guidance scales still lead to a robust recovery. We'd be glad to clarify our writing to describe unambiguously that our experiments do prove that this works.\n\n> For multiple key capacities, the experimental analysis needs to be provided.\n\nWe've now formalized the detection process, please see our global response. In this framework, the detection produces rigorous p-values. Armed with this setup, the question of multiple keys, which we raised as a limitation in our current submission, can be simply reduced to the question of a multiple comparisons test. Standard methods, such as the Holm–Bonferroni approach, can now be applied to rigorously test against an arbitrary number of keys.\n\nThank you again for your thoughtful review. We made a significant effort to address your feedback including experiments, and we would appreciate it if you would consider raising your score in light of our response. Do you have any additional questions we can address?"}]

Author rebuttal:
We would like to thank all the reviewers for their time and for writing thoughtful reviews of our work. We've added a section about deriving P-value below and also attached a PDF containing additional figures.

Based on questions about theoretically justified threshold values, we've formalized the detection event as a hypothesis test from which we can derive explicit P-values via a non-centered chi-squared distribution. Such a formal, interpretable P-value can be used to understand whether the observed watermark could have occurred in a natural image by random chance. This derivation allows the user to set a threshold of detection, i.e., the watermark is 'detected' when $p$ is below a chosen P-value threshold $\alpha$. By doing so, one can control the false positive rate $\alpha,$ making false accusations statistically unlikely.

Below, we provide a detailed formulation:

### Deriving P-values
We construct a statistical test for the presence of the watermark that produces a rigorous P-value. The forward diffusion process is designed to map images onto Gaussian noise, and so we assume a null hypothesis in which the entries in the array $x_T'$ obtained for a natural image are Gaussian. We find that this assumption holds quite well in practice, see Figure 1 in the rebuttal PDF.

For any test image $x_0'$, we compute the approximate initial vector $x_T'$ and then set $y= \mathcal{F}(x'_T)$. We then define the following null hypothesis

$$
\begin{equation}
    H_0: \textit{$y$ is drawn from a Gaussian distribution $\mathcal{N}(\mathbf{0}, \sigma^2 I_\mathbb{C})$.}
\end{equation}
$$

Here, $\sigma^2$ is an unknown variance, which we estimate for each image using the formula $\sigma^2 = \frac{1}{M} \sum_{i \in M} |y_i|^2.$

To test this hypothesis, we define the score
$$
\begin{equation}
    \eta = \frac{1}{\sigma^2} \sum_{i \in M} |k_i^* -  y_i |^2.
\end{equation}
$$
When $H_0$ is true, the distribution of $\eta$ is exactly a *noncentral* $\chi^2$ *distribution* [1], with $|M|$ degrees of freedom and non-centrality parameter $\lambda=\frac{1}{\sigma^2}\sum_{i} |k_i^*|^2$. 

We declare an image to be watermarked if the value of $\eta$ is too small to occur by random chance.

The probability of observing a value as small as $\eta$ is given by the cumulative distribution function $\Phi_{\chi^2}$  of the noncentral $\chi^2$ distribution:
$$
\begin{equation}
    p=\Pr \left(\chi^2_{|M|, \lambda}\leq \eta \middle| H_{0} \right) = \Phi_{\chi^2}(z).
\end{equation}
$$
$\Phi_{\chi^2}$ is a standard statistical function [2], available in `scipy` and many other statistics libraries. 

We show qualitative examples of the proposed watermarking scheme and accompanying P-values in Figure 2 in the rebuttal PDF. For each prompt, we show the generated image with and without the watermark, and also a watermarked image subjected to a transformation. For each image, we report a P-value. As expected, these values are large for non-watermarked images, and small (enabling rejection of the null hypothesis) when the watermark is present. Transformations reduce the watermark strength as reflected in the increased P-value.

[1] P. B. Patnaik. The Non-Central X2- and F-Distribution and their Applications. Biometrika, 36(1/2): 202–232, 1949. ISSN 0006-3444. doi: 10.2307/2332542. URL https://www.jstor.org/stable/2332542.
[2] Paul Glasserman. Monte Carlo Methods in Financial Engineering, volume 53 of Stochastic Modelling and Applied Probability. Springer, New York, NY, 2003. ISBN 978-1-4419-1822-2 978-0-387-21617-1. doi: 10.1007/978-0-387-21617-1. URL http://link.springer.com/10.1007/978-0-387-21617-1.

Meta-review: This paper proposes a watermarking method for diffusion generative models. The idea is adding the watermark in the initial seed noise that starts the diffusion generation. Since the diffusion can be inverted, the watermark is checked in the seed noise space, as opposed to the usual pixel space. 
It is interesting (and surprising ) that this is robust to manipulations in the image and also in inverting without the caption used for generation.
This method is only applicable for diffusions so its a unique type of watermarking, as the authors explain.
Overall the authors did a great job in responding to all the comments in great detail.
Decision: Accept (poster)