Attack-Resilient Image Watermarking Using Stable Diffusion

Authors: lijun  zhang
xiao  liu
antoni  viros
cindy  xiong bearfield
yuriy  brun
hui  guan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 19:51:44

Abstract: Watermarking images is critical for tracking image provenance and proving ownership. With the advent of generative models, such as stable diffusion, that can create fake but realistic images, watermarking has become particularly important to make human-created images reliably identifiable. Unfortunately, the very same stable diffusion technology can remove watermarks injected using existing methods.
To address this problem, we present ZoDiac, which uses a pre-trained stable diffusion model to inject a watermark into the trainable latent space, resulting in watermarks that can be reliably detected in the latent vector even when attacked. We evaluate ZoDiac on three benchmarks, MS-COCO, DiffusionDB, and WikiArt, and find that ZoDiac is robust against state-of-the-art watermark attacks, with a watermark detection rate above 98% and a false positive rate below 6.4%, outperforming state-of-the-art watermarking methods. We hypothesize that the reciprocating denoising process in diffusion models may inherently enhance the robustness of the watermark when faced with strong attacks and validate the hypothesis. Our research demonstrates that stable diffusion is a promising approach to robust watermarking, able to withstand even stable-diffusion-based attack methods. ZoDiac is open-sourced and available at https://github.com/zhanglijun95/ZoDiac.

Main paper: 
 introduction: 
 Digital image watermarking, a technique for subtly embedding information within digital images, has become increasingly crucial and beneficial in the context of content protection and authenticity verification [36, 10, 5, 4] . The advance in generative AI technologies [28, 30, 29] , such as stable diffusion, further underscores the need for watermarking solutions to distinguish between AI-generated and human-created images.Given an existing image, many methods have been proposed to embed a watermark that is invisible and robust to watermark removal attacks. Conventional watermarking strategies have employed various methods, such as embedding information in texture-rich regions [3] , manipulating the least significant bits [43] , or utilizing the frequency domain [21] . The emergence of deep learning has introduced neural network (NN)-based watermarking methods [53, 24, 35, 48, 47, 19] , which have shown promise in achieving high invisibility and robustness against traditional attacks, such as adding Gaussian noise or applying JPEG compression. Figure 1 : The watermark detection rate of existing methods and our ZoDiac before and after the diffusion-based attack Zhao23 [51] . Two example images show that ZoDiac's watermarks are perceptually invisible.Unfortunately, the recent advent of powerful image generation techniques can circumvent existing watermarking methods. The most recent work [51] shows that stable diffusion can be used in a watermark removal attack, and none of the existing watermarking techniques are robust enough to that attack. Figure 1 shows that the watermark detection rate (WDR) of six existing watermarking methods (DwtDct, DwtDctSvd [9] , RivaGAN [48] , SSL [16] , CIN [25] , StegaStamp [35] ) before and after the stable-diffusionbased watermark removal attack [51] drops from 79%-100% before attack to only 0%-48% after attack on the MS-COCO dataset [23] .To address the problem, we propose a novel stable-diffusion-based watermarking framework called ZoDiac. ZoDiac takes as input an existing image and uses a pre-trained stable diffusion model to inject into the image a watermark that can be reliably detected even when attacked. The rationale behind ZoDiac is that a pre-trained stable diffusion maps a latent vector into an image and that more than one latent vector can be mapped to perceptually the same image. Given an existing image, ZoDiac identifies a latent vector that contains a watermark pattern and can also be mapped to the same image using a pre-trained stable diffusion model. ZoDiac further allows mixes of the original image and the watermarked image to enhance image quality with a minor influence on watermark robustness. To detect watermarks, ZoDiac applies a diffusion inversion process that remaps the image to the latent vector and then detects the watermark pattern in the latent vector by statistical test. Recent trend [11, 15, 42] has used diffusion models for watermarking, but it can only encode a watermark into synthetically-generated images when they are being generated. By contrast, ZoDiac can watermark existing and real-world images.ZoDiac has two distinctive features that make it robust and effective. First, ZoDiac injects watermarks in the latent space that stable diffusion models operate on when sampling random noise to generate synthetic images, making the watermarks both invisible and robust to even the most advanced stablediffusion-based attack method. We hypothesize that ZoDiac exhibits strong robustness because the image generation process of the diffusion model serves as a powerful attack-defense mechanism by default and empirically validate it in §4.3. Second, while existing watermarking methods typically require the training of a dedicated model on a representative training dataset, ZoDiac builds on top of pre-trained stable diffusion models, forgoing the time-consuming training process.Our main contributions are:• ZoDiac -a novel framework for embedding invisible watermarks into existing images using pre-trained stable diffusion. To the best of our knowledge, ZoDiac is the first watermarking method that is robust to the most advanced generative AI-based watermark removal attack. • Empirically Demonstrated Strong Robustness Against Watermark Attacks -Our evaluation on images from diverse domains, MS-COCO [23] , DiffusionDB [41] , and WikiArt [27] datasets, shows that ZoDiac is robust against the state-of-the-art watermark attack mechanism, with watermark detection rate (i.e., true positive rate) above 98% and false positive rate below 6.4%, outperforming state-of-the-art watermarking methods. ZoDiac also maintains invisible image quality degradation, underscoring its efficacy in achieving robust watermarking with quality preservation (see examples in Figure 1 ). • Robustness Against Combined Attacks -Prior watermarking method evaluations [42, 51] focused on robustness to only a single attack at a time. We show that in a more realistic scenario, where the attacker can combine multiple attacks, ZoDiac significantly outperforms all existing methods. For example, when combining all attacks other than image rotation, ZoDiac retains a detection rate above 50% while all existing methods fail with a detection rate of 0%. Only one of the existing methods, SSL [16] , is effective against rotation, but it is ineffective against other attacks. We propose a method for making ZoDiac robust to rotation (above 99%), only slightly increasing its false-positive rate (from 0.4% to 3.4%) on MS-COCO dataset with appropriate hyperparameter settings.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: We proposed a robust image watermarking method based on stable diffusion models called ZoDiac. In the abstract and introduction, we provide our motivation in Figure 1 and elaborate on its robust and effective features with experimental results. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
preliminary and related work: 
 We first introduce the image watermarking problem and then the diffusion model background necessary to understand ZoDiac. We place our work in the context of the most related research; Appendix A discusses related work in depth.The Robust Image Watermarking Problem. Image watermarking aims to embed a predefined, detectable watermark into an image while ensuring that the watermarked image is similar to the original. This similarity is usually quantified using metrics such as the Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity Index (SSIM) [40] . Malicious attacks can attempt to remove the watermark, again, without significantly changing the image. A robust image watermarking method should be able to detect the watermark even in attacked images.Both image watermarking and watermark removal attacks have been studied extensively. The development of deep neural networks (DNN) resulted in advanced learning-based watermarking approaches, such as RivaGAN [48] , StegaStamp [35] , and SSL [15] . These methods train a dedicated DNN model on large-scale representative image datasets to watermark images. Meanwhile, advances in generative AI techniques have produced new attacks based on Variational AutoEncoders [2, 8] and Stable Diffusion [29] . None of the existing watermarking approaches are sufficiently robust to these advanced diffusion-based attacks.Diffusion Models and DDIM Inversion. Diffusion models [31] , which both the most sophisticated attacks and our approach build on, are a class of generative AI models that generate high-resolution images. To explain diffusion models, we first explain the forward diffusion process, in which a data point sampled from a real data distribution x 0 ∼ q(x) is gradually converted into a noisy representation x T through T steps of progressive Gaussian noise addition. This transformation yields x T as an isotropic Gaussian noise, i.e., x T ∼ N (0, I). Specifically, a direct generation of x t from x 0 is:EQUATIONwhere ᾱt = t i=0 (1 -β i ), β i ∈ (0, 1) is the scheduled noise variance that controls the step size, and ϵ ∼ N (0, I).Diffusion models reverse this forward process, learning to retrieve the original image x 0 from the noise x T by estimating the noise at each step and iteratively performing denoising. The Denoising Diffusion Implicit Model (DDIM) [33] is a prominent denoising method, known for its efficiency and deterministic output. Formally, for each denoising step t, a learned noise predictor ϵ θ estimates the noise ϵ θ (x t ) added to x 0 , leading to an approximate of x 0 . Then DDIM reintroduces ϵ θ (x t ) to determine x t-1 :EQUATIONIn this way, DDIM could deterministically recover the same image x 0 from the specified noise x T . DDIM also enables an inversion mechanism [14] that can reconstruct the noise x T from an image x 0 . DDIM inversion adheres to the forward diffusion process in Eq. ( 1), substituting ϵ t with ϵ θ (x t ) at each timestep.We denote the denoising process, that is, the image generation process as G and its inversion as G ′ . A stable diffusion model [29] operates in the latent space. We use Z T and Z 0 in the latent space to represent the noise x T and the image x 0 respectively. A pre-trained Variational Autoencoder (VAE) carries out the transformation between the noise x T and its latent vector Z T (and also the image x 0 and its representation Z 0 ). Throughout this paper, we refer to Z T as the latent vector.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We discuss the robustness limitation when facing rotation attack and attack with full knowledge of our pipeline in Appendix C. We also mention that ZoDiac focuses on zero-bit watermarking without meaningful information in the conclusion. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
the zodiac watermarking framework: 
 This section introduces ZoDiac, our novel watermarking technique that uses pre-trained stable diffusion models to achieve watermark invisibility and strong robustness to attacks. The core idea is to learn a latent vector that encodes a pre-defined watermark within its Fourier space, and can be mapped by pre-trained stable diffusion models into an image closely resembling the original. Figure 2 : Overview of ZoDiac with watermark embedding and detection phases. There are three major steps in the embedding phase: 1) latent vector initialization, 2) watermark encoding, and 3) adaptive image enhancement. In the detection phase, the watermark is decoded by performing DDIM inversion, Fourier transformation, and statistical testing.Figure 2 illustrates both the watermark embedding and detection phases of the ZoDiac framework. We first explain the high-level idea, and then §3.2 and 3.3 detail each step.Watermark Embedding. Watermark embedding consists of three main steps, latent vector initialization, watermark encoding, and adaptive image enhancement. Algorithm 1 lists the pseudocode for watermark embedding.Require: original image x 0 , watermark W Require: pre-trained diffusion model G, and its inversion G ′ Require: diffusion steps T, latent update steps N Require: SSIM threshold s * Ensure: watermarked image x01: Z T ← G ′ (x 0 ) 2: for i = 1 to N do 3: x0 ← G(Z T ⊕ W) 4:Take gradient descent on ∇ Z T L(x 0 , x0 ) {Eq. (6)} 5: end for6: x0 = x0 + γ(x 0 -x0 ) {Eq. (7)} 7: Search γ * ∈ [0, 1] s.t. S(x 0 , x 0 ) ≥ s * {Eq. (8)} 8: return x0The original image x 0 first undergoes a DDIM inversion process to identify its latent vector Z T (Latent Vector Initialization, Line 1, §3.2.1). Zo-Diac then encodes a watermark in the latent vector Z T and trains the watermarked latent vector such that a pre-trained stable diffusion model can use it to generate a watermarked image similar to the original image (Watermark Encoding, Lines 2-5, §3.2.2). To encode the watermark into Z T , ZoDiac converts Z T to the Fourier space, encodes a given watermark W, and then transforms it back to the spatial domain prior to being fed into the diffusion model. Zo-Diac preserves the visual similarity between the generated image x0 and the original image x 0 by optimizing the latent vector via a carefully designed reconstruction loss. The diffusion model remains frozen and operates in inference mode during the optimization. The gradient flow during backpropagation is indicated by the dashed red arrows in Figure 2 . After image generation, ZoDiac mixes the watermarked image x0 with the original one x 0 to further enhance image quality (Adaptive Image Enhancement, Lines 6-7, §3.2.3).Watermark Detection. Watermark detection detects watermarks. ZoDiac first reconstructs the latent vector of the image under inspection via DDIM inversion, transforms the latent vector into the Fourier space, and then conducts a statistical test to detect a potential watermark ( §3.3).Finding a good initialization for the latent vector Z T allows a stable diffusion model to reproduce the original image x 0 rapidly, and is thus critical to reducing the time spent on optimizing that latent vector. ZoDiac employs DDIM inversion to initialize the latent vector Z T = G ′ (x 0 ), from which DDIM can then remap to the original image. Empirically, we find that using a randomly initialized latent vector can require hundreds of iterations when remapping, while using an initial latent vector from DDIM inversion converges in dozens of iterations while achieving a higher image quality (see details in Appendix B.1).The watermark encoding step aims to encode a watermark into an image with minimal impact on its visual quality. ZoDiac injects the watermark into the Fourier Space of the initialized latent vector from §3.2.1, and then optimizes the watermarked latent vector to ensure that it can be used to synthesize an image resembling the original. Our fundamental insights are two-fold: (1) injecting watermarks into the latent space of images can effectively improve the watermark robustness thanks to the potential attack-defense capability provided by the diffusion process (see §4.3); (2) injecting watermarks into the Fourier representation of a latent vector helps preserve the quality of the watermarked image (see Appendix B.7).Watermark Injection. ZoDiac injects a concentric ring-like watermark into the Fourier space of the latent vector, leading to a circularly symmetric watermark embedding in the low-frequency domain. Such a pattern is proved to be resistant to geometric transformations and common image distortions [32, 42] .ZoDiac assumes that elements of a watermark W are generated by randomly sampling from a complex Gaussian distribution, noted as CN (0, 1). Elements that are of the same distance to the center of the latent vector have the same value, making the watermark "ring-like". Formally, let F(Z T ) ∈ C ch×w×h be the Fourier transformed latent vector, where ch is the number of channels, and w and h are the width and height. Let p = (i, j) be a coordinate, c = (h/2, w/2) be the latent vector's center, and d(p, c) be the Euclidean distance from a coordinate to the center. Each element in the watermark W ∈ C w×h is:W p = w ⌈d(p,c)⌉, where w ⌈d(p,c)⌉ ∼ CN (0, 1).ZoDiac also needs a binary mask that indicates the location where the watermark will be applied.Mathematically, let M ∈ {0, 1} w×h be a binary mask with a predefined radius d * . Each item in the mask M isEQUATIONFinally, the watermark W is applied to the Fourier-transformed latent vector F(Z T ) with the binary mask M,EQUATION)where ic is the watermark injecting channel and ⊙ denotes the element-wise product. We denote the latent vector after watermarking as Z T ⊕ W.Latent Vector Optimization. ZoDiac then seeks to find a latent vector Z T such that, when being watermarked by W and then processed by the denoising process G, generates an image x0 that is the most similar to the original image x 0 . To solve the optimization problem, we design a reconstruction loss L that allows ZoDiac to iteratively refine Z T via standard backpropagation:EQUATIONwhere x0 = G(Z T ⊕ W), L 2 is the Euclidean distance, L s represents the SSIM loss [50] , L p corresponds to the Watson-VGG perceptual loss [12] , and λ s , λ p are weighting coefficients. These coefficients are set to balance the scale of each loss component.Adaptive image enhancement aims to improve the visual quality of the image x0 generated from the watermark encoding step. To do this, it adaptively mixes x0 with the original image x 0 such that the mixed image can meet a desired image quality threshold. Mathematically, the mixed image x0 is:EQUATIONwhere γ ∈ [0, 1] is a modulating factor. A higher γ improves image quality at the cost of potential watermark diminishing. Adaptive image enhancement automatically identifies the smallest γ that results in desired image quality through binary search. It optimizes the following objective:EQUATIONwhere s * is the desired image quality and S is an image similarity metric. We use the SSIM metric by default.In the watermark detection phase, our primary objective is to verify whether a given image x 0 contains watermark W. The detection process starts with transforming x 0 to y = F(G ′ (x 0 ))[-1, :, :] ∈ C w×h , representing the last channel of the Fourier transformed latent vector. It then detects the presence of W in y via a statistical test procedure. The statistical test computes a p-value, an interpretable statistical metric that quantifies the likelihood of the observed watermark manifesting in a natural image by random chance. A watermark is considered detected when the computed p-value falls below a chosen threshold.To compute the p-value, we first define the null hypothesis as H 0 : y ∼ N (0, σ 2 , I C ), where σ 2 is estimated for each image from the variance of y masked by the circular binary mask M, i.e.,σ 2 = 1 M (M ⊙ y) 2. This is because the DDIM inversion G ′ maps any test image x 0 into a Gaussian distribution [31] , and the Fourier transformation of a Gaussian distribution remains Gaussian.To test this hypothesis, we define a distance score η that measures the disparity between W and y in the area defined by the binary mask M:EQUATION)Under H 0 , η follows a non-central chi-squared distribution [26] , characterized by M degrees of freedom and a non-centrality parameter λ = 1 σ 2(M⊙W) 2 . An image is classified as watermarked if the value of η is too small to occur by random chance. The probability of observing a value no larger than η, i.e., the p-value, is derived from the cumulative distribution function of the non-central chi-squared distribution. Non-watermarked images will exhibit higher p-values, while watermarked images yield lower values, indicating a successful rejection of H 0 and confirming the watermark's presence. In practice, we treat (1 -p)-value as the likelihood of watermark presence and set up a detection threshold p * to determine the watermark presence An image with (1-p) > p * is considered to be watermarked.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [No] Justification: We do not include theoretical results. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
empirical evaluation: 
 This section evaluates ZoDiac's efficacy using a diverse domain of images including real photographs, AI-generated content, and visual artwork.Datasets. Our evaluation uses images from three domains, photographs, AI-generated images, and visual artwork. For each domain, we randomly sample 500 images from well-established benchmarks, including MS-COCO [23] , DiffusionDB, and WikiArt. ZoDiac Settings. We use the pre-trained stable diffusion model stable-diffusion-2-1-base [29] with 50 denoising steps. We show ZoDiac is compatible with other diffusion models in § 4.4. We optimize the trainable latent vector for a maximum of 100 iterations using the Adam optimizer [20] . It takes Table 1 : Image quality in terms of PSNR, SSIM, and LPIPS, and watermark robustness in terms of Watermark Detection Rate (WDR) before and after attacks, on MS-COCO, DiffusionDB, and WikiArt datasets. We evaluate on ten individual attacks and two composite attacks: "All" that combines all the individual attacks, and "All w/o ⟳" that excludes the rotation attack. ↑ and ↓ indicate whether higher or lower values are better. For each attack, we highlighted in gray the techniques with the maximum WDR, and those within 2% of the maximum; ZoDiac is the only method within 2% of the maximum WDR for all attacks, except Rotation. ZoDiac dominates all methods when facing the most advanced Zhao23 attack (recall Figure 1 ) and the combined All w/o ⟳. . We calibrate the weights for the SSIM loss λ s and the perceptual loss λ p to 0.1 and 0.01, respectively, to balance the scales of the various loss components. We set the watermark injecting channel ic to be the last channel of the latent representation and the watermark radius d * to 10. To balance the image quality and detectability of the watermark, we set the SSIM threshold s * to 0.92 and the detection threshold p * to 0.9, except where explicitly noted otherwise. We evaluate other hyperparameter settings in the ablation study (see §4.4 and Appendix B).Watermarking Baselines. We compare ZoDiac to six watermarking methods.• Traditional Methods: Two conventional techniques, DwtDct and DwtDctSvd [9] , utilize frequency decomposition. DwtDctSvd has been used to watermark stable diffusion models [29] .• Training-Based Methods: RivaGAN [48] is a pre-trained GAN-based model incorporating an attention mechanism. SSL [16] is a latent-space-based watermarking method that uses selfsupervised learning for network pre-training. CIN [25] combines invertible and non-invertible mechanisms to achieve high imperceptibility and robustness against strong noise attacks. StegaStamp [35] applies adversarial training and integrates both CNN and spatial transformer techniques.Considering the above methods inject bitstrings as the watermark, we use 32-bit messages for DwtDct, DwtDctSvd, RivaGAN, SSL, and CIN, and 96-bit messages for StegaStamp. Detection thresholds were set to reject the null hypothesis (i.e., H 0 : the image has no watermark) with p < 0.01, requiring correct detection of 24/32 and 61/96 bits for the respective methods to form a fair comparison [51] .Watermark Attack Methods. We evaluate ZoDiac's robustness against a comprehensive set of attacks used in recent watermarking evaluations [51, 1] . The set of attacks employed includes:• Adjustments in brightness or contrast with a factor of 0.5.• JPEG compression with a quality setting of 50.• Image rotation by 90 degrees.• Addition of Gaussian noise with a std of 0.05.• Gaussian blur with a kernel size of 5 and std of 1.• BM3D denoising algorithm with a std of 0.1.• VAE-based image compression models, Bmshj18 [2] and Cheng20 [8] , with a quality level of 3.• A stable diffusion-based image regeneration model, Zhao23 [51] with 60 denoising steps.Since the attack strength tuned by the hyper-parameters of different attack methods will influence the presence and detection of the injected watermark, we evaluate the effectiveness of ZoDiac under other settings in the ablation study (see §4.4 and Appendix B). Besides, unlike prior work, we go deeper and evaluate against the composite attack that combines all the aforementioned attacks (named "All") and a variant without rotation (named "All w/o ⟳").We also examine the robustness of ZoDiac under pipeline-aware attack, where the attacker has partial or full knowledge of our watermarking method, in Appendix C.2. Our results confirm the robustness of ZoDiac in the face of partial knowledge attacks and underscore the critical importance of safeguarding the model weights and watermark configurations.Evaluation Metrics. We evaluate the quality of the watermarked image (x) compared to the original image (x) using three metrics: Peak Signal-to-Noise Ratio (PSNR)(x, x) = -10 log 10 (MSE(x, x)), Structural Similarity Index (SSIM) [40] with range [0, 1], and LPIPS [49] that measures perceptual similarity. For PSNR and SSIM, larger values correspond to higher image similarity; for LPIPS, lower values do. To assess watermark robustness, we use the Watermark Detection Rate (WDR) for watermarked images, which is the same as True Positive Rate (TPR), and False Positive Rate (FPR) for non-watermarked images. Both WDR and FPR range from 0 to 1. We expect to achieve high WDR and low FPR.Table 1 reports the watermarked image quality and WDR before and after attacks for ZoDiac and baselines.Image Quality. ZoDiac achieves satisfactory image perceptual similarity, as defined by [7] and [34] , of approximately 30dB PSNR and 0.92 SSIM. ZoDiac's image quality outperforms the previously most robust watermarking method, StegaStamp. Figure 11 in Appendix E illustrates the visually imperceptible effect of ZoDiac's and other methods' watermarks. §4.4 examines how increasing ZoDiac's SSIM threshold can further improve image quality, trading off WDR.Watermark Robustness. ZoDiac consistently exhibits high detection rates, within 2% of the maximum WDR, against watermark removal algorithms as highlighted by the gray cells (except for the Rotation attack). Traditional approaches, such as DwtDct and DwtDctSvd, consistently fail with brightness and contrast changes, rotation, and advanced generated-AI-based attacks. RivaGAN and SSL fail for advanced attacks, Bmshj18, Cheng20, and Zhao23. StegaStamp, despite being robust across most attacks, fails for rotation and demonstrates significantly diminished detection rates under the diffusion model-based attack Zhao23. When facing the composite attack excluding rotation ("All w/o ⟳" column), ZoDiac maintains a detection rate of about 0.5, while all baselines fail, with a WDR close to 0. Appendix B.8 provides more discussions on composite attacks.The exception of rotation is noteworthy. All methods, apart from SSL which incorporates rotation during training, fail for rotational disturbances. In ZoDiac, this limitation stems from the non-rotationinvariant nature of the DDIM inversion process; the latent representation derived from an image and its rotated version differ significantly. However, rotation is not an invisible attack and its effects are readily reversible, allowing users to manually correct image orientation to facilitate watermark detection. Appendix C.1 presents an extended discussion on overcoming rotational attacks.Recall that the image generation process is an iterative algorithm that progresses from x T to x 0 , involving the prediction of an approximate x 0 at each step and the subsequent addition of noise to obtain the noisy input for the next step. We hypothesize that this reciprocating denoising process may inherently enhance the robustness of the watermark when faced with strong attacks. To validate the hypothesis, we experiment with different denoising steps when reconstructing the watermarked image on the MS-COCO dataset. In addition to the commonly-used 50 denoising steps, we further evaluate denoising steps of 10, 1, and even 0, where 0 means utilizing only the well-trained image autoencoder in the diffusion model without the diffusion process.Table 2 reports the watermarked image quality in terms of PSNR when setting the SSIM threshold s * = 0.92 and the robustness when subjected to three representative attacks. Appendix B.2 contains the full table with all attacks. We make two observations. First, the number of denoising steps do not significantly influence the watermarking performance. Regardless of the attack methods employed, ZoDiac exhibits consistently high WDR while maintaining similar image quality with different denoising steps. Second, even when reduced to a single denoising step, ZoDiac significantly outperforms the variant without the diffusion process (i.e., when the denoising step is 0) in terms of watermark robustness, thereby verifying the hypothesis that the diffusion model contributes substantially to the robustness of the embedded watermark.Varying the SSIM Threshold s * . While a high image quality threshold s * can yield an image with negligible quality loss, it conversely raises the risk of watermark elimination. This study explores the impact of varying s * on image quality and watermark robustness. Figure 3 illustrates the trade-off curves on the MS-COCO dataset, subject to four advanced attack methods, by adjusting SSIM thresholds s * ∈ [0.8, 0.98] in increments of 0.03. Appendix B.3 reports results for eight other attacks and two other datasets. Overall, against advanced watermark removal techniques, ZoDiac consistently outperforms all baseline methods in terms of robustness. Although the adaptive image enhancement step can also enhance the watermarked image quality of StegaStamp, the most robust baseline method, it causes a significant decline in its watermark robustness, as shown by the pink line in Figure 3 . StegaStamp injects watermarks in the image space, making it more sensitive to the enhancement. By contrast, ZoDiac injects watermarks in the latent space of images, benefiting from the enhancement step and achieving a balance between high image quality and watermark robustness.Varying the Detection Threshold p * . Recall from §3.3 that an image is considered watermarked if its (1-p)-value exceeds p * . A larger p * imposes a stricter criterion for watermark detection, reducing the probability of both watermark detection for watermarked images, measured by watermark detection rate (WDR), and false detection for non-watermarked images, measured by false positive rate (FPR). Table 3 reports the effects of detection thresholds p * ∈ {0.90, 0.95, 0.99} on WDR and FPR before and after three representative attacks on MS-COCO dataset. Appendix B.4 contains the full table with all attacks for the three datasets and also provides the FPR of baselines for reference. The data show that p * = 0.9 maintains a high WDR and an acceptable FPR, in practice.Varying the Backbone Models. ZoDiac is compatible with different pre-trained stable diffusion models. In this study, we extend our evaluation to the stable-diffusion-v1-4 and stable-diffusion-xlbase-1.0 (in addition to stable-diffusion-2-1-base). Figure 4 shows the trade-off curves between the watermarked image quality and the watermark detection rate without attack as well as with three representative attacks, Rotation, Zhao23, and All w/o ⟳. Appendix B.5 shows additional results addressing other attacks. The data shows consistent performance (WDR and image quality in SSIM) between the two models, as indicated by the almost overlapped solid and dashed lines, suggesting that ZoDiac can seamlessly integrate with other pre-trained stable diffusion models, maintaining its state-of-the-art efficiency, regardless of the backbone model employed.Varying the Attack Strength. We also compare ZoDiac with the strongest baseline StegaStamp under different hyper-parameter settings of the attack methods, in terms of WDR under different FPR levels. Table 4 reports the image quality in terms of PSNR and the watermarking robustness when we adjust the Brightness and Contrast attacks with a factor of 0.2 to 1.0 following WAVES [1] . As the adjustment factors for brightness and contrast decrease from 0.9 to 0.2, the PSNR of the attacked watermarked images drops significantly-from 37.79 to 11.41 for brightness and from 38.57 to 14.54 for contrast. Despite this degradation, ZoDiac consistently maintains a WDR above 0.99 across all scenarios. Even at extreme settings, where PSNRs fall to 11.41 for brightness and 14.54 for contrast, ZoDiac significantly outperforms StegaStamp, achieving higher WDR and lower FPR. This clearly demonstrates ZoDiac 's robustness even under severe quality degradation. Appendix B.6 includes additional results when tuning other attacks.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: We provide the pseudocode of ZoDiac in Algorithm 1 to help with implementing the watermarking method, and also provide configurations for all the necessary hyperparameters in Section 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
contributions: 
 We have presented ZoDiac, a robust watermarking framework based on pre-trained stable diffusion models. ZoDiac hides watermarks within the trainable latent space of a pre-trained stable diffusion model, enhancing watermark robustness while preserving the perceptual quality of the watermarked images. An extensive evaluation across three datasets of photographs, AI-generated images, and art demonstrates ZoDiac's effectiveness in achieving both invisibility and robustness to an array of watermark removal attacks. In particular, ZoDiac is robust to advanced generative-AI-based removal methods and composite attacks, while prior watermarking methods fail in these scenarios. This robustness, coupled with the ability to achieve high-quality watermarked images, positions ZoDiac as a significant advancement in the field of image watermarking. ZoDiac focuses on zero-bit watermarking [17] , only hiding and detecting a mark. ZoDiac is constrained to adhere to a Gaussian distribution in the latent vector of the diffusion model, but future work will explore encoding meaningful information, such as a message, in the watermark while preserving ZoDiac's robustness.Digital Image Watermarking. Digital watermarking, particularly in the image domain, has been a cornerstone in computer vision for decades. Traditional methods have predominantly relied on frequency decomposition techniques such as Discrete Fourier Transform (DFT) [37] , Discrete Cosine Transform (DCT) [6] , Discrete Wavelet Transform (DWT) [44] , or their combinations [9] . These frequency-based methods are advantageous due to their inherent resilience to common image manipulations.The development of deep neural networks (DNN) has introduced novel learning-based watermarking approaches. HiDDeN [53] employs joint training of watermark encoder and decoder alongside noise layers that mimic image perturbations. It leverages an adversarial discriminator for enhanced visual quality and has been expanded for arbitrary image resolutions and message lengths in [22] . Regarding the watermark robustness, StegaStamp [35] uses differentiable image perturbations during training to improve noise resistance and incorporates a spatial transformer network for minor perspective and geometric changes. Distortion Agnostic [24] further brings robustness to unforeseen transformations by introducing adversarial training. Considering the watermarking as an image generation process, RivaGAN [48] , SteganoGAN [47] , and ARWGAN [19] resort to generative AI, particularly Generative Adversarial Network (GAN), for enhanced performance and robustness in watermarking. A comprehensive review is provided in [38] . With the demonstrated efficacy of diffusion models in image generation [14] , our work pioneers the use of pre-trained diffusion models for robust image watermarking.Image Watermarking Attack. Watermarking attacks are typically categorized into two types [51] . Destructive attacks treat the watermark as part of the image, seeking to remove it through image corruption. Common methods include altering image brightness or contrast, applying JPEG compression, introducing Gaussian noise, and so on. Constructive attacks, on the other hand, view the watermark as noise superimposed on the original image, focusing on its removal via image purification techniques such as Gaussian blur [18] , BM3D [13] , and learning-based methods like DnCNNs [46] . Recently, regeneration attacks have emerged, leveraging the strengths of both destructive and constructive approaches. These attacks corrupt the image by adding Gaussian noise to its latent representation, then reconstruct it using generative models such as Variational AutoEncoders [2, 8] or Stable Diffusion [29] . We aim to effectively counter watermark removal attacks especially advanced generative AI-based ones when watermarking given images.Diffusion Model Watermarking. Rapid evolution in deep generative models has led to methods capable of synthesizing high-quality, realistic images [14] . These models pose threats to society due to their potential misuse, prompting the necessity to differentiate AI-generated images from those created by humans. Recent efforts resort to watermarking the model itself, which focuses on enabling these models to automatically generate watermarked images for easier identification later. One straightforward approach is to fine-tune diffusion models with datasets containing watermarked images, leading these models to inherently produce watermarked outputs [11, 39, 52, 45] . Alternatives like Stable Signature [15] focus on training a dedicated watermark decoder to fine-tune only the latent decoder part of the stable diffusion model. Tree-Rings [42] takes a different route by embedding watermarks in the initial noisy latent, detectable through DDIM inversion [14] . Although these methods effectively lead diffusion models to generate watermarked images, they are not equipped for watermarking existing images, a functionality our approach is designed to provide. The result shows the advantage of the latent vector initialization step, which achieves a higher quality image, measured by the L2 distance between the image and its original counterpart within only 20 iterations. In contrast, a randomly initialized latent vector requires more than hundreds of iterations to produce a similar image whose quality (L2 = 0.0036) still largely falls behind. The watermarking performance comparisons between three pre-trained stable diffusion models, stable-diffusion-2-1-base, stable-diffusion-v1-4, and stable-diffusion-xl-base-1.0. The tradeoff curves between the watermarked image quality and the watermark detection rate with and without attack are provided.shows the trade-off curves between the watermarked image quality and the watermark detection rate with and without attack. Overall, the three backbone models exhibit similar watermarking performance.We conducted additional experiments under various attack levels, following the settings in WAVES [1] . The experiments span multiple attack scenarios:• Tables 8 and 9 : Adjustments in brightness or contrast with a factor of 0.2 to 1.0 (i.e., the watermarked images without being attacked). • Table 10 : JPEG compression with a quality setting from 10 to 90 • Table 11 : Gaussian noise with a standard deviation varying from 0.02 to 0.1 • Table 12 : Gaussian blur with a kernel size of 5 to 19.• Tables 13 and 14 : Two VAE-based image compression models, Bmshj18 and Cheng20 with quality levels of 2 to 6. • Table 15 : A stable diffusion-based image regeneration model, Zhao23 with 40 to 200 denoising steps.We compared ZoDiac and the strongest baseline, StegaStamp, in terms of WDR under different FPR levels, leading to several insights: Brightness and Contrast Attacks: As the adjustment factors for brightness and contrast decrease from 0.9 to 0.2, the PSNR of the attacked watermarked images drops significantly-from 37.79 to 11.41 for brightness and from 38.57 to 14.54 for contrast. Despite this degradation, ZoDiac consistently maintains a WDR above 0.99 across all scenarios. Even at extreme settings, where PSNRs fall to 11.41 for brightness and 14.54 for contrast, ZoDiac significantly outperforms StegaStamp, achieving higher WDR and lower FPR. Specifically, ZoDiac records a WDR of 0.994 at an FPR of 0.032 compared to StegaStamp's 0.852 at an FPR of 0.056 for brightness, and a WDR of 0.998 at an FPR of 0.032 compared to StegaStamp's 0.730 at an FPR of 0.056 for contrast. This clearly demonstrates ZoDiac 's robustness even under severe quality degradation.JPEG, G-Noise, G-Blur, Bmshj18, and Cheng20 Attacks: With varying attack parameters, including the quality settings for JPEG, Bmshj18, Cheng20, the standard deviation for G-Noise the kernel size for G-Blur, the PSNR of the attacked watermarked images remains within the range of 20 to 40. In these scenarios, both ZoDiac and StegaStamp consistently sustain a high WDR of around 0.98. The only exception is JPEG with a quality setting of 10. Under this condition, the WDR of both ZoDiac and StegaStamp slightly decreases from their usual range of 0.956-0.986 to 0.746-0.834 for ZoDiac, and from 0.996-1.0 to 0.758-0.806 for StegaStamp.Zhao23: As the number of image regeneration steps in Zhao23 increases, the PSNR of the attacked watermarked images decreases from 27.15 to 22.44. Despite this degradation, ZoDiac remains highly resistant across most scenarios, with its WDR slightly decreasing from 0.998 to 0.898. In contrast, StegaStamp significantly underperforms, completely failing when the regeneration steps exceed 120. The rotation attack presents a significant challenge for most existing watermarking methods, including ZoDiac (see Table 1 ). There are two strategies to mitigate this issue. The first approach, exemplified by SSL, integrates rotation into the model's training phase as a type of data augmentation. However, this method is incompatible with ZoDiac's zero-shot nature. The second approach involves rectifying the image's orientation prior to watermark detection. This correction can be achieved manually or via an automated process by systematically rotating the image through a series of predetermined angles, spanning the full 360-degree spectrum and performing watermark detection at each increment. We call this the "rotation auto-correction" defense. In this study, we test rotation auto-correction with ZoDiac. We conduct experiments using intervals of 5, 10, and 30 degrees and report the Watermark Detection Rate (WDR) for watermarked images subjected to the 90-degree rotation attack, and the False Positive Rates (FPR) for unmarked images. Table 18 reports the results on the MS-COCO dataset. As expected, applying rotation auto-correction enables ZoDiac to achieve a higher WDR. However, it also increases the risk of detecting a watermark in unmarked images, leading to a higher FPR. A higher detection threshold p * = 0.99 can greatly suppress FPR with a slight decrease in WDR, decreasing from 0.244 -0.676 to 0.014 -0.034. Therefore in practice, to deploy the rotation auto-correction defense, the higher detection threshold is appropriate.Conducting a pipeline-aware attack involves simulating what a knowledgeable attacker would do if they were aware of how our watermarking works. The goal is to evaluate the robustness of ZoDiac against real-world threats where attackers might have insights into the defense strategies used. Specifically, we assume the attacker can inject another watermark to potentially disrupt or overwrite the original one by using ZoDiac, while testing whether the original watermark can still be preserved. Table 19 reports the Watermark Detection Rate (WDR) for both the original and newly injected watermarks under four scenarios according to the extent of information available to the attacker. When the attacker has only partial knowledge of our watermarking method, such as lacking the model weights (i.e., the second row) or being unaware of the watermark injection configurations, including the injection channel and watermark radius (i.e., the third row), or both (i.e., the first row), the original watermark remains undisturbed by the new one and it is successfully detected with a high WDR. However, when the attacker has complete knowledge of the watermarking pipeline used for injecting the original watermark (i.e., the fourth row), the original watermark is unfortunately overwritten by the new one. The results confirm the robustness of our proposed ZoDiac in the face of partial knowledge adaptive attacks and also underscore the critical importance of safeguarding the model weights and watermark configurations to ensure security.The GPU usage of ZoDiac is 15,552MiB when operating with 32-bit floating-point precision. This memory is primarily attributed to the activation maps and model parameters required for the diffusion model computations, where the model parameters take 5,308MiB memory. All the experiments are conducted with 1 NVIDIA RTX8000.Since ZoDiac is a zero-shot watermarking method based on the well-trained diffusion model, the time cost is solely attributed to optimizing the noisy latent representation of the given image embedded with the watermark. Therefore the time cost is influenced by the selected denoising step during the reconstruction of the watermarked image. The completion of reconstruction is determined when the SSIM loss falls below 0.2 (i.e., the SSIM metric for the watermarked image quality exceeds 0.8) or the maximum number of iterations (i.e., 100) is reached. As shown in Table 20 , the larger the denoising step, the longer each iteration will take, consequently increasing the overall time cost. Regarding average convergence iterations, as the denoising step decreases, ZoDiac requires slightly fewer iterations to achieve similar performance. This might be attributed to the shallower steps of gradient backpropagation, leading to faster convergence. Notice that when the denoising step is decreased to 1, it becomes harder to reconstruct a watermarked image with a quality above 0.8, making it more challenging to trigger the early exit condition based on the SSIM loss threshold. Consequently, it leads to longer convergence iterations in this case. Additionally, we have demonstrated that the denoising steps do not significantly impact the watermarking performance. The detailed results and analysis are provided in Appendix B.2. Therefore, we could always choose small denoising steps to effectively embed robust watermarks while maintaining a reasonable time cost.Figure 11 shows several image examples, taken from each of our three datasets. For each image, we show the original, watermarked versions with seven watermarking methods, including ZoDiac, and the residual images (i.e., the watermarked image minus the original image). We observe that ZoDiac does not introduce significant visible degradation. We can hardly observe image differences for the DwtDct, RivanGAN, and SSL methods, a small amount of differences for DwtDctSvd, CIN, and our proposed ZoDiac, and a significant amount for StegaStamp. Notably, DwtDct, RivanGAN, and SSL suffer from poor robustness against advanced attacks, while ZoDiac achieves a favorable trade-off between visual quality preservation and robust watermarking. Our work advances the robust image watermarking research field. In particular, we produce the first diffusionmodel-based image watermarking method that is resilient to advanced watermark removal attacks, such as a diffusion-model-based attack and attacks composed of multiple, individual attack methods. Our method enables content protection and authenticity verification, positively impacting visual artists and other visual content creators. Our technique may use machine learning and is thus susceptible to some of the pitfalls of machine learning, such as bias, which may result in negative societal impact.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: The code Github link is provided in the abstract. Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We provide configurations for all the necessary hyperparameters in Section 4.1 and also additional ablation studies for the choice of these hyperparameters in Section 4.4 and Appendix B. Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 7. Experiment Statistical Significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [No] Justification: We do not report error bars, since our watermarking method is a deterministic process that will inject the watermark into the given image. The watermark detection rate will only be changed when using different SSIM thresholds and detection thresholds, and we discuss the influence of different configurations in Section 4.4 and Appendix B. Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: This paper has received consistent feedback from all three reviewers. The reviewers engaged in thorough discussion and rebuttal, with most of their concerns being addressed, ultimately reaching a consensus. The paper proposes a framework for embedding invisible watermarks into existing images using a pretrained stable diffusion model. The experiments are comprehensive and effective. Therefore,  the AC  has decided to accept this paper.

Reviews: review: paper_summary: paper_summary: This paper presents ZoDiac, a novel image watermarking technique leveraging a pre-trained stable diffusion model to inject watermarks into a trainable latent space, enhancing watermark robustness against image attacks. Extensive experiments on three modern benchmarks demonstrate ZoDiac’s state-of-the-art watermark detection accuracy, outperforming existing methods in the face of traditional and generative model-based watermark removal attacks.
strength_weakness: strength_weakness: 1. While ZoDiac demonstrates excellent robustness under various attacks, as shown in Table 1, it sacrifices image quality, achieving only slightly better visual quality than StegaStamp.

2. As the authors mentioned, the main weakness of the current method is that it is limited to zero-bit watermarking. How to increase the length of the embedded information is an issue that needs to be addressed.
questions: questions: See the weaknesses.
limitations: limitations: N.A.
paper_summary: paper_summary: The paper introduces ZoDiac, an image watermarking framework leveraging pre-trained stable diffusion models to embed concentric ring-like zero-bit watermark into existing images. ZoDiac operates by injecting the watermark into the trainable latent space. The method is extensively evaluated on three benchmarks MS-COCO, DiffusionDB, and WikiArt, demonstrating strong effectiveness and robustness, outperforming existing methods. ZoDiac is notable for its ability to watermark both AI-generated and real-world images without requiring extensive retraining, and it remains effective even against combined attack scenarios.
strength_weakness: strength_weakness: - The method seems inefficient as it needs 45.4 - 255.9s to watermark one image. 
- While the paper demonstrates empirical robustness, it lacks theoretical support to explain why the approach is resilient to various attacks. The hypothesis about the reciprocating denoising process enhancing robustness of the watermark is not rigorously proven.
- The paper lacks experiments with watermark of different patterns apart from the concentric ring-like one.
questions: questions: - Since previous works use different types of watermark, such as StegaStamp uses hyperlink bitstring, there is a concern regarding comparison fairness, hence the demonstrated results might be misleading. It would be better if the authors can provide clarification on this. 
- It will be interesting to see if ZoDiac can work with watermark of different patterns. 
- Other than UNet-based latent diffusion models like Stable Diffusion, it will be exciting to see if ZoDiac can work with Transformer-based ones like Pixart-alpha [1] or DiT [2].

[1]. Chen, J., Yu, J., Ge, C., Yao, L., Xie, E., Wu, Y., Wang, Z., Kwok, J., Luo, P., Lu, H., Li, Z.: PixArt-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In: ICLR (2024)

[2]. William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
limitations: limitations: The authors have adequately addressed the limitations and potential negative societal impact of their work.
paper_summary: paper_summary: This paper presents a new image watermarking method, ZoDiac, which leverages pretrained stable-diffusion model to generate watermark in the latent space (derived by using DDIM to process the original image). The author claims that ZoDiac achieves better robustness than the state-of-the-art (SOTA) watermarking methods.
strength_weakness: strength_weakness: Lack of sufficient details for the benchmark methods (both other watermark competitors and the attack methods), making it questionable if the comparison of robustness is fair and significant; the experiments and the logic conveyed in this paper is not convincing enough to support the claim that ZoDiac is more robust than the existing SOTA method, i.e.., StegaStamp. Please see “Questions” below for detailed comments.
questions: questions: 1. Are all watermarking methods compared fairly?

As stated in line 238 and 252, the watermark detection for the proposed ZoDiac relies on a threshold of (1-p). Analogously, for all other watermark competitors considered in this paper (DwtDctSvd, RivaGAN, SSL, CIN, StegaStamp), the watermark detection relies on thresholding the bitwise accuracy (or bit-error-rate in CIN) between the decoded bitstring and the ground truth. Altering these thresholds will lead to different trade-off points of True-Positive-Rate (TPR) and False-Positive-Rate (FPR) in the watermark detection performance. 

The authors seem to be aware of this trade-off as in line 252 they report the detection threshold for ZoDiac; in line 11 they stress that ZoDiac has more than 98% TPR and less than 6.4% FPR; and in Table 3 they present an ablation study for ZoDiac. However, the detection threshold and FPR number for other watermarking methods are never mentioned in this paper (including Appendix). Imagine if another method (let’s call it B) achieves 97% TPR and 3% FPR, is ZoDiac or B more robust? As a result, I do not think the current evaluation is fair to compare the robustness of all watermarking methods.

Considering the fact that the detection threshold for each method may not be comparable, it would be reasonably fairer to make the following comparison to demonstrate robustness—let all methods achieve the same TPR then compare their FPR; or vice versa. In fact, the best demonstration here would be profiling the TPR-FPR curve for all watermarking methods to show the whole picture.

2. Has the definition of robustness clearly explained in this paper?

According to line 94-96, the robustness is defined as “watermark being detectable in attacked images” and attacked images should be “without significantly changing the image”. Have the authors defined and justified what is “significant”? Without properly addressing the question of “significance”, the robustness is essentially a trade-off between three quantities: (1) TPR, (2) FPR, and (3) attack image quality (inverse significance). Imagine that the attacked images are all with very severe additive gaussian noise such that no watermark can be correctly detected by any method considered, can we conclude that all watermarks are equally (non)robust? An example of justification can be found in [1], e.g., the contribution (1) and (2) mentioned. This may not be perfect but explains why [1] performs the robustness evaluation as Fig 3. I suggest the authors carefully consider this question, make the justification of their consideration and adjust their evaluation experiment accordingly, which may change/bring new insights to their robustness conclusions.

3.  Can you justify the selection of the attack methods used in testing the robustness? What is the criterion/consideration in selecting these attacks?

All the benchmark attacks considered in this paper (Brightness, Contrast, JPEG, Gaussian noise, Gaussian blur, BM3D, Bmshj18, Cheng20, Zhao23) have tunable parameters (e.g., corruption level, standard deviation, compression rate, kernel size, etc.) to adjust the attack strength. However, the author did not provide any information on what levels of these attacks are used in the experiment. Altering these parameters will inevitably affect the TPR-FPR-imageQuality trade-off. It is essential for the authors to provide details of the choice of these parameters and justify how it is related to the robustness considered in this paper.

[1] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.
limitations: limitations: N/A. There's no possible negative social impact that needs to discuss explicitly.

Comments: title: Reponse to Reviewer AMWU
Final rating updated
Reponse to Reviewer AMWU
Response to Reviewer mbRv
Reponse to Reviewer r9M9
Response to the reviewer
Response to the reviewer (3/3)
Response to the reviewer (2/3)
Response to the reviewer (1/3)
Reply to the authors
Response to the reviewer (3/3)
Response to the reviewer (2/3)
Response to the reviewer (1/3)
Reply to the authors
Response to the reviewer (2/2)
Response to the reviewer (1/2)
Reply to the authors
comment: Thank you for your kind words and positive feedback. We are glad that most of your concerns have been addressed, and we greatly appreciate your final recommendation for acceptance. We will ensure that the agreed-upon results are included in the final version.
Dear authors,

Your efforts are much appreciated. Most of my concerns have been addressed sufficiently. Therefore, my final recommendation for this paper has been updated as **accept**. If the paper is finally accepted, I hope the authors can find enough space to include what we all have agreed on.
With the author-reviewer discussion period ending later today, we wanted to again thank you for a constructive discussion, and for helping us improve our manuscript.  We also would like to make a plea. We believe our paper accomplishes all important requirements to be accepted. Specifically, our paper (1) tackles an important problem, (2) makes considerable progress on that problem over the state of the art, (3) makes precise concrete claims that capture that progress, without overclaiming, and (4) supports those claims with empirical evidence. 

We believe from our discussion that you, too, agree that our paper accomplishes all four of those requirements and makes considerable progress on this important problem. **We would like to ask you to please consider supporting the acceptance of our paper more strongly**, as we believe its inclusion in the NeurIPS program will improve both the program and the state of science, and we hope you, too, agree with that statement.  Thank you!
With the author-reviewer discussion period ending later today, we want to reach out to thank you again for your careful review and feedback. Regarding your concern about fair comparison with baselines encoding different types of messages, we would like to relay more justifications, which we will include in our manuscript.

The fundamental problem we tackle is to **mark the images to claim ownership and effectively detect the encoded watermark in images even after being subjected to malicious attacks**. For this problem, solutions can use zero-bit or multi-bit watermarking methods.  Our evaluation compares robustness within the context of this problem in a fair way (as we explained in our global rebuttal).  Other problems may require multi-bit watermarks, but these are not the problems our paper tackles.  A single NeurIPS paper cannot do everything, and incorporating more complex watermarks while retaining ZoDiac's robustness to attacks is important future work. We are committed to making research progress in this direction.  Existing state-of-the-art evaluations, such as WAVE [1], also compare watermarking methods with different types of messages, including zero-bit method Tree-Rings and multi-bit method StegaStamp. We believe that we form a fair comparison following similar settings **for the specific problem our paper targets.**

We believe our paper makes considerable progress on this important problem and hope you can support the acceptance of our paper. Publishing this significant advance in watermarking will enable future research into more complex watermarks.  Thank you again for recognizing our work's value.

[1] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.
With the author-reviewer discussion period ending later today, we wanted to reach out to thank you again for your careful review. 

ZoDiac uses zero-bit watermarking to solve important problems of tracking image provenance and proving ownership (as our introduction properly scopes).  There are other problems that require multi-bit watermarks that other papers have tackled, but a single NeurIPS paper cannot solve all problems related to watermarking. Incorporating more complex watermarks while retaining ZoDiac's robustness to attacks is important future work, but one paper cannot do everything. We conclusively demonstrate that ZoDiac significantly improves watermark robustness, while still achieving better image quality than the strongest baseline, StegaStamp. All watermarking methods sacrifice a little image quality -- that's the nature of the underlying problem.  

We are hopeful that you can support our paper’s acceptance. Publishing this significant advance in watermarking will enable future research into more complex watermarks.  Thank you for your help.
Thank you for your thoughtful feedback and for acknowledging our efforts in the rebuttal.
We understand your concern about the comparison of different methods, particularly given the variations in message types. We will address this by adding explicit explanations in the revised paper, clarifying how the message types differ and discussing the implications of these differences in our comparisons.

Thank you again for your constructive comments and for helping us improve our work.
**Table 10.** The PSNR of attacked images and the WDR of ZoDiac and StegaStamp under different strengths of **the Bmshij18 attack**. 
| Bmshj18 quality level | Detection Threshold |  FPR  |   2   |   3   |   4   |   5   |   6   |
|:---------------------:|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|          PSNR         |          -          |   -   | 30.70 | 32.26 | 33.90 | 35.44 | 37.17 |
|       StegaStamp      |        61/96        | 0.056 | 0.996 | 0.998 | 0.998 |  1.0  |  1.0  |
|       StegaStamp      |        60/96        | 0.094 | 0.998 |  1.0  |  1.0  |  1.0  |  1.0  |
|         ZoDiac        |      $p^*=0.95$     | 0.032 | 0.986 | 0.986 |  0.99 |  0.99 | 0.992 |
|         ZoDiac        |      $p^*=0.90$     | 0.062 | 0.968 | 0.992 | 0.992 | 0.996 | 0.996 |

**Table 11**. The PSNR of attacked images and the WDR of ZoDiac and StegaStamp under different strengths of **the Cheng20 attack**. 
| Cheng20 quality level | Detection Threshold |  FPR  |   2   |   3   |   4   |   5   |   6   |
|:---------------------:|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|          PSNR         |          -          |   -   | 31.79 | 33.22 | 35.07 | 36.58 | 37.98 |
|       StegaStamp      |        61/96        | 0.056 | 0.994 |  1.0  |  1.0  |  1.0  |  1.0  |
|       StegaStamp      |        60/96        | 0.094 | 0.996 |  1.0  |  1.0  |  1.0  |  1.0  |
|         ZoDiac        |      $p^*=0.95$     | 0.032 |  0.97 | 0.978 |  0.98 | 0.992 | 0.994 |
|         ZoDiac        |      $p^*=0.90$     | 0.062 | 0.984 | 0.986 |  0.99 |  0.99 | 0.994 |

**Table 12.** The PSNR of attacked images and the WDR of ZoDiac and StegaStamp under different strengths of **the Zhao23 attack.**
| Zhao23denoising step | Detection Threshold |  FPR  |   40  |   60  |   80  |  100  |  120  |  140  |  160  |  180  |  200  |
|:--------------------:|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|         PSNR         |          -          |   -   | 27.15 | 26.28 | 25.56 | 24.91 | 24.35 | 23.81 | 23.33 | 22.86 | 22.44 |
|      StegaStamp      |        61/96        | 0.056 | 0.588 | 0.286 | 0.098 | 0.032 |  0.01 | 0.002 | 0.002 |  0.0  |  0.0  |
|      StegaStamp      |        60/96        | 0.094 | 0.674 | 0.386 | 0.144 |  0.06 | 0.024 | 0.004 | 0.002 |  0.0  |  0.0  |
|        ZoDiac        |      $p^*=0.95$     | 0.032 |  0.98 | 0.974 |  0.95 |  0.94 | 0.924 | 0.912 | 0.894 | 0.866 | 0.818 |
|        ZoDiac        |      $p^*=0.90$     | 0.062 | 0.988 | 0.988 | 0.964 |  0.97 | 0.952 | 0.946 | 0.938 | 0.926 | 0.898 |
**Table 6.** The PSNR of attacked images and the WDR of ZoDiac and StegaStamp under different strengths of **the Brightness attack**.
| Brightness factor | Detection Threshold |  FPR  |  0.2  |  0.3  |  0.4  |  0.5  |  0.6  |  0.7  |  0.8  |  0.9  |  1.0  |
|:-----------------:|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|        PSNR       |          -          |   -   | 11.41 | 13.21 | 14.56 | 21.89 | 27.95 | 30.84 | 33.58 | 37.79 | 100.0 |
|     StegaStamp    |        61/96        | 0.056 | 0.852 | 0.954 | 0.988 | 0.998 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |
|     StegaStamp    |        60/96        | 0.094 | 0.890 | 0.990 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |
|       ZoDiac      |      $p^*=0.95$     | 0.032 | 0.994 | 0.994 | 0.994 | 0.996 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 |
|       ZoDiac      |      $p^*=0.90$     | 0.062 | 0.996 | 0.996 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 |

**Table 7.** The PSNR of attacked images and the WDR of ZoDiac and StegaStamp under different strengths of **the Contrast attack**. 
| Contrast factor | Detection Threshold |  FPR  |  0.2  |  0.3  |  0.4  |  0.5  |  0.6  |  0.7  |  0.8  |  0.9  |  1.0  |
|:---------------:|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|       PSNR      |          -          |   -   | 14.54 | 17.50 | 23.03 | 28.62 | 30.56 | 33.05 | 36.57 | 38.57 | 100.0 |
|    StegaStamp   |        61/96        | 0.056 | 0.730 | 0.918 |  0.98 | 0.998 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |
|    StegaStamp   |        60/96        | 0.094 | 0.768 | 0.956 | 0.998 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |
|      ZoDiac     |      $p^*=0.95$     | 0.032 | 0.998 | 0.996 | 0.996 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 |
|      ZoDiac     |      $p^*=0.90$     | 0.062 | 0.994 | 0.996 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 |

**Table 8.** The PSNR of attacked images and the WDR of ZoDiac and StegaStamp under different strengths of **the JPEG attack**.
| JPEG quality | Detection Threshold |  FPR  |   10  |   20  |   30  |   40  |   50  |   60  |   70  |   80  |   90  |
|:------------:|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|     PSNR     |          -          |   -   | 28.05 | 30.82 | 32.28 | 33.23 | 34.02 | 34.68 | 35.62 | 36.88 | 39.09 |
|  StegaStamp  |        61/96        | 0.056 | 0.758 | 0.996 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |
|  StegaStamp  |        60/96        | 0.094 | 0.806 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |
|    ZoDiac    |      $p^*=0.95$     | 0.032 | 0.746 | 0.956 | 0.982 | 0.986 | 0.992 | 0.994 | 0.994 | 0.994 | 0.996 |
|    ZoDiac    |      $p^*=0.90$     | 0.062 | 0.834 | 0.986 |  0.99 |  0.99 | 0.992 | 0.996 | 0.996 | 0.996 | 0.998 |

**Table 9.** The PSNR of attacked images and the WDR of ZoDiac and StegaStamp under different strengths of **the G-Blur attack**. 
| G-Blur kernel size | Detection Threshold |  FPR  |   5   |   7   |   9   |   11  |   13  |   15  |   17  |   19  |
|:------------------:|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|        PSNR        |          -          |   -   | 32.17 | 29.42 | 28.00 | 27.04 | 26.22 | 25.60 | 25.08 | 24.59 |
|     StegaStamp     |        61/96        | 0.056 |  1.0  |  1.0  | 0.998 | 0.998 | 0.998 | 0.998 | 0.996 |  0.99 |
|     StegaStamp     |        60/96        | 0.094 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  | 0.996 |
|       ZoDiac       |      $p^*=0.95$     | 0.032 | 0.996 | 0.994 | 0.994 | 0.994 | 0.994 | 0.994 | 0.994 | 0.992 |
|       ZoDiac       |      $p^*=0.90$     | 0.062 | 0.996 | 0.996 | 0.994 | 0.994 | 0.994 | 0.994 | 0.994 | 0.994 |
We really appreciate your decision and enjoy the discussion with you. 

In response to your suggestions, we conducted additional experiments under various attack levels, following the settings in WAVES, and will include these findings in the revised paper. The experiments span multiple attack scenarios:
- **Tables 6 and 7**: Adjustments in brightness or contrast with a factor of 0.2 to 1.0 (original image).
- **Table 8**: JPEG compression with a quality setting from 10 to 90
- **Table 9**: Gaussian blur with a kernel size of 5 to 19.
- **Tables 10 and 11**: Two VAE-based image compression models, Bmshj18 and Cheng20 with quality levels of 2 to 6.
- **Table 12**: A stable diffusion-based image regeneration model, Zhao23 with 40 to 200 denoising steps.

We compared ZoDiac and the strongest baseline, StegaStamp, in terms of WDR under different FPR levels, leading to several insights:
- **Brightness and Contrast Attacks**: As the adjustment factors for brightness and contrast decrease from 0.9 to 0.2, the PSNR of the attacked watermarked images drops significantly—from 37.79 to 11.41 for brightness and from 38.57 to 14.54 for contrast. **Despite this degradation, ZoDiac consistently maintains a WDR above 0.99 across all scenarios.** Even at extreme settings, where PSNRs fall to 11.41 for brightness and 14.54 for contrast, ZoDiac significantly outperforms StegaStamp, achieving higher WDR and lower FPR. Specifically, ZoDiac records a WDR of 0.994 at an FPR of 0.032 compared to StegaStamp's 0.852 at an FPR of 0.056 for brightness, and a WDR of 0.998 at an FPR of 0.032 compared to StegaStamp's 0.730 at an FPR of 0.056 for contrast. **This clearly demonstrates ZoDiac’s robustness even under severe quality degradation.**
- **JPEG, G-Blur, Bmshj18, and Cheng20 Attacks**: With varying attack parameters, including the quality settings for JPEG, Bmshj18, Cheng20, and the kernel size for G-Blur, the PSNR of the attacked watermarked images remains within the range of 25 to 40. **In these scenarios, both ZoDiac and StegaStamp consistently sustain a high WDR of around 0.98.** The only exception is JPEG with a quality setting of 10, where the PSNR drops to 28.05. Under this condition, the WDR of both ZoDiac and StegaStamp slightly decreases from their usual range of 0.956-0.986 to 0.746-0.834 for ZoDiac, and from 0.996-1.0 to 0.758-0.806 for StegaStamp.
- **Zhao23**: As the number of image regeneration steps in Zhao23 increases, the PSNR of the attacked watermarked images decreases from 27.15 to 22.44. **Despite this degradation, ZoDiac remains highly resistant across most scenarios**, with its WDR slightly decreasing from 0.998 to 0.898. In contrast, StegaStamp significantly underperforms, completely failing when the regeneration steps exceed 120.

In summary, the additional ablation study demonstrates that ZoDiac achieves either competitive or superior robustness performance compared with StegaStamp across a wide range of attack scenarios, highlighting its robustness and reliability even under severe conditions.
I appreciate the authors' effective rebuttal. Most of my concerns have been addressed. However, my doubt regarding Q1 still remains. Since different methods use different types of message (32-bit messages for DwtDct, DwtDctSvd, RivaGAN, SSL, and CIN, and 96-bit messages for StegaStamp as stated), it is rather improper or misleading to compare them on the same scale as in Table 1 in the main paper without having any explanation or implication. I suggest to at least have some clarifications for them using different type of messages in the revision. Therefore, I keep my ratings unchanged.
Thank you for the prompt response. After carefully reading your response and checked your original paper, I decide to temporarily raise the rating to **weak accept**. Below are the explanations:

>About definition of robustness, acceptable attack level and authors' claim about the robustness of ZoDiac.

It is still logically unclear how the PSNR threshold in multi-media compression/signal transmission can transfer to the application of watermarks. However, setting the acceptable level in practice (e.g., to what level of content preservation can be considered copyright protected) may not be a pure technical problem, I view this logical leap a secondary issue.

In the original paper, the authors only claim that ZoDiac is a robust watermarking method, which I do agree. The avg. WDR in the rebuttal, however, did misled me with some biased impression of overclaiming. In this sense, the authors should get a fair credit based on the original paper content.

>About all examples I provided above related to fair comparison about thresholding, FPR, image quality, attack type and level, etc.

First of all, given all the experiment results so far (including ones in the paper and the additional ones in the rebuttal), I do not question the robustness of ZoDiac. The examples I gave above, however, is not for questioning the performance under a specific case. Instead, they are to explain why multiple detection thresholds (FPR/TPR cutoffs), attack levels are necessary to show more holistic pictures about robustness, which may in turn support the authors claim.

>Summary of what affects the current rating.

**Positives**: the proposed method is novel; claims are appropriate based on the current empirical evaluations.

**Negatives**: more comprehensive evaluation is still possible and not covered by the authors, and it is possible that the authors' claims may be even strengthened after such evaluation.

A further improvement on the rating is still possible if the holistic evaluation of robustness can be provided.
**Q4:** If "PSNR < 20 is deemed unacceptable" is the standard, why don't we tune the hyper-parameters of all attack methods to achieve the attack images with PSNR=20 first, then test the watermark detection performance using these attacked images and calculate the WDR?    
**Q5:** Problems related to image quality (generated by attack methods): if G-Noise attack level is doubled, it is unclear if ZoDiac (p=0.9) can maintain a 0.996 detection rate. It would be unclear if the performance of ZoDiac will reduce to 0.5; what if StegaStamp can still maintain 1 at the same time? Similar concerns can be raised for all other attacks.

**A4&5:** We agree that conducting additional experiments with more hyperparameter settings in attack methods could provide further empirical information on how ZoDiac and alternative approaches perform on attacked images with various image quality levels. Before we report some of these experiments in this direction to clarify the concerns, **we want to first emphasize that our current experiments already form a fair comparison and can faithfully reflect the robustness of ZoDiac.** Our experiment in the paper compares ZoDiac and alternative approaches against attacking methods (e.g., G-Noise) under the same hyper-parameter settings (e.g., Gaussian variance). After going through the SAME attack, our comparison assumes that the most robust watermarking method is the one that achieves the highest WDR/TPR while maintaining a low FPR, which is the same evaluation setting in many prior works including WAVES [2].

To help clarify the reviewer’s concerns on how different image quality will affect the performance of ZoDiac and alternative approaches, we compare ZoDiac and the strongest baseline StegaStamp under different hyper-parameter settings in G-Noise. Specifically, **we add Gaussian random noise (i.e., the G-Noise attack) with a standard deviation varying from 0.02 to 0.1 following WAVE \[2] and compare the WDR of ZoDiac with StegaStamp**. Below we provide the image quality of the attacked images compared to the watermarked images before the attack in Table 4. Under the same attack hyperparameter setting, the image quality of the attacked images for ZoDiac and StegaStamp are basically the same. The FPR and WDR comparisons are in Table 5. 

**Table 4. The image quality of the attacked watermarked images compared to those without being attacked in terms of PSNR.**
| G-Noise std |  0.02 |  0.03 |  0.04 |  0.05 |  0.06 |  0.07 |  0.08 |  0.09 |  0.1  |
|:-----------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|  StegaStamp | 34.00 | 30.55 | 28.10 | 26.21 | 24.67 | 23.38 | 22.26 | 21.29 | 20.42 |
|    ZoDiac   | 34.04 | 30.59 | 28.14 | 26.26 | 24.71 | 23.41 | 22.29 | 21.32 | 20.45 |

**Table 5. The WDR of ZoDiac and StegaStamp under different strengths of the G-Noise attack.** 
| G-Noise std | Detection Threshold / Accurate Bit Ratrio |  FPR  |  0.02 |  0.03 |  0.04 |  0.05 |  0.06 |  0.07 |  0.08 |  0.09 |  0.1  |
|:-----------:|:-----------------------------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|  StegaStamp |                   61/96                   | 0.056 |  1.0  |  1.0  |  1.0  | 0.998 | 0.998 | 0.998 | 0.998 | 0.998 | 0.996 |
|  StegaStamp |                   60/96                   | 0.094 |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  |  1.0  | 0.998 |
|    ZoDiac   |                 $p^*=0.95$                | 0.032 | 0.998 | 0.998 | 0.998 | 0.996 | 0.996 | 0.996 | 0.996 | 0.996 | 0.994 |
|    ZoDiac   |                 $p^*=0.90$                | 0.062 |  1.0  | 0.998 | 0.998 | 0.996 | 0.996 | 0.996 | 0.996 | 0.996 | 0.996 |

The results provide empirical evidence to answer Q4 and Q5 under the G\_noise attack: 

(1) Under the PSNR of 20 for attacked images, i.e., the G-Noise attack with std of 0.1, ZoDiac achieves competitive WDR with StegaStamp under similar FPR, which is consistent with the observations reported in the paper.

(2) When G-Noise strength is doubled from std of 0.05 to 0.1, ZoDiac can still maintain its high WDR.
**Q3:** Without reporting different FPR points for other methods (e.g., StegaStamp of CIN), it is unclear if when FPR=0.062, these methods will have much more robust performance under attack.

**A3:** We agree that comparisons under more FPR points would give a more comprehensive picture of how ZoDiac and alternative approaches perform. 

**First of all, we’d like to clarify that existing results already demonstrate the comparable robustness in terms of WDR, if not better, of ZoDiac compared to the alternative approaches under the considered FPR levels in Table 1 of the global response.** Note that the considered FPR levels for baselines are already higher than the FPR of 0.001 considered in WAVE [2] and the FPR of 0.01 in prior work [3]. This means that, if we decrease the FPR level of the baselines to be at the same level as ZoDiac (e.g., FPR=0.004), the WDR/TPR of the baselines would be even worse compared to ZoDiac. Even if we assume that their WDR/TPR will remain the same as we see in Table 1, we can still conclude that ZoDiac achieves 0.288-0.46 higher WDR/TPR than CIN against attack methods, JPEG, BM3D, Bmshj18, Cheng20, and Zhao23, and only 0.01-0.11 lower WDR/TPR against other methods. And ZoDiac demonstrates 0.688 higher WDR/TPR than StegaStamp under Zhao23 and 0.376 higher WDR/TPR under Rotation, with slightly (i.e., 0.004-0.012) lower WDR/TPR in other cases.

Second, we acknowledge that Table 1 in the global response does not provide evidence for the advantage of ZoDiac over baselines on much higher FPR levels (e.g., FPR=0.062). **To help clarify the reviewer’s doubt, we conducted more experiments comparing ZoDiac with StegaStamp and CIN under a similar FPR of 0.062 as shown in Table 3 below**. Here we cannot get the exact same FPR of 0.062 since the threshold for CIN and StegaStamp are controlled by the number of correct bits, resulting in more coarse-grained FPR levels. Instead, we report the performance of CIN at FPR levels between 0.026-0.102 and StegaStamp between 0.056-0.094. We used the advanced attack approach Bmshj18, Cheng20, and Zhao23.

Overall, ZoDiac still outperforms StegaStamp at FPR=0.094 against Zhao23 by 0.602 higher WDR, and outperforms CIN at FPR=0.102 against the three advanced attacks by 0.108-0.308 higher WDR. The result further confirms the advantages of ZoDiac over StegaStamp and CIN even at relatively high FPR levels. 

**Table 3. Comparison of WDR against the advanced attack methods under a similar high FPR of 0.062**
|   Methods  | Detection Threshold / Accurate Bit Ratrio |  FPR  | Bmshj18 | Cheng20 | Zhao23 |
|:----------:|:-----------------------------------------:|:-----:|:-------:|:-------:|:------:|
|     CIN    |                   24/32                   | 0.026 |  0.662  |  0.666  |  0.478 |
|     CIN    |                   23/32                   | 0.056 |  0.748  |  0.784  |  0.598 |
|     CIN    |                   22/32                   | 0.102 |  0.848  |  0.878  |  0.680 |
| StegaStamp |                   61/96                   | 0.056 |  0.998  |  1.000  |  0.286 |
| StegaStamp |                   60/96                   | 0.094 |  1.000  |  1.000  |  0.386 |
|   ZoDiac   |                 $p^*=0.90$                | 0.062 |  0.992  |  0.986  |  0.988 |

[2] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.    
[3] Wen, Yuxin, et al. "Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust." NeurIPS 2023.
Thank you for your valuable suggestions. Below we provide our responses and additional experimental results to the key questions.

**Q1:** How [1] can support the authors' claim that "PSNR < 20 is deemed unacceptable" 

**A1:** [1] mentioned that “In image and video compression quality degradation, the PSNR value varies from 30 to 50 dB for 8-bit data representation and from 60 to 80 dB for 16-bit data. In wireless transmission, the accepted range of quality loss is approximately 20 - 25 dB”. 

In wireless transmission, maintaining signal quality while allowing for some degradation is crucial for ensuring that transmitted data remains intelligible and useful. It is similar to the watermark attack methods whose goal is to remove the watermark while preserving the usability of content despite some level of degradation. Therefore, we believe that 20dB could be a reasonable PSNR threshold in our case.

[1] Sara U, Akter M, Uddin M S. Image quality assessment through FSIM, SSIM, MSE and PSNR—a comparative study\[J]. Journal of Computer and Communications, 2019, 7(3): 8-18.


**Q2:** Why WDR from different attacks can be cross-comparable and why WDR is a valid measure? Why try to use Avg. WDR as a unified score and claim the best?

**A2:** WDR is not used to compare across different attacks but to compare the robustness of different watermarking methods under the SAME attack. It is identical to TPR and we use WDR and TPR interchangeably in the rebuttal.  WDR/TPR is a commonly-used and valid metric.
The Avg. WDR provided in the global response is for reading the comparison more easily. We agree with you that it may not be a good choice to do averaging since the different attacks results in images with different image quality. **In fact, we didn’t use Avg. WDR in the result analysis in our paper and we clearly compare  ZoDiac with alternative watermarking approaches under each attack method in lines 293-299 in the paper.**
Thank you for the prompt response. However, it seems to me that most of my concerns still remain. Please allow me to be brief:

>1. About acceptable attack quality of image.

The authors have pointed to reference [1]. However, I do not see how [1] can support the authors claim that "PSNR < 20 is deemed unacceptable" in the watermark application. Can the authors give any further logical argument how they arrive from [1] to this claim, for example, in terms of content protection in watermarking problems which has been mentioned as the motivation of this work in the introduction?

>2. About **WDR**

I do not have question how WDR is calculated. My concern is **why WDR is a valid** measure logically. That is to say, what is the logical argument that the watermark detection results from different attacks can be cross-comparable and averaged?

For example, if "PSNR < 20 is deemed unacceptable" is the standard, why don't we tune the hyper-parameters of all attack methods to achieve the attack images with PSNR=20 first, then test the watermark detection performance using these attacked images and calculate the WDR?  To clarify, I do not mean that WDR is a sensible measure under this condition, but it seems to me a fairer setup than the authors have practiced.

>3. About the detailed discussion/comparison of performance between ZoDiac and other existing methods provided by the authors in the reply (excluding content related to WDR).

This detailed discussion is indeed sensible, e.g., identifying ZoDiac performs better under Zhao23 but worse on other more traditional image editing (such as jpeg, G-Noise), and is in fact more constructive and informative than trying to use WDR as a unified score and claim "the best". I would suggest the authors consider using these discussions in their main paper.

>4. About suggesting following the evaluation proposed by [2]

This suggestion is largely due to the unsettled logical augment of 1. and 2. Under this condition, faithfully showing the influence of robustness caused by image quality, FPR and TPR is necessary.

Again, using the table at the top of the rebuttal (**False Positive Rate on original images and Watermark Detection Rate**) as an example. 

**(1)** Problems related to TPR/FPR point: without reporting different FPR point for other methods (e.g., StegaStamp of CIN), it is unclear if when FPR=0.062, these methods will have much more robust performance under attack.

**(2)** Problems related to **image quality (generated by attack methods)**: if G-Noise attack level is doubled, it is unclear if ZoDiac (p=0.9) can maintain 0.996 detection rate. It would be unclear if the performance of ZoDiac will reduce to 0.5; what if StegaStamp can still maintain 1 at the same time? Similar concerns can be raised for all other attacks.

Therefore, extra experiments are necessary to confirm the above cases are not happening and verify the robustness of the proposed method. 

I appreciate the authors acknowledgment and promises of adding the experiments that I suggested. Unfortunately, the rating can be only given to what is available in the submitted paper and rebuttal. The change of rating should be subject to timely addressing these concerns.


[1] Sara U, Akter M, Uddin M S. Image quality assessment through FSIM, SSIM, MSE and PSNR—a comparative study. Journal of Computer and Communications, 2019, 7(3): 8-18.

[2] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.
**For Q2**, we apologize for not including the TPR-FPR curve in the rebuttal due to limited time. However, **we believe we have formed a fair comparison, and the results show that ZoDiac demonstrates better robustness compared to the baselines, especially against advanced watermark attack methods.**

To ensure clarity, let's first define the terms TPR, FPR and the Avg. WDR for consistent understanding:
1. **TPR/WDR**: The watermark detection rate on watermarked images. Higher values are better. When the watermark detection threshold increases (i.e., is more tight), TPR decreases.
2. **FPR**: The watermark detection rate on clean images. Lower values are better. When the watermark detection threshold increases, FPR decreases.
3. **Avg. WDR**: The averaged watermark detection rate aross all the attack methods, considering the evaluation importance of the different attacks are the same. Notice that we didn’t sum with FPR on the original images. 

Table 1 has shown, ZoDiac achieves lower FPR with a higher TPR than alternative approaches on many attack methods, implying that ZoDiac can achieve even higher TPR under the same FPR level. We will use the following example to elaborate on the statement. .
-  **DwtDct vs ZoDiac**: 
   - When DwtDct’s FPR is 0.052, its WDR under Zhao23 attack is 0 and its highest WDR is achieved under G_noise attack which is 0.687. 
   - When ZoDiac’s FPR is 0.03, its WDR under Zhao23 attack is 0.974 and its WDR under G_noise attack is 0.996.  
   - Implication: Decreasing the detection threshold for ZoDiac will increasing both FPR and TPR. The WDR of ZoDiac will be even higher than that of DwtDct under the same FPR level against all attack methods. 
- **DwtDctSvd vs ZoDiac**:
   - When comparing ZoDiac (with a FPR of 0.03) with DwtDctSvd (with a FPR of 0.018), ZoDiac significantly outperforms DwtDctSvd in WDR on 8 out of 10 attack methods with a 0.246-0.97 higher WDR. 
   - For the rest two attack methods (G-noise and G-Blur), ZoDiac achieves a slightly lower WDR with a maximum 0.004 difference. Increasing ZoDiac’s FPR to be of the same level as DwtDctSvd will further reduce the gap.  

We can get the conclusion for the rest of the methods in a similar way:.
- **RivaGAN vs ZoDiac**: ZoDiac achieves 0.376-0.976 higher TPR against advanced methods (Bmshj18, Cheng20, Zhao23, Rotation) and competitive TPR against others.
- **SSL vs ZoDiac**: ZoDiac shows 0.932-0.988 better TPR against JPEG, G-Noise, Bmshj18, Cheng20, and Zhao23 where SSL directly fails, and competitive results against Brightness, Contrast, and G-Blur. SSL outperforms ZoDiac in Rotation, which we analyze in lines 300-305 and Appendix C.1.
- **CIN vs ZoDiac**: ZoDiac (with a FPR of 0.004) achieves 0.288-0.46 higher TPR than CIN (with a FPR of 0.026) against attack methods, JPEG, BM3D, Bmshj18, Cheng20, and Zhao23, and 0.01-0.11 lower TPR against other methods.
- **StegaStamp vs ZoDiac**: ZoDiac (with a FPR of 0.03) demonstrates 0.688 higher TPR than StegaStamp (with a FPR of 0.056) under Zhao23 and 0.376 higher TPR under Rotation, with 0.004-0.012 lower TPR in other cases.

In summary, the results in Table 1 provide sufficient evidence that ZoDiac demonstrates better robustness when facing different types of watermark removal attacks compared with the baselines.    
With that said, we believe that the paper could be further strengthened with more comparisons between ZoDiac and alternative approaches under (1) the exactly same FPR level by tuning detection threshold, and (2) different image quality degradations from attack methods by tuning attack strengths, as suggested by the reviewer. We will add these experiments in the revision. 

Finally, we agree with you that [3] is a very solid work in evaluating image watermarking approaches. We will refer to [3] when adding the above mentioned experiments. Fortunately, we reach to similar insights:  (1) there is always a tradeoff between the image quality of watermarked images and the watermark robustness and (2) StegaStamp achieves the strongest watermark robustness while introducing artifacts into images.     
Compared to StegaStamp, our proposed ZoDiac is more robust under the advanced watermark removal attack Zhao23 and achieves competitive WDR under other attacks as analyzed above. In terms of the image quality, ZoDiac has similar performance as StegaStamp as shown in Table 1 in the paper. From the visual comparison in Figure 11, we can observe a small amount of differences in the residual images for our proposed ZoDiac, and a significant amount for StegaStamp.

[3] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.
Thank you for the prompt review and feedback.

**For Q1**, we acknowledge that we followed the attack settings in [1], which was published in the ICML23 Workshop, instead of a peer-reviewed conference. To better understand "image quality within an acceptable range," we refer to the PSNR guidelines in [2], where PSNR < 20 is deemed unacceptable. Our experiment so far ensures the PSNR of the images after the attack to be larger than 20dB. We will add experiments that vary the parameters of the watermark attack methods to provide a more comprehensive analysis on  attacked image with different PSNR quality.

[1] Zhao, Xuandong, et al. "Invisible image watermarks are provably removable using generative ai." arXiv 2023. OR Zhao X, Zhang K, Wang Y X, et al. Generative autoencoders as watermark attackers: Analyses of vulnerabilities and threats. 2023.    
[2] Sara U, Akter M, Uddin M S. Image quality assessment through FSIM, SSIM, MSE and PSNR—a comparative study. Journal of Computer and Communications, 2019, 7(3): 8-18.
I appreciate the authors' effort in the rebuttal. However, the response does not sufficiently address my concerns. Here are the reasons:

>**1. How is table 1 (image quality of the attacked watermarked image) related to your claim "image quality within an acceptable range"? Or, what is an acceptable range?**

If I take it as "introducing changes to the images that are undetectable by human eyes", I am pretty sure a number of perturbations tested (e.g., brightness, G-Noise, maybe more) can be visibly detected. If I take it as PSNR $\ge 20$, then the question is why PSNR $= 20$ is a reasonable cutoff. Can you further clarify your argument and make it rigorous?

>**2. I do not see how the provided response support a "fair comparison".**

**First**, comparison at the same watermark detection threshold is hardly ever a fair strategy. Let me give you an example with two watermarking systems, where both rely on thresholding the bitwise accuracy (with some abuse of notation, let's use $I$ as un-watermarked image, $I_w$ watermarked images and $D_i$ as the decoder):

1. System (1):   $D_1(I) = 0, ~ \forall I$ and $D_1(I_w) = 1, ~ \forall I_w$
2. System (2):   $D_2(I) = 0.4, ~ \forall I$ and $D_2(I_w) = 0.6, ~ \forall I_w$

System (1) and (2) essentially provide the same performance (if measured by TPR-FPR curve for example). Will setting a threshold $p=0.01$ and compare reach the same conclusion?

The above example only uses bitwise acc., not to mention that the proposed method "ZoDiac" is thresholding on a template matching distance.

**Second**, comparison at equivalent FPR levels is not correct. The only rigorous way in this direction is to tune the threshold $p$ of each method and achieve the exact **same FPR** level (control of variables), then TPR (not only for $I$, but also $I_w$ under different attacks) can be meaningfully compared. If we want to cross-compare the performance among different attacks, the **level of attack** (e.g., image quality loss) **should also be explicitly controlled to the same level**. Otherwise, they are not cross-comparable as my comment in the previous iteration.

**Third**, why the numbers in Table: **False Positive Rate on original images and Watermark Detection Rate (WDR)** can be meaningfully sum and average to come up with metric Avg. WDR?

For example, I can make a counter-argument that (1) rotation, contrast, G-noise and Zhao23 introduce visible corruptions by human eyes and should not be considered; (2) Bmshj18 and Cheng20 are essentially similar methods and should be combined as one; etc. Of course, my argument above is debatable and not convincing, but so as yours at current level. Can you provide further logical justification why this is a valid metric? Otherwise, it falls into my comment above (in "Second").

**Fourth**, if we take one step back and acknowledge that no single watermark currently outperforms the others under every attack, would you agree that showing and discussing the difference in behavior (e.g., XX is better under G-Noise but worse under Cheng20) is more sensible than trying to get a score and claim the overall superiority?

**Lastly**, I acknowledge the paper [1] the authors pointed to as their major support of "fair" comparison setups. Unfortunately, it is not a published work in any peer-reviewed conferences yet and it also suffers the non-rigorous evaluation setups as well. It does not sufficiently justify the evaluation approach you have taken in this paper. I highly recommend the authors to [2] and consider how they evaluated the watermarking systems and why.

[1] Zhao, Xuandong, et al. "Invisible image watermarks are provably removable using generative AI." arXiv 2023.

[2] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.