RAW: A Robust and Agile Plug-and-Play Watermark Framework for AI-Generated Images with Provable Guarantees

Authors: xun  xian
ganghua  wang
xuan  bi
jayanth  srinivasa
ashish  kundu
mingyi  hong
jie  ding
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 19:02:25

Abstract: Safeguarding intellectual property and preventing potential misuse of AI-generated images are of paramount importance. This paper introduces a robust and agile plug-and-play watermark detection framework, referred to as RAW.
As a departure from existing encoder-decoder methods, which incorporate fixed binary codes as watermarks within latent representations, our approach introduces learnable watermarks directly into the original image data. Subsequently, we employ a classifier that is jointly trained with the watermark to detect the presence of the watermark.
The proposed framework is compatible with various generative architectures and supports on-the-fly watermark injection after training. By incorporating state-of-the-art smoothing techniques, we show that the framework also provides provable guarantees regarding the false positive rate for misclassifying a watermarked image, even in the presence of adversarial attacks targeting watermark removal. 
Experiments on a diverse range of images generated by state-of-the-art diffusion models demonstrate substantially improved watermark encoding speed and watermark detection performance, under adversarial attacks, while maintaining image quality. Our code is publicly available [here](https://github.com/jeremyxianx/RAWatermark).

Main paper: 
 introduction: 
 In recent years, Generative Artificial Intelligence, notably in computer vision, has made significant strides. The adoption of diffusion models (DM) in applications like Stable Diffusion [1] and DALLE-2 [2] has greatly improved image generation quality. However, these advancements also raise concerns about potential misuse, seen in instances such as DeepFake [3] and copyright infringement [4] .To mitigate the potential misuse of diffusion models, the incorporation of watermarks emerges as a promising solution. Watermarked images, tagged with crafted signals, act as markers to identify their machine-generated origin. Watermarking techniques designed for generative models can be generally classified into two categories: model-specific [5, 6, 7, 8] and model-agnostic [9, 10, 11] , as outlined in Table 1 . Model-specific methods refer to those tailored for particular generative models, often offering a better tradeoff between watermarked image quality and watermark detection performance. However, this characteristic may potentially restrict their applicability across diverse use cases. For instance, the Tree-Ring watermark [8] is tailored for specific samplers, e.g., DDIM [12] , used for image generation within diffusion models. The feasibility of adapting this method to other commonly used samplers remains open to discussion.Table 1 : Summary of features of several representative watermark techniques. The second column denotes the method's suitability for real-time on-the-fly implementation. / denotes cases where the watermark is embedded during the generative process. The third column evaluates whether the watermarking method provides provable guarantees on false-positive rates (FPRs) under adversarial attacks in a distribution-free manner.Model agnostic On-the-fly deployment Provable FPRs under adversarial attacks DwtDctSvd [13] ✓ ✓ ✗ RivaGan [10] ✓ ✗ ✗ StegaStamp [11] ✓ ✗ ✗ Stable Signature [5] ✗ / ✗ Tree Ring [8] ✗ / ✗ RAW (Our method) ✓ ✓ ✓In contrast, model-agnostic approaches directly watermark generated content without modifying the generative models. These approaches can be categorized into two types. The first, from traditional signal processing, e.g., DwTDcTSvd [13] , embeds watermarks in specific parts of images' frequency domains. However, they can be vulnerable to strong image manipulations and adversarial attacks for removing watermarks [14] . The second type employs deep learning techniques, utilizing encoderdecoder structures to embed watermarks, such as binary codes, in latent spaces. For instance, RivaGan [10] trains the watermark and watermark decoder jointly as learned models, enhancing transmission and robustness. However, these methods demand greater computational resources for watermark injection, limiting real-time on-the-fly deployment. For example, when injecting watermarks into images, the RivaGan requires over 15× the time needed by the DwtDct methods [15] .Additionally, there has been a sustained emphasis on precisely evaluating false-positive rates (FPRs) and/or the Area Under the Receiver Operating Characteristic curve (AUROC) for each utilized watermarking strategy [16] , given the potential economic implications associated with watermark implementation. To establish an explicit theoretical formulation for FPRs, many studies have assumed that the binary watermark code extracted from unwatermarked images exhibits a pattern where each digit is an independent and identically distributed (IID) Bernoulli random variable with equal probabilities. This assumption enables the explicit derivation of the FPRs when comparing the extracted binary code with the predefined actual binary watermark code. However, such an assumption may not hold as empirically observed in [17] , and thus the corresponding formulation for FPRs might be incorrect.In this paper, we introduce a Robust, Agile plug-and-play Watermark framework, abbreviated as RAW. RAW is designed for both adaptability and computational efficiency, providing a modelagnostic approach for real-time, on-the-fly deployment of image watermarking (which can be straightforwardly extended to video watermarking as outlined in Section 4.5). This dedication to adaptability is to ensure the accessibility for third-party users, e.g., artists and generative model providers. Moreover, this adaptability is fortified by the integration of state-of-the-art smoothing techniques for achieving provable guarantees on the FPRs for detection, even under adversarial attacks.A new framework for robust and agile plug-and-play image watermark learning In contrast to encoder-decoder techniques that insert fixed binary watermarks into latent spaces, we propose to embed learnable watermarks, matching the image dimensions, into both the frequency and spatial domains of the original images. To differentiate between watermarked and unwatermarked samples, we utilize a classifier, e.g., a convolutional neural network (CNN), and perform joint training for both the watermarks and the classifier. The proposed framework offers several benefits, including enhanced computational efficiency achieved through batch processing for watermark injection post joint training. For instance, our experimental results demonstrate time efficiency enhancements, approximately 30× (200×) faster than the frequency-based (encoder-decoder based) method, respectively. Moreover, it can be readily integrated with other state-of-the-art techniques to further enhance robustness and generalizability, such as adversarial training [18, 19] , and contrastive learning [20, 21] .Provable guarantees on FPRs even under adversarial attacks without distributional assumptions By integrating methods from the conformal prediction literature [22, 23] into our RAW framework, we showcase its ability to offer rigorous, distribution-free assurances regarding the FPRs. Additionally, we develop a novel technique, inspired by the randomized smoothing [24, 25] , to further enhance our provable guarantees. This technique ensures certified guarantees on FPRs under arbitrary perturbations, including those adversarial with bounded norms. That is, as long as any transformations on test images stay within a predefined range, our guarantee on FPRs remains valid.Extensive empirical studies on various datasets & extension to video watermark We evaluate the effectiveness of our proposed method across various generative data scenarios, such as the DBDiffusion [26] and MS-COCO [27] . Additionally, we discuss how to extend the proposed framework for video watermarking and provide empirical results to demonstrate its effectiveness.Our assessment includes detection performance, robustness against image manipulations/attacks, the computational efficiency of watermark injection, and the quality of generated images. Experimental results consistently affirm the excellent performance of our approach, as evidenced by significantly improved watermark encoding speed and notable enhancements in detection performance against state-of-the-art adversarial attacks aimed at removing watermarks.Classical watermarking techniques for images Image watermarking has long been a fundamental problem in both signal processing and computer vision literature [9, 28] . Methods for image-based watermarking typically operate within either the spatial or frequency domains [9, 29] . Within the spatial domain, methodologies span from basic approaches, such as the manipulation of the least significant bit of individual pixels, to more complex strategies like Spread Spectrum Modulation [30, 31] . In the frequency domain, watermark embedding [9, 32] involves modifying coefficients generated by transformations such as the Discrete Cosine Transform [33] .The advent of advanced deep learning techniques has opened up new avenues for watermarking. Many of these methods [34, 35, 36, 10, 17, 11] , are based on the encoder-decoder architecture. In this model, the encoder embeds a binary code into images in latent representations, while the decoder takes an image as input and generates a binary code for comparison with the binary code injected for watermark verification. For example, the HiDDeN technique [36] involves the simultaneous training of encoder and decoder networks, incorporating noise layers specifically crafted to simulate image perturbations. While these methods improve robustness over traditional watermarking, they might not be suitable for real-time watermark injection due to the lengthy feed-forward process in the encoder, especially with larger architectures. Most recently, there has been a line of work focusing on designing watermarks specifically for diffusion models [5, 8, 37] . These methods benefit from improved robustness due to their model-specific design, but the applicability to other types of models is not yet well established.Deep neural networks represent valuable intellectual assets, given the significant resources invested in their training and data collection processes [1] . For example, training stable diffusion models consumes approximately 150,000 GPU hours, amounting to roughly $600,000 in costs [38] . Given their broad applications in realworld scenarios, ensuring copyright protection and facilitating their identification is crucial in both normal and adversarial contexts [39, 40] . Some approaches embed watermarks directly into model parameters [41] , necessitating white-box access for inspection. Others [42, 43] rely on backdoor attacks [44, 45, 46, 47] , injecting triggers into training data to enable targeted predictions during testing. These methods primarily focus on safeguarding model intellectual property rather than the generated outputs. 
preliminary: 
 Notations. We consider the problem of embedding watermarks into images and then detecting the watermarks as a binary classification problem. Let X = [0, 1] C×W ×H be the input space, with C, W , and H being the channel, width and height of images, respectively. We denote Y = {0, 1} to be the label space, with label 0 indicating unwatermarked and 1 indicating watermarked versions, respectively. For a vector v, we use ∥v∥ to denote its ℓ 2 -norm.Threat Model. We consider the following use scenario of watermarks between a third-party user Alice, e.g., an artist, a generative model provider Bob, e.g., DALLE-2 from OPENAI, and an adversary Cathy.(1) Alice selects a diffusion model (DM) from Bob's API interface and sends an input (e.g., a prompt for text-to-image diffusion models) to Bob for generating images. (2) Upon receiving Bob generated images X ∈ X , Alice embeds a watermark into X, denoted as X ′ ∈ X , and releases it to the public. (3) Adversary Cathy applies (adversarial) image transformation(s), e.g., rotating and cropping, on X ′ to obtain a modified version X ′ (∈ X ). (4) Alice decides if X ′ was generated by herself or not.Problem Formulation. From the above, the watermark problem for Alice essentially boils down to a binary classification or hypothesis testing problem:H 0 : X ′ was generated by Alice (Watermarked) ; H 1 : X ′ was NOT generated by Alice (Unwatermarked) .To address this problem, Alice will build a detector given byEQUATIONwhere τ ∈ R is a threshold value and V θ (to be defined later) is a scoring function which takes the image input and output a value in [0, 1] to indicate its chance of being a sample generated by Alice.Alice's Goals. Alice's objective is to design watermarking algorithms that fulfill the following objectives: (1) Quality: the quality of watermarked images should closely match that of the original, unwatermarked images; (2) Identifiability: both watermarked and unwatermarked content should be accurately distinguishable;(3) Robustness: the watermark should be resilient against various image manipulations.Cathy's (Adversary) Goals. Cathy aims to design attack algorithms to meet the following objectives:(1) Watermark Removal: the watermarks embedded by Alice can be effectively eliminated after the attacks;(2) Quality: the attacks cannot significantly alter the images.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: Discussions on the limitations of this work are provided in Section 5 in the main text.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
raw: 
 In this section, we formally introduce our RAW framework. At a high level, the RAW framework comprises two consecutive stages: a training stage and an inference stage, as outlined in Figure 2 in Appendix. In the following subsection, we first provide an in-depth description of the training stage.Suppose Alice obtains a batch of diffusion model-generated images. The unwatermarked data are denoted as D uwm ≜ {X i } n i=1 for i = 1, . . . , n. Alice will need to embed watermarks into these images to protect intellectual property. Definition 3.1 (Watermarking Module). A watermarking module E w (•) maps X to itself, with parameters w ∈ R d1 .The watermarking module can take the form of an encoder with an attention mechanism, as seen in the RivaGan [10] , or it can involve Fast Fourier Transformation (FFT) followed by frequency adjustments and an inverse FFT, as employed in DwtDct.In our RAW framework, we propose to add two distinct watermarks into both frequency and spatial domains:EQUATIONwhere v, u ∈ X are two watermarks, c 1 , c 2 > 0 determine the visibility of these watermarks, and F(F -1 ) represents the Fast Fourier Transformation (FFT) (inverse FFT), respectively. For simplicity of notation, in the rest of this paper, we will denote w ≜ [u, v]. For implementation, we clip the watermarked image E w (X) to be within the range [0, 1], making E w (X) a non-linear function of X.The rationale for adopting the above approach is to simultaneously enjoy the distinct advantages offered by watermarks in both domains. In particular, the incorporation of watermark patterns in the frequency domain has been widely recognized for its effectiveness against certain image manipulations such as translations and resizing. Moreover, our empirical validation corroborates the improved robustness of spatial domain watermarking in the presence of noise perturbations.We denote the watermarked dataset E w to be D wm ≜ {E w (X i )} n i=1 for i = 1, . . . , n. Alice now wishes to distinguish the combined dataset D ≜ D uwm D wm with a verification module, which is a binary classifier. Definition 3.2 (Verification Module). A verification module is a mappingV θ (•) : X → [0, 1] parameterized by θ ∈ R d2 .The score generated by the verification module for an input image can be understood as the chance of this image being generated by Alice.To fulfill Alice's first two goals, Alice will consider jointly training the watermarking and verification modules parameterized by w and θ, with the following loss function:EQUATIONwhere X is the training image and Y ∈ {0, 1} is the label indicting X is watermarked or not.Recall that Alice also aims to enhance the robustness of the watermark algorithms. As a result, we consider transforming the combined datasets with different data augmentations M 1 , . . . , M m to obtainD 1 ≜ M 1 (D), . . . , D m ≜ M m (D), respectively.Here, the data augmentations M 1 , . . . , M m are defined as follow.). An image modification module is a map M : X → X .To sum up, the overall loss function for our RAW framework is specified as:EQUATIONwhere BCE(•) denotes the binary cross entropy loss as specified in Equation (3). The loss function above is composed of two terms: BCE(D), which corresponds to the cross-entropy on the original combined datasets D, and m k=1 BCE(D k ), signifying the cross-entropy on the augmented datasets D 1 , . . . , D m . In our experiments, inspired by contrastive learning such as those presented in [20, 21] , we adopt a two-view data augmentation approach by setting m = 2.We describe the overall training algorithm below. For completeness, we provide the full pseudo-code as summarized in Algorithm 3 in the appendix. We consider conducting the following two steps alternatively.• Optimize the verification module V θ based on the overall loss L raw by stochastic gradient descent (SGD):θ t+1 ← θ t -µ t ∇ θ L raw (θ t , w), where µ t > 0 is the step size at each step t.• Optimize the watermark w based on L 0 , defined as the first term on the right-hand side of Equation ( 4), with sign-based stochastic gradient descent (SignSGD):EQUATIONwhere sign(•) outputs the sign of each component, and ν t > 0 is the step size.We provide two remarks on the optimization procedure for the watermark update, as outlined in Eq. ( 5). First, we choose signSGD over conventional SGD, based on empirical findings suggesting that (sign-based) first-order methods can improve both training and test performance in data-level optimization tasks [19, 48, 49] . In our context, this refers to cases where the optimized variable is the watermark rather than the model parameters. Second, we focus the watermark optimization solely on the first term because the other terms, BCE(D k ) for k = 1, 2, . . . , m, may contain non-differentiable elements with respect to the watermark. These non-differentiable parts make gradient-based optimization impractical.We present a generic approach for Alice to obtain provable guarantees on the FPRs when using the previously trained V θ on test images, even amidst adversarial perturbations.To begin with, we first examine a scenario where the future test data X test ∈ X adheres to an IID pattern with the watermarked data D wm generated by Alice, without undergoing any image modifications. In this case, Alice can employ conformal prediction to establish provable guarantees on the FPRs. The main idea is that, by utilizing the trained V θ as a scoring mechanism, the empirical quantile of the watermarked data's distribution will converge to the population counterpart. This convergence is guaranteed by the uniform convergence of cumulative distribution functions (CDFs).To be more specific, we set the threshold τ defined in Equation ( 1) to be the α-quantile (with finitesample corrections) of the predicted scores for watermarked data,V θ (D wm ) ≜ {V θ (E w (X i ))} n i=1. Following [23, 46] , it can be shown that, with high probability, the probability of the resulting detector g misclassifying a watermarked image X test is upper-bounded by α, given that X test is IID with D wm . For completeness, a rigorous statement and its proof are provided in Appendix A.2.The above argument assumes that the future test image X test follows an IID pattern with the original watermarked data D wm . However, if the test image X test undergoes manipulation or attack, denoted by A(X test ), with A : X → X being an adversarial image manipulation, then it can deviate from the distribution of D wm . This deviation from IID will render the previous argument invalid. Moreover, in practice, Alice is unaware of the adversarial transformation A employed by the attacker, thus making it even more challenging to control the FPRs.To address this problem, we propose to consider a robust version of the originally trained V θ , denoted as Ṽ, such that X and A(X) stay close under Ṽ, namelyEQUATIONfor all X and a small η > 0. The reason for finding such Ṽ is because we can relate Ṽ(A(X test )) back to Ṽ(X test ) which is IID with Ṽ(D wm ) (accessible to Alice) to establish the FPRs with previous arguments.To develop the robust version from the base V θ , we will build upon the following result from the randomized smoothing literature. Denote N (µ, Σ) to be the normal distribution with mean µ and covariance Σ respectively, and Φ -1 (•) to be the inverse of the cumulative distribution function of a standard normal distribution. Lemma 3.4 ([50, 51] ). Let h : R → [0, 1] be a continuous function. Let σ > 0, andH(X) = E Z∼N (0,σ 2 I) [h(X + Z)]. Then the function Φ -1 (H(X)) is σ -1 -Lipschitz with respect to ℓ 2 norm.The above result suggests that for any (continuous) base verification module (classifier) V θ , we can obtain a smoothed version withEQUATIONand it is guaranteed that| Ṽ(X) -Ṽ(Y )| ≤ σ -1 ∥X -Y ∥, for any X, Y ∈ X .Suppose the attacker employs an adversarial attack A such that ∥X -A(X)∥ ≤ γ. We haveEQUATION)Remark 3.5 (A can not be excessively adversarial). We emphasize that the transformation A should not be excessively adversarial. In other words, the parameter γ should be a very low value for both theoretical and practical reasons. From a theoretical perspective, an overly adversarial transformation A can result in trivial TPRs/FPRs. For instance, if watermarked images are transformed into a completely uniform all-white or all-black state, it becomes impossible to detect the watermark. From a practical standpoint, an excessively adversarial transformation A tends to overwrite the original content within the images. This directly contradicts the intentions of attackers and may not achieve the desired stealthy modifications.Given a pair of (E w , V θ ), a desired robust range γ > 0, and a smoothing parameter σ > 0, Alice now will set the thresholding value τ , as introduced in Equation ( 1), to satisfy:EQUATIONwhere δ ∈ (0, 1) is a violation rate describing the probability that the FPRs exceeds α, andF is the empirical cumulative distribution function of { Ṽ(E w (X i ))} n i=1 , where Ṽ(E w (X i )) ∆ = Φ -1 E Z∼N (0,σ 2 I) [V θ (E w (X i ) + Z)] .The next result shows that if a future test input comes from the same distribution as the watermarked data D wm , the above procedure can be configured to achieve any pre-specified false positive rate α with high probability. Theorem 3.6 (Certified FPRs of g based on threshold in Equation ( 9)). Given any watermarked dataset D wm and its associated verification module V θ , suppose that the test data X test are IID drawn from the distribution of D wm . Given any δ ∈ (0, 1), γ > 0 and σ > 0, for any (adversarial) image transformations A such that ∥A(X) -X∥ ≤ γ for all X ∈ X , the robust verification module Ṽ specified in Eq. ( 7) and its corresponding hard-label detector g(•) introduced in Eq. ( 1), with the threshold τ as specified in Eq. ( 9) satisfyP g( Ṽ (A(X test ))) = 0 (Unwatermarked) | D wm ≤ α,with probability at lease 1 -δ for any α ∈ (0, 1) such that α > (log(2/δ)/(2n)).Due to space constraints, the proof is provided in Appendix A.1. The above result shows that by using the decision rule as specified in Equation ( 9), Alice can obtain a provable guarantee on the Type I error rate in terms of detecting future test input X test even X test is adversarially perturbed within γ-range (as measured by ℓ 2 -norm), under the condition that the future test input X test is independently and identically distributed as the D wm , namely watermarked samples generated by the artist. Moreover, the result above only addresses the detector's performance in terms of FPRs and does not account for True Positive Rates (TPRs). To achieve high TPRs, careful selection of γ and σ is essential. For instance, an excessively large σ will cause the smoothed classifier Ṽ to lose its detection capability.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: We have provided details, including GitHub links, on how to implement the results for our proposed methods.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experiments: 
 In this section, we conduct a comprehensive evaluation of our proposed RAW, including (1) detection performance, (2) robustness, (3) watermarking speed, (4) the quality of watermarked images, and (5) the provable FPRs guarantees under adversarial attacks. Our findings reveal significantly enhanced robustness in RAW while preserving the quality of generated images. Furthermore, a substantial reduction in watermark injection time, indicates the suitability of RAW for on-the-fly deployment. All the experiments were conducted on machines equipped with Nvidia Tesla A100s.Datasets (1) In line with the previous work [8] , we employ Stable Diffusion-v2-1 [1] , an open-source text-to-image diffusion model, with DDIM sampler, to generate images. All the prompts used for image generation are sourced from the MS-COCO dataset [27] . (2) We further evaluate our RAW utilizing DBdiffusion [26] , a dataset consisting of 14 million images generated by Stable Diffusion. This dataset encompasses a wide array of images produced under various prompts, samplers, and userdefined hyperparameters. Ablation studies examining various generative models, such as SDXL-1.0, are detailed in Appendix D.Verification Modules/Classifiers In terms of verification modules, for all the results reported in the main text, we utilize ResNet 18 [52] . For training the verification modules, 500 images are randomly selected for training for each dataset. Subsequently, we evaluate the trained watermarks and associated models on 1000 new, unwatermarked images and their watermarked versions. Ablation studies on using different models, such as VGG [53] as verification modules, and different number of training data are provided in Section 4.4 and Appendix D. To assess the detection performance, we adhere to the convention of reporting the area under the curve of the receiver operating characteristic (AUROC) [17, 5] .(2) For evaluating the quality of the watermarked images, we adopt both the Frechet Inception Distance (FID) [54] and the CLIP score [55] , following the methodology outlined in [8] . Additionally, we also evaluate the quality of watermarked images using metrics such as PSNR and SSIM, and include the results in the appendix. For our watermark scheme, each independent run (including training and testing) corresponds to a unique watermark. All metrics are averaged across 5 independent runs, with a standard deviation of 0.03 for all detection performances (AUROC) and 0.7 for image quality metrics. Robust detection performance We assess the robustness of our proposed RAW against six common data augmentations and three adversarial attacks in this subsection. The data augmentation set comprises: color jitter with a brightness factor of 0.5, JPEG compression with quality 50, rotation by 90 • , addition of Gaussian noise with 0 mean and standard deviation 0.05, Gaussian blur with a kernel size of (7, 9) and bandwidth 4, and 70% random cropping and resizing. For adversarial attacks, we select three state-of-the-art methods for removing watermarks, with two VAE-based attacks Bmshj2018 [14] (VAE Att1) and Cheng2020 [56] (VAE Att2) from CompressAI [57] with compression factors are set to 3 for both models, and one diffusion-model attack with noise steps of 60 following [58] .The averaged results are in the 'AUROC (Adv)' column of Table 2 and the detailed results are summarized in Table 3 . Our approach demonstrates superior performance compared with alternative methods. Specifically, across both datasets, the average AUROC for our RAW increased by 70% and 13% for nine image manipulations/attacks, surpassing frequency-and encoder-decoder-based methods. We note that Stegastamp demonstrates similar averaged robust detection performance compared to ours. However, this comes at the expense of reduced image quality, as evidenced by markedly increased FID scores and/or lowered SSIM/PSNR scores. method, respectively. This is attributed to streamlined batch operations in our RAW. This highlights the suitability of our approach for on-the-fly deployment.We assess the certified FPRs performance of our proposed RAW by varying the FPRs rate α prespecified by Alice. We set the adversaril radius γ = 0.001 and the smoothing parameter σ = 0.05. We summarize the results of five independent runs in Figure 1 (a) and report the mean (with standard error < 0.002). The results show that the FPRs of RAW consistently matches the theoretical upper bounds (i.e., α), supporting the result presented in Theorem 3.6.Effect of training sample size on prediction performance We manipulate the sample size of the watermarked training dataset D wm to assess its impact on detection performance. These findings are illustrated in Figure 1 (b), where we note that satisfactory detection performance can be achieved with a reasonably small training dataset, e.g., 100 images.Trade-off between robustness and image quality We explore the trade-off between robustness and image quality by adjusting the watermark strength parameters c 1 (= c 2 ), as illustrated in Figure 1 (c) & (d) below. We note that with increasing values of c 1 and c 2 , the average AUROC, under 9 (adversarial) images manipulations, also increases, while the image quality only exhibits a slight degradation, as indicated by the slightly increased FID value. These findings also highlight the stability of the watermark hyperparameters c 1 and c 2 in our proposed RAW, a desirable characteristic for real-world deployments, as preferred by practitioners. We discuss how to extend the proposed watermark framework from AI generated images to videos. The main idea is to embed the same pre-trained watermark either in every frame or at intervals, such as every three frames, of the video. The corresponding verification module, i.e., the watermark classifier, is then used to detect the watermark in each frame where it was embedded. The overall presence of the watermark in the video is determined by the detection results from these frames.We evaluate the pre-trained (on the MS-COCO dataset) watermark and its corresponding classifier using short videos created by the stable-video-diffusion-img2vid-xt model, which generates videos from images. The images used to produce the videos are sourced from the DiffusionDB dataset. This guarantees that the testing videos were not exposed to either the watermark or the classifier during the training process. The results are summarized in Table 5 below, where we observed that our proposed method achieves a similar detection rate compared to RivaGan, while providing a significantly faster watermark encoding speed (×60 times faster).  
conclusion: 
 In this study, we introduce the RAW framework as a versatile watermarking approach essential for protecting intellectual property and mitigating potential misuse of AI-generated images. The proposed RAW framework offers several notable features, including significantly enhanced watermark encoding speed and/or detection performance, along with the assurance of provable guarantees on false positive rates even under adversarial perturbations in test images. Experimental findings across various datasets validate its advantages.Limitation & Future Work One potential limitation of the proposed method is that when applied to large-scale systems with millions of users, the associated training loss will be high. One interesting direction is to study how to efficiently fine-tune from one pair of watermark and its corresponding verification module to a new pair. Other directions include investigating the maximum number of concurrent watermarks learnable in a single training session and optimal smoothing strategies for wider certified radii.Broader Impact This paper aims to contribute to the advancement of trustworthy machine learning, particularly in ensuring the safe and legitimate use of contemporary generative artificial intelligence. Our efforts could have several positive societal implications, such as protecting intellectual property and preventing potential misuse of AI-generated images.The Appendix contains proofs, experimental details, and ablation studies. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: All experiment details are listed.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: Error bars are reported.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: The computation resources are listed in the appendix.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: Discussions on potential societal impacts are provided in Section 5 of the main text.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have properly cited all the datasets, models, and GitHub repositories (code) used in this work.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: This paper does not release any new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: This paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: This paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In Section A, we outline the formal proofs supporting the theoretical findings introduced in the main text. In Section B, we present omitted details, encompassing pseudocode and additional deliberations regarding the construction of RAW. In Section C, we offer implementation details of our experiments.In Section D, we present additional ablation studies on different hyperparameters.In this section, we provide the proof for Theorem 3.6. To begin with, we first revisit the overall inference procedure employed by our RAW and provide several detailed discussions. For the reader's convenience, we include the pseudo-code outlining the inference stage protocol of RAW in Algorithm 1 below.Input: querying input Xtest, watermarked dataset D wm = {Ew(Xi)} n i=1 , verification module V θ , desired false positive rate α ∈ (0, 1), violation rate δ ∈ (0, 1), desired adversarial robust range γ > 0, smoothing parameter σ > 0, and the smoothed/robust verification moduleṼ(•) ≜ Φ -1 (E Z∼N (0,σ 2 I) [V θ (Ew(•) + Z)])1: Receiving a future query sample Xtest 2: for i = 1 to n do 3: Calculate si ≜ Ṽ(Ew(Xi)) // Xi ∈ D wm 4: end for 5: Select the decision threshold τ according to Equation (10) The overall inference process for the proposed RAW mainly involves determining a decision threshold value τ . This threshold is calculated based on the provided watermarked dataset D wm and the corresponding (smoothed) verification module Ṽ, and it should satisfy the following condition:EQUATIONwhere δ ∈ (0, 1) is the violation rate describing the probability that the (FPRs) exceeds α, and F is the empirical cumulative distribution function of the watermarked dataset under Ṽ, i.e., { Ṽ(E w (X i ))} n i=1 . In the case where (log(2/δ)/(2n)) > α, we set the thresholding value τ to be the maximum of Ṽ(E w (X i ))} n i=1 . We note that under such cases there will be no theoretical guarantees in term of FPRs.The proof of Theorem 3.6 is built upon the techniques established in our previous work [46] . In [46] , we developed similar provable guarantees under the IID assumption of future test data. However, in our case of Theorem 3.6, such an IID assumption is no longer valid and hence raise up new technical challenges. To prove Theorem 3.6, we will use the following result concerning the convergence properties of empirical cumulative distribution functions (ECDFs). Lemma A.1 (Dvoretzky-Kieffer-Wolfowitz inequality). Given a natural number n, let X 1 , X 2 , . . . , X n be real-valued independent and identically distributed random variables with cumulative distribution function F (•). Let F (•) denote the associated empirical distribution function.The interval that contains the true CDF, F (x), with probability 1 -δ is specified asF (x) -ε ≤ F (x) ≤ F (x) + ε where ε = ln 2 δ 2n .Proof of Theorem 3.6. Note that X test is IID drawn from the watermarked data distribution and we haveEQUATIONEQUATION) = α -ε + ε (15) = α,holds with probability at least 1 -δ. The equation ( 11) is because of the decision rule as specified in Algorithm 1, and the inequality ( 12) is due to the lipschitz condition of Ṽ with parameter σ-1 , namely | Ṽ(X) -Ṽ(Y )| ≤ σ -1 ∥X -Y ∥, for any X, Y ∈ X .Additionally, the F in equation ( 13) represents the CDF of the watermarked data under Ṽ(•), i.e., Ṽ(E w (X)), while F in ( 14) denotes the empirical CDF obtained from D wm , i.e., { Ṽ(E w (X i ))} n i=1 under Ṽ(•). The inequality in (14) arises from the DKW inequality as specified in Lemma A.1. Furthermore, the Equation ( 15) is based on the fact that the CDF follows a uniform distribution (a result of the probability integral transformation) and the selection of the thresholding value specified in Equation (10) .In this section, we present the omitted results of the provable FPRs concerning the watermark detection performance of our RAW in the absence of adversarial attacks, as elaborated in Line 225 of the main text. All the techniques used in this section are based on our previous work [46] , as discussed earlier.As there are no anticipated adversarial attacks on test images, there is consequently no requirement to apply smoothing/robustification to the trained verification model V θ . The corresponding pseudo-code is outlined in Algorithm 2 below. Similarly, the updated thresholding value τ is selected to satisfy theInput: querying input Xtest, watermarked dataset D wm = {(Ew(Xi))} n i=1 , verification module V θ , desired false positive rate α ∈ (0, 1), violation rate δ ∈ (0, 1) 1: Receiving a future query sample Xtest 2: for i = 1 to n do 3: Calculate si ≜ V θ (Ew(Xi)) // Xi ∈ D wm 4: end for 5: Select the decision threshold τ according to Equation (10) EQUATIONwhere δ ∈ (0, 1) is the violation rate describing the probability that the (FPRs) exceeds α, and F is the empirical cumulative distribution function of the watermarked dataset under V θ , i.e., {V θ (E w (X i ))} n i=1 . Contrasting the selection of τ under adversarial attacks in Equation ( 10), we note that the term γ/σ is omitted because of the absence of adversarial attacks. In the case where (log(2/δ)/(2n)) > α, we set the thresholding value τ to be the maximum of {V θ (E w (X i ))} n i=1 . Theorem A.2 (Certified FPRs under no adversarial attacks). For any watermarked dataset D wm and its associated verification module V θ , suppose that the test data (X test , Y test ) are IID drawn from the distribution of D wm . Given δ ∈ (0, 1), the detectorg(•) (defined in Line 231 in the main text) with the threshold τ as specified in Equation ( 16) satisfies P(g(X test )) = 0 (Unwatermarked) | D wm ) ≤ α with probability at least 1 -δ for any α ∈ (0, 1) such that α > (log(2/δ)/(2n)).Proof. The proof follows the same approach as the proof for Theorem 1, with the exclusion of the γ/σ term.In this section, we initially present the omitted pseudo-code for the training algorithm, followed by further discussions regarding the design of RAW. We report the PSNR and SSIM of the watermarked images in the Table 6 below. For PSNR and SSIM, our method outperforms the StegaStamp method but is behind DwtDctSvd and RivaGAN. These results are consistent with our expectations. The StegaStamp method achieves more robust detection by sacrificing image quality through injecting a large amount of noise into the image, while DwtDctSvd and RivaGAN maintain image quality slightly better at the cost of reduced robustness against (adversarial) image perturbations.In this section, we first list the implementation details for the results presented in the main text. Next, we present ablation studies on the performance of our RAW by using different hyper-parameters.Watermark setup Recall that the watermarking module in our RAW takes the form ofE w (X) = F -1 (F(X) + c 1 × u) + c 2 × v,where u, v ∈ X are two watermarks injected into spatial and frequency domains, respectively. For the results presented in the main text, we set the watermark strength parameters c 1 and c 2 to be 0.05 each. To enhance invisibility further, we implement a circular mask with a radius of 150 for the frequency domain watermarks, inspired by the procedure outlined in [8] .Verification module setup In terms of the verification module V θ , throughout all the experiments in the main text, we adopt the pre-trained ResNet 18 [52] architecture.Training data augmentation Two types of data augmentations are employed during the training phase. (1) To enhance the robustness of the trained verification model V θ against Gaussian noise, ensuring its sustained predictive effectiveness post-smoothing, we introduce Gaussian noise into the training data. This augmentation entails the addition of noise with a mean of zero and a standard deviation of 0.5 during the training process. (2) Additionally, we employ standard image augmentations such as random cropping and flipping to facilitate training. In this section, we assess the watermark detection performance of our RAW using diverse model architectures: ResNet 9, ResNet 34, VGG 16 [53] and ViT [61] . The summarized results on both Stable-Diffusion-generated MS-COCO and DBDiffusion dataset can be found in Table 7 below. FID and CLIP scores are omitted, given that the watermark strength parameters c 1 and c 2 are consistent with those in the main text (i.e., both set to 0.5). Notably, we observed an improvement in both benign and adversarial detection performances with a more complex model, ResNet 34, which is reasonable as complex models often possess greater learning capacity. In this section, we shift our focus to a more realistic scenario where we fine-tune both the classifier and the watermarks using pre-trained models. We note that the new scenario differs from the one discussed in the main text, where the classifier is trained from scratch with randomly initialized weights.To elaborate, we initially pretrain a set of watermarks along with their corresponding classifiers using a dataset such as MS-COCO. Subsequently, we fine-tune this pair using a new dataset. We present the results of this strategy by varying the number of new training data and summarize the outcomes in Table 8 below. In contrast to the training from scratch scenario described in the main text, our RAW framework demonstrates a significant improvement with a reasonably small training dataset size. In this section, we examine the watermark detection results of our RAW approach across various generative models, specifically utilizing two widely recognized architectures: SDXL and BriXL. We maintain consistent settings as detailed in the main text. The AUROC outcomes are consolidated in Table 9 . Our findings indicate that our RAW method achieves high AUROC scores for both MS-COCO and DBDiffusion datasets, underscoring its broad applicability. In this section, we present additional visual examples in Figure 5 and 4 resulting from our proposed RAW method. Our observations reveal no notably discernible differences between the original and watermarked images, highlighting the effectiveness of our approach.Pixel Differences (5×) Justification: The claims in the abstract and introduction accurately reflect the paper's contributions and scope.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Justification: We conformed with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).38th Conference on Neural Information Processing Systems (NeurIPS 2024). 


Decision: 3x BA. This paper proposes to embed watermarks into images after generation by adding learnable watermarks directly on the original image and employing a classifier to detect the presence of the watermark. The reviewers agree on the (1) important topic, (2) novel method, and (3) clear improvements. Most of the concerns, such as the insufficient evaluation, insufficient recent baselines, and insufficient analysis against image manipulations, have been addressed by the rebuttal. Therefore, the AC leans to accept this submission.

Reviews: review: paper_summary: paper_summary: This paper proposes a post-processing watermarking strategy that embeds watermarks into images after generation. The strategy is designed to be computationally efficient and model-agnostic. It involves training a learnable watermark and embedding it into both the frequency and spatial domains of the original image. The embedded watermark is later verified by a classifier, which is trained jointly with the watermark.
strength_weakness: strength_weakness: * The survey of existing works in the "Related Works" section is incomplete. Numerous works on watermark frameworks for diffusion models are not included.
* It would be helpful if the authors provided a figure depicting an overview of the proposed method.
* The authors should clarify why they did not use signSGD to optimize the verification model if it is better. What are the data-level optimization problems?
* The reasoning for optimizing the watermark only based on $L_0$ instead of $L_{raw}$ needs to be explained.
* The method is not plug-and-play; it requires training the watermark and the verifier for each target generative model.
* Experimental evaluation:
  * For the SDXL model, no comparison with baselines is made.
  * Metrics for image quality comparison are limited. The authors should also include metrics like SSIM.
questions: questions: see above.
limitations: limitations: see above.
paper_summary: paper_summary: This paper proposes a Robust, Agile plug-and-play Watermarking (RAW) framework, which adds learnable watermarks directly on the original image and employs a classifier to detect the presence of the watermark. This design enhances both adaptability and computation efficiency, providing an model-agnostic for real-time applications. This paper show that RAW achieves provable guarantees on the false positive rate (FPR) for detection, even under adversarial attacks.
strength_weakness: strength_weakness: 1. In Equation (5), the watermark is also optimized, which indicates that this method can only add the same watermark. This limits the application.
2. The proposed method still embeds the watermark into the carrier image and is not specifically designed for AI-generated images.
3. In Table 2, some columns lack bolded data, and some bolded entries are not the best results.
4. The compared methods are outdated, failing to highlight the robustness and quality of this approach.
5. Since the carrier is involved in the method, PSNR should be evaluated.
questions: questions: N/A
limitations: limitations: N/A
paper_summary: paper_summary: This paper introduces a novel image watermarking technique that injects signals into both the frequency and pixel domains of images. The watermark is specifically designed for the detection of AI-generated images and incorporates smoothing techniques to provide provable error guarantees against attacks under certain conditions. Additionally, the method's simplicity leads to faster watermarking speeds compared to existing techniques.
strength_weakness: strength_weakness: - A comparison against more recent watermarking techniques is needed. In Table 4, the paper does not provide performance results (i.e., AUROC) against attacks compared to recent methods such as TreeRing [1], Trustmark [2], and RoSteLAS [3]. The included watermarking techniques (i.e., DwtDCT, RivaGAN) are known to be weaker against attacks compared to state-of-the-art watermarks.

- The paper lacks a comprehensive analysis of performance against image manipulations. While Table 4 illustrates performance on various attacks, it is unclear whether the hyper-parameters for these attacks have been set to their full strength. For instance, what hyper-parameters are being used for the VAE and Diff attacks? It is recommended that the authors include image quality metrics for the attacks (e.g., PSNR, SSIM) to demonstrate that their method maintains acceptable performance even under strong attacks that preserve image quality. Additionally, including a wider variety of diffusion attacks (e.g., pixel space and latent space) and potentially adversarial attacks (e.g., model substitution attacks), as discussed in the literature [4][5], would provide a more comprehensive evaluation.



[1] Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust, 2023.

[2] Tu Bui, Shruti Agarwal, and John Collomosse. Trustmark: Universal watermarking for arbitrary resolution images, 2023.

[3] Tu Bui, Shruti Agarwal, Ning Yu, and John Collomosse. Rosteals: Robust steganography using autoencoder latent space, 2023.

[4] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., and Feizi, S. Robustness of ai-image detectors: Fundamental limits and practical attacks, 2023.

[5] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., Zhu, S., Mohamed, A., Wen, Y., Goldstein, T., et al.: Benchmarking the robustness of image watermarks, 2024.
questions: questions: - Based on Table 3, RAW achieves a comparably low FID and high CLIP score, suggesting it applies an imperceptible and low-budget watermark signal to the images in the current evaluations. However, a more detailed evaluation of RAW's robustness against provable diffusion attacks [1][2] that claim to break imperceptible watermarks would be valuable. I am curious to know the authors' opinion on how their method might perform against such attacks.

[1] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., and Feizi, S. Robustness of ai-image detectors: Fundamental limits and practical attacks, 2023.

[2] Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai, 2023.
limitations: limitations: - As noted in the limitations section, this technique is designed for use with a single watermark key (w) rather than multiple values, which is a disadvantage compared to existing methods. However, given the importance of the task of detecting AI-generated images, this focus on a single key value is justified and does not limit the value of this work.

Comments: title: Thanks for the response
Final Comments
comment: The authors have addressed my concerns about the evaluation metric, SDXL model, and optimizer. Thanks! However, I still have concerns about the rest of the questions and consider them as weaknesses.  Hence, I decided to keep my score.
My concerns have not been well addressed, and even some have gone unanswered. I decide to lower the score and lean towards Borderline reject.