Invisible Image Watermarks Are Provably Removable Using Generative AI
Authors: Xuandong Zhao
Kexun Zhang
Zihao Su
Saastha Vasan
Ilya Grishchenko
Christopher Kruegel
Giovanni Vigna
Yu-Xiang Wang
Lei Li
Venue: NeurIPS 2024 poster (2024)
Keywords: Image Watermark
AI Safety
Primary area: safety_in_machine_learning

Abstract: Invisible watermarks safeguard images' copyrights by embedding hidden messages only detectable by owners. They also prevent people from misusing images, especially those generated by AI models.
We propose a family of regeneration attacks to remove these invisible watermarks. 
The proposed attack method first adds random noise to an image to destroy the watermark and then reconstructs the image. 
This approach is flexible and can be instantiated with many existing image-denoising algorithms and pre-trained generative models such as diffusion models. Through formal proofs and extensive empirical evaluations, we demonstrate that pixel-level invisible watermarks are vulnerable to this regeneration attack.
Our results reveal that, across four different pixel-level watermarking schemes, the proposed method consistently achieves superior performance compared to existing attack techniques, with lower detection rates and higher image quality.
However, watermarks that keep the image semantically similar can be an alternative defense against our attacks.
Our finding underscores the need for a shift in research/industry emphasis from invisible watermarks to semantic-preserving watermarks. Code is available at https://github.com/XuandongZhao/WatermarkAttacker

Invisible Image Watermarks Are Provably Removable Using Generative AI
Xuandong Zhao UC Berkeley
Kexun Zhang Carnegie Mellon University
Zihao Su UC Santa Barbara
Saastha Vasan UC Santa Barbara
Iyy Grishchenko UC Santa Barbara
Christopher Kruegel UC Santa Barbara
Giovanni Vigna UC Santa Barbara
Yu-Xiang Wang
Lei Li Carnegie Mellon University
UC San Diego
Abstract
Invisible watermarks safeguard images' copyrights by embedding hidden messages only detectable by owners, They also prevent people from misusing images, especially those generated by AI models. We propose family of regeneration attacks to remove these invisible watermarks. The proposed attack method first adds random noise to an image destroy the watermark and then reconstructs the image. This approach is flexible and can be instantiated with many existing image- denoising algorithms and pre-trained generative models such as diffusion models. Through formal proofs and extensive empirical evaluations, we demonstrate that pixel-level invisible watermarks are vulnerable to this regeneration attack. Our results reveal that, across four different pixel-level watermarking schemes, the proposed method consistently achieves superior performance compared to existing attack techniques, with lower detection rates and higher image quality. However, watermarks that keep the image semantically similar can be an alternative defense against our attacks. Our finding underscores the need for shift in research/industry emphasis from invisible watermarks to semantic- prenuticppreervinngg watermarks. Code is available https:/ // h////gtttt githuh..bb.... com/ /wwaa//ngnaaao/////aa/a wwtpe//aaaaeeeeeaeaaaa..
Introduction
Generative models like DALL-E [45], Imagen [50], and Stable Diffusson [47] can produce images that are often visually indistinguishable from those created by human photographers and artists, potentially leading to misunderstandings and false beliefs due to their visual similarity. To address this, government leaders [30, 26, 29] advocate for the responsible use AI, emphasizing the importance of identifying Al- Al-generated content. In response, leading Al companies such Google [20]. Microsoft [61], and OpenAl [4] have pledged to incorporate watermarks into their Al-generated images. Invisible watermarks [62, 17, 9] preferred they preserve image quality and less likely to be removed by laypersons. However, abusers are aware of these watermarks and may attempt remove them, making crucial for invisible watermarks to be robust against evasion attacks.
Existing attacks can be categorized into two types. The first is destructive, where the watermark is removed by corrupting the image. Typical destructive attacks include modifying the brightness, JPEG compression, and adding Gaussian noise. These approaches are effective at removing watermarks, but
'Equal contribution. Email addresses: xuandongzhao@berkkll.y edu kexunzBandrev. cmu, edu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Attacker removes the watermark
Original Image
Watermarked Image
Attacked Image
Add
Map
Noise
Regenerate
Watermark
8
Embedding
Noisy Embedding
Watermark
Detector
No watermark
Detected
Undetected
Removing invisible watermarks: The proposed attack first maps the watermarked image its embedding, which is another representation of the image. Then the embedding is noised to destruct the watermark. After that, regeneration algorithm reconstructs the image from the noisy embedding.
Figure 1:
they result in significant quality loss. The second type of attack is constructive, where the watermark is treated attacks include image-denoising techniques like Gaussian blur [25], BM3D [11], and toariing-basee approaches [65]. However, they cannot remove resilient watermarks easily. To counter these attacks, learning-based watermarking methods [67, 66, 15] were proposed to explicitly train against the
noise the original image and removed by purifying the image. Constructive
known attacks be robust. But how about other attacks? What is the end of this cat-and-mouse game? In this paper, we ask a more fundamental question:
Is an invisible watermark necessarily non-robust?
To be more precise, is there fundamental trade-off between the invisibility of watermark and its resilience to any attack that preserves the image quality to certain-level?
To address this question, propose regeneration attack that leverages the strengths of both destructive and constructive approaches. The pipeline of the attack is given in Figure I. Our attack first corrupts the image by adding Gaussian noise to its latent representation. Then, reconstruct the image from the noisy embedding using generative model. The proposed regeneration attack is flexible in that it can be instantiated with various regeneration algorithms, including traditional denoisers and deep generative models such as diffusion models [23]. Ironically, the recent advances in generative models that created the desperate need for invisible watermarks also making watermark removal easier when integrated into the proposed regeneration attack.
Surprisingly, prove that the proposed attack guarantees the removal of certain invisible watermarks such that no detection algorithm could work. In other words, our attack is provably effective in removing any watermarks that perturb the image within limited range of f2-distance, regardless of whether they have been proposed or have not yet been invented. We also show that our attack maintains image quality comparable to the unwatermarked original.
To validate our theory, we conduct extensive experiments on four widely used invisible water- marks [41, 67, 53, 15]. Our proposed attack method significantly outperforms five baseline methods terms of both image quality and watermark removal. For the resilient watermark scheme RivaGAN, our regeneration attacks successfully remove 98% of the invisible watermarks while maintaining a PSNR above 30 compared to the original images. With the empirical results and the theoretical guarantee, claim that pixel-level invisible image watermarks vulnerable to our regeneration attacks.
Given the vulnerability of invisible watermarks, we explore another option: semantic watermarks. Semantic watermarks do not limit the perturbation to be within an éy-distance. As long as the watermarked image looks similar and contains similar content, it is considered suitable for use. One instance of such semantic watermarks is Tree-Ring [60], which has shown resilience against our attacks. While not perfect solution, the watermark becomes somewhat "visible", semantic watermarking offers a potential path forward for protecting the proper use of Al-generated images when invisible watermarks provably ineffective.

---

Our contributions can be summarized as follows:
We propose family of regeneration attacks for image watermark removal that can be instantiated with many existing denoising algorithms and generative models.
We prove that the proposed attack is guaranteed to remove certain pixel-based invisible watermarks and that the regenerated images are close to the original unwatermarked image.
We evaluate the proposed attack on various invisible watermarks to demonstrate their vulnerability and its effectiveness compared with strong baselines.
We explore other possibilities embed watermarks in visible yet semantically similar way. Empirical results indicate that this approach works better under our attack and is worth investigating an alternative.
2 Related Work and Background
2.1 Related Work
Image watermarking and steganography. Steganography and invisible watermarking are key techniques in information hiding, serving diverse purposes such as copyright protection, privacy- preserved communication, and content provenance. Early works in this area employ hand-crafted methods, such as Least Significant Bit (LSB) embedding [62], which subtly hides data in the lowest order bits of each pixel in an image. Over time, numerous techniques have been developed to imperceptibly embed secrets in the spatial [17] and frequency [24, 44] domains of an image. Additionally, the emergence of deep learning has contributed significantly to this field. Deep learning methods offer improved robustness against noise while maintaining the quality of the generated image.
SteganoGAN [66] uses generative adversarial networks (GAN) for steganography and perceptual image optimization. RivaGAN [67], further improves GAN-based watermarking by leveraging attention mechanisms. SSL watermarking [15], trained with self-supervision, enhances watermark features through data augmentation. Stable Signature [14] fine-tunes the decoder of Latent Diffusion Models to add the watermark. Tree-Ring [60] proposes a semantic watermark, which watermarks generative diffusion models using minimal shifts of their output distribution. This work focuses on the removal of invisible watermarks, as opposed to visible watermarks, for several reasons. Visible watermarks straightforward for potential adversaries to visually identify and locate within an image. Additionally, removal techniques for visible watermarks are already extensively studied in prior work such [35, 21, 10]. In contrast, invisible watermarks do not have explicit visual cues that reveal their presence or location. Developing removal techniques for imperceptible embedded watermarks presents unique challenges.
Deep generative models. The figh-dimensional nature of images poses unique challenges to generative modeling. In response to these challenges, several types of deep generative models have been developed, including Variational Auto-Enet Encoders (VAEs) [56, 55], Generative Adversarial Networks (GANs) [18], flow-based generative models [46], and diffusion models [23, 47]. These models leverage deep latent representations to generate high-quality synthetic images and approximate the true data distribution. One particularly interesting of generative models is for data purification, ie., removing the adversarial noise from a data sample. The purification is similar to watermark removal except that purification is defense strategy while watermark removal is attack. The diffusion-based approach in [43] is similar to an instance of our regeneration attack, but the usage is different in our paper and our theoretical guarantee of watermark removal is stronger. In this paper, we aim to demonstrate the capability of these deep generative models in removing invisible watermarks from images by utilizing the latent representations obtained through the encoding and decoding processes.
2.2 Problem Setup
This section defines invisible watermarks and the properties of an algorithm for their detection. It then discusses the threat model for the removal of invisible watermarks.
Definition 2.1 (Invisible watermark). Let E X be the original image and x., Watermark(s;; aux) be the watermarked image for watermarking scheme that is function of x and any auxiliary information aux, e.g.,& secret key. We say that the watermark is A-invisible on clean image a w.n.t. "distance" function dist: X x X + R2 if dist(x, xw) < A.
3

---

Definition 2.2 (Watermark detection). A watermark detection algorithm Detect : X' x aux - {0. 10 determines whether an image being aux. Detect may make two types of mistakes, false positives (classifying unwatermarked image as watermarked) and false negatives (classifying a watermarked image as unwatermarked), could be drawn from either the null distribution Po or watermarked distribution P1- We define Type I error (or false positive rate) é == Prz-Po (Detect(z) = and Type error (or false negative rate) e2 PI [Detect(z) 0j.
X' is watermarked with the auxiliary information (secret key)
watermarking scheme is typically designed such that P1 is different from Po so the correspond- ing (carefully designed) detection algorithm can distinguish them almost perfectly, that is, to ensure él and E2 are nearly O. An attack on a watermarking scheme aims at post-processing a possibly watermarked image which changes both Po and Pi with the hope of increasing Type I and Type II error at the same time, hence evading the detection.
We consider the following threat model for removing invisible watermarks from images:
Adversary's capabilities. We assume that adversary only has access to the watermarked images. The watermarking scheme Watermark, the auxiliary information aux, and the detection algorithm Detect unknown the adversary. The adversary can make modifications these already watermarked images it has access to using arbitrary side information and computational resources, but it cannot rely on any specific property of the watermarking process and cannot query Detect.
Adversary's objective. The primary objective of an adversary is to render the watermark detection algorithm ineffective. Specifically, the adversary aims to produce image ă from the watermarked image zw which causes the Detect algorithm to always have a high Type error (false positive rate) high Type II error (false negative rate). Simultancously, the output image I should maintain comparable quality to the original, non non-watermarked image. The adversary's 's objective will be formally defined later.
2.3 Invisible Watermark Detection
Watermarking methods implant k-bit secret information into images. The detection algorithm Detect uses an extractor for this hidden data, applying statistical test to ascertain if the extracted message matches the secret. This test evaluates the number of matching bits M (m, m²) between the extracted m' and original m E {0, 1 messages. watermark is detected M (m, m') 2 for predefined threshold [63, 38). Formally, we distinguish whether an image was watermarked (H,) or not (Hn). Assuming under null hypothesis Ho that the extracted bits behave like independent Bernoulli variables with a 0.5 probability. M mm, follows binomial distribution B 0.5). The false positive rate (e1) corresponds to the probability that M (m, m²) exceeds т. This has a closed form using the regularized incomplete beta function Ia (a; b):
1 E (T) = (M m') > T Ho) = 2
Σ
k -1/2(T + +1,
1
Decision threshold. We consider watermark be detected, we can reject the null hypothesis Ho with p-value less than 0.01. In practice, for a k 32-bit watermark, we require least 23 bits to be extracted correctly in order to confirm the presence a watermark. This provides a reasonable balance between detecting real watermarks and avoiding false positives.
3 The Proposed Regeneration Attack
Our attack method first destructs a watermarked image by adding noise to its representation, and then reconstructs it from the noised representation.
Specifically, given an embedding function Φ RR + Rd, regeneration function A Rd R". and a noise level o, the attack algorithm takes watermarked image Ew E IR. and returns
destructive
f A ((xw) N3(0
(I)
constructive

---

Algorithm/ 1 Regeneration Attack Instance: Removing invisible watermarks with diffusion model
input The watermarked image Tw, time step tn determining the level of noise added. output A reconstructed clean image fi.
I: 20 ((F // map the watermarked image zw to latent space
2: 6 ~ NO,, Ia) // sample a random normal Gaussian noise
3: 2-- + Va(t))zo + vI- a(t))e // add noise to the latent, noise level determined by 8*
4: 8g solve 2t , tt, S, g) // denoise the noised latent reconstruct clean latent
5: 2 t 0(ān) // map the reconstructed latent back to watermark-frre image
6: return
The first step of the algorithm is destructive. It maps the watermarked image Tu to an embedding ((xw) (which is possibly different representation of the image), and adds i.i.d. Gaussian noise. The explicit noise shows the destructive nature of the first step. The second step of the algorithm constructive. The corrupted image representation ((xw) (O 12) is passed through regeneration function A to reconstruct the original clean image.
There are various different choices for & and A that can instantiate our attack. e can be as simple as identity map, or as complicated as deep generative models including variational autoencoders [31]. A can be traditional denoising algorithms from image processing and recent AI models such as diffusion [23]. The choice of Φ and A may change the empirical results, but does not affect the theoretical guarantee. In the following sections, introduce three combinations of and A to instantiate the attack. Among the three, the diffusion instantiation is the most complicated and we describe it with pseudocode Algorithm
3.1 Attack Instance 1: Identity Embedding with Denoising Reconstruction
Set ø to be identity map. then A can be any image denoising algorithm, c.g., BM3D [11], TV- denoising [48], bilateral filtering [54], DnCNNs [65], or learned natural image manifold [6, 11, 65, 64]. A particular example of interest is "denoising autoencoder" [57]. which takes ø to be identity, adds noise to the image deliberately, and then denoises by attempting reconstruct the image. Observe that for "denoising autoencoder" we do not need to add additional noise.
3.2 Attack Instance 2: VAE Embedding and Reconstruction
The regeneration attack in Equation can be instantiated with a variational autoencoder (VAE). VAE [31] consists of an encoder (=(=x) that maps a sample to the latent space and decoder po(x|z) that maps latent back to the data space x. Both the encoder and decoder are parameterized with neural networks. VAEs are trained with a reconstruction loss that measures the distance from the reconstructed sample to the original sample and a prior matching loss that restricts the latent to follow pre-defined prior distribution.
Instead of mapping x directly to the encoder maps it to the mean p(x) and variance (xx) of Gaussian distribution and samples from it. Therefore, VAE already adds noise during the encoding stage (though its variance depends on the sample x, which is not exactly the same as defined in Equation 1), so there is no need to add extra noise. Note that this is similar to the situation of denoising autoencoders described in Section 3.1, as the denoising autoencoder is a trivial case of VAE where u(x) is identity.
3.3 Attack Instance 3: Diffusion Embedding and Reconstruction
The regeneration attack can also be instantiated with diffusion models. Diffusion models [23] define generative process that learns to sample from unknown true distribution p(2.). This process is learned by estimating original samples from randomly noised ones. In other words, diffusion models trained to denoise, which makes them candidates for the regeneration function A in the proposed attack. The embedding function & can either be identity [23] or latent embedding [47], depending on the space where diffusion is trained.
For diffusion models, the process of adding noise to clean sample is known as the forward process. Likewise, the process of denoising noisy sample is known as the backward process.
5

---

The forward process is defined by the following stochastic differential equation (SDE): de f(z, t)dt + (())d (2), where E [0, 1]. E R. w(t) R is a standard Wiener process. and g are real-valued functions. The backward process can then be described with its reverse SDE: d: [f(2, 1) - V log (5)] di + (()) (3), where i is a reverse Wiener process. Diffusion models parameterize Va log (e() with neural network t(z,t). By substituting s(z, t) into Equation 3, the backward SDE becomes known and solvable using numerical solvers [34, 36],
2g = t2, f.))
Among many ways to define f and g in Equation 2, variance preserving SDE (VP-SDE) is commonly
used [23, 47]. Under this setting, the conditional distribution of the noised sample is the following Gaussian [51]: p0=) = N (((a((f)=o, 1 a<t)) (4), where a(t) pre-defined noise schedule. The variance of the original distribution p(zo) is preserved at any step.
As defined in Algorithm 1, our algorithm removes the watermark from the watermarked image Ta using diffusion models. Tw is first mapped to the latent representation Fg. which is then noised the time step t*, A latent diffusion model is then used to reconstruct the latent žu, which is mapped back to an image
Similar to denoising autoencoders, in either diffusion or VAEs, the noise-injection is integral to the algorithms themselves, and no additional noise-injection is needed.
Theoretical Analysis
We show in this section that the broad family of regeneration attacks as defined Equation enjoy provable guarantees their ability to remove invisible watermarks while retaining the high quality of the original image. Our proofs are deferred to Appendix D. More discussion on the implications and interpretation of our theoretical analysis can be found in Appendix C.
4.1 Certified Watermark Removal
How do we quantify the ability of an attack algorithm to remove watermarks? We argue that if after the attack, no algorithm is able distinguish whether the result is coming from watermarked image or the corresponding original image without the watermark, then we consider the watermark certifiably removed. More formally:
Definition 4.1 (f-Certified (Certified-Watermarkk Watermark-Freet We say that watermark removal attack is f-Certified- Watermark-Frees (or f (-CWF) against a watermark scheme for non-increasing function f: [o, 1] [0, 1]. if for any detection algorithm Detect X' x aux {0, 1}. the Type II error (false negative rate) of Detect obeys that 62 2 f(1) for all Type I error 0 s S 1.
Let us also define a parameter to quantify the effeet of the embedding function d. Definition 4.2 (Local Watermark- -Specifiii Lipschitz property). We say that an embedding function Φ X + R satisfies T.z, Watermark-Specine Lipschitz property if for watermark scheme w that generates Iy with x,
( - ((x)) S Laz,o ( x/|.
The parameter Lr, measures how much the embedding compresses the watermark added on particular clean image v. If & is identity, then La.w 1. .. If ø is a projection matrix to a linear subspace then 0 S Lew I depending on the magnitude of the component z ø in this subspace. For a neural image embedding é, the exact value of Læ, is unknown but given each aw and x can be computed and estimated efficiently. We defer more discussion to the Appendix A.
Theorem 4.3. For a A-invisible watermarking scheme with respect lz-distance. Assume the embedding functiin & of the diffusion model is we..w-lollly Lipschitz. The randomized algorithm A(g(.) + (0, o2Ta) produces a reconstructed image which satisfies f-CWF with
f(G) = ( (
where de is the Cumulative Density Function function of the standard normal distribution.

---

Figure 2: Theoretical and empirical trade-off functions for DwtDetSvd watermark detectors after our attack. Trade- off functions indicate how much less Type II error (false negative rate) the detector gets in return by having more Type I error (false positive rate). Theoretically, after the attack, no detection algorithm can fall in the Impossibility Region and have both Type error and Type II error at low level. Empirically, the watermark detector performs even worse than the theory, indicating the success of our attack and the validity of the theoretical bound. We use 500 watermarked MS-COCO images with an empirically valid upper bound of L 1 and noise level a = 1.164. An additional example for the RivaGAN watermark provided in Figure 12.
1.0
1.16 Indistinguishable Diffusion ttack
Impossibility Region
error
0.6
0.4
0.2
Type I error
Figure illustrates what the tradeoff function looks like. The result says that after the regeneration attack, it is impossible for any detection algorithm to correctly detect the watermark with high confidence. In addition, it shows that such detection is hard telling the origin of single sample Y from either of the two Gaussian distributions (00, 1) and (LL A/o, 1). Specifically, when there is uniform upper bound L 2 Lz, ws can calibrate such that the attack is provably effective for prescribed ( watermarks (sce specific constructions in Appendix A).
guarantee for all input images and all i-invisible
The proof, deferred to Appendix D, leverages an interesting connection to a modern treatment of differential privacy [13] known as the Gaussian differential privacy [12]. The work of [12] itself is refinement and generalization of the pioneering work of [59] and [28] which established tradeoff-function view.
4.2 Utility Guarantees
In this section, we prove that the regenerated image is close to the original (unwatermarked) image To. This is challenging because the denoising algorithm only gets access to the noisy version of the watermarked image. Interestingly, we can obtain a general extension lemma showing that for any black-box generative model that can successfully denoise noisy yet unwatermarked image with high probability, the same result also applies to the watermarked counterpart, except that the failure probability is slightly larger.
Theorem 4.4. Let To be an image with n pixels and Φ T Rd be an embedding function. Let A be image generation denoising algorithm such that with probability at least 1 - 8, [|A(()z))) + NO0 oala) - xoll S E Then for any A-invisible watermarking scheme that produces zw from a clean image Fo. then à A(((xw)) ) o Ia)) satisfies that
| zoll S Exo.0,5
with a probability at least 1 5. where min.ER 8 + do
wm 3
c
(-
no 3
in
25
which & denotes the standard normal CDF and Â = Lizg,
The theorem says that if a generative model is able to denoise noisy version of the original image, then the corresponding watermark removal attack using this generative model provably produces an image with similar quality.
22 42
Corollary 4.5. The expression for 3 above can be (conservatively) simplified to 8 Se 81/2,
For example if x Lu then this is saying that if Ez,.0,5 depends logarithmically on 1/5, the same exponential tail holds for denoising the watermarked image.
The above result is powerful in that makes no assumption about what perturbation the watermarking schemes could inject and which image generation algorithm we use. We give few examples below.
For denoising algorithms with theoretical guarantees, e.g.. TV-denoising [27, Theorem 2], our results imply provable guarantees on the utility for the watermark removal attack of the form, "w.h.p.,
1 | - 2oll² = ò [TVV2a(zo) i.e., vanishing mean square error (MSB) as n gets bigger.
7

---

Original Image
DwtDctSvd WM
Brightness
JPEG
Gaussian blur
VAE attack
Diffusion attack
Figure 3: Examples of attacks on DwtDetSvd watermarking, including destructive attacks (e.g.. brightness change and JPEG compression), constructive attacks (e.g., Gaussian blur), and regeneration attacks using VAES and diffusion models. Brightness change, JPEG compression, VAE attack, and diffusion attack successfully remove the watermark. The VAE attack over-smooths the image, resulting in blurriness. The diffusion attack maintains high image quality while removing the watermark. Additional attack examples for other watermarking schemes are in Figures 8, 9, 10, 11.
For modern deep Bearning-based image denoising and generation algorithms where worst-case guarantees are usually intractable, Theorem 4.4 is still applicable for each image separately. That is to say. as long as their empirical denoising quality is good on an unwatermarked image, the quality should also be good its watermarked counterpart.
5 Evaluation
Datasets. We evaluate our attack on two types of images: real photos and Al-generated images. For real photos, we use 500 randomly selected images from the MS-COCO dataset [33]. For Al- generated images, we employ the Stable Diffusion on-uu.iiiiiii..... v2.1 1 model from Stable Diffusion [47], a state-of-the- -he--tte-rt generative model capable of producing high-fidelity images. Using prompts from the Stable Diffusion Prompt (SDP) dataset', we generate 500 images encompassing both photorealistic and artistic styles. This diverse selection allows for comprehensive evaluation of our attack on invisible watermarks. All experiments are conducted on Nvidia A6000 GPUs.
Watermark settings. We evaluate four publicly available pixel-level watermarking methods: DwtDetSvd [41], RivaGAN [67], StegaStamp [53], and SSL watermark [15]. These methods represent a variety of approaches, ranging from traditional signal processing to recent deep learning techniques, as introduced in Section 2.1. To account for watermarks of different lengths. we use k 32 bits for DwtDetSvd, RivaGAN, and SSL watermark, and k = 96 bits for StegaStamp. For watermark detection, we set the decision threshold to reject the null hypothesis with p < 0.01, requiring the detection of 23 out of 32 bits or 59 out of 96 bits, respectively, for the corresponding methods, described in Section 2.3. For watermark extraction, we use the publicly available code for each method with default inference and fine-tuning parameters specified in their papers.
Attack baselines. To thoroughly evaluate the robustness of our proposed watermarking method, we test against a diverse set of baseline attacks representing common image perturbations. We select both geometric/quality distortions and noise manipulations that could potentially interfere with embedded watermarks. Specifically, the baseline attack set includes: brightness adjustments with enhancement factors of [2, 4, 6, 8, 12], contrast adjustments with enhancement factors of [2, 3, 4. 5, 6, 7J, JPEG compression at quality levels [10, 20, 30, 40, 50, 60], Gaussian noise addition with
2-1-base -roompts

---

Cherg2000
PPG
Gasszin
Brigionsss
Centrst
DctDrrvdd watermaking
KivOOAR Hezemarking
SSL notermoking
StapaStamo Notemar
0.3
0.3
5.0
0.3
000
6.0
FSMK
FSAR
PSWK
PS
Oc1DwiBed watermarking
NauGAAN wazermarkng ki
5SL notermaking
StagaStamp notermarke
1.0
14
3.0
3.0
0.4
0.3
0.6
0.2
82
0.5
Figure 4: Quality-deteettbbliiy tradeoff for four watermarking schemes under eight attack methods on the MS-COCO dataset. Regeneration attacks (Diffusion model, VAE-Cheng2020, and VAE- Bmshj2018) are highlighted for their performance. The x-axis shows image quality metries (SSIM and PSNR, higher values indicate better quality), while the y-axis represents the detection metric True Positive Rate at 1% False Positive Rate (TPR FPR=0.01, lower values are better for attackers). The strongest attacker should appear in the lower right corner of these plots. Regeneration attacks demonstrate superior performance compared to other attack methods, achieving both lower TPR and higher image quality. Quality-detectabiiity tradeoff results for the SDP dataset are in Figure 7.
mean of O and standard deviations of [5, 10, 15, 20, 25, 30), and Gaussian blur with radii of [2, 4, 6. 10, 12]. Further details are provided in Appendix B.
Proposed attacks. For regeneration attacks using variational autoencoders, evaluate two pre-trained image compression models from the CompressAI library [5]: Bmshj2018 [3] and Cheng2020 [7]. Compression factors are set to [1, 2, 3, 4, 5, 6], where lower factors correspond to more heavily degraded images. For diffusion model attacks, we use the Stable Diffusion-v2 model. The number of noise steps is set to [10, 30, 50, 100, 150, 200] (with =00.10, 0.17, 0.23, 0.34, 0.46, 0.57J), and we employ pseudo numerical methods for diffusion models (PNDMs) [34] to generate samples. By adjusting the compression factors and noise steps, achieve varying levels of perturbation for regeneration attacks.
Evaluation metrics. We evaluate the quality of attacked and watermarked images compared to the original cover image using two common metrics: Peak Signal-to-Noiss Ratio (PSNR) defined as PSNR (I, x²) 110--11oogo (MSE (z, 2'),, for images x, x' E [0, 1exhxx and Structural Similarity Index (SSIM) [58] which measures perceptual similarity. To evaluate the diversity and quality of watermarked images, we use Fréchet Inception Distance (FID) [22] between the distributions of watermarked and unwatermarked images. To evaluate the robustness of the watermark, we compute the True Positive Rate (TPR) at a fixed False Positive Rate (FPR), specifically TPR @ FPR=0.01. The detection threshold corresponding configuration: correctly decoding 23/32, 59/96, and 32/48 bits for the respective watermarking methods, described in the watermark settings.
FPR=0.01 is set according to each watermark's default
5.1 Results and Analysis
This section presents detailed results and analysis of the regeneration attack experiments on different watermarking methods. Some attacking examples are shown in Figure 3.

---

220-
Image Latent
did in Imagle
Ex Ci Latens
Table I: Performance of different watermarking meth-
ods. All methods successfully detect the embedded watermark.
10-
25-
MSS: COCO Dullace
SDP Genernted Datoset
DwtDctSud
RivaGAN
StagiStamp
Moreraark
PSNRT
SSIMT F1D1 1P--01--- |SNNR
PSNN SSIMI ETD,, 1PR0/18-0001
20.38
0.583
5.28
1,000
32-13
0,972
9.52
1.030
Figure 5: The lz distances between original images and watermarked ones. StegaStamp watermarked images are much more different in both pixel space and latent space.
RisGDAN
40.55
0..79
1.030
SSL
41.79
23,87
StepStuap
1,000
1.000
Watermarking performance without attacks. Table I summarizes the watermarked image qual- ity and detection rates for images watermarked by the four methods without any attacks. Each Biiioodd-wwDetSdddd RivaGAN, SSL, and
embeds and retrieves mes- sages from the images. These methods are post-processing techniques, adding watermarks to existing images. Among them, SSL achieves the highest PSNR and SSIM values, indicating superior percep- tual quality and minimal visual distortion compared to the original images. DwtDetSvd achieves the lowest FID scores, suggesting the watermarked images maintain fidelity similar to clean images. In contrast, StegaStamp exhibits a noticeable drop in quality, with the lowest PSNR and highest FII) scores. As illustrated in Figure 6, StegaStamp introduced noticeable blurring artifacts.
Watermark removal effectiveness and image quality reservation. Figure 4 and 7 present the quality- deteitbbietet tradeoff results from applying various regeneration attacks to remove water- marks. The VAE and diffusion model-based attacks (VAE-Bmshj2018, VAE-Cheng2020, Diffusion) consistently achieve over 99% removal rates for the DetDwtSvd, RivaGAN, and SSL watermark- ing methods, demonstrating their high effectiveness. In contrast, StegaStamp exhibits the highest robustness, with effective removal only achieved by the diffusion model with substantial noise. This resilience is partially attributed to StegaStamp's 's lower visual quality and higher perturbation levels (as indicated in Table 1, Figure 6 and Figure 5). Yet, increasing the noise level in diffusion models reduces StegaStamp's resistance. As demonstrated in Figure and 7, higher noise levels improve the removal rate on StegaStamp, aligning with the theory that larger perturbations necessitate more noise for effective removal. This also aligns with the trade-off of lower image quality higher noise levels. Overall, the consistently high removal rates across various watermarking schemes demonstrate the effectiveness of regeneration attacks for watermark removal. In terms of image quality preservation, the regeneration attacks generally maintain high quality, as indicated by PSNR and SSIM metrics. VAE models yield higher PSNR and SSIM scores, suggesting superior perceptual quality from a GAN-based perspective. However, qualitative inspection of example images in Figure 3 reveals the VAE outputs exhibit some blurring compared to the diffusion outputs. Since PSNR and SSIM are known to be insensitive to blurring artifacts [42, 58), conclude that the choice of using regeneration with diffusion models should be guided by the specific requirements each application.
Potential defense. Although we show that the proposed attack guaranteed remove any pixel- based invisible watermarks, it is not impossible to detect Al-generated images. Semantic watermarks offer a viable detection method, with further details and results discussed Appendix
6 Conclusion
We proposed a regeneration attack on invisible watermarks that combines destructive and constructive attacks. Our theoretical analysis proved that the proposed regeneration attack is able to remove certain invisible watermarks from images and make the watermark undetectable by any detection algorithm. We showed with extensive experiments that the proposed attack performed well empirically. The proofs and experiments revealed the vulnerability of invisible watermarks. Given this vulnerability, we explored an alternative defense that uses visible but semantically similar watermarks. Our findings on the vulnerability of invisible watermarks underscore the need for shifting the research/industry emphasis from invisible watermarks to their alternatives.
10

---

References
[1] Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng. Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, et al. Benchmarking the robustness of image watermarks. arXiv preprint arXiv:2401.08857. 2024.
[2] Borja Balle and Yu-Xiang Wang. Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In International Conference on Machine Learning. pages 394-403. PMLR, 2018.
[3] Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang. and Nick Johnston. Variational image compression with scale hyperprior. arXiv preprint 10X.:00000 2018.
[4] Diane Bartz and Krystal Hu. Openai, google, others pledge to watermark ai content for safety, white house says, Jul 2023.
[5] Jean Bégaint, Fabien Racapé, Simon Feltman, and Akshay Pushparaja. Compressai: py- torch library and evaluation platform for end-to-end compression research. arXiv preprint arXiv:2011.03029 2020.
[6] Antoni Buades, Bartomeu Coll, and J-M Morel. non-local algorithm for image denoising. In 2005 IEEE computer society conference computer vision and pattern recognition (CVPR 055), volume 2, pages 60-65. leee, 2005.
[7] Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7939-7948, 2020.
[8] Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. Digital watermarking and steganography. Morgan kaufmann, 2007.
(9] Ingemar J Cox and Matt Miller. The first 50 years of electronic watermarking. EURASIP Journal Advances in Signal Processing, 2002:1-7, 2002.
[10] Xiaodong Cun and Chi-Man Pun. Split then refine: stacked attention-guided resunets for blind single image visible watermark removal. In Proceedings of the AAAI conference on artificial infelligence, 2021.
[11] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3-d transform-domen collaborative filtering. IEEE Transactions on image processing. 2608))2080-2095, 2007.
[12] Jinshuo Dong, Aaron Roth, and Weijie Su. Gaussian differential privacy. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):3-31 3-73,,:3377333 2022.
[13] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Crypiography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings pages 265-284. Springer, 2006.
[14] Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The sta- ble signature: Rooting watermarks in latent diffusion models. arXiv preprint arXiv:2303.154 154.5 2023.
[15] Pierre Fernandez, Alexandre Sablayrolles, Teddy Furon, Herv'e J' J'egou, and Matthijs Douze. Watermarking images in self-supervised latent spaces. ICASSP 2022 2022 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 3054-3058, 2021.
[16] Teddy Furon and Patrick Bas. Broken arrows. EURASIP Journal on Information Security. 2008:1-13. 2008.
[17] Kazem Ghazanfari, Shahrokh Ghaemmaghami, and Saeed R Khosravi. Lsb++; An improvement to Isb+ steganography. TENCON 201 1-2011 IEEE Region 10 Conference, pages 364-368. IEEE, 2011.

---

[18] Ian Goodfellow, Jean Pouget-Abadic Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(1 1):19--144, 2020.
[19] lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint orXiv:1412.6572. 2014.
[20] Google. Google keynote (google i/o 23). Google blog. 2023.
[21] Amir Hertz, Sharon Fogel, Rana Hanocka, Raja Giryes, and Daniel Cohen-Or. Blind visual motif removal from a single image. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 6858-6867, 2019.
[22] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[23] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6855 2020.
[24] Vojtëch Holub and Jessica Fridrich. Designing steganographic distortion using directional filters. In 2012 IEEE International workshop information forensics and security (WIFS), pages 234-239, IEEE, 2012.
[25] Osama Hosam. Attacking image watermarking and steganography- survey. International Journal of Information Technology and Computer Science, 1 11(3):23-37, 2019.
[26] The White House. President biden issues executive order on safe, secure, and trustworthy artificial intelligence. 2023.
[27] Jan-Christian Hütter and Philippe Rigollet. Optimal rates for total variation denoising. In Conference on Learning Theory, pages 1115-1146. PMLR, 2016.
[28] Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The composition theorem for differential privacy. In International conference machine learning, pages 1376-13888 PMLR, 2015.
[29] Makena Kelly. Chuck schumer calls on congress to pick up the pace on ai regulation, Jun 2023.
[30] Makena Kelly. White house rolls out plan to promote ethical ai, May 2023.
[31] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv: /3/2.6/ 14, 2013.
[32] Xinyu Li. Diffwa: Diffusion models for watermark attack. ArXiv, abs/2306. 12790, 2023.
[33] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[34] Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations, 2021.
[35] Yang Liu, Zhen Zhu, and Xiang Bai. Wdnet: Watermat--dccomposiion network for visible watermark removal. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3685-3693, 2021.
[36] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dol-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[37] Nils Lukas, Abdulrahman Diaa, Lucas Fenaux, and Florian Kerschbaum. Leveraging optimiza- tion for adaptive attacks image watermarks, ArXiv, abs/2309. 16952, 2023.
12

---

[38] Nils Lukas and Florian Kerschbaum. Ptw: Pivotal tuning watermarking for tre-trained image generators. arXiv preprint arXiv:2304.0736/. 2023.
[39] Matthew L Miller, Ingemar Cox, and Jeffrey A Bloom. Informed embedding: exploiting image and detector information during watermark insertion. In Proceedings 2000 International Conference on Image Processing (Cat. No. 00C113701), volume 3, pages 1-4. IEEE, 2000.
[40] Ilya Mironov. Rényi differential privacy, In IEEE computer security foundations symposium (CSF), pages 263-275. IEEE, 2017.
[41] KA Navas, Mathews Cheriyan Ajay, M Lekshmi, Tampy S Archana, and M Sasikumar. Dwt- dct-svd based watermarking. In 2008 3rd International Conference on Communication Systems Software and Middleware and Workshops (COMSWARE'OSS 088), pages 271 1-44. IEEE, 2008
[42] Peter Ndajah, Hisakazu Kikuchi, Masahiro Yukawa, Hidenori Watanabe, and Shogo Muramatsu. Ssim image quality metric for denoised images. In Proc. 3rd WSEAS Int. Conf. on Visualization, Imaging and Simulation, pages 53-58, 2010.
[43] Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashree Anand- kumar. Diffusion models for adversarial purification. In International Conference on Machine Learning, pages 16805-16827. PMLR, 2022.
[44] Tomáš Pevný, Tomáš Filler, and Patrick Bas. Using high-dimensional image models to perform highly undetectable steganography. In Information Hiding: 12th International Conference, IH 2010, Calgary, AB, Canada, June 28-30, 2010, Revised Selected Pupers 12, pages 161-177. Springer, 2010.
[45] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06/25, 2022.
[46] Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference machine learning, pages 1530-1538. PMLR, 2015.
[47] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[48] Leonid Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena, 60(1-4):259-266 1992.
[49] Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Malemir Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. ArXiv, abs/2310.000 2023.
[50] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479-3666 2022.
[51] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
[52] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties neural networks, arXiv preprint 2013.
[53] Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical photographs. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2117-2126, 2020.
[54] Carlo Tomasi and Roberto Manduchi. Bilateral filtering for gray and color images. In Sixth international conference on computer vision (IEEE Cat. No. 98CH36271), pages 846.846.
IEEE, 1998.
13

---

[55] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[56] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096-1100 1103, 2008.
[57] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful representations in deep network with a local denoising criterion. Journal of machine learning research, 11(12), 2010.
[58] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE Transactions on Image Processing, 13(4):600-61, 2004.
[59] Larry Wasserman and Shuheng Zhou. A statistical framework for differential privacy. Journal of the American Statistical Association, 105(489):375-388 2010.
[60] Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030, 2023.
[61] Kyle Wiggers. Microsoft pledges to watermark ai-generated images and videos. TechCrunch blog, 2023.
[62] Raymond B Wolfgang and Edward J Delp. watermark for digital images. In Proceedings of 3rd IEEE International Conference on Image Processing, volume 3, pages 219 1-22. IEEE, 1996.
[63] Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In Proceedings of the IEEE/CVF International conference on computer vision, pages 14448-14457, 2021.
[64] Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Lue Van Gool, and Radu Timofte. Plug-and- play image restoration with deep denoiser prior. IEEE Transactions Pattern Analysis and Machine Intelligenee 44(10):6360 030166660666666,666 2021.
[65] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning deep enn for image denoising. IEEE transactions on image 2017.
processing,
[66] Kevin Alex Zhang, Alfredo Cuesta-Infantee Lei Xu, and Kalyan Veeramachaneni. Steganogan: High capacity image steganography with gans, arXiv preprint arXiv: 00389, 2019.
[67] Kevin Alex Zhang. Lei Xu, Alfredo Infaate, and Kalyan Veeramachaneni. Robust invisible video watermarking with attention. ArXiv,
2019.
14

---

A Additional Discussion in FAQ style
How do we control a certain degree of Certified Watermark Freeness apriori? We note that our guarantee in Theorem 4.3 depends on the specific watermark injected on specific image instance through the unknown local Lipschitz parameter Zz Specifying a fixed CWF level requires us to have a uniform upper bound of Lr, independent to r and w. To give two concrete examples, when the embedding Φ chosen trivial to be the identity map, then we can take Lega S 1. When & is a lower pass filtering, e.g., Fourier transform than removing all high-freque -reeueryyyncy components except the top & dimension, then we can bound La.w I Vk/n n where is the number of pixels. More generally, any linear transformation with bounded operator norm works.
Is there a uniform bound for neural embedding e such as those from VAE and diffusion models? We believe there is, but these bounds might be too conservative use in practice. In particular, they might be >>> 1 due to the widely observed adversarial examples [52, 19]. If the injected watermarks are carefully designed such that the injected perturbation is aligned with an adversarial perturbation, then the resulting watermarked images will be more resilient to our attacks that leverage the neural embeddings. They will not be more resilient to our attacks that do not use neural embeddings though. In practice, we found that in all existing watermarks, neural embeddings result in substantially smaller distortions to the original images while suppressing Lzz,0r substantially (Figure 5).
Other ways of achieving CWF without adding Gaussian noise. Research from the golden age of digital watermarking [39, 9, 16, 8] has proposed methods such as quantization for removing watermarks that provide similar tradeoff's in practice to the watermarks that we experimented with in this paper. However, these methods, in general, do not come with provable CWF guarantees. It is critical that the watermark removal attack is randomized to enjoy similar properties to what we have in Theorem 4.3. That said, the classical removal attacks can be used in constructing better embedding function á which may help reducing the local Lipschitz parameter and, thus, improving the tradeoff between certified removal and utility of the regenerated image. Besides Gaussian noise, we can also add Laplace noise and other well-known perturbation mechanisms from the differential privacy and cryptography research community. Thoroughly investigating the impact of the choice of noise (and randomized quantization approaches) is an interesting direction of future research.
B
Additional Method Details
B.1
Invisible Watermarking Methods
In this section, we review several well-established invisible watermarking schemes that are evaluated in our experiments (Section 5). These approaches cover a range of methods, including traditional signal processing techniques and more recent deep learning methods. They include the default watermarking schemes employed by the widely used Stable Diffusion models [47]. We show some invisible watermarking examples in Figure
DwtDetSvd watermarking. The DwtDetSvd watermarking method [41] combines Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), and Singular Value Decomposition (SVD) to embed watermarks in color images. First, the RGB color space of the cover image is converted to YUV. DWT is then applied to the Y channel, and DCT divides it into blocks. SVD is performed on each block. Finally, the watermark is embedded into the blocks. DwtDctSvd is the default watermark used by Stable Diffusion.
RivaGAN watermarking, RivaGAN [67] presents a robust image watermarking method using GANs. It employs two adversarial networks to assess watermarked image quality and remove watermarks. An encoder embeds the watermark, while a decoder extracts it. By combining these, RivaGAN offers superior performance and robustness. RivaGAN is another watermark used by Stable Diffusion.
StegaStamp watermarking. StegaStamp [53] is robust CNN-based watermarking method. It uses differentiable image perturbations during training to improve noise resistance. Additionally, incorporates spatial transformer network to resist minor perspective and geometric changes like cropping. This adversarial training and spatial transformer enable StegaStamp withstand various attacks.
15

---

Original Image
Dwt WM
RivaGAN WM
SSL WM
StegaStamp WM
Figure More examples of different invisible watermarking methods.
SSL watermarking, SSL watermarking [15] utilizes pre-trained neural networks' latent spaces to encode watermarks. Networks pretrained with self-supervised learning (SSL) extract effective features for watermarking. The method embeds watermarks through backpropagation and data augmentation, then detects and decodes them from the watermarked image or its features.
B.2 Existing Attacking Methods
In this section, review common attacking methods that aim degrade remove invisible termarks in images. These methods are widely used to measure the robustness of watermarking algorithms against removal tampering [67, 53, 15, 14, 60). The attacking methods be cate- gorized into destructive attacks, where the watermark considered part of the image and actively removed by corrupting the image, and constructive attacks, where image processing techniques like denoising are used to obscure the watermark.
Destructive attacks intentionally corrupt the image to degrade or remove the embedded watermark. Common destructive attack techniques include:
Brightness/Contrast adjustment. This attack adjusts the brightness and contrast parameters of the image globally. Adjusting the brightness/contrast makes the watermark harder to detect.
JPEG compression. JPEG a common lossy image compression technique. It has a quality factor parameter that controls the amount of compression. A lower quality factor leads to more loss of fine details and a higher chance degrading the watermark.
Gaussian noise. This attack adds random Gaussian noise to each pixel of the watermarked image. The variance of the Gaussian noise distribution controls the strength of the noise. higher variance Icads to more degradation of the watermark signal.
16

---

CHC
PPG
Brigionoss
DcIDerSed watermrking
NivOOAN Hozemarkig
SSL notermaking
StapaStamo Notemar
0.3
8.0
0.3
0.0
0.3
12
FSMK
FSAR
PSNK
PS
watormarking
wozerarking
5SL notermaking
StagaStamp notermarke
1.0
1.0
3.0
3.0
0.4
0.4
0.6
0.2
0.5
Figure 7: Trade-off between quality and detectability for four watermarking schemes tested against eight attack methods the Stable Diffusion Prompt (SDP) dataset.
Constructive attacks aim remove the watermark by improving image quality and restoring the original unwatermarked image. Examples include:
Gaussian blur. Blurring the image by convolving it with Gaussian kernel smoothens the watermark signal and makes it less detectable. The kernel size and standard deviation parameter control the level of blurring.
Other recent attacking methods [32, 49, 37, 1] aim to remove invisible image watermarks using generative models. Concurrent to work, [32] use propose DiffWA, conditional diffusion model with distance guidance for watermark removal. However, DifIWA only works for low-resolution images and lacks theoretical guarantees. [49] train substitute classifier and conduct projected gradient descent (PGD) attacks on to deceive black-box watermark detectors. However, their approach requires multiple queries the target generator. [37] propose adaptive attack that locally replicates secret watermarking keys by creating differentiable surrogate keys used to optimize the attack parameters. However, they assume the attacker knows the watermarking method, which is a stronger assumption than ours. Our method removes invisible watermarks from high-resolution images, provides thcoretical justifications, and does assume knowledge of the watermarking algorithm.
C Remarks on Our Theoretical Guarantee
In this section, we discuss more about the theoretical guarantees in Section For the readers' convenience, we first restate the definitions and theorems and then discuss their implications and interpretation. The proofs can be found in Appendix D.
C.1 Certified Watermark Removal
How do we quantify the ability of an attack algorithm to remove watermarks? We argue that if after the attack, algorithm is able distinguish whether the result is coming from watermarked image or the corresponding original image without the watermark, then we consider the watermark certifiably removed. More formally:
Definition C.1 (f (CCrrtifiee- Certified-Watermarffe WVttermakk-Free), We say that a watermark removal attack is / f-Certified-
Watermark- FFree (or /-CWF) against watermark scheme for non-increasing function f: [0, [0, 1), if for any detection algorithm Detect x aux > (0, 1}. the Type II error (false negative rate) of Detect obeys that E2 2 f(e1) for all Type I error 0 < < 1.
17

---

Original Image
DwrDctSvd WM
Brightness
JPEG
Gaussian blur
VAE attack
Diffusion attack
Figure 8: More examples of watermarking attacks against DwiDetSvd.
Let us also define parameter to quantify the effect of the embedding function Definition C.2 (Local Watermark-Specinn Lipschitz property). We say that an embedding function 6: R satisfies LLocll Watermark-Specife Lipschitz property if for watermark scheme w that generates Ia with x,
[1(())) ((x))1 S L | x||--
The parameter L,, measures how much the embedding compresses the watermark added particular clean image x. If o is identity, then La,w = 1. If ø is a projection matrix to a linear subspace then 0 La < depending subspace, For a neural image embedding é, the exact value of Lg is unknown but given each & and x can be computed efficiently.
the magnitude of the component of zw æ in this
Theorem C.3. For a A-invisible watermarking scheme with respect to lz-dintance. Assume the embedding function & of the diffusion model is Uez.u-lccally Lipschitz. The randomized algorithm A(ő(-) +N, o² La)) produces a reconstructed image & which satisfies f-CWF with
f(e1) = (5
Lr
where de is the Cumulative Density Function function of the standard normal distribution.
Figure illustrates what the tradeoff function looks like. The result says that after regeneration attack, it is impossible for any detection algorithm to correctly detect the watermark with high confidence. In addition, shows that such detection is as hard as telling the origin of single sample Y from either of the two Gaussian distributions (0 1) and (L A/o, 1).
18

---

Oreaia
diaces
PRO
WZ stuck
Dificies afact
Orien PAY
50
PRG
Desttonike
WAT ctock
Difvcike oPah
OO
Figure 9; Examples of different attacks against RivaGAN watermark.
Figure 10: Examples of different attacks SSL watermark.
Orggnal rapt Stepiarp WM Br prttte
Gns
WA stace
Defuries attact
1.0
Indistinguishable
Diffusion attack
type orror
0.6
0.4
Impossibility Region
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Type eerrr
Figure 11: Examples different attacks against StegaStamp watermark.
Figure 12: Theoretical and empirical trade-off functions for RivaGAN watermark.
The proof in Section D leverages an interesting connection to a modern treatment differential privacy [13] known the Gaussian differential privacy [12]. The work of [12] itself is a refinement and generalization of the pioneering work [59] and [28] which established tradeoff-function view.
Discussion. Let us instantiate the algorithm with a latent diffusion model by choosing o V(1 (1-a(tt)))/e ()/(()) (see Algorithm 1) and discuss the parameter choices.
Remark C.4 (Two trivial cases). Observe that when a<(!) = O, the result of the reconstruction does not depend on the input .. thus there is no information about the watermark in i()), i.e., the trade-off function is f(1)) =1 Eg - perfectly watermark-
however, the information about x (through xw) is also lost. When att)) 1, the attack trivially returns 2(0) Zw. which does not change the performance of the original watermark detection algorithm at all (and it could be perfect, ie., 6 C2 = 0).
Remark C.5 (Choice of 1*). In practice, the best choice 1" in between the two trivial cases, i.e., one should choose it such that Lz, Aatat)/((t))) a(t))) small constant. The smaller the constant, the more thoroughly the watermark is removed. The larger the constant, the higher the fidelity of the regenerated image w.r.t. zw (thus 2y too).
Remark C.6 (VAE). Strictly speaking, Theorem 4.3 does not directly apply to VAE because the noise added on the latent embedding depends the input data. So if it chooses (xx)) 1 and
19

---

(xw)) = O, then it is easy to distinguish between the two distributions. We can still provide provable guarantees for VAE if either the input image is artificially perturbed (so VAE becomes a denoising algorithm) or the latent space is artificially perturbed after getting the embedding vector. When (x) itself could be stable, more advanced techniques from differential privacy such as Smooth Sensitivity or Propose- Teroos-TTes--eeee Test-Release can be used to provide certified removal guarantees for the VAE attack. In practice, we find that the VAE attack is very effective in removing watermarks as is without adding additional noise.
Remark C.7 (The role of embedding function Φ). Readers may wonder why having embedding function 6 is helpful for removing watermarks. We give three illustrative examples.
Pixel quantization. This ø is effective against classical Least Significant Bit (LSBB watermarks. By removing the lover-significance bits ((x) = ((xw) thus L O.
Low-pass filtering. By choosing Φ to be a low-pass filter, one can effectively remove or attenuate watermarks injected in the high-frequency spectrum of the Fourier domain, hence resulting in Lew < 1 for these watermarks.
feep-learning-bsedd image embedding. Modern deep- fiep-learnig--oee based image models effectively encode a "natural image manifold," which allows a natural image x to pass through while making the added artificial watermark smaller. To be more concrete, consider e to be linear projection to d-dimensional "natural image subspace". For a natural image x and watermarked image The = x + E, have )(..)) = (xx) <(c)
p(c). If e (O, u In) then E [[]](()) [[] This projection compresses the magnitude of the watermark substantially while preserving the signal, thereby boosting the effect of the noise added in the embedding space in obfuscating the differences between watermarked and unwatermarked images.
do? < no
Finally, we note that while Theorem 4.3 and 4.4 are specific to lz-distance, the general idea applics to other distance functions (e.g., & distance). fg-distance is natural for the Gaussian noise natively introduced by diffusion and VAE-based regeneration attacks.
C.2 Utility Guarantees
In this section, we prove that the regenerated image é is close to the original (unwatermarked) image Tp. This is challenging because the denoising algorithm only gets access to the noisy version of the watermarked image.
Interestingly, we can obtain a general extension lemma showing that for any black-box generative model that can successfully denoise a noisy yet unwatermarked image with high probability, the same result also applies to the watermarked counterpart, except that the failure probability is slightly larger.
Theorem C.8. Let To be an image with n pixels and & IR." Rdd be an embedding function. Let A be an image generation denoising algorithm such that with probability at least [|A(()(z)) + NO, oá)a)) zoll S E. Then for any A-invisible watermarking scheme that produces Zw from a clean image To, then & = A(ő(Iw) + +N o² oa])) satisfies that
|| zoll S Erg.o.8
with probability least 1 1 3 where
δ to
Â
8 = min VER
8.e" +
- e
vo
2a
A
2x
A
in which & denotes the standard normal CDF and Â = A.
The theorem says that if a generative model is able to denoise noisy version of the original image, then the corresponding nutermark-remoove attack using this generative model provably produces an image with similar quality.
Corollary C.9. The expression for 8 above can be (conservatively) simplified to
1.2 122
<
81/2
For example, if x TT A, then this saying that if a. depends logarithmically on 1/8, the same exponential tail holds for denoising the watermarked image.
20

---

The above result is powerful in that it makes no assumption about what perturbation the watermarking schemes could inject and which image generation algorithm we use. We give few examples below.
For denoising algorithms with theoretical guarantees, e.g. TV-denoising [27, Theorem 2]. our results imply provable guarantees the utility for the watermark removal attack of the form, "w.h.p.,
+| 1= = Ô oTV2a(=o) ", i.e., vanishing mean square error (MSE) as gets bigger.
For modern deep learning- based image denoising and generation algorithms where worst-case guarantees are usually intractable, Theorem 4.4 is still applicable for each image separately. That is to say, as long as their empirical denoising quality is good on an unwatermarked image, the quality should also be good on its watermarked counterpart.
D Proofs of Technical Results
Proof of Theorem 4.3. By the conditions on the invisible watermark and the local Lipschitz assump- tion on (b, we get that
(1)())) (f lll S II
This can be viewed as the fz-local-sensitivity of @ at zo in the language of differential privacy literature.
The two candidate distributions are (()(z)), I and ((((z=), Let T(P,Q) [0, 1] [0, be the tradcoff' function for distinguishing between distributions / and Q as Definition 2.1 of [12]. The tradeoff function outputs the Type II eтror of the likelihood ratio test of the two point hypotheses as function of the Type error. By the Seyman-Pearsonn lemma, the likelihood ratio test is uniform most powerful, thus the tradeoff function provides a lower bound of any test in distinguishing P and Q
First notice that translation and scaling by any non-zero constant (a non-zero linear transformation) does not change the tradeoff function, thus
= ((
(l o(zo) 1)
((
T ())
σ²
Next, observe that the likelihood ratio of the multivariate isotropic normal distributions remains the same when we collapse it to the 1-dimension along the vector @(50) - ((Tw), i.e.,
T ((OO.)) of (( ((30) .)) (N 1). (0(xw)) - ((wo)ll .1).
(5)
o²
Lemma 2.9 of [12] states that for any post- post-processing function h, T(((P),((Q)) 2 T(P.Q)
(6)
which is known to information theorists as the information processing inequality of the Hockey- Stick divergence.
Consider a particular randomized post-processing function h that first adds N/O, v) then divides VI+
(1mm))))))))) < (N(O, Log T (N. o2
11)
(5)
2² (v1
for spocinic
Leg.w A o2(vI+e)
To see the last identity, observe that we can choose v > such that thanks to the local Lipschitz assumption. To put things together, we get
L o2 .1) )
T (N
The left hand side is the tradeoff function of two univariate Gaussian with variance 1, by Equation (6) of [12], the tradeoff function can be written as ( - a.)
where is the cumulative L o²
density function of the standard Gaussian distribution.
21

---

Finally, by applying the the postprocessing property (6) again by instantiating h to be the re- e--geeraation procedure, we get that the regenerated image ş also satisfies the same tradeoff function as stated above.
Proof of Theorem 4.4. The key idea is to use the definition of indistinguishabiiity (differential privacy, but for fixed pair of neighbors, rather than for all neighbors). So say two input x; x' are (v, w)- indistinguishable using the output of a mechanism. M if for any event S,
(r(M(() Ss < e" Pr[M(x') E S] w.
and the same also true when x'' are swapped. In our case, we have already shown that (from the proof of Theorem 4.3) for any post-processing algorithm A, zo and Tu are indistinguishable using á in the trade-off function sense. [2] obtained "dual" characterization which says that the same Gaussian mechanism satisfies (u, u) iediiniistinguuhhabllyy with
Å 2x
v0
vo
w +
20
Â
-
e
Â
for all U R. By instantiating the event be that || xo
E then we get
Pr[l
soll Exp.0,6) S e. Pr[ll || - xoll > l w
Fio
20
E" 8 w.
This completes the proof.
Proof of Corollary 4.5. The w, v "privacy profile" implies a Renyi-divergence bound (one can also get that directly from the Renyi-DP of gaussian mechanism) which implies (by Proposition 10 of [40]) that
uāa 267
(w-1)// 1)/u
< min u21
e
The stated result is obtained by setting = 2.
E Defense with Semantic Watermarks
In this section, discuss possible defenses that resilient to the proposed attack, and although Theorem 4.3 has guaranteed that no detection algorithm will be able to detect the watermark after our attack, the guarantee is based on the invisibility with respect to distance. Therefore, by relaxing that invisibility constraint and thus making the watermark more visible, we may be able to prevent the watermark from being removed. One less-harmful way loosen the invisibility constraint is with semantic watermarks. As shown in Figure 13, baseldbased watermarks such as DwtDetSvdd keep the image almost intact, while semantic watermarks change the image significantly but retain its content.
E.1 Tree-Ring Watermarks
Tree-Ring Watermarking [60] is new technique that robustly fingerprints diffusion model outputs in a way that is semantically hidden in the image space. An image with a Tree-Ring watermark does not look the same as the image, but is semantically similar (in Figure 13, both the original and the semantically watermarked image contain an astronaut riding a horse in Zion National Park). Unlike existing methods that perform post-hoc modifications images after sampling, Tree-Ring Watermarking subtly influences the entire sampling process, resulting in a model fingerprint. The watermark embeds pattern into the initial noise vector used for sampling. These patterns are structured in Fourier space so that they are invariant to convolutions, crops, dilations, flips, and rotations. After image generation, the watermark signal is detected by inverting the diffusion process to retrieve the noise vector, which is then checked for the embedded signal. [60] demonstrated that Tree-Ring Watermarking can easily applied to arbitrary diffusion models, including text- conditioned Stable Diffusion, plug-in with negligible loss in FID.
22

---

(a) No Watermark
(b) Pixel WM
Semantic WM
Figure 13: The image with a pixel- based watermark such as DwtDetSvd looks almost the same as the original. The image with a semantic watermark such as Tree-Ring contains the same content but is visibly different from the original. Original image generated with the prompt "an astronaut riding horse in Zion National Park" from [60].
Table Tree-Ring watermarks are robust against the regeneration attacks.
Attacker
MS-COCO TPR @ FPR=0.011
SDP Generated TPR @ FPR=0.011
Brightness-2
1.000
1.000
Contrast-2
1,000
1,000
JPEG-50
1.000
0.994
Gaussian noise-5
1.000
0,996
Gaussian blur-6
1.000
1.000
VAE- B0shj201883
0.998
0.994
2AF-C
1.000
0.994
Diffusion model-60
1.000
0.998
E.2 Defense Experiments
To evaluate Tree-Ring as an alternative watermark, we use the same datasets from the previous experiments in Section 5 MS- IS-COCC and SDP. However, since Tree-Ring adds watermarks during the generation process of diffusion, it cannot directly operate on Al-generated images. Instead, needs textual inputs that describe the content of the images. We use captions from MS-COCO and the user prompts of SDP datasets as the input prompts. The selected set of attacks (including our proposed attack) is applied to the Tree-Ring watermarked images.
As shown in Table Tree-Ring watermarks show exceptional robustness against all the attacks tested. However, such robustness does not come for free. We depict the fz-distances between original images and watermarked ones in Figure 14. Images with Tree-Ring watermarks are significantly more different from the original images in both the pixel space and the latent space. These results indicate that Tree-Riee Ring, as an instance of semantic watermarks, shows the potential to be an alternative solution to the image watermarking problem. However, as our theory predicts, the robustness comes the price of more visible differences.
320
Iimage Latent
250
230
dis
130
120
DwtDerSvd
RivaGAN
SSL
StegaStamp
Tree-Ring
Figure 14: The l distances between original images and watermarked ones. Tree-Ring watermarked images are much more different in both pixel space and latent space, making Tree-Ring a visible watermark.
23

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our major claim is that pixel-level invisible watermarks are vulnerable to our proposed attack. It's clearly stated in abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
We have pointed out one class of watermarks that can resist our attacks semantic-preserving ones. We have evaluated our method on these watermarks and discussed them in detail in Appendix F.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [Yes]
Justification: Our theoretical results are in Section 4, with our assumptions in Section 4. Our proofs can be found in Appendix D. We have also discussed the implications and interpretation of our theoretical analysis AppendiXC.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: Our experiment setting is detailed in Section 5, We have also provided our implementation in the submitted supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
25

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Our code is provided in the supplementary material. We use openly accessible data in our experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cccccccpp. wwwpcppwpcc//eeeecceeeceeeeeetle wptppccguuuesss///p/pppppppaaasl "oodeSubmissiiooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
We have specified the details in Section 5 for readers to understand the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Although we haven't 't reported error bars, the effectiveness of our proposed attack much better than the existing basclines.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
26

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We discussed the compute resources we used in Section 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpp://neurt:: newrips.....//
Answer: [Yes]
Justification:
Yes it does.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the socictal impacts in our abstract, introduction, conclusion, and Appendix F.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
27

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not release any models or data.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have properly cited the dataset we use, MS-COCO in our paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/com//ttt/o datasttss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The code provided in our supplementary material is well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
29

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "oivndV3ZSR", "reviewer": "Reviewer_n9vc", "summary": "This paper proposes regeneration attacks, which adds destructive Gaussian noise to the latent representation of the watermarked image, and then reconstructs the corrupted latent to reconstruct the original clean image. The paper provides theoretical guarantee that shows the trade-off function between the Type I error and the Type II error after the attack, in addition to the theorem that shows the ability of the regeneration attack's capability to produce images with similar quality as the generative model that reconstructs the corrupted latent. The paper also introduces a potential defense mechanism that can survive the regeneration attack.", "questions": "Major concerns are already addressed in Appendix A, so here are some minor questions that may be slightly out of the scope. \n1. For in-processing watermarking, semantic watermarking is a great alternative for preserving visual quality as well as robustness to regeneration attacks. However, for post-hoc watermarking, changing the semantic content leads to unsatisfactory results. How do you envision balancing the robustness of semantic watermarks with their increased visibility in practical applications? Are there any strategies you are exploring to minimize the visual impact while maintaining robustness under regeneration attacks?\n2. The Stegastamp method seems to be able to withstand the proposed attack due to a relatively high l2 distance in both pixel and latent space. Figure 5 shows that with greater noise levels, StegaStamp fails to withstand the regeneration attacks, but the reduced PSNR and SSIM indicate significant degradation in image quality. Since StegaStamp's image quality is already low, enforcing regeneration attacks that further degrade image quality seems unrealistic in real-world scenarios. Given that PSNR and SSIM are pixel-level metrics, such degradation is expected after multiple perturbations in the semantic space. I wonder if the semantically meaningful content is still preserved after regeneration attacks capable of breaking StegaStamp. Specifically, a visualization for Figure 5 or an LPIPS curve might be helpful for understanding the preservation of semantic content.\n\n-------------------Post rebuttal Edit------------------------\n\nI appreciate the author for the comments. My questions has been well addressed. I have increased my rating accordingly.", "limitations": "potential negative societal impact has not been addressed but the author does provide potential defense mechanisms for the proposed attack method.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and easy to read. The presentation is clear and the empirical result is thorough and informative. Besides empirical results, the author also provides theoretical guarantees on the claim. Besides proposing the regeneration attack, the author also provides a potential defense mechanism that sheds light on future watermarking research under the proposed attack.", "weaknesses": "The proposed method relies on the upper bound $L \\geq L_{x,w}$ to effectively calibrate $\\sigma$, which is slightly unrealistic considering the embedding function $\\phi$ and original image $x$ is unknown to the attacker, even though a uniform upper bound may exist, this could potentially affect the attack performance or image quality since the attacker normally has no access to the decoding scheme thus unable to verify the performance of the attack while maintaining the image quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We appreciate the opportunity to address the specific points raised and provide further clarification.\n\n### 1. Calibration of $\\sigma$ and Attack Performance\n   \n   > \"Relies on the upper bound $L \\geq L_{x,w}$ to effectively calibrate $\\sigma$… this could potentially affect the attack performance or image quality since the attacker normally has no access to the decoding scheme, thus unable to verify the performance of the attack.\"\n   \n   Access to the decoding scheme is not necessary for calibrating $\\sigma$. Although we cannot calibrate by verifying if an attack is successful, we can still choose the optimal $\\sigma$ through a binary search. Typically, the quality of the attacked image degrades as $\\sigma$ increases, allowing us to use binary search to find the largest $\\sigma$ that maintains acceptable image quality. Additionally, we discuss controlling a certain degree of Certified Watermark Freeness (CWF) a priori in Appendix lines 522-529. Here is the original content for your reference:\n\n- \"Our guarantee in Theorem 1 depends on the specific watermark injected into a specific image instance through the unknown local Lipschitz parameter $L_{x,w}$. Specifying a fixed CWF level requires a uniform upper bound of $L_{x,w}$ independent of $x$ and $w$. For example, when the embedding $\\phi$ is trivial (identity map), we can take $L_{x,w} \\leq 1$. When $\\phi$ involves lower pass filtering, such as a Fourier transform that removes all high-frequency components except the top \\(k\\) dimensions, we can bound $L_{x,w} \\leq \\sqrt{k}/n$, where $n$ is the number of pixels. Generally, any linear transformation with a bounded operator norm is suitable.\"\n\n### 2. Balancing Robustness and Visibility of Semantic Watermarks\n\n   > \"How do you envision balancing the robustness of semantic watermarks with their increased visibility in practical applications? Are there any strategies you are exploring to minimize the visual impact while maintaining robustness under regeneration attacks?\"\n   \n   We appreciate your interest in future developments. Yes, we are actively exploring methods to enhance the robustness of post-hoc watermarking. One promising approach involves leveraging powerful image editing techniques to add or remove unimportant subjects or alter textures. These modifications are visible but appear as normal image content without the watermark key.\n\n### 3. Visualization or LPIPS Curve for Figure 5\n\n   > \"Specifically, a visualization for Figure 5 or an LPIPS curve might be helpful for understanding the preservation of semantic content.\"\n   \nWe have included a visualization image in the Rebuttal PDF. This visualization demonstrates that the semantically meaningful content remains preserved.\n\n----\nOverall, we hope these clarifications address your concerns and demonstrate the strength and potential of our approach. We appreciate your consideration and look forward to your stronger support.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S5GHu7p7bw", "reviewer": "Reviewer_V7G3", "summary": "The main idea proposed in this paper is that regenerating images using other (pretrained) generative AI (e.g., vae, diffusion models) can provably remove any invisible watermarks embedded in a given image. Accompanying this, the paper can be divided into the following parts: (1) intro of the proposed regeneration methods; (2) proof of the removal guarantee (by any regeneration methods falls into the definition of this paper, not limited to vae or diffusion; and (3) empirical experiments to support their claim \"the regeneration methods is effective, especially using defussion models.\"", "questions": "1. Similar to Fig. 2, can you also provide (1) watermarks other than DwtDctSVD and (2) curves that are achieved by other regeneration attacks (e.g., VAE, denoising autoencoder)? Current illustration is insufficient to support the claim in the caption “indicating the success of our attack and the validity of the theoretical bound”.\n\n2. As the authors repeatedly claim the superior attack performance of their proposed method, the visualization w.r.t only DwtDctSVD is not sufficient, as DwtDctSVD appears to be the least robust watermark among all the selected watermarks in this paper (see Table 2). Can you provide complete set of attack visualization of all methods on different watermarks considered in this paper? This will also help to clarify \"Weakness 2\" stated above.\n\n3. As the authors proposed denoising reconstruction (attack instance 1), I think it is necessary to include DiffPure as a baseline attack in the experiment and discuss its performance. This has been applied as a watermark attack method in a published paper [2].\n\n[2] Saberi, Mehrdad, et al. \"Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks.\" The Twelfth International Conference on Learning Representations. 2023.", "limitations": "N/A. The work does not have potential negative impact that needs to be explicitly discussed.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea proposed in this work original.\nThe paper is clearly written. The use of math does not create burden in readability but helps to understand the main idea.", "weaknesses": "The weakness is listed below in order, from major to minor:\n\nWeakness 1 (Major): Experiment settings and evaluation are not rigorous.\n\nThe key experiment and result to support the authors claim of the effectiveness of their method will be those that can show the “strength” of their proposed attacks, but the current evaluation experiment is poorly designed. Reporting the watermark detection acc. (as in Table 2) using a fixed attack point (e.g., JPEG 50, or the selected VAE’s, etc., where they all have tunable parameters) cannot faithfully show the strength. This make the PSNR value reported in Table 2 meaningless. Instead, if there is a tradeoff between the image quality and the watermark detectability expected, the authors should report the maximal PSNR (the best image quality) for each attack method when the same detection ability (e.g., fail the watermark detection) of the decoder is achieved (e.g., with the same bitwise acc.) by tuning the hyperparameters of the attack methods (e.g., the JPEG quality factor, the compression index of the selected VAE model used in this paper, the noise level of the diffusion regeneration as Fig. 5, etc.). In this regard, the attack method with the best image quality can be argued as the strongest reasonably. Alternatively, the authors may consider profiling the quality and detectability tradeoff as proposed in [1]. The current evaluation experiment is not sufficient to support the claim made in line 71-73.\n\nWeakness 2 (Major): Conclusion with insufficient support.\n\nIn line 13-14 “Our findings underscores … to semantic-preserving watermarks” is not convincing. For example, let’s say we want to prevent misusing generated images. In Fig 6., the authors visualize the StegaStamp watermark, where there are obvious abnormal patterns embedded into the image (in another word, they are not `invisible’ and humans can tell these images are suspicious). I would suspect that the attacked images of StegaStamp after applying the methods proposed by the authors will still have visible artifacts that are enough to raise human caution and suspect they are not original images—so it may be reasonable to argue that StegaStamp can successfully prevent image misuse and do not need a shift as the authors claimed. My suspicions are raised by the following results presented by the authors: (i) The visualizations of the attack methods proposed by the authors (e.g., Diffusion Attack in Fig 3. And 7.) are on DwtDctSVD watermarks and they already contain visible artifacts; (ii) corresponding PSNR values in Table 2 for DwtDctSVD is higher than StegaStamp; whereas in Fig 5., the PSNR of diffusion regeneration on StegaStamp is low. So, I would imagine that the attacked images of StegaStamp will have more visible artifacts but the author did not include any visualization of this. Thus, I consider the current conclusion an overclaim without persuasive argument and suggest the authors provide more evidence to support their claim.\n\nWeakness 3 (Minor) --- inaccurate statements.\n\nIn line 1-3 “Invisible watermark safeguards images’ copyrights … prevent people from misusing images … ”.  These are largely believed to be only possible applications that people are thinking of how to use invisible watermarks, but not affirmative conclusions.\nLine 4 “ The proposed attack method first adds random noise to an image…”. According to Eq. 1, the proposed methods add noise to the latent feature of the image (attack instance 2 & 3), not the image itself.\n\n\n[1] An, Bang, et al. \"WAVES: Benchmarking the Robustness of Image Watermarks.\" Forty-first International Conference on Machine Learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback. We appreciate the opportunity to clarify our contributions and address your concerns.\n> W1: Experiment settings and evaluation are not rigorous. The authors may consider profiling the quality and detectability tradeoff.\n\nWe appreciate your suggestion. We have conducted a comprehensive evaluation with various parameter settings for each attacking method. The results are now included in the attached rebuttal PDF.\n\nSpecifically, **we have plotted the quality-detectability tradeoff for five watermark schemes across eight different attacking methods**. The x-axis represents quality metrics (SSIM and PSNR, higher is better), while the y-axis shows the detection metric True Positive Rate at a fixed False Positive Rate (TPR@FPR=0.01, lower is better from an attacker's perspective). The strongest attacker should appear in the lower right corner of these plots. We used the same watermark settings as described in the original paper (Section 5, Watermark Setting).\n\n**Key findings**\n\n- Our proposed regeneration attack consistently outperforms other methods across all five watermarking scenarios.\n- For DctDwtSvd, RivaGAN, and SSL watermarks, our regeneration attack instance 2 (VAE) achieves the best Pareto front of quality and attack detectability.\n- Our regeneration attack instance 3 (diffusion model) performs best against Stable Signature and shows strong results across all scenarios. It offers the additional benefit of ease of use and can achieve good attacking results with different noise levels.\n\nThese results strongly support our claim that the regeneration attack is a very effective method for attacking various watermarks compared to strong baselines.\n\n> W2: Conclusion with insufficient support. It may be reasonable to argue that StegaStamp can successfully prevent image misuse and does not need a shift as the authors claimed.\n\nWe appreciate the reviewer's perspective on StegaStamp. Our conclusion in lines 13-14 is based on both theoretical analysis and empirical evidence:\n\n- Theoretical guarantee: Our regeneration attack is proven to remove certain pixel-based invisible watermarks that perturb the image within a limited range of l2 distance. This guarantee applies to both existing and future watermarking techniques that fall within this category.\n- StegaStamp specifics: As the reviewer noted, StegaStamp introduces \"visible artifacts that are enough to raise human caution and suspect they are not original images\". Figure 4 shows that the l2 distance for StegaStamp is quite large, making it not \"invisible\" if we set the l2 distance to be small.\n- Empirical results: To address the reviewer's concerns, we have **included the attacked images for different watermarks, including StegaStamp, in the attached PDF**. These images demonstrate that our regeneration attack (diffusion model) produces high-quality results while successfully evading StegaStamp detection.\n\nOur framework is self-consistent, and we have not hidden any results or overclaimed our conclusions. The additional visual examples provide further support for the effectiveness of our approach across different watermarking techniques.\n\n\n> W3: Inaccurate statements: \"Invisible watermark safeguards images' copyrights … prevent people from misusing images … \". These are largely believed to be only possible applications.\n\nWe appreciate the reviewer's attention to detail. To clarify, these are not merely possible applications but **are already in use in prominent real-world scenarios**:\n\n- Stable Diffusion [1], a widely used open-source image generation model, employs invisible watermarks to protect generated images.\n- DeepMind [2] utilizes SynthID in images to safeguard copyrights and prevent image misuse.\n\nThese examples demonstrate that invisible watermarks are actively being used for copyright protection and preventing image misuse in real applications.\n\n\n> W3: Line 4 \"The proposed attack method first adds random noise to an image…\". According to Eq. 1, the proposed methods add noise to the latent feature of the image (attack instance 2 & 3), not the image itself.\n\nWe thank the reviewer for pointing out this potential source of confusion. To clarify:\n\nThe regeneration attack is a family of attacks with multiple instances:\n- Instance 1 adds noise directly to the image.\n- Instances 2 and 3 add noise to the latent feature of the image.\n\nTo improve clarity, we will revise the statement to: \"adds random noise to an image or its latent feature...\"\n\n> Q1 \n\n**We have added an example of the RivaGAN watermark with our regeneration attack in the attached PDF** to further illustrate the theoretical guarantee's applicability across different watermarking techniques.\n\n> Q2\n\nWe acknowledge the reviewer's point about the need for more comprehensive visualizations. We have now included a complete set of attack visualizations for all methods on the different watermarks considered in our paper. These can be found in the attached PDF.\n\n> Q3\n\nRegarding DiffPure, we appreciate the suggestion to include it as a baseline. While DiffPure is conceptually similar to our regeneration attack instance 3 (diffusion model), we have conducted additional experiments to compare their performance:\n\nUsing the RivaGAN watermark as an example, we fixed TPR<0.05 at FPR=0.01 for both regeneration and DiffPure attacks, then compared PSNR:\n- Regen-Diff: PSNR = 23.33\n- DiffPure: PSNR = 23.07\n\nThese results demonstrate that our Regeneration-Diffusion approach achieves better image quality while maintaining the same level of watermark removal effectiveness.\n\n----\nWe hope these clarifications and additional results address the reviewer's concerns and further strengthen our paper's contributions. We are grateful for the opportunity to improve our work and would be happy to provide any additional information or clarifications if needed.\n\n----\n\n[1] https://github.com/Stability-AI/stablediffusion\n\n[2] https://deepmind.google/technologies/synthid/", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hlZl5D0eWa", "reviewer": "Reviewer_ii78", "summary": "This paper investigates the resilience of invisible watermarks embedded in images against removal attacks. The authors propose a new class of attacks, called regeneration attacks, which combine adding random noise to the image and then reconstructing the image using generative models. The study demonstrates that these attacks can effectively remove invisible watermarks, including the resilient RivaGAN, while maintaining image quality.", "questions": "N/A", "limitations": "N/A", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Introduces a new Image Watermarks attack leveraging generative models, providing a fresh perspective on watermark removal.\n\nOffers formal proofs to demonstrate the effectiveness of the proposed attacks.\n\nProvides extensive empirical results showing the success of the attacks across different watermarking methods.\n\nThe writing is clear and well-organized.", "weaknesses": "While invisible watermarks are highly vulnerable, semantic watermarks are less affected by the proposed attacks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your positive feedback and the opportunity to address your concern.\n\n> \"While invisible watermarks are highly vulnerable, semantic watermarks are less affected by the proposed attacks.\"\n\nThe primary purpose of this paper is not to propose an attack that can remove any type of watermark. Our goal is to raise awareness about the vulnerability of all invisible watermarks to some extent. We believe we have successfully demonstrated this point and provided valuable insights into alternative solutions in light of these vulnerabilities. Therefore, this additional information should not be viewed as a weakness but rather as a significant contribution to the community. Our attack is important to motivate the community to study semantic watermarks.\n\nThank you again for your feedback, and we are looking forward to your stronger support!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
To address reviewer n9vc's question 2 and reviewer V7G3's several concerns, we have included more figures in the attached PDF.


## Figure 1: Quality-Detectability Tradeoff


During the rebuttal, we conducted a comprehensive evaluation with various parameter settings for each attacking method. 


For Figure 1, we plotted the quality-detectability tradeoff for five watermark schemes across eight different attacking methods. The x-axis represents quality metrics (SSIM and PSNR, higher is better), while the y-axis shows the detection metric True Positive Rate at a fixed False Positive Rate (TPR@FPR=0.01, lower is better from an attacker's perspective). The strongest attacker should appear in the lower right corner of these plots. We used the same watermark settings as described in the original paper (Section 5, Watermark Setting)


**Attack Methods and Parameters**

*Regeneration Attacks:*
- Diffusion model: noise steps {10, 30, 50, 100, 150, 200}
- VAE-Cheng2020: compression factors {1, 2, 3, 4, 5, 6}
- VAE-Bmshj2018: compression factors {1, 2, 3, 4, 5, 6}

*Baseline Attacks:*
- JPEG compression: quality {10, 20, 30, 40, 50, 60}
- Gaussian blur: kernel size {2, 4, 6, 8, 10, 12}
- Brightness enhancement: change of {2, 4, 6, 8, 10, 12}
- Gaussian noise: standard deviation {5, 10, 15, 20, 25, 30}
- Contrast enhancement: change of {0.5, 2, 3, 4, 5, 7}

**Key findings**
- Our proposed regeneration attack consistently outperforms other methods across all five watermarking scenarios.
- For DctDwtSvd, RivaGAN, and SSL watermarks, our regeneration attack instance 2 (VAE) achieves the best Pareto front of quality and attack detectability.
- Our regeneration attack instance 3 (diffusion model) performs best against Stable Signature and shows strong results across all scenarios. It offers the additional benefit of ease of use and can achieve good attacking results with different noise levels. 

These results strongly support our claim that the regeneration attack is a very effective method for attacking various watermarks compared to strong baselines.




## Figure 2-4: Additional Visualizations
- Figure 2: More visualizations of attacked images on RivaGAN watermark. All regeneration attacked images can evade detection.
- Figure 3: More visualizations of attacked images on SSL watermark. All regeneration attacked images can evade detection.
- Figure 4: More visualizations of attacked images on StegaStamp watermark. The diffusion model of the regeneration attacked images can evade detection.

## Figure 5: Theoretical and Empirical Trade-off Functions


Figure 5 shows another example similar to Figure 2 in our original paper. It presents theoretical and empirical trade-off functions for RivaGAN watermark detectors after our attack. Trade-off functions indicate how much less Type II error (false negative rate) the detector gets in return by having more Type I error (false positive rate). Theoretically, after the attack, no detection algorithm can fall in the *Impossibility Region* and have both Type I error and Type II error at a low level. Empirically, the watermark detector performs even worse than the theory, indicating the success of our attack and the validity of the theoretical bound. We use 500 watermarked MS-COCO images with an empirically valid upper bound of $L=1$ and noise level $\sigma = 0.57\Delta$.


----
We believe these additional evaluations and visualizations further enhance our paper's quality and address the reviewers' concerns.

Meta-review: After discussion, all the reviewers provided positive feedback on this submission, therefore we recommend accepting it.
Decision: Accept (poster)