Leveraging Optimization for Adaptive Attacks on Image Watermarks
Authors: Nils Lukas
Abdulrahman Diaa
Lucas Fenaux
Florian Kerschbaum
Venue: ICLR 2024 poster (2024)
Keywords: watermarking
adaptive attacks
optimization
stable diffusion
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We leverage optimization to break five image watermarks for Stable Diffusion models using adaptive, learnable attacks.

Abstract: Untrustworthy users can misuse image generators to synthesize high-quality deepfakes and engage in unethical activities. Watermarking deters misuse by marking generated content with a hidden message, enabling its detection using a secret watermarking key. A core security property of watermarking is robustness, which states that an attacker can only evade detection by substantially degrading image quality. Assessing robustness requires designing an adaptive attack for the specific watermarking algorithm. When evaluating watermarking algorithms and their (adaptive) attacks, it is challenging to determine whether an adaptive attack is optimal, i.e., the best possible attack. We solve this problem by defining an objective function and then approach adaptive attacks as an optimization problem. The core idea of our adaptive attacks is to replicate secret watermarking keys locally by creating surrogate keys that are differentiable and can be used to optimize the attack's parameters. We demonstrate for Stable Diffusion models that such an attacker can break all five surveyed watermarking methods at no visible degradation in image quality. Optimizing our attacks is efficient and requires less than 1 GPU hour to reduce the detection accuracy to 6.3% or less. Our findings emphasize the need for more rigorous robustness testing against adaptive, learnable attackers.

Published as a conference paper at ICLR 2024
LEVERAGING OPTIMIZATION FOR ADAPTIVE ATTACKS ON IMAGE WATERMARKS
Nils Lukas, Abdulrahman Diaa", Lucas Fenaux', Florian Kerschbaum University of Waterloo, Canada (nlukas, abdul rahman diaa, lucas. fenaux, florian, www.:kkkeeebeeeeeeeeeeeeoe
ABSTRACT
Untrustworthy users misuse image generators synthesize high-quality deep- fakes and engage in unethical activities. Watermarking deters misuse by marking generated content with hidden message, enabling its detection using secret termarking key. A core security property of watermarking is robustness, which states that an attacker age quality. Assessing robustness requires designing an adaptive attack for the specific watermarking algorithm. When evaluating watermarking algorithms and their (adaptive) attacks, it is challenging to determine whether an adaptive attack is optimal, i.c., the best possible attack. We solve this problem by defining an objec- tive function and then approach adaptive attacks as an optimization problem. The core idea of our adaptive attacks to replicate secret watermarking keys locally by creating surrogate keys that are differentiable and can be used optimize the attack's s parameters. We demonstrate for Stable Diffusion models that such an at- tacker can break all five surveyed watermarking methods no visible degradation in image quality. Optimizing our attacks is efficient and requires less than GPU hour to reduce the detection accuracy 6.3% or less. Our findings emphasize the need for more rigorous robustness testing against adaptive, learnable attackers.
only evade detection by substantially degrading im-
INTRODUCTION
Deepfakes are images synthesized using deep image generators that can be difficult to distinguish from real images. While deepfakes can serve many beneficial purposes if used ethically, for example, in medical imaging (Akrout et al.) 2023) education (Peres al. 2023). they also have the potential to be misused and erode trust in digital media. Deepfakes have already been used disinformation campaigns (Bonch et al., 2019;; Barrett et al., 2023) and social engineering attacks (Mirsky & Lee, 2021), highlighting the need for methods that control the misuse deep image generators.
Watermarking offers solution to controlling misuse by embedding hidden messages into all gener- ated images that are later detectable using a secret watermarking key. Images detected as deepfakes can be flagged by social media platforms news agencies, which can mitigate potential harm (Grin- baum & Adomaitis 2022). Providers large image generators such as Google have announced the deployment of their own watermarking methods (Gowal & Kohli 2023) enable the detection of deepfakes and promote the ethical use of their models, which was also declared as one of the main goals in the US govermment's "AI Executive Order" (Federal Register 2023).
A core security property of watermarking is robustness, which states that an attacker can evade de- tection only by substantially degrading the image's quality. While several watermarking methods have been proposed for image generators (Wen et al., 2023 Zhao et al. 2023) Fernandez et al.) 2023), none of them are certifiably robust (Bansal al., 2022) and instead, robustness is tested em- pirically using limited set of known attacks. Claimed security properties of previous watermarking methods have been broken by novel attacks (Lukas et al.) 2022), and no comprehensive method exists to validate robustness, which causes difficulty in trusting the deployment of watermarking in practice. We propose testing the robustness of watermarking by defining robustness using objective
Equal Contribution

---

Published as a conference paper at ICLR 2024
1.) KeyGen
2.) ttratiiely Optimize Attack Parameters
3.1 KeyGen
4.) Embed
5.) Generate
Verify
"A teday bear
Washington"
Surrogate
Quality Less
Secret
Embedded
Key z'
Key
Message
Generator
Attack Preparation
Generation
8.) Compute p-value
7.) Extract Message
6.) Evasion with Optimized Parameters
D
Extracted
Extract from Image
0g
Verification
Message
Evasion
Figure I: An overview of adaptive attack pipeline. The attacker prepares their attack by gener- ating surrogate key and leveraging optimization to find optimal attack parameters 0A (illustrated here encoder & and decoder D) for any message. Then, the attacker generates watermarked images and applies a modification using their optimized attack to evade detection. The attack is successful if the verification procedure cannot detect the watermark in high-quality images.
function and approaching adaptive attacks as an optimization problem. Adaptive attacks are specific the watermarking algorithm used by the defender but have access to the secret watermarking key. Knowledge the watermarking algorithm enables the attacker to consider range of surrogate keys similar to the defender's key. This also presents a challenge for optimization since the attacker only has imperfect information about the optimization problem. Adaptive attackers had previously been shown to break the robustness of watermarking for image classifiers (Lukas et al. 2022). but attacks had to be handcrafted against each watermarking method. Finding attack parameters through an optimization process can challenging when the watermarking method not easily optimiz- able, for instance, when it is not differentiable. Our attacks leverage optimization by approximating watermark verification through differentiable process. Figure [ shows that our adaptive attacker prepare their attacks before the provider deploys their watermark. We show that adaptive, learn- able attackers, whose parameters can be optimized efficiently, can evade watermark detection for billion parameter Stable Diffusion models negligible degradation in image quality.
2 BACKGROUND
Latent Diffusion Models (LDMs) are state-of-the-art generative models for image synthesis (Rom-) bach et al. 2022). Compared to Diffusion Models (Sohl-Dickstein et al.) 2015). LDMs operate in a latent space using fixed, pre-trained autoencoder consisting an image encoder and decoder D. LDMs use forward and reverse diffusion process across T steps. In the forward pass, real data point Zo encoded into latent point 20 = E(zp) and progressively corrupted into noise via Gaussian perturbations. Specifically.
((=/==-))= = (N V (E t 10,1,..,,TT1
(1)
where B, is the scheduled variance. the reverse process, a neural network fe guides the denoising, taking and time-step as inputs to predict 2,1 fo (z,, t). The model is trained to minimize the mean squared error between the predicted and actual 2--1- The outcome is latent &0 resembling 20 that can be decoded into it D(=o). Synthesis in LDMs can be conditioned with textual prompts.
2.1 WATERMARKING
Watermarking embeds a hidden signal into medium, such as images, using secret watermarking key that is later extractable using the same secret key. Watermarking can characterized by the medium used by the defender to verify the presence of the hidden signal. White-hox and black-
2

---

Published as a conference paper at ICLR 2024
box watermarking methods assume access to the model's parameters query access via API. respectively, and have been used primarily for Intellectual Property protection (Uchida et al. 2017
No-box watermarking (Lukas & Kerschbaum 2023 assumes more restrictive setting where the defender only knows the generated content but does not know the query used to generate the image. This type of watermarking has been used to control misuse by having the ability to detect any image generated by the provided image generator (Gowal & Kohli, 2023). Given a generator's parameters Dg. no-box watermarking method defines the following three procedures.
T + KEYGEN(OG): randomized function to generate a watermarking key т.
0 EMBED(OG, m)) For generator Ic. watermarking key and a message m, return parameters 0% of watermarked generato that only generates watermarked images.
p + VERIFY T, m): This function (i) extracts a message m' from æ using T and (ii) returns the p-value to reject the null hypothesis that and m' match by random chance.
A watermarking method set of algorithms that specify (KEYGEN, EMBED, VERIFY). A wa- termark is hidden signal in an image that can be mapped to message m using a secret key т. The key refers to secret random bits of information used in the randomized verification algorithm to detect a message. Adaptive attackers know the watermarking method but not the key message pair. Carlini & Wagner (2017) first studied adaptive attacks in the context of adversarial attacks.
In this paper, we denote the similarity between two messages by their Ln-norm difference. We use more meaningful similarity measures when M allows it, such as the Bit-Error-Rate (BER) when the messages consist of bits. A watermark is retained in an image if the verification procedure returns p < 0.01, following Wen et al. (2023). Adi et al. (2018) specify the requirements for trustworthy watermarking, and we focus on two properties: Effectiveness and robustness. Effectiveness states that a watermarked generator has a high image quality while retaining the watermark, and robustness means that a watermark is retained in an image unless the image's quality is substantially degraded. We refer to Lukas & Kerschbaum (023) for security games encoding effectiveness and robustness.
2.2 WATERMARKING FOR IMAGE GENERATORS
Several works propose no-box watermarking methods to prevent misuse for two types of image gen- erators: Generative Adversarial Networks (GANs) (Goodfellow et al.) 2020) and Latent Diffusion Models (LDMs) (Rombach et al., 2022). We distinguish between post-hoc watermarking meth- ods that apply an imperceptible modification to an image and semantic watermarks that modify the output distribution of an image generator and are truly "invisible" (Wen et al. 2023).
For post-hoe watermarking, traditional methods hide messages using the Discrete Wavelet Transform (DWT) and Discrete Wavelet Transform with Singular Value Decomposition (DWT- SVD) (Cox et al., 2007) and are currently used for Stable Diffusion. RivaGAN (Zhang et al. 2019) watermarks by training a deep neural network adversarially to stamp pattern on image. Yu ee al. (2020 202T) propose two methods that modify the generator's 's training procedure but require expensive re-training from scratch. Lukas & Kerschbaum (2023) propose watermarking method for GANs that can be embedded into a pre-trained generator. Zhao et al. (0023) propose a general method to watermark diffusion models (WDM) that uses method similar to Yu ct al. (2020). which trains an autoencoder to stamp a watermark on all training data before also re-training the generator from scratch. |ernandez al. (2023) pre-train autoencoder to encode hidden messages into the training data and embed the watermark by fine-tuning the decoder D component of the LDM.
Wen et al. (202)) are the first to propose semantic watermarking method for LDMs they call Tree- Rings Watermarks (TRW). The idea is to mark the initial noise IT with a detectable, tree-ring-like pattern m in the frequency domain before generating an image. During detection, they leverage the property of LDM's that the diffusion process is invertible, which allows mapping an image back to its original noise. The verification extracts a message m' by spectral analysis and tests whether the same tree-ring patterns m are retained in the frequency domain of the reconstructed noise.
Uchida et al. (017) study watermarking image classifiers. Our categorization is independent of the task. Embedding can alter the entire generation process, including adding pre- and post-processors.
3

---

Published as a conference paper at ICLR 2024
Surveyed Watermarking Methods. In this paper, evaluate the robustness of five watermarking methods: TRW, WDM. DWT. DWT-SVD, and RivaGAN. DWT, DWT-SVD, and RivaGAN are default choices when using Stability Stable Diffusion repository and WDM and TRW are two recently proposed methods for Stable Diffusion models. However, WDM requires re-training a Stable Diffusion model from scratch, which can require 150-1000 GPU days (Dhariwal & Nichol, [021] and is not replicable with limited resources. For this reason, instead of using the autoencoder on the input data, we apply their autoencoder as post-processor after generating images.
3 THREAT MODEL
We consider a provider capable of training large image generators who make their generators acces- sible many users via black-box API, such OpenAl with DALL-E. Users can query the gener- ator by including a textual prompt that controls the content of the generated image. We consider an attack by an untrustworthy user who wants to misuse the provided generator without detection.
Provider's Capabilities and Goals (Model Capabilities) The provider fully controls image gener- ation, including the ability to post-proces generated images. (Watermark Verification) In a no-box setting, the defender must verify their watermark using a single generated image. The defender aims for an effective watermark that preserves generator quality while preventing the attacker from evading detection without significant image quality degradation.
Attacker's Capabilities. (Model Capabilitiess) The user has black box query access to the provider's watermarked model and also has white- access to less capable, open-source surrogate generators, such as Stable Diffusion on Huggingface. We assume the surrogate model's image quality is inferior to the provided model; otherwise, there would be no need to use the watermarked model. Our at- tacker does not require access to image generators from other providers, but, of course, such access may imply access to surrogate models as our attack does require. (Data Access) The attacker has un- restricted access to real-world image and caption data available online, such as LAION-5B (Schuh-) mann et al. 2022). (Resources) Computational resources are limited, preventing the attacker from training their own image generator from scratch. (Queries) The provider charges the attacker per image query, limiting the number of queries they can make. The attacker can generate images either unconditionally or with textual prompts. (Adaptive) The attacker knows the watermarking method but lacks access to the secret watermarking key T and chosen message m.
Attacker's Goal. The attacker wants to use the provided, watermarked generator to synthesize im- ages (i) without a watermark that (ii) have a high quality. We measure quality using a perceptual similarity function Q : x X + IR between the generated, watermarked image and perturbed im- age after the attacker evades watermark detection. We require that the defender verifies the presence of watermark correctly with p-value of at least p < 0.01, same as Wen et al. (2023).
CONCEPTUAL APPROACH
As described in Section 2] a watermarking method defines three procedures (KeyGen, Embed, Verify). The provider generates secret watermarking key +- KEYGEN(OG) that allows them to watermark their generator so that all its generated images retain the watermark. To embed a watermark, the provider chooses message (we sample message and modifies their generator's 's parameters 0 + EMBED(OG. T, m). Any image generated by 8% should retain the watermark. For any + GENERATE((?)) we call watermark effective if (i) the watermark is retained, i.e., VERIFY(Z, r, m) < 0.01 and (ti) the watermarked images have a high perceptual quality. The attacker generates images t GENERATE((2) and applies image-to- image transformation, A X' - X' with parameters A to evade watermark detection by perturbing + A(x). Finally, the defender verifies the presence of their watermark in â, shown in Figure
M uniformly at random)
Let W W(O; 7', m) = EMBED m)) be the watermarked generator after embedding with key 7' and message m and Gw GENERATE(WW (W (0c, 7', m)) denotes the generation of an image using the watermarked generator parameters. For any high-quality Gw, the attacker's 's objective becomes:
max
E Y' (EY MEM
(VERIFY(A(GW)) ). m)) + Q(A(GW)) Gw]]
(2)
4

---

Published as a conference paper at ICLR 2024
This objective seeks to maximize (i) the expectation of successful watermark evasion over all po- tential watermarking keys r' and messages m (since the attacker does not know which key-message pair was chosen) and (ii) the perceptual similarity of the images before and after the attack. Note that this paper, we define image quality as the perceptual similarity to the watermarked image before the attack. There two obstacles for attacker to optimize this objective: (1) The attacker has imperfect information about the optimization problem and must substitute the defender's 's image generator with less capable, open-source surrogate generator. When KEYGEN depends lg, then the distribution of keys differs, and the attack's 's effectiveness must transfer to keys generated using ag. (2) The optimization problem might be hard to approximate, even when perfect information available, e.g., when the watermark verification procedure is not differentiable.
4.1 MAKING WATERMARKING KEYS DIFFERENTIABLE
We overcome the two aforementioned limitations by (1) giving the attacker access to a similar (but less capable) surrogate generator lg, enabling them to generate surrogate watermarking keys, and (2) by creating a method GKEYGEN(OG) that creates a surrogate watermarking key 0x through which we can backpropagate gradients. A simple but computationally expensive method of creating differentiable keys OD is using Algorithm [ to train a watermark extraction neural network with parameters O predict the message m from an image.
Algorithm GKEYGEN: A Simple Method to Generate Diffcrentiable Keys
Require: Surrogate generator e Watermarking method (KeyGen, Embed, Verify). N steps
1: T + KEYGEN(OG)
> The surrogate key
2: for +-11 1 to N do
3:
m ~ M
b Sample a random message
4:
0g + EMBED T, m)
D. Embed the watermark
5:
+ GENERATE (0:)
6:
m' < (XTRACT(E;
7:
90,, + Veolim- m'l1
Compute gradients using distance between messages
8:
OD + OD Adam(Op..0))
9: return Op
The surrogate key
Algorithm generates surrogate key (line 1) to embed a watermark into the surrogate generator and use it to generate watermarked images (lines 3-5). The attacker extracts the message (line 6) and updates the parameters of the (differentiable) watermark decoder using an Adam optimizer (Kingma & Ba 2014). The attacker subsequently uses the decoder's parameters Op as inputs to VERIFY. Our adaptive attacker must invoke Algorithm [ only for the non-differentiable watermarks DCT and DCT-SVD (Cox al. (007). The remaining three watermarking methods TRW (Wen al. 2023 WDM (Zhao et al.J (203)) and RivaGAN (Zhang et al.] (2019) do not require invoking GKEYGEN. In our work, we the parameters On of ResNet-50 decoder (see Appendix A.3 for details).
4.2 LEVERAGING OPTIMIZATION AGAINST WATERMARKS
Equation requires finding attack parameters 0A against any watermarking key 7' KEYGEN (fa) which can computationally expensive the attacker has invocate GKEYGEN many times. We find empirically that generating many keys is unnecessary, and the attacker can find effective attacks using only a single surrogate watermarking key 0D +- GKEYGEN((c).
We propose two learnable attacks Ai, A2 whose parameters 0 A, can be optimized efficiently. The first attack, called Adversarial Noising, finds adversarial examples given an image using the surrogate key as a reward model. The second attack called Adversarial Compression, first fine-tunes the parameters of a pre-trained autoencoder in preparation stage and uses the optimized parameters during an attack. The availability of a pre- trained autoencoder is realistic assumption if the attacker has access to surrogate Stable Diffusion generator, the autoencoder is detachable component of any Stable Diffusion generator. Access to a surrogate generator implics the availability of a pre-trained autoencoder at additional cost in computational resources for the attacker.
5

---

Published as a conference paper at ICLR 2024
Algorithm 2 Adversarial Noising
Algorithm 3 Adversarial Compression
Require: surrogate 8c, budget e, image x adversarial perturbation
Require: surrogate fc. strength a image 2
I: 0A
I: 0A (0,00))
Compressor parameters
2: 0D GKEYGEN(0E)
surrogate key OD t- GKFYGEN(0G)
3: m < EXTRACT(E;
3: for j <- to N do
4: for j < 1 to N do
4:
m ~ M
5: m' < EXTRACT(2 + AA, 0p)
5:
0/ + EMBED(OG. 0D, 0p,
6: 99 + -Veallm m
I GENERATE
7: 0A <- P.(OA Adam (A 904))
7:
x' + D(E(I; 0A)
>> compression
return + +OA
m' + EXTRACT (x'
9:
A + Vo(LLPIPS (2))==/mmmmm'
10: 0A t 0A Adam (A. (A.)o.)
return ((E(x;
Adversarial Noising. Algorithm 2 shows the pseudocode of adversarial noising attack. Given surrogate generator 0G. budget IR+ for the maximum allowed noise perturbation, and watermarked image generated using the provider's watermarked model, the attacker wants to compute perturbation within an e-ball of the Loo norm that evades watermark detection. The attacker generates a local surrogate watermarking key (line 2) and extracts a message m from æ (line 3). Then, the attacker computes the adversarial perturbation by maximizing the distance the initially extracted message m while clipping the perturbation into an e-ball using P. (line 7).
Adversarial Compression. Algorithm shows the pseudocode of our adversarial compression attack. After generating surrogate watermarking key (line 2), the attacker generates images con- taining a random message (lines 4-6) and uses their encoder-decoder pair to compress the images (line 7). The attacker iteratively updates their model's 's parameters by (i) minimizing quality loss, which we set to the LPIPS metric (Zhang et al. 2018), and (ii) maximizing the distance between the extracted and embedded messages (line 9). The output O of the optimization loop between lines and 10 only needs to be run once, and the weights t can be re- re-used in subsequent attacks.
We highlight that the attacker optimizes approximation of Equation 2 since they only have access to surrogate generator êG. but not the provider's generator fg. This may lead general- ization gap of the attack at inference time. Even an attacker can find optimal attack parameters 0 that optimizes Equation (2) using 6G. the attacker cannot test whether their attack remains effective when the defender uses a different model 0g to generate watermarking keys.
5 EXPERIMENTS
Image Generators. We experiment with Stable Diffusion, open-source, state-of-the-art latent diffusion model. The defender deploys a Stable Diffusion-v2.00 mode trained for 1.4m steps in total on subset of LAION-5B (Schuhmann et al. 2022). The attacker uses less capable Sta- ble Diffusion-vl. checkpoint, trained for 431k steps in total on LAION-2B and LAION-HD. All experiments were conducted on NVIDIA A100 GPUs.
Images are generated using DPM solver (Lu et al. 2022) with 20 inference steps and default guidance scale of 7.5. We create three different watermarked generators for each surveyed water- marking method by randomly sampling watermarking key - +- KEYGEN(DG) and message m M, used to embed watermark. Appendix A.I contains descriptions of the watermarking keys. All reported values represent the mean value over three independently generated secret keys.
Quantitative Analysis. Similar to Wen et al. (2023), we report the True Positive Rate when the False Positive Rate is fixed to 1,, called the TPR @ FFPR. Appendix A.2 describes statistical tests used the verification procedure each watermarking method to derive p-values. We report the
base

---

Published as a conference paper at ICLR 2024
Watermark Accuracy versus Perceptual Distance
Watermark
1.0
>+
Adaptive Attacks
Accuracy (TPR@1%FPR)
0.8-
Non-Adaptive
x
No Attack
0.6-
WDM
0.4
x
TRW
RivaGAN
0.21
DWT
DWT-SVD
0.0
160
0.0
0.1
0.2
0.3 Perceptual Distance (LPIPS)
0.4
0.5
0.6
Figure 2: The effectiveness of our attacks against all watermarks. We highlight the Pareto front for each watermarking method by dashed lines and indicate petptivv//oo-ddaapiii attacks by colors.
Fréchet Inception Distance (FID) (Heusel et al., 2017 which measures the similarity between real and generated images. Additionally, we report the CLIP score (Radford al. (02T) that measures the similarity of a prompt to an image. We generate Ik images to evaluate TPR @ 1%FPR and 5k
images evaluate FID and CLIP score on the training dataset of MS-COCO-2011 (Lin al. 2014).
5.1 EVALUATING ROBUSTNESS
Figure shows a scatter plot of the effectiveness of our attacks against all surveyed watermarking methods. We evaluate adaptive and non-adaptive attacks. Similar to Wen et al.] (2023), for the non- adaptive attacks, we use Blurring, JPEG Compression, Cropping, Gaussian noise, Jittering. Quan- tization, and Rotation but find these attacks be ineffective at removing the watermark. Figure 2 highlights Pareto optimal attacks for pairs of (i) watermark detection accuracies and (ii) perceptual distances. We find that only adaptive attacks evade watermark detection and preserve image quality.
Table summarizes the best attacks from Figure when we set the lowest acceptable detection accuracy to 10%. When multiple attacks achieve a detection accuracy lower than 10%. we pick the attack with the lowest perceptual distance to the watermarked image. We observe that adver- sarial compression an effective attack against all watermarking methods. TRW also evaded by adversarial compression, but adversarial noising 2/255 preserves higher image quality.
TRW
WDM
DWT
DWT-SVD
RivaGAN
Best Attack
Adv. Noising
Compression
Compression =
Compression r= 1
Compression r=
Parameters
= 2/255
LPIPS 4 3.2e-2
5.6e-2
7.7e-2
7.5e-2
7.3e-2
Accuracy
5.2%
2.0%
0.8%
1.9%
6.3%
Table 1: A summary of Pareto optimal attacks with detection accuracies less than 10%. We list the attack's name and parameters, the perceptual distance before and after evasion, and the accuracy (TPR@) @ (FFPR). is the maximal perturbation in the L norm and is the number of compressions.
5.2 IMAGE QUALITY AFTER AN ATTACK
Figure shows the perceptual quality after using our adaptive attacks. We show cutout of the top left image patch with high contrasts the bottom right to visualize noise artifacts potentially introduced by our attacks. We observe that, unlike adversarial noising, the compression attack visible artifacts (see also Appendix (reendixA.44fe for more visualizations).
introduces
The FID and CLIP scores the watermarked images and the images after using adversarial noising and adversarial compression remain unchanged (see Table 2 in the Appendix). We calculate the quality using the best attack configuration from Figure 2 when the detection accuracy is less than 10%. Adversarial Noising is ineffective removing WDM and RivaGAN for < 10/255.

---

Published as a conference paper at ICLR 2024
TRW
WDM
DWT
DWT-SVD
RivaGAN
Adversarial Noise
No Watermark
27255, 0,009
8/255, 0.13
6/25, 0.18
4/255, 0.29
8/255, 0.05
Adversarial Compression
0.50
1.00
Figure 3: A visual analysis of two adaptive attacks. The left image shows the unwatermarked output, including a high-contrast cutout of the top left corner of the image to visualize noise artifacts. On the right images after evasion with adversarial noising (top) and adversarial compression (bottom).
Ablation Epsilon Budget Adversarial Noising
Ablation
Repetitions Adversarial Compression
100
TRN
100-
THN
Aucunay (1PR03N/PM0
WCM
Accurasy (1PHG)38119)
WDM
DWT
DWT
DWTT-VDD RlvoGAN
DWT-SVD
60-
AhvaAN
40-
91
1/255
6,255
10/255
No Optimiottion
Epelon Budget IL
Number Repelliionns))
Figure 4: Ablation studies over (left) the maximum perturbation budget e in T for adversarial noising and (right) the number of adversarial compressions against each watermarking method. "No Optimizations" means we did not optimize the parameters 0 of the attack.
5.3 ABLATION STUDY
Figure shows ablation studies for our adaptive attacks over the (i) maximum perturbation budget and (ii) the number of compressions applied during the attack. TRW and DWT-SVD are highly vulnerable to adversarial noising, whereas RivaGAN and WDM are substantially more robust to these types of attacks. We believe this is because keys generated by RivaGAN and WDM suf- ficiently randomized, which makes our attack (that uses only a single surrogate key) less effective unless the surrogate key uses similar channels as the secret key hide the watermark. Adversarial compression without optimization of the parameters 0 is ineffective at evading watermark detec- tion against all methods except DWT. After optimization, adversarial compression evades detection from all watermarking methods with only single compression.
DISCUSSION & RELATED WORK
Attack Scalability. The presented findings clearly indicate that even with a less capable surrogate generator, an adaptive attacker can remove all surveyed watermarks with minimal quality degra- dation. Our attackers generate single surrogate key and are able to evade watermark verification, which indicates design flaw since the key seems have little impact. If KEYGEN were sufficiently randomized, breaking robustness should not be possible using single key, even if the provided and surrogate generators are the same. An interesting question emerging from our study relates to the maximum difference between the watermarked and surrogate generators for the attacks to remain effective. We used a best-effort approach, by using two public checkpoints with the largest reported quality differences: Stable Diffusion v1.1 and v2. More research is needed study the impact the effectiveness of our attacks (i) using different models (ii) or limiting the attacker's knowledge of the method's public parameters. Our attacks evaluate the best parameters suggested by the authors.
8

---

Published as a conference paper at ICLR 2024
Types of Learnable Attacks. Measuring the robustmess against different types of learnable attacks crucial in assessing the trustworthiness of watermarking. We explored (i) Adversarial Exam- ples, which rely solely the surrogate key, and (ii) Adversarial Compression, which additionally requires the availability of a pre-trained autoencoder. We believe this requirement is satisfied in practice, given that (i) training autoencoders is computationally less demanding than training Stable Diffusion, and many pre-trained autoencoders have already been made publicly available (Podell al., 2023). Although autoencoders enhance an attacker's ability to modify images, our study did not extend to other learnable attacks such as inpainting (Rombach et al. (022) or more potent image editing methods (Brooks et al. (023) which could further enhance an attack's s effectiveness.
Enhancing Robustness using Adaptive and Learnable Attacks. Relying on non -on-adaptive attacks for evaluating a watermark's robustness inadeuuate as underestimates the attacker's capabili- ties. To claim robustness, the defender could (i) provide a certification of robustness (Bansal et al.) 2022]. or (ii) showcase empirically that their watermark withstands strong attacks. The issue is that we lack strong attackers. Although Lukas et al. (2022) demonstrated that adaptive attackers can break watermarks for image classifiers, their attacks were handcrafted and did not scale. Instead, propose better method of empirically testing robustness by proposing adaptive learnable at- tackers that require only the specification of a type of learnable attack, followed by an optimization procedure find parameters that minimize objective function. We believe that any watermarking method proposed in the future should evaluate robustness using our attacks and expect that future watermarking methods can enhance their robustness by incorporating our attacks.
Limitations. Our attacks are based on the availability of the watermarking algorithm and an open- source surrogate generator to replicate keys. While providers like Stable Diffusion openly share their models, and replicas of OpenAl's DALL-E models are publicly available (Dayma el al. (021), not all providers release information about their models. To the best of our knowledge, Google has not released their generators (Saharia et al. 2022).. but efforts to replicate are ongoing Providers like Midjourney, who keep their image generation algorithms undisclosed, prevent adaptive attackers altogether but may be vulnerable to these attacks by anyone to whom this information is released.
Outlook. An adaptive attacker can instantiate more effective versions of their attacks with knowl- edge of the watermarking method's algorithmic descriptions (KEYGEN, EMBED, VErIfY). Our attacks require no interaction with the provider. Robustness against adaptive attacks extends to robustness against non-adaptive attacks, which makes studying the former interesting. Adaptive attacks will remain a useful tool for studying a watermarking method's 's robustness, even if the wa- termarking method is kept secret. Previous works did not consider such attacks, and we show that future watermarking methods must consider them if they claim robustness empirically.
6.1 RELATED WORK
Jiang et al. (2033) propose attacks that use (indirect) access to the secret watermarking key via access to the provider's VERIFY method. Our attacks require no access to the provider's secret wa- termarking key, as our attacks optimize over any key message pair (see Equation (2). These threats related since both undermine robustness but are orthogonal due to different threat models. Peng (ct al. (2023);(Cur CCi et al. (2023) propose black-box watermarking methods that protect the Intellectual Property of Diffusion Models. We focus no-box verifiable watermarking methods that control misuse. Lukas & Kerschbaum (202)); Yu et al.] (2020) (021) propose watermarking methods but only evaluate GANs. We focus on watermarking methods for pre-trained Stable Diffusion models with much higher output diversity and image quality (Dhariwal & Nichol 2021)
7 CONCLUSION
We propose testing the robustness of watermarking through adaptive, learnable attacks. Our empiri- cal analysis shows that such attackers can evade watermark detection against all five surveyed image watermarks. Adversarial noising evades TRW (Wen et al. (023) with e = 2/255 but needs to add visible noise to evade the remaining four watermarking methods. Adversarial compression evades all five watermarking methods using only single compression. We encourage using these adaptive attacks to test the robustness of watermarking methods in the future more comprehensively.
https: httppp////ggthbbbccc////caaaaaaa hagga//iiiiriiiaaaaaaaaaeeeeeeee magen- -otttree---e pytorch

---

Published as a conference paper at ICLR 2024
8 ETHICS STATEMENT
The attacks we provide target academic systems, and the engineering efforts to attack real systems substantial. We make it harder by releasing our code publicly. We will, however, release our code, including pre-trained checkpoints, upon carefully considering each request. Currently, there are no known security impacts of our attacks since users cannot yet rely on the provider's use of watermarking. The use of watermarking is experimental and occurs at the provider's own risk, and our research aims to improve the trustworthiness of image watermarking by evaluating it more comprehensively.
REFERENCES
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weak- ness into strength: Watermarking deep neural networks by backdooring. In 27th USENIX Secu- rity Symposium (USENIX Security 18), pp. 1615-1631, 2018.
Mohamed Akrout, Bálint Gyepesi, Péter Holló, Adrienn Poór, Blága Kincsö, Stephen Solis, Katrina Cirone, Jeremy Kawahara, Dekker Slade, Latif Abid, et al. Diffusion-basee data augmentation for skin discase classification: Impact across original medical datasets to fully synthetic images. arXiv preprint arXi::2311008802 2023.
Arpit Bansal, Ping-yeh Chiang. Michael J Curry, Rajiv Jain, Curtis Wigington, Varun Manjunatha, John P Dickerson, and Tom Goldstein. Certified neural network watermarks with randomized smoothing. In International Conference Machine Learning, pp. 1450-1465. PMLR, 2022.
Clark Barrett, Brad Boyd, Ellie Burzstein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, et al. Identifying and mitigating the security risks of generative ai. arXiv preprint vrXiv:2308.148/0 2023.
Dan Boneh, Andrew Grotto, Patrick McDaniel, and Nicolas Papernot, How relevant is the turing test in the age of sophisbots? IEEE Security & Privucy, 176))64-71,2019)
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392-18402, 2023.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten de- tection methods. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pp. 3-14, 2017.
Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. Digital watermark- ing and steganography. Morgan kaufmann, 2007.
Yingqian Cui, Jie Ren, Han Xu, Pengfei He, Hui Liu, Lichao Sun, and Jiliang Tang. Diffusion- shield: A watermark for copyright protection against generative diffusion models. arXiv preprint a30iv:2306. 04642, 2023.
Boris Dayma, Suraj Patil, Pedro Cuenca, Khalid Saifullah, Tanishq Abraham, Phúc Le Khac, Luke Melas, and Ritobrata Ghosh. Dall mini, 7 2021. URL https://g thub.com/
forisdayma/dall -ininl
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Federal Register. Safe, secure, and trustworthy development and use of artificial intelligence.
sa a http::/wwww...ppppaaoaaaaaaaeee-
intelligentl 2023. Accessed 4 Nov. 2023.
Pierre Fermandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks latent diffusion models. arXiv preprint arXiv:2303.1544 2023.
10

---

Published as a conference paper at ICLR 2024
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139-14) 2020.
Sven Gowal and Pushmeet Kohli. Identifying ai-generated images with synthid, 2023. URL
httpss:/ae http///wwwda. https://wwww.deepmmnn. wwtp:/ddepmiin.com/bbb.b
fying generated- -ewpaatedddiiiieeseeeee
with-synthid Accessed: 2023-09-23.
Alexei Grinbaum and Laurynas Adomaitis. The ethical need for watermarks in machine-generated language. arXiv preprint arXiv:2209.03118, 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai-generated content. arXiv preprint arXiv:2305.03800 2023.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint orXiv:1412.6986 2014.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer ViCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022.
Nils Lukas and Florian Kerschbaum. Ptw: Pivotal tuning watermarking for pre-trained image gen- erators. 32nd USENIX Security Symposium, 2023.
Nils Lukas, Edward Jiang, Xinda Li, and Florian Kerschbaum. Sok: How robust image classifi- cation deep neural network watermarking? In 2022 IEEE Symposium on Security and Privacy (SP), pp. 787-804. IEFE, 2022.
Yisroel Mirsky and Wenke Lee. The creation and detection of deepfakes: A survey, ACM Computing Surveys (CSUR), 54(1):1-41, 2021.
Sen Peng, Yufei Chen, Cong Wang, and Xiaohua Jia. Protecting the intellectual property of diffusion models by the watermark diffusion process. arXiv preprint arXiv:2306.0334 2023.
Renana Peres, Martin Schreier, David Schweidel, and Alina Sorescu. On chatgpt and beyond: How generative artificial intelligence may affect research, teaching, and practice. International Journal of Research in Marketing, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 2023.
Alec Radford, Jong Wook Kim. Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Informa- fion Processing Systems, 35:36479-36499 2022.

---

Published as a conference paper at ICLR 2024
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278-25294 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265. PMLR, 2015.
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on international conference on multimedia retrieval, pp. 269-277, 2017.
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fin- gerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030, 2023.
Ning Yu, Vladislav Skripniuk, Dingfan Chen, Larry Davis, and Mario Fritz. Responsible disclosure of generative models using scalable fingerprinting. arXiv preprint arXiv:2012.09726, 2020.
Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. Artificial fingerprinting for gen-
erative models: Rooting deepfake attribution in training data. In Proceedings of the IEEE/CVF International conference on computer vision, pp. 14447, 2021.
Kevin Alex Zhang, Lei Xu, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Robust invisible video watermarking with attention. arXiv preprint vrXiv:1909.02280 2019.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 386-595, 2018.
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, and Min Lin. A recipe for watermarking diffusion models. arXiv preprint arXiv:2303. 2023.
A APPENDIX
A.1 PARAMETERS FOR WATERMARKING METHOОS
Tree Ring Watermark (TRW) Wen et al. 2023 We evaluate the Tree-RingRings method, which the authors state "delivers the best average performance while offering the model owner the flexibil- ity of multiple different random keys". Using the author's 's implementation we generate and verify watermarks using 20 inference steps, where we use no knowledge of the prompt during verification and keep the remaining default parameters chosen by the authors.
Watermark Diffusion Model (WDM) (Zhao al., 2023): As stated in the paper, instead of stamp- ing the model's training data to embed a watermark, we apply the pre-trained encoder to generated image as post-proee post-processing step. We choose messages with n 40 bits and use the encoder archi- tecture proposed by YYu et al. 2021), followed by ResNet-50 decoder. Each call to KEYGEN (0c), where 0G is the surrogate generator, trains new autoencoder from scratch.
DWT, DWT-SVD (Cox al. 2007) and RivaGAN (Zhang al. 2019). We 32-bit messages and keep the default parameters set in the implementation used by the Stable Diffusion model
A.2 STATISTICAL TESTS
Matching Bits. WDM Zhao et al. 2023). DWT, DWT-SVD (Cox et al. 2007) and Riva-
GAN (Zhang al. 2019 encode messages E M by bits and our goal is verify whether
https://github.... http:///w...cceeaeeeeee/ee/eeeea httpe///htttaccccccc.eeeme/////s
12

---

Published as a conference paper at ICLR 2024
m . is present in E X' using key We extract m' from x and want to reject the following null hypothesis.
Ho m and m' match by random chance.
For a given pair of bit-strings of length n, if we denote the number of matching bits as k, the expected number of matches by random chance follows binomial distribution with parameters and expected value 0.5. The p-value for observing at least & matches is given by:
p = 1 - CDF(k - 1; 0.5)
Where CDF represents the cumulative distribution function of the binomial distribution.
Matching Latents. TRW (Wen et al. 2023 leverages the forward diffusion process of the diffusion model to reverse an image to its imitial noise representation 2T. This transformation is represented by m' F(TT). where F denotes Fourier transform. The authors find that reversed real images and their representations in the Fourier domain are expected to follow Gaussian distribution. The watermark verification process aims to reject the following null hypothesis:
Ha y originates from a Gaussian distribution V(0,07c)
Here, y is subset of m' based on watermarking mask chosen by the provider, which determines the relevant coefficients. The test statistic, y) denotes the normalized sum-of-squares difference between the original embedded message m and the extracted message m', which can be complex- valued due to the Fourier transform. Specifically,
mms -m/ Σ 0
(4)
And,
p= Pr (XMMIA Sn Ho) xx²
(5)
Where y2 represents the cumulative distribution function of the noncentral x² distribution. We refer to Wen et al. (2023) for more detailed descriptions of these statistical tests.
A.3 DETAILS ON GKEYGEN
This section provides more details on Algorithm [ VERIFY consists of sub-procedure EXTRACT, which maps an image to a message using the secret key, as stated in Section 2.1 The space of messages is specific the watermarking method. We consider two message spaces M: multi- bit messages and messages in the Fourier space. All surveyed methods except TRW are multi-bit watermarks, for which use the categorical cross-entropy to measure similarity, and for TRW, use the mean absolute error as a similarity measure between messages (line 7 of Algorithm I.
As stated in the main paper, we instantiate GKEYGEN only for the DWT and DWT-SVD watermark- ing methods. We train ResNet-50 decoder On in Algorithm [ to predict a bit vector of the same length the message and calculate gradients during training using the cross-entropy loss. Attack- ing TRW, WDM, and RivaGAN only requires invoking GEN, as the keys are the parameters of (differentiable) decoders. We train these keys from scratch for WDM and RivaGAN, For TRW, the key generation does not require any training, as the key only specifies elements in the Fourier space that encode the message. The forward diffusion process used in VERIFY already differentiable.
A.4 QUALITATIVE ANALYSIS OF WATERMARKING TECHNIQUES
We refer to Figure 5 for examples non-watermarked, watermarked, and attacked images using the attacks summarized in Table | We show three images for each of the five surveyed watermarking methods: an image without a watermark, one with a watermark, and the watermarked image after an evasion attack. We show the prompt that was used to generate these images and label each image with the v-auuue with which the expected message was detected in the image using the secret watermarking key and the VERIFY procedure.
13

---

Published as a conference paper at ICLR 2024
A.5 QUALITY EVALUATION
Table Ssoow the FID and CLIPScore of all five surveyed watermarking methods without a water- mark (first row), with watermark (second row), after our adaptive noising attack (third row) and after our adversarial compression attack (fourth row). All results are reported as the mean value over three independent runs using three different secret watermarking keys. We observe that the degradation in FID and CLIPScores is statistically insignificant, as seen in Figure
TRW
WDM
DWT
DWT-SVD
RivaGAN
FID
CLIP
FID
CLIP
FID
CLIP
FID
CLIP
FID
CLIP
No WM
23.32
31.76
23.48
31.77
23.48
31.77
23.48
31.77
23.48
31.77
WM
24.19
31.78
23.43
31.72
23.16
32.11
23.10
32.15
22.96
31.84
A-Noise
23.67
32.15
N/A
N/A
23.55
32.46
22.89
32.50
N/A
N/A
A- A-Comp
24.36
31.87
23.27
32.01
23.16
32.17
23.06
31.92
23.25
31.86
Table Quality metrics before and after watermark evasion. FID represents the Fréchet Inception Distance, and CLIP represents the CLIP score, computed on 5k images from MS-COCO-2015 N/A means the attack could not evade watermark detection, and we do not report quality measures.
A.6 ATTACK EFFICIENCY
From a computational perspective, generating a surrogate watermarking key with methods such as RivaGAN or WDM is the most expensive operation, as it requires training watermark encoder- decoder pair from scratch. Generating a key for these two methods takes around GPU hours each on single A100 GPU, which is still negligible considering the total training time of the diffusion model, which takes approximately 150-1000 GPU days (Dhariwal & Nichol, (021). The optimization of Adversarial noising takes less than second per sample, and tuning the adversarial compressor's parameters takes less than 10 minutes on single A100 GPU.
A.7 DOUBLE-TAIL DETECTION
Jiang et al. 2023) propose a more robust statistical test that uses two-tailed detection for multi- bit messages. The idea is to test for the presence of watermark with message
message
m (all bits flipped). We implemented the double-tail detection described by Jiang et al. (2023) and adjusted the statistical test in VERIFY Figure Table summarizes the resulting TPR @ FFPR with single or double-tai detection. Since TRW
double-tail detection on the same images used in
(Wen et
2023) is not multi-bit watermarking method, we omit its results.
Attack Method
WDM
DWT
DWT-SVD
RivaGAN
Adv. Noising (5 1/255)
100% / 100%
78.8% / 75.9%
100% / 100%
100% / 100%
Adv. Noising (s 2/255)
99.9% 99.0%
79.1% / 75.0%
99.5% / 99.0%%
99.9% 99.9%
Adv. Noising (E 6/255)
68.0% 68.7%
50.2%
49.5%
0.0%
23.3%
96.3% 96.6%
Adv. Noising (s 10/255)
29.9% 36.9%
0.0%
57.3%
0.0%
11.8%
67.0% 63.3%
Adv. Compression
2.0% 1.8%
0.8% 0.5% %
1.9% 2.5 %
6.3% 5.7 %
Table 3: Summary of TPR@ 1 1&FPR using single-tail (left) and double-tail detection (right). We
mark attacks bold if their TPR @ 1%FPR is less than 10%.
Table shows that double-tail detection increases the robustness of DWT and DWT-SVD against adversarial noising, which is the same effect that Jiang al. (023) find in their paper. We find that Adversarial Compression remains effective against all attacks in the presence of double-tail detection. 4 that adversarial noising highly effective against TRW but is ineffective against the remaining methods because an attacker has to add visible noise (see Figure 3 An attacker would always use the Adversarial Compression attack in real-world attack.
14

---

The p-values and text Published as a conference paper at ICLR 2024
No Watermark
Watermarked
Attacked
TRW: "Cars are parked on the street near an old building"
P-value 0.28
P-value 1.77e-09
P-yalue 0.52
WDM: "A bench at the beach next to the sea"
P-value 0.13
P-value 3.73-11
P-value 0.08
go bridge"
P-value 0.30
P-vauu 2.33-10
P v-aauue 0.57
DWT-SVD: "A white horse standing on top field."
P-value 0.30
P-yalue 2.33-10
P-yalue 0.05
RivaGAN: "Donuts frosting glazed toppings
A
Figure 5: Qualitative showcase of three kinds of images: non- won-watermarked watermarked with men- tioned technique, and attacked images with the strongest attack from Table [ prompts are also provided.
15


Reviews:
[{"review_id": "dokA3TomtD", "reviewer": "Reviewer_f7pQ", "summary": "Untrustworthy users can misuse image generators to synthesize high-quality deep-\nfakes and engage in online spam or disinformation campaigns. Watermarking de-\nters misuse by marking generated content with a hidden message, enabling its\ndetection using a secret watermarking key. A core security property of water-\nmarking is robustness, which states that an attacker can only evade detection by\nsubstantially degrading image quality. Assessing robustness requires designing\nan adaptive attack for the specific watermarking algorithm. A challenge when\nevaluating watermarking algorithms and their (adaptive) attacks is to determine\nwhether an adaptive attack is optimal, i.e., it is the best possible attack. We solve\nthis problem by defining an objective function and then approach adaptive attacks\nas an optimization problem. The core idea of our adaptive attacks is to replicate\nsecret watermarking keys locally by creating surrogate keys that are differentiable\nand can be used to optimize the attack’s parameters. The authors demonstrate for Stable\nDiffusion models that such an attacker can break all five surveyed watermarking\nmethods at negligible degradation in image quality. These findings emphasize the\nneed for more rigorous robustness testing against adaptive, learnable attackers.", "questions": "Why is this topic important?\n\nWhat are the future work directions of this work?\n\nWhy is the comparative analysis limited\n\nWhat is the complexity of the algorithms?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Paper is well formatted\n\nTopic is interesting\n\nGood balance of theory and experiments", "weaknesses": "Please improve readability\n\nPlease number all equations\n\nPlease discuss figures, tables and algorithms clearly in the text\n\nPlease add a security analysis to known attacks in this domain", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9VXGOMwie7", "reviewer": "Reviewer_Agid", "summary": "Authors emphasize the significance of watermarking in countering misuse by marking generated content with hidden messages. The core security property of watermarking, robustness, is investigated in this paper. The authors assert that evaluating robustness involves creating adaptive attacks tailored to specific watermarking algorithms. To this end, one of the paper's contributions is the proposed approach to assess the optimality of adaptive attacks by framing them as an optimization problem and defining an objective function. The paper presents evidence that such attackers can effectively break all five surveyed watermarking methods with negligible degradation in image quality.", "questions": "1.\tABSTRACT: The text should include more details to the proposed methodology, numerical results achieved, and comparison with other methods\n2.\tCould you tell me the limitations of the proposed method? How will you solve them? Please add this part to the manuscript.\n3.\tThe abbreviations must appear at the very first place that the terminology is introduced and the way of introducing the terms must be consistent throughout the manuscript from abstract to conclusion.\n4.\tThe conclusions should be improved such that the authors add some analytical terms.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "First, on the aspect of the paper’s organization, this manuscript is well-organized and easy to follow. Second, on the aspect of clarity, the proposed method is clearly defined using schematics and pseudo-code descriptions. Third, this paper provides an approach to evaluating adaptive attacks and the demonstration of their effectiveness provide a fresh perspective on the challenges faced in countering image manipulation.", "weaknesses": "The motivation and importance of the proposed method are not clear enough, e.g., what problems did the previous works exist? Besides, the experiments comparison and discussion are weak. Experiment section should expand the scope of discussion, compare with more advanced methods, and provide in-depth discussions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nc2XU96cKr", "reviewer": "Reviewer_LPhx", "summary": "This paper proposes a method for performing adaptive attacks against image watermarking methods, allowing for more accurate evaluations of the robustness of watermarking methods against motivated attackers. To enable standard adversarial optimization attacks against known but non-differentiable watermarking methods, the authors propose to train surrogate (differentiable) watermark detection networks. Experiments show that adaptive attacks crafted with the proposed method significantly degrade the effectiveness of all evaluated watermarking methods and outperform non-adaptive attacks while preserving the overall perceptual quality of attacked images.", "questions": "Did the authors train surrogate detectors for all the evaluated watermarking methods to create true no-box attacks?\n\nDid the authors experiment with different degrees of attacker knowledge -- e.g., what if the attacker does not know the length of the watermark embedded by the actual generator? Would training on 32-bit messages cause attacks on a 64-bit message system to fail?\n\nRather than training surrogate detectors to reconstruct embedded messages, did the authors consider simply training a binary classifier on watermarked and un-watermarked images from the surrogate generator and then performing an adversarial attack on the binary classifier? This seems like the simplest no-box approach.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The issue of watermarking the outputs of generative models is timely and interesting.\n\nThe idea of training differentiable surrogates for arbitrary watermarking methods is an interesting threat model.\n\nThe selection of baseline watermarking methods is reasonable and includes both \"post-hoc\" (low-perturbation) and \"semantic\" (high-perturbation) methods.\n\nThe autoencoder/compression-based attack is interesting and seems to effectively remove watermarks while retaining high perceptual quality.", "weaknesses": "I think there is a terminology issue in the paper that could be confusing for readers. It appears the watermark \"key\" referenced in the paper more closely matches the concept of a watermark \"detector\" algorithm in methods such as RivaGAN and Tree-Rings; many methods often use \"key\" and \"message\" interchangeably to refer to the hidden signal. If this is true, the authors' proposed training of differentiable surrogate \"keys\" can be understood as training differentiable surrogate detector networks that predict the key/message concealed in a watermarked image --  allowing for gradient-based optimization attacks on non-differentiable detectors. The pseudocode in Algorithm 1 strongly suggests this. If this is the case, I urge the authors to revise their terminology to make this more clear.  \n  \n- - - -\n\nIt should be clarified that in the general no-box watermarking scenario, the second step $\\mathrm{EMBED}$ need not modify the parameters of the generator model, just endow it with watermarking capabilities (e.g. by applying a post-hoc watermarking algorithm to its outputs). As far as I can tell, none of the methods evaluated in the paper modify the generator parameters directly. Overall, the no-box watermarking steps and their instantiations for each of the evaluated watermarking methods are not clearly explained.\n  \n- - - -\n\nThe proposed method is similar to that of Jiang et al. [5] in that it adversarially parameterizes image transformations to remove watermarks. The authors claim that the method of Jiang et al. requires access to a watermarking \"key;\" however, Jiang et al. propose attacks under the explicit assumption that the attacker does not have access to the ground-truth key/message (which is either approximated via the detector model's predictions or sampled at random) -- from pp.6, \"the attacker does not have access to the ground-truth watermark $w$\".  On the other hand, if I am correct that the authors take \"key\" to mean \"detector,\" this claim makes more sense in that Jiang et al. use full access (white-box) or query access (black-box) to the detector to craft attacks. \n\nThe authors should clarify their statements about the prior work of Jiang et al. and the distinctions between their methods. And while additional experiments may not be feasible at this point for various reasons, I think the paper would be much stronger if it included comparisons between the proposed approach and either or both attack variants proposed by Jiang et al. This would also require modifying the proposed attack to evade two-tailed detection, as discussed by Jiang et al.\n\n- - - -\n\nIf the attacker \"does not need to invoke GKEYGEN\" for TRW/WDM/RivaGAN, as stated in section 4.2, does this mean the attacker does not train a differentiable surrogate detector? In this case, doesn't the proposed approach just become a standard white-box attack, as in Jiang et al.?\n\n- - - -\n\nThe authors do not provide any details of the architecture of the surrogate detector networks $\\theta_D$ trained by the adversary. This seems like a crucial aspect of the proposed approach, so it is strange that it is not discussed.\n\n- - - -\n\nThe related work section mixes references to image classifier watermarks [1][2] and watermarks for generative models [3], which are very different: the former aims to protect the intellectual property of a model developer, typically through query- or trigger-based verification, while the latter is embedded in all outputs of a generative model to distinguish real from fake content. This paper is concerned with the latter kind of watermark, so I'm confused by the emphasis on works in the former area. At the very least, these two different types of works should be clearly distinguished from one another in the related work section.\n\n- - - -\n\nAs far as I can tell, the term \"Adaptive Attack\" comes from the adversarial example literature -- the authors should explain what distinguishes an adaptive attack from a non-adaptive attack and probably cite the original work [4].\n\n- - - -\n\nFigure 1 is missing step #8 (it skips from 7 to 9).\n\n- - - -\n\nThis is a much smaller concern, but the substitution of the Stable Diffusion v1 generator for v2 does not seem like a very difficult obstacle for the attacker to overcome, given the general similarities in architecture and training. Attacks on post-hoc methods probably shouldn't be affected too much by the choice of surrogate generator, but Tree-Rings is deeply intertwined with the generator structure. Therefore, it would be interesting to see how attacks on TRW fare when there is a more substantial mismatch between the actual and surrogate generator.\n\n- - - -\n\nOverall, I think the central idea -- no-box watermark attacks with differentiable surrogates -- is very interesting, and the experimental results look very strong. However, I think the paper has many issues that still need to be addressed.\n\n\n[1] Nils Lukas, Edward Jiang, Xinda Li, and Florian Kerschbaum. Sok: How robust is image classification deep neural network watermarking? In 2022 IEEE Symposium on Security and Privacy\n(SP), pp. 787–804. IEEE, 2022.\n\n[2] Arpit Bansal, Ping-yeh Chiang, Michael J Curry, Rajiv Jain, Curtis Wigington, Varun Manjunatha, John P Dickerson, and Tom Goldstein. Certified neural network watermarks with randomized smoothing. In International Conference on Machine Learning, pp. 1450–1465. PMLR, 2022.\n\n[3] Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030, 2023.\n\n[4] Nicholas Carlini and David Wagner. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods. In AISec '17: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp.3-14. 2017.\n\n[5] Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai-generated content. arXiv preprint arXiv:2305.03807, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7WYeBeluqb", "reviewer": "Reviewer_2NFv", "summary": "This paper introduces an objective function and approaches adaptive attacks as an optimization problem to evaluate the robustness of watermarking algorithms.  The core idea of the proposed adaptive attack is to replicate secret watermarking keys locally by creating surrogate keys that are differentiable and can be used to optimize the attack’s parameters. The experiments reveal that this type of attacker can successfully compromise all five surveyed watermarking methods with minimal degradation in image quality. These findings underscore the necessity for more comprehensive testing of the robustness of watermarking algorithms against adaptive, learnable adversaries.", "questions": "1 The paper shows that generates a single surrogate key and can evade watermark verification which indicates that the private key seems to have little impact. Is it possible that the less impact comes from the high similarity between the surrogate generator and the watermark generator?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a practical method of empirically testing the robustness of different watermarking methods. The proposed adaptive attack is shown to be effective against different watermarking methods.", "weaknesses": "1. The attack requires the knowledge of the watermarking algorithm. \n2 in the experiments the surrogate generator and the watermark generator exhibited a high degree of similarity, which may not be as practical in real-world scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)