Image Watermarks are Removable using Controllable Regeneration from Clean Noise
Authors: Yepeng Liu
Yiren Song
Hai Ci
Yu Zhang
Haofan Wang
Mike Zheng Shou
Yuheng Bu
Venue: ICLR 2025 Poster (2025)
Keywords: Watermark
Detection
Robustness
Diffusion Model
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We propose a controllable regeneration method for effective image watermark removal. We aim for our method to serve as a benchmark to evaluate and enhance the robustness of future watermarking techniques.

Abstract: Image watermark techniques provide an effective way to assert ownership, deter misuse, and trace content sources, which has become increasingly essential in the era of large generative models. A critical attribute of watermark techniques is their robustness against various manipulations. In this paper, we introduce a watermark removal approach capable of effectively nullifying state-of-the-art watermarking techniques. Our primary insight involves regenerating the watermarked image starting from a \textbf{clean Gaussian noise} via a controllable diffusion model, utilizing the extracted semantic and spatial features from the watermarked image. The semantic control adapter and the spatial control network are specifically trained to control the denoising process towards ensuring image quality and enhancing consistency between the cleaned image and the original watermarked image. To achieve a smooth trade-off between watermark removal performance and image consistency, we further propose an adjustable and controllable regeneration scheme. This scheme adds varying numbers of noise steps to the latent representation of the watermarked image, followed by a controlled denoising process starting from this noisy latent representation. As the number of noise steps increases, the latent representation progressively approaches clean Gaussian noise, facilitating the desired trade-off. We apply our watermark removal methods across various watermarking techniques, and the results demonstrate that our methods offer superior visual consistency/quality and enhanced watermark removal performance compared to existing regeneration approaches. Our code is available at \url{https://github.com/yepengliu/CtrlRegen}.

Published as a conference paper at ICLR 2025
IMAGE WATERMARKS ARE REMOVABLE USING CON- TROLLABLE REGENERATION FROM CLEAN NOISE
Yepeng Liu', Yiren Song", Hai Ci,,
Yu Zhang?, Haofan Wang',
Mike Zheng Shou', Yuheng Bu'
University of Florida SSho Lab, National University Singapore TTnggii University *InstantX Team
{yepeng. Liu, peyuuhnng@ufl... edu, fcihai03, mike. zheng. shou @emmai 1mmm...coomm yi virenênus. edu.sg sg
ABSTRACT
Image watermark techniques provide effective way assert ownership, de- ter misuse, and trace content sources, which has become increasingly essential the era of large generative models. critical attribute of watermark techniques is their robustness against various manipulations. In this paper, we introduce watermark removal approach capable of effectively nullifying state-of-the-as -hr watermarking techniques. Our primary insight involves regenerating the water- marked image starting from a clean Gaussian noise a controllable diffusion model, utilizing the extracted semantic and spatial features from the watermarked image. The semantic control adapter and the spatial control network are specifi- cally trained to control the denoising process towards ensuring image quality and enhancing consistency between the cleaned image and the original watermarked image. To achieve smooth trade-off between watermark removal performance and image consistency, we further propose an adjustable and controllable regen- eration scheme. This scheme adds varying numbers of noise steps the latent representation of the watermarked image, followed by a controlled denoising pro- cess starting from this noisy latent representation. increases, the latent representation progressively approaches clean Gaussian noise, facilitating the desired trade-off. We apply our watermark removal methods across various watermarking techniques, and the results demonstrate that our methods offer superior visual consistency/quality and enhanced watermark removal perfor- mance compared to existing regeneration approaches. Our code is available at nttps://gitine com http::/gg..httt.ooooo/gggggnnn/d
the number of noise steps
INTRODUCTION
As the large generative models continue to advance, the realism of AI- Al-generated content has reached unprecedented levels. Those Al-generated contents nearly indistinguishable from content cre- ated by humans. While this technological progress brings about excitement and efficiency, the diffi culty of distinguishing Al-generated and human-made content can lead severe problems, such as the spread of misinformation and the potential erosion of trust in digital media. In response, water-
marking Al-generated content (Wen et al., 2024 Zhao al., 2023a Saberi et al., 2023 Zhao al.)
20236
Lukas et al. 2023 Yang et al., 2024 Ci et al. 2024b a Kirchenbauer et al. [023] Zhang al. 2024a Liu & Bu 2024 Rezaei al.
He al. 2024) has emerged an effective solution, providing proactive and reliable method to embed hidden information into Al-generated content for identification and traceability.
2024
Recently, several promising watermarking methods for Al-generated images have emerged, typi- cally embedding watermarks by perturbing the pixels or latent representations of the image. Low perturbation watermarks (Fernandez et al. 2023 Zhu 2018 Fernandez et al.) 2022 Zhang et al.) 2019 Cox et al.) 2007) lead to a small &g distance between watermarked and un-watermarked im- ages both pixel and latent space, making them potentially more vulnerable. On the other hand, watermark methods that induce high perturbation (Saberi et al. [023)] Zhao et al., (023a) usually significantly modify the image its representation, thereby resulting in enhanced robustness against

---

Published as a conference paper at ICLR 2025
Emage 1.2 Distance
a Latent Distanee
Regen Chrieggen
150
Dosance
0.6
100
0.6
0.4
Dethhrsdd
RAsGAN
SST.
Stateeignttin
SteptSuap
DeeRing
DedDaSee
RasGAN
SSL
Sudhifiigaatone
SkegeStamp
Treekiig
Figure 1: The left chart computes the & distance between watermarked and un- watermarked images across different watermarking methods in both pixel and latent spaces, identifying StegaStamp and TreeRing the high perturbation watermarks. The right chart shows the watermark removal perfor- mance of our method versus existing regeneration attacks across different watermarking techniques, highlighting the challenges of neutralizing high-pu -egttreeturbtiion watermarks with current methods.
The robustness of image watermarks, being a crucial attribute, targeted by numerous attacks aimed at removing the watermark.
various malicious manipulations, as shown in Figure T
For low perturbation watermarks, the regeneration attack (Zhao et al. 2023a; Saberi et al. 2023) is proposed remove the invisible watermark that disturbs the image within a bounded &g distance range. Specifically, this method encodes the watermarked image from pixel to latent space, then uses the diffusion model's (Rombach et al. 2022) forward and reverse processes add limited noise to the latent representation and denoise it to reconstruct the image. This method is effective for low- perturbation watermarks, which only cause limited pixcl/reprcsentation perturbation and can thus be easily removed through limited noising and denoising steps. However, it struggles with high- perturbation watermarks like StegaStamp (Tancik et al.) 2020) and TreeRing (Wen et al.) 2024), which significantly alter the l distance between watermarked and un watermaareee images both pixel and latent spaces, as illustrated in Figure The main reason is that the starting noise used for image reconstruction is derived by adding only a small amount of noise the latent representation of the watermarked image. As result, some hidden watermark information may still be retained and could be diffused into the regenerated image during the reverse process.
In terms of high perturbation watermarks, increasing the number of noising steps of regeneration or implementing the regeneration multiple times (An et al., 2024) can make the starting noise more closely resemble the pure Gaussian noise. This additional noise helps further disrupt the watermark structure, thereby enhancing the effectiveness of the watermark removal. However, as the number of noising steps increases, there is a significant sacrifice in image quality, as illustrated in Figure and This results in visible distortion and artifacts, and the image may even lose its semantic integrity.
The existing regeneration method struggles with balance: few noising steps are inadequate to elim- inate high perturbation watermarks, while more noising steps significantly compromise the image quality and consistency between the watermarked and cleaned images. Therefore, it is significant challenge to implement a regeneration attack that effectively removes watermark structures for both low and high perturbation watermarks while maintaining image quality and consistency.
In this paper, we introduce novel watermark removal method, CtrlRegen, designed to remove im- age watermarks through a controllable regeneration process. Our core idea is use clean noise sampled from the Gaussian distribution as a starting point for the demoising process of the diffu- sion model. This strategy ensures that watermark information is thoroughly cleaned up from both pixel and latent spaces. One significant challenge is controlling the diffusion model to generate the cleaned image from a clean Gaussian noise while maintaining image quality and content consis- tency. To address this, we train a pluu-and--p and-play semantic control adapter and spatial control network. These components extract semantic and spatial information from the watermarked image and use this information conditions to guide the denoising process towards a high degree of consistency between the watermarked and cleaned images.
To offer a versatile solution that adapts to varying degrees of watermark robustness, we further propose an adjustable and controllable regeneration method, CtriRegen+, as shown in Figure 2 This method begins by noising the latent representation of the image, and then controls the forward process starting from this noised latent representation. We can adjust the number of noising steps to regulate the degree of watermark destruction. With an increased number of noising steps. the latent representation becomes closer to pure noise, resulting in a more thorough destruction of the watermark information. It is worth noting that, unlike the uncontrolled regeneration method, our
2

---

Published as a conference paper at ICLR 2025
CtrlRegen
Controllable Generation
Latent Space
Denoise
Denoise
Clean Noise
T-1
Watermarked Image
Cleaned Image
CtrlRegen+
Latent Space
Map
Add Noise
Denoise
Zg
Watermarked Image
Cleaned Image
Controllable Generation
Figure 2: Overview of the proposed method. CirlRegen controls the regeneration of watermarked images from clean noise without any watermark information. CtrlRegen+ first encodes wa- termarked image into latent representation and introduces varying levels of noise based on the robustness of the watermark. then controls the denoising process to reconstruct the image.
controllable method still preserves high image quality and visual similarity with the watermarked image, even with a high number of noising steps, as presented in Figure
We conduct experiments to implement our watermark removal methods across various watermark methods, including low and high perturbation watermarks. The results demonstrate that our Ctrl- Regen effectively reduces the detection performance (TPR@) 1%FPR) of StegaStamp from 1.00 to 0.01 and of TreeRing from 0.99 0.12. Conversely, the uncontrolled regeneration method proves less effective for these two watermark methods. Moreover, our CtriRegen+ achieves better image quality/cconsistecyy while maintaining the same watermark removal performance compared to the uncontrolled regeneration approach.
RELATED WORK
2.1 IMAGE WATERMARK METHONS
Watermarking techniques offer an active and reliable method to trace the source of images pro- tect copyright. Traditional methods embed watermark information directly into the generated images (Rouhani et al. 2018 Chen et al., 2019: Jia al. (021). With the development of large genera- tive models, such as Stable Diffusion (Rombach et al.) (022), watermark information can now also be integrated scamlessly into the process of creating digital content. We broadly categorize water- marking methods into post-hoc methods and in in-generation methods based on the stage at which the watermark is embedded. Post-hoc methods embed watermarks into given image using techniques such as encoder-decoder (e.g., HiDDeN (Zhul 2018), Stegastamp (Tancik et al. 2020). optimization (e.g., SSL (Fernandez al. al.) 2022 wavelet transforms (e.g., DwtDetSvd (Cox al. 2007)). In contrast, in-generation methods introduce watermarks during image generation by modifying com- ponents of the generative model, such initial noise (e.g., Tree-Ring (Wen al. 2024), RingID (Ci
et al. 2024b)) or the VAE decoder (e.g., StableSignature (Fernandez et al.) 2023). WMAdapter (Ci al. 2024a)). We evaluated the effectiveness of our proposed methods on both post-hoc and generation watermarking methods.
2.2 IMAGE WATERMARK REMOVING METHODS
Robustness, a crucial attribute image watermarking, is assessed through a range of watermark removal attacks (Hu et al. 2024 Kassis & Hengartner 2024). including editing attacks, regenera-

---

Published as a conference paper at ICLR 2025
tion attacks, and adversarial attacks. The editing attack comprises typical image manipulations such as cropping, compression, rotation, brightness/contrast adjustments, and the addition of Gaussian noise. These manipulations simulate common alterations that images might undergo in practice. Most existing watermarking methods demonstrate robustness against these basic operations. Re- cently, regeneration attack (Zhao et al.] 2023a Saberi et al. 2023), as no-box attack, proposes to remove the image watermark through the noising and denoising process of pre-trained diffusion model or encoding and decoding process of variational autoencoder (VAE). The regeneration attack demonstrates a powerful ability to remove watermarks for methods that cause minimal perturbation to the pixel or latent space of a watermarked image. However, tends to be less effective under limited noising steps for watermarking methods that induce high perturbation. For those high per- turbation watermarking methods, the adversarial attacks (Saberi et al. 2023; Lukas et al., [ Jiang et al., 2023) provide an effective method to generate adversarial images that evade watermark detection. These methods treat the watermark detector as a classifier and introduce adversarial per- turbations to the watermarked image through optimization to deceive the detector. However, this strategy demands more unrealistic capabilities from attackers, including knowledge of the water- marking method (Lukas al., 2023), access the watermarked large generative models in the white-box setting (Saberi et al. 2023), or the ability to make multiple uninterrupted queries to the API of watermark detector (Jiang al., 2023). Moreover, these methods are image-specific and watermark sppecfff,,,,,, which means that attackers need to tailor the perturbation for each image according different watermarking methods. This process both cimeuunuu and compu- tationally intensive. In this paper, we focus on the regeneration attack and propose a controllable regeneration attack combined with our proposed control techniques.
2.3 DIFFUSION MODELS
Diffusion probability models (Song et al. 2020 Ho et al.) 2020) are advanced generative mod- els that restore original data from pure Gaussian noise by learning the distribution of noisy data at various levels of noise. With their powerful capability to adapt to complex data distributions, diffusion models have achieved outstanding achievements in several domains, including image syn- thesis (Rombach et al.) 2022, Peebles& & Xie 2023), image editing (Brooks et al. 2023 Hertz et al.) 2022 Zhang al. 2024c d) and even 3D content creation (Poole et al. (022). Among them, Stable Diffusion (Rombach et al. 2022) (SD), a notable example, employs a UNet architecture and iter- atively generates images with impressive text-to-image capabilities through extensive training on large-scale text-image datasets. Alongside these developments, controllabl image generation has seen enhancements from methods like ControlNet (Zhang & Agrawala 2023) 721-adapter (Mou iet al. 2023), which utilize multimodal inputs such as depth maps and segmentation maps to signif- icantly increase the controllability over the generated images. Furthermore, subject-driven image generation techniques now range from those requiring test-time fine-tuning (Gal et al. 2022 Ruiz
al. 2022] Kumari et al. 2023 Hu et al.] (022) to those operating entirely tine-tuning-ff free (Ye iet al. 2023, Zhang et al. (024b)), each offering varying degrees of adaptability and computational demand. In this paper, we propose an image watermarking removal algorithm for watermark re-
moval based on ControlNet Zhang& & Agrawala (2023) and IP-Adapter Ye al. (2023).
THE PROPOSED CONTROLLABLE REGENERATION ATTACK
3.1 OVERVIEW
The core idea behind our proposed method is to regenerate the watermarked image starting from clean noise. A controllable diffusion model is then designed to maintain the consistency between the watermarked image and the cleaned image during the denoising process, with the watermarked image serving as a conditional input.
The workflow of our method is presented as Figure 3 Specifically, given watermarked image Ia E IRN, starting latent representation z, and the generation function G IR" x RRN - R.N. we can obtain the cleaned image:
i = (2, +w),
where i.. = (x.)) E R represents the edge-detected image obtained from x using edge detection algorithm. For CtrlRegen, input z is e (0 In)- For CtrlRegen+, input is given by 2μ-
4

---

Published as a conference paper at ICLR 2025
① Semantic Control
Image Encoder
Projection Network
Image Embedding
Input: watermarked Image
Latent Space
T' steps
Pixel Space
U-Net
Edge detection
Xy: Clean
Zo
Output:
Gaussian Noise
Cleaned Image
2
Spatial Control
Spatial-Not
Canny edges Image
Figure 3: Workflow of the controllable regeneration. The red modules represent the trainable pa- rameters, while the blue modules represent the pre-trate pre-trained and fixed parameters. Semantic control is applied through inserted cross-attention modules using extracted image embedding as a condition. Spatial control is achieved by integrating the outputs from the convolution layers of Spatial-Net into the decoder blocks of the U-Net.
var-20 20 + -1- where t* E [0, is the number of noising steps, = ((zw) is the latent representation obtained by encoding the watermarked image using an encoder, and a+- represents the variance schedule of the noise ~ NNO0, I,), which gradually decreases from 1 to 0 as t increases from 0 to
Compared to the previous uncontrolled regeneration methods, one important goal of our approach to control the generation process starting from to maintain the consistency between Ve and J. To achieve this, we propose two methods designed to control the semantic content and spatial distribution the images, respectively.
3.2
REGENERATION CONTROL MODEL TRAINING
3.2.1 SEMANTIC CONTROL.
One key challenge in semantic control is preserving the semantic content of watermarked images while effectively destroying the watermark information. Previous uncontrolled regeneration meth- ods achieved this by initially destructing the 2n with random Gaussian noise via the forward process of the diffusion model, followed by reconstructing the image through an uncontrolled reverse pro- cess. While this method destructing through a limited number of noising and denoising steps proves effective for removing low perturbation watermarks, it falls short in eliminating watermark that causes high perturbation in the latent space. Drawing inspiration from SD model, which em- ploys a text encoder to convert text prompts into text embeddings and then using the cross-attentins mechanism (Vaswani 2017) to ensure the generated image semantically aligns with the text, we compress the watermarked image into an image embedding that preserves only semantic content. This embedding then used to control the generation process via cross-attention, ensuring the re- generated image retains semantic accuracy without the watermark information.
We train a semantic control adapter to facilitate semantic control as depicted in the upper branch of Figure 3 which includes image encoder, projection network, and newly implemented decoupled cross-attention layer. Our training approach for the projection network and cross-attention layer employ
draws strategies similar those used in the IP-Adapter (Ye et al. 2023). Specifically, pre-trained DINOv2 (Oquab et al. (2023) as our image encoder to extract the image feature from the watermarked image, capitalizing its high-performance capability to extract rich visual features.
5

---

Published as a conference paper at ICLR 2025
A trainable projection network is then used to transform the extracted image feature into image embedding, which is used to control the generation through cross- attention in the following step.
To preserve the original SD model and control generation using image embedding an additional trainable image cross-attention layer is integrated into the U-Net within the SD model. This layer is designed to compute the attention using the image embedding ((x). such that
condition,
Q(K") Vd
Attention(4) (Q, K' V*) softmaax
V"
where W.: W(z), K' Wk "0x)) and V y'=W; W ((x)). Here, the y=(²) is the intermediate representation within the U-Net. Wkk and Wyy are learnable matrices specifically for image cross- attention. WQ the same projection matrix employed in the text cross-attention During the training and inference process, text vooss-attention and image cross-attention operate concurrently. The com- bined attention mechanism expressed as Z Attent ion(Q. K. +)+ in ( Q, K" V"), where K and V are the key and value matrices associated with text cross-attention and K" and V pertain to the image cross-attention. Specifically, we set the text prompt to be empty during both the training and inference process. Therefore, only the image embedding influences and controls the generation process.
During the training process, the parameters of the image encoder 0g and the SD 0g are kept frozen, while only the projection network 8p and the image cross-attention layer Og are trained. Given the training image dataset X, the training objective be expressed
Le = E.,.e.
egg (Zt, 0, (zt, (c),()13 13)
where x E X is the input image, 09..9, (x) maps the input image to an embedding. Fort I. = 1,... ,T, 2, is the latent representation at different time steps, EØs (Zt, " represents sequence of U-Net used to predict the noise given the input 2z, and condition Φu_.0, (z), where Os includes fixed 0₂ and Oc. and trainable Op and 0g. We use only the image part of the dataset as the generation condition.
3.2.2 SPATIAL DISTRIBUTION CONTROL
The semantic control adapter facilitates a coarse gearse-grained semantic content alignment between the watermarked image and the regenerated image, ensuring that most of the semantic content is pre- served. However, it struggles to control the finer details and layout during the generation process. To address this, enhancing spatial control of regeneration is another key challenge. Incorporating an edge-detected image as an additional condition offers an effective solution, as it provides crucial spatial information without introducing extraneous watermark information. To better leverage spa- tial information from the edge-detected image, we propose to use spatial control network. This network is designed to extract spatial features, which are then integrated into the denoising process of U-Net, thus enhancing spatial distribution in the regenerated image.
An Edge-detected image emphasizes the boundaries and contours within an image by identifying the rapid changes in intensity, which correspond to object edges. Typically rendered as a binary image, it features edges marked in white against non-edge areas in black. Specifically, we use Canny edge images extracted from the watermarked image using the Canny detection method (Canny) 1986). In method, we adopt the ControlNet (Zhang & Agrawala, 2023) structure for spatial control network, as shown in the lower branch of Figure 3 After applying the spatial control network to U-Net, the output of the neural blocks within U-Net is expressed as:
2, t) = Fe. (z), 0e,.0, (x), t) + Hag (ē', t), < 0, 2I),
(4)
where of is the intermediate representation between different neural blocks within U-Net, is the original image, á = -f(x) represents the edge-detected image, Fo_ is the original neural block of U-Net, (-) represents the neural blocks and convolution layers of spatial network, 4 refers to the intermediate representation within the spatial control network that corresponds dimensionally with z/, in which 3d' is derived from through encoder and convolution layer.
To enhance compatibility and integration among the components, we combine the semantic control adapter, spatial control network, and SD within unified framework. We fix the parameters of the already trained semantic control adapter and the SD and tune the spatial control network Ag to opti- mize its performance in conjunction with the other components. Now, with both image embedding

---

Published as a conference paper at ICLR 2025
and edge-detected image conditions, the training objective is:
Lec No...ll cσ(21,
(5)
where E2 00.00, 0, (z), a,t) t) is the U-Net with 2 and as input and 0, (x) and a as condition, 0 includes both 0g and Dg.
3.3 REGENERATION WITH CONTROL
Once semantic control adapter and spatial control network are fully trained, they are ready to be deployed for direct watermark removal, requiring only a single watermarked image. The entire inference process of CtrlRegen outlined in Algorithm[] Appendix B We sample the initial noise from a pure Gaussian distribution. Then the extracted image embedding and edge-detected image serve as conditions, providing the necessary semantic and spatial information. Two well-traines networks are integrated with the backbone SD to denoise across T timesteps. Finally, the cleaned image is obtained by decoding the denoised latent representation.
The inference process of CirlRegen+ is detailed in Algorithm of Appendix B Unlike CtriRegen, which utilizes pure Gaussian noise as a starting point, Ctrlkegen+ first adds noise to the latent representation and then uses this noised representation The noising step can be selected based on the strength of different watermark methods.
starting point to reconstruct the image.
4 EXPERIMENTS
4.1 EXPERIMENT SETTING
Datasets. We evaluate our watermark removal performance using two datasets. For the post-hoc watermarking methods, we sample 1000 real photos from the MIRFLICKR (Huiskes & Lew 2008), which consists of comprehensive image collection sourced from the social photography platform Flickr. For the in in-generation watermarking methods, we sample 1000 prompts from a large-scale text-to-image prompt dataset, DiffusionDB (Wang et al., 2022). to generate watermarked images using generative models. Additionally, we train the semantic control adapter using 10 million images sampled from LAION-2B (Schuhmann et al., 2022) and COYO-700M| The spatial control network is trained using 1 118 image-canny pairs from MSCOCO (Lin et al. 2014).
Image Watermark Methods. To demonstrate the effectiveness of our watermark removal method, we evaluate it against seven diverse watermarking methods, including TreeRing (Wen et al.
2024).
StableSignature (Fernandez al. (023). , StegaStamp (Tancik et al., (020), HiDDeN (Zhu 2018
SSL (Fernandez et al.) 2022), RivaGAN (Zbang et al. 2019) and DwtDetSvd (Cox et al. 2007
These methods encompass both low and high-perturbation watermarks, providing comprehensive evaluation of our approach's 's capabilities. For multi-bit watermarking methods, the specific number of bits used for each method is provided in Table of Appendix AA
Image Watermark Removal Baselines. Our method is compared with two regeneration methods Regen (Zhao et al. (2023a) and Rinse (An al. 2024). Regen employs the diffusion model to regen- erate watermarked images through a process of noising and denoising. Rinse iteratively applies the Regen method multiple times to improve the efficacy of watermark removal. For the experimental results shown in Table we set the noising and denoising steps to 70 for Regen, while Rinse applies the Regen process twice.
Implementation Details. We employ Stable Diffusion-vil (Rombach al. 2022 the backbone for our model, maintaining its parameters in a frozen state to preserve the original capabilities. For the semantic control adapter, we integrate DINOv2-giant (Oquab et al. 2023) as the image encoder, also keeping its parameters frozen to leverage its pre-trained strengths. The training of the semantic control adapter is conducted on 8 NVIDIA A 100 GPUs, and the batch size is set to 8 per GPU. The training of the spatial control network is carried out on 8 NVIDIA A100 GPUs with batch size of 4 per GPU, At the inference stage, we conduct experiments on single NVIDIA RTX 4090.
Evaluation Metries. The watermark removal performance is evaluated using the TPR@ 1%FPR, which calculates the True Positive Rate (TPR) when the False Positive Rate (FPR) constrained
7

---

Published as a conference paper at ICLR 2025
to 1%. It is crucial to ensure low rate of misclassification of the unwatermarked image as water- marked. Moreover, we calculate the average bit accuracy for multi-bit watermark methods, which measures the percentage of conrectly recovered watermark bits. For better comparison, the average bit accuracy and TPR FPPR before (Avg Bit Ace B and T@ 1 1%F B) and after (BitAcc and T@ 1 1%F A) the attack are both presented.
To evaluate visual similarity and quality, we employ two types metrics: full-reference and non- reference assessment. For full-reference assessment, and Peak Signal-to-Noise Ratio (PSNR) to evaluate the visual similarity between watermarked im- ages and regenerated images. For non-reference assessment, we adopt two state-of-art methods, Q- Align (Wu et al. 2023) and LIQE Zhang et al. (2023), to evaluate the quality of regenerated images. Q-align uses large multi-modality models to evaluate image quality by aligning machine-genen generated scores with human judgment by leveraging discrete text-defined levels for scoring. The LIQE uses multitask learning approach, combining scene classification and distortion identification with CLIP- derived embeddings, to assess image quality in non-reference manner. These metrics allow us to comprehensively analyze the image quality both with and without a reference image, ensuring a thorough assessment of the visual outcomes.
CLIP-FID (Kynkäänniemi al. (022)
Table 1: Comprehensive Performance of attacks between our CtrlRegen and other methods.
Watermarks
Attacks
BitAce B
BitAce A
Tē RSISFB. BT
T@1SFAL A CLIP-FID
PSNR
Q-Align t LIQE
1.00
0.64
1.00
0.39
8.91
26.01
3.34
2.82
DwiDerSvd
Rinse
1.00
0.53
1.00
0.11
11.50
23.83
2.95
2.27
CtriRegen
1.00
0.46
0,00
3.63
3.76
Regen
0.55
1.00
0.07
3.26
2.68
RivaGAN
Rinse
1.00
0.50
1.00
0.02
7.39
23.72
2.87
2.11
CtriRegen
1.00
0.48
1.00
0.00
4.24
19.53
3.62
3.69
Regen
0.99
0.68
1.00
0.39
6.06
22.25
2.66
2.54
0.59
0.10
1.93
CtriRegen
0.99
0.56
1.00
0.06
5.64
19.07
3.22
3.15
0.99
0.49
1.00
0.02
1.91
24.16
3.86
3.67
StableSignature
Rinse
0.99
0.47
1.00
0.10
4.15
21.85
3,50
2.98
CtriRegen
0.49
0.02
3.97
4.02
Regen
1.00
0.88
1.00
0.99
22.34
3.06
3.53
StegaStamp
Rinse
1.00
0.77
1.00
0.94
10.73
21.31
2.67
2.71
CtriRegen
1.00
0.49
1.00
0.01
5.27
19.
3.62
3.77
Regen
0.99
0.87
2.84
25.59
4.03
3.96
TreeRing
Rinse
0.61
23.28
3.69
3.29
CtriRegen
0.99
0.12
1.63
19.32
4.17
4.31
4.2
MAIN RESULTS
4.2.1
CTRLREGEN
Watermark Removal Performance CtrlRegen. Table ! presents the watermark detection per- formance across various watermarking methods, both before and after the application of different watermark removal attacks. shows that all watermarking methods have great detection perfor- mance before attacks. For low perturbation watermarking methods such DwtDetSvd, RivaGAN, SSL and StableSignature, the Regen and Rinse exhibit effective watermark removal performance. However, for high perturbation methods like StegaStamp and TreeRing, Regen and Rinse become ineffective. This is because StegaStamp and TreeRing induce significant disturbances in both pixel- space and latent-space. Specifically, Regen's 's noising and denoising process applied to the latent representation of the watermarked image, and the limited number of noising steps is insufficient to completely disrupt the watermark structure embedded within the latent representation. This limi- tation hinders its ability to effectively manage substantial perturbations in the latent space. Rinse extends the approach by running Regen multiple times in an attempt to further disrupt the watermark structure. Despite the enhancement, the effectivvness is still limited, especially for StegaStamp, which still has 0.94 TPR %FPR. Our method regenerates the watermarked image by starting from clean Gaussian noise in the latent space, which inherently contains watermark structure within the latent representation. Therefore, our watermark removal approach achieves notable per- formance, reaching 0.01 and 0.12 TPR @ F&R for StegaStamp and TreeRing, respectively.
8

---

Published as a conference paper at ICLR 2025
Watermarked Image
Regen & Residual
Rinse & Residual
Ours Residual
DwrDDrSSdd
RivaGAN
SSL
StegaStamp StableSignature
TreeRing
Figure 4: Examples different watermark removal attacks on different watermarking methods.
Visual Similarity and Quality of CtrlRegen, The regenerated images are assessed from two key aspects: visual similarity and quality. Table presents these measurements for various watermark removal methods. For visual similarity, CirlRegen achieves a lower CLIP-FID score compared to baselines. This lower score suggests that the distribution of our regenerated images more closely approximates that of the original watermarked images, reflecting better preservation of visual char- acteristics and overall image integrity. CtrlRegen exhibits a lower score for pixel-level measurement, PSNR, compared to baselines. However, it does necessarily indicate severe degradation sim- ilarity. The changes that lead to lower PSNR can actually harmonize well with the content of the image without making the regenerated image appear unnatural. On the contrary, although the Regen and Rinse may achieve higher PSNR scores, it results in visible artifacts and degradation in the regenerated images compared the original watermarked images, as illustrated Figure 4 We provide more discussion and examples (Figure X) for this problem in Appendix [A] To further substantiate that images regenerated by our method exhibit superior image quality, we employ two image quality measurements: Q-Align and LIQE. These metrics are used quantitatively assess and compare the quality of the regenerated images. From Table I our method achieves better image quality compared to Regen and Rinse.
4.2.2 CTRLREGEN+
We evaluate our CtrlRegen+ against Regen across various aspects, including watermark removal effectiveness, visual resemblance, and image quality, by varying the number noising steps. For high-perturbation watermarks, such StegaStamp and TreeRing, (100, 200, 300, 400, 500, 1000) and sample from pure Gaussian noise. We do not set noising step number larger than 500 for Regen, this number of noising steps, the noised latent representation approaches pure Gaussian noise. Consequently, the uncontrolled regeneration process would likely produce image completely unrelated to the original watermarked image.
set the noising steps to be

---

Published as a conference paper at ICLR 2025
27-
StegaStamp
25
3.1
21-
2.9
Ct Ctegee Reges
Ctri Cerine
Rogen
2.7
CoriRegene
Rogen
17
15
2.3
TRRg
TRRgENFRR
TPRgINFP8
4.3
4.3-
25
TreeRing
3.9
19-
3.7-
CorlSegan-
0.0
0.2
0.4
6.6
6.8
1.0
0.0
2
0.4
0.3
1.9
8.8
9.2
6.4
0.8
TPR9INPPR
TPR8INIPR
TPR8INFPR
38-
4.8
DwtDctSvd
25-
0-Align
3.6
27-
3.4
Ctruteeer Regen
Corllggene Ragen
3.2-
Cir Uteggn
Regen
25
0.4
0.6
9.8
0.0
0.2
9.4
9.6
3.8
9.2
0.4
0.8
TPRGTNFA
Performance of CtrlRegen+ compared to Regen with varying number of noising steps on high and low perturbation watermarks, respectively. We invert the CLIP-FID score ensure that the lep-left represents better performance across all figures.
Figure 5:
The first two rows of Figure 5 illustrate the relationship between watermark removal performance and visual similarity/quality across different noising steps for high perturbation watermarks. Com- pared Regen, CtrlRegen+ not only maintains high image quality and similarity but also achieves superior watermark removal performance. Specifically, for Regen, using high number of noising steps can indeed remove the watermark but also lead to a significant degradation the quality of images, shown in Figure of Appendix A It will make the watermark removal meaningless.
For low- now-perturbation watermarks, small number of noising steps is sufficient to remove the water- mark. Therefore, we evaluate the performance of CtrlRegen+ using a range of relatively low noising steps, i.e., {20, 40, 60, 80, 100, 120, 140). As shown in the third row of Figure 5 even with fewer noising steps, our CtrlRegen+ method still demonstrates superior image quality and similarity com- pared to the Regen approach while achieving comparable watermark removal performance. More results of CtrlRegen+ on low perturbation watermarks are presented in Figure of Appendix [A]
Furthermore, we conduct an ablation study detailed in Appendix semantic and spatial control on consistency.
to investigate the impact of
CONCLUSION
this paper, we introduce a controllable regeneration attack that, combined with proposed control techniques, effectively removes image watermarks by thoroughly eliminating the watermark infor- mation in the latent space. Our experiments demonstrate improved visual consistency and image quality compared to existing uncontrolled regeneration attacks under the same removal performance. We note that our attack is a no-box approach, meaning that the attacker only needs one watermarked image remove watermarks without requiring knowledge of the watermarking scheme, detection rules, or access to the detector. By demonstrating the ability to defeat robust watermarking tech- niques, highlight the urgent need develop stronger watermarking solutions that can withstand this attack. In this work, the consistency of regeneration may be constrained to certain extent by the backbone model we employed. In future work, we will explore more advanced backbone models, such as SD-v3 (Esser et al. 2024), and refine our control techniques for further improvement.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
Our primary motivation for developing a more effective watermark removal method to uncover the vulnerabilities in current watermarking techniques. By demonstrating the weaknesses in ex- isting watermarking methods, we aim to underscore the urgent need for more robust and resilient watermarking solutions. Additionally, robustness of future watermarking techniques and fostering their advancement. The proposed watermark removal techniques may pose ethical risks, including potential copyright infringement, undermining digital rights management, and enabling the unauthorized use of protected content. essential that our methods should be used responsibly and in compliance with local reg- ulations, ensuring this work contributes to strengthening, rather than compromising, digital watermarking security.
methods will serve
benchmark for assessing the
ACKNOWLEDGEMENT
The UF team acknowledges UFIT Research Computing for providing computational resources and support that have contributed to the research results reported in this publication. The NUS team is only supported by the Ministry of Education, Singapore, under the Academic Research Fund Tier (FY2023).
REFERENCES
Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, et al. Benchmarking the ro- bustness of image watermarks. arXiv preprint arXiv:2401.08573 2024.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392-18402, 2023.
John Canny. A computational approach edge detection. JEEE Transactions on pattern analysis and machine intelligence, (6):679-69) 698, 1986.
Huili Chen, Bita Darvish Rouhani, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models. In Proceedings of the 2019 on International Conference on Multimedia Retrieval, pp. 105-113, 2019.
Hai Ci, Yiren Song, Pei Yang, Jinheng Xie, and Mike Zheng Shou. Wmadapter: Adding watermark control latent diffusion models. arXiv preprint 24Xx:22006.08337, 2024a.
Hai Ci, Pei Yang, Yiren Song, and Mike Zheng Shou. Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification. arXiv preprint arXiv:2404. 14055, 2024b.
Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. Digital watermark- ing and steganography. Morgan kaufmann, 2007.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
Pierre Fernandez, Alexandre Sablayrolles, Teddy Furon, Hervé Jégou, and Matthijs Douze. Wa- termarking images self-supervised latent spaces. In ICASSP 2022-2022 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp.
IEEE, 2022.
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks latent diffusion models. ICCV, 2023.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint 2rXiv:2208.016688 2022.

---

Published as a conference paper at ICLR 2025
Haiyun He, Yepeng Liu, Ziqiao Wang, Yongyi Mao, and Yuheng Bu. Universally optimal water- marking schemes for llms: from theory to practice. arXiv preprint arXiv:2410.02890, 2024.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01620 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-666 6851, 2020.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL
net wttpiiewwwee/foouummmmo forum?
Id=nZevKeeFYff,
Yuepeng Hu, Zhengyuan Jiang, Moyang Guo, and Neil Gong. A transfer attack to image water- marks. arXiv preprint arXiv:2403. 15365, 2024.
Mark J Huiskes and Michael S Lew. The mir flickr retrieval evaluation. In Proceedings of the 1st ACM international conference on Multimedia information retrieval, pp. 39-43, 2008.
Hengrui Jia, Christopher Choquette- Choo, Varun Chandrasekaran, and Nicolas Papernot. En- tangled watermarks as defense against model extraction. In 30th USENIX security symposium (USENIX Security 21), pp. 1937-1954, 2021.
Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai-generated content. In Proceedings of the 2023 ACM SIGSAC Conference Computer and Communications Security, pp. 1 1168-1181 1181, 2023.
Andre Kassis and Urs Hengartner. Unmarker: A universal attack on defensive watermarking, arXiv preprint arXiv:2405.0830 2024.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pp- 17061-17084. PMLR, 2023.
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun- Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference Com- puter Vision and Pattern Recognition, pp. 1931-1941, 2023.
Tuomas Kynkäänniemi, Tero Karras, Miika Aittala, Timo Aila, and Jaakko Lehtinen. The role of imagenet classes in fr', 'echet inception distance. arXiv preprint arXiv:2203.0600 2022.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Yepeng Liu and Yuheng Bu. Adaptive text watermark for large language models. arXiv preprint
1wX1v:2401..3327. 2024.
Nils Lukas, Abdulrahman Diaa, Lucas Fenaux, and Florian Kerschbaum. Leveraging optimization for adaptive attacks on image watermarks. arXiv preprint arXiv:2309.16955 2023.
Chong Mou, Xintao Wang, Liangbin Xie, Jian Zhang. Zhongang Qi. Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.08455 2023.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa. Alaacldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiy preprint arXiv:2304.07193, 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEFE/CVF International Conference on Computer Vision, pp. 4195 4205, 2023.
12

---

Published as a conference paper at ICLR 2025
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Ahmad Rezaci, Mohammad Akbari, Saeed Ranjbar Alvar, Arezou Fatemi, and Yong Zhang. Lawa: Using latent space for in-generation image watermarking. arXiv preprint arXiv:2408.05868, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. Deepsigns: A generic watermarking framework for ip protection of deep learning models. arXiv preprint
2018.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfrr Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. 2022.
Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. arXiv preprint arXiv:23 2023.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278-25294, 2022.
Jiaming Song. Chenlin Meng. and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical pho- tographs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Ashish Vaswani. Attention is all you need. arXiv preprint arXiv: 2017.
ZijieJ J Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: A large-scale prompt gallery dataset for text-to-image generative models. arXiv preprint arXiv:2210. 14896, 2022.
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-rings watermarks: Invis- ible fingerprints for diffusion images, Advances in Neural Information Processing Systems, 36, 2024.
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Chunyi Li, Liang Liao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guangtai Zhai, and Weisi Lin. Q-align: Teaching Imms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.170996 2023.
Zijin Yang, Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, and Nenghai Yu. Gaussian shad- ing: Provable performance-lossles image watermarking for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for -xt-too-mmae diffusion models. arXiv preprint arXiv:2308.0672.1 2023.
Kevin Alex Zhang, Lei Xu, Alfredo Cufsta-Infante, and Kalyan Veeramachameni. Robust invisible video watermarking with attention. arXiv preprint 1rXhv:1909001285. 2019.
Lijun Zhang, Xiao Liu, Antoni Viros Martin, Cindy Xiong Bearfield, Yuriy Brun, and Hui Guan. Robust image watermarking using stable diffusion. arXiv preprint wrX1v:2401.04247, 2024a.
Lvmin Zhang and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. arXiv preprint arXiv:2302.055 00554, 2023.
13

---

Published as a conference paper at ICLR 2025
Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. Blind image quality assess- ment via lision-anguagge correspondence: A multitask learning perspective. In IEEE Conference Computer Viston and Pattern Recognition, pp. 14071-1408 2023.
Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Sst-encoder: Encoding selective subject representation for subject- driven generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8069-8078, 2024b.
Yuxuan Zhang, Lifu Wei, Qing Zhang, Yiren Song, Jiaming Liu, Huaxia Li, Xu Tang, Yao Hu, and Haibo Zhao. Stable-makeup: When real-world makeup transfer meets diffusion model. arXiv preprint arXiv:2403.07766, 2024c.
Yuxuan Zhang, Qing Zhang, Yiren Song, and Jiaming Liu. Stable-hair: Real-world hair transfer via diffusion model. arXiv preprint 2xXiv:2407.1400) 2024d.
Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai. arXiv preprint arXiv:2306.0195, 2023a.
Yunqing Zhao, Tianyu Pang, Chao Du Xiao Yang, Ngai-Man Cheung, and Min Lin. A recipe for watermarking diffusion models. arXiv preprint crXiv:2303.10137 2023b.
J Zhu. Hidden: hiding data with deep networks. arXiv preprint arXiv: 107,00937,,,0018
14

---

Published as a conference paper at ICLR 2025
A ADDITIONAL EXPERIMENTA RESULTS AND ABLATION STUDY
CtrlRegen+ for Low perturbation watermarks. Figure 6 displays the results of CtrlRegen+ com- pared to Regen across three low- pertuurbbbbainn watermarks: RivaGAN, SSL, and StableSignature. For SSL, the number of noising steps set to {20, 60, 100, 140, 180, 220, 260, 300, 340). From the fig- ure, it is evident that SSL remains relatively robust compared to other low perturbation watermarks when number of noising steps is small. Our CtriRegen+ presents better consistency compared to Regen at the same TPR @ 1 1%FPR Additionally, the quality of images regenerated by Regen deteri- orates with an increase in noising steps, whereas the quality of images from CtrlRegen+ improves. This improvement oceurs because SSL. introduces visible perturbations gradually removed as the number of noising steps increases in the CtrlRegen+ process. For Riva- GAN and StableSignature, the number of noising steps is set to {20, 40, 60, 80, 100, 120, 140). StableSignature is the most vulnerable to regeneration attacks, with even just 20 noising steps being sufficient remove its watermark.
the image, which
38
2.5
29
RivaGAN
28-
3.66
4,8
4,5-
28-
Regine
Regen
5.0
Rerlee Regtn
Rerinegene
Regen
25
5.5
3.3
0.0
TRRgINEPR
TFRgINFFRR
TPRgIaFRR
30
1.0
2.5
3.1
26-
4,0
3.0
SSL
P3W
24
5.5
7.8-
20-
Begrne Regen
8.5-
Etrlilegace Reges
Segena
Segen
18
6.0
0.4
0.6
30.
0.0
8.2
0.4
0.6
0.8
1.0
2.5
0.0
0.2
0.4
0.5
TPRgINFPR
TPRgINFPR
TPRg1NFPR
38
8.0
4.3
StableSignature
28.
8.5
F1D
2386
1.9
24
Ctrlnggen
Ct-LRapess
ctriiflapen
Regen
Regen
Regen
28
0.8
8.2
9.4
5.6
8.8
1.8
2.0
9.8
8.2
9.4
6.6
1.8
3.8
0.0
6.2
0.4
6.6
6.8
FPR
1PRBUMFPR
TPRyINFPR
Figure 6: Performance of CtrlRegen+ compared Regen on the remaining low perturbation water- marks. The noising step number set to (20, 60, 100, 140, 180, 220, 260, 300, 340} for SSL and {20, 40, 60, 80, 100, 120, 140} for RivaGAN and StableSignature.
Examples of CtrlRegent. Figure shows the image regenerated by CtrlRegen+ and Regen at various noising steps. From those examples, we can observe that, the number of noising steps in- creases, images regenerated by Regen become noisy and distorted. Conversely, images regenerated by CurlRegen+ maintain high visual consistency and quality.
Semantic Control and Spatial Control. Our controllable regeneration process comprises two cru- cial components: semantic control and spatial control. The semantic control adapter extracts the semantics from the watermarked image, ensuring the basic semantic integrity of the regenerated im- age, though it may lack detailed accuracy. The spatial control network, on the other hand, manages more detailed aspects of the regeneration process. To demonstrate this, we regenerate the water- marked image using only semantic control and then again using both semantic and spatial controls together, allowing us observe the enhancements in visual similarity and quality. Table 2 displays the corresponding results. It demonstrates that the usage of spatial control not only enhances the consistency of the image but also improves its quality.
15

---

Published as a conference paper at ICLR 2025
Watermarked Image T
200
300
400
Regen
StegaStamp
CtriRegen+
CtriRegen+ Regen
TreeRing
Figure 7: Visual examples of CtrlRegen+ compared to Regen at different numbers of noising steps are illustrated, with the noising step set to { {100, 200, 300, 400, 500]. The image is watermarked using StegaStamp and TreeRing. It shows that our CirlRegen+ preserves high visual consistency and quality, even across various noising steps.
The visual similarity and quality performance between approaches using only semantic control versus those incorporating both semantic and spatial control. The results reveal that semantic control alone provides only -oarse-grained guidance, while incorporating both semantic and spatial controls significantly enhances image consistency and quality across various watermarking methods.
Table 2:
Watermarks
Attack Settings
CLIF-FID
Q-Aliga
Semantic Only Semantic&Sattal
13.06
15.41
2.84
2.55
DwiDetSed
8.68
19.13
3.63
3.76
Semantic Only
9.06
15.43
2.86
2.55
RivaGAN
Semantic&Spataal
4.24
19.53
3.62
3.69
Semantic Only Semantic&Spatial
11.36
15.72
2.38
1.78
SSL
5.64
19.07
3.22
3.15
Semantic Only Semantic&Sppatil
4.04
14.37
3.09
2.71
StableSignature
3.97
StegaStamp
Semantic Only
9.97
15.72
2.78
2.52
Semantic&Spatial
5.27
19.10
3.62
3.77
Semantic Only
TreeRing
Semantic&Spatial
1.63
19.32
4.17
4.34
Inference time of our watermark removal methods. To evaluate the inference time of our re- generation methods, we conducted experiments to compute the average inference time, including the backbone SD model using text as input, CtrlRegen, and CtrlRegen+ with different noising steps (200, 400, 600, 800, 1000). The results are presented in the following table. From Table 3 it is evi- dent that our regeneration method (CtrlRegne, CtrlRegen+ 800, and CtrlRegen+ 1000) introduces a negligible time delay (less than 1 second) compared to the backbone Stable Diffusion model. Addi- tionally, for CtrlRegen+ with noising steps smaller than 600, the inference time even shorter than that of the backbone SD model. This demonstrates the efficiency of our approach.
16

---

Published as a conference paper at ICLR 2025
Table Inference time comparison of backhone SD, CtrlRegen, and CtrlRegen+ with varying nois- ing steps for watermark removal. It highlights the efficiency of our methods.
Setting
Backhone SD CtriRegen CarlRegen+ (200) CtriRegen+ (400) ChriRegen+ (600) CtriRegen+ (800) ChrlRegens (1000)
Generation Timels
1.74
2.56
1.71
2.15
Discussion about CtrlRegen Visual Similarity. In order to further illustrate that CtrlRegen pre- serves the overall content and ensures the perturbation integrates well with the image, provide a visual comparison between CtrlRegen and Regen at the same PSNR level. Our experiments were conducted StegaStamp watermarks, with Regen using noising step of 360, while CtrlRegen generates images starting from clean noise. Under these settings, both methods achieve comparable PSNR values. However, shown in Figure similar PSNR levels, Regen introduces noticeable distortions to the image content, whereas our method maintains strong alignment with the water- marked image. This comparison effectively demonstrates that the relatively lower PSNR of our approach does not compromise the overall integrity of the watermarked image content.
Watermarked
CHINGE WANI BYVRD
Regen
EuR YOO SRIM
Ours
CHINGE YAAM
Figure 8: Visual examples of CtriRegen compared to Regen at similar PSNR levels, where Regen uses a noising step of 360. The image is watermarked using StegaStamp. Regen introduces no- ticeable distortions to the image content, whereas our method maintains strong alignment with the watermarked image. This comparison effectively demonstrates that the relatively lower PSNR of our approach does not compromise the overall integrity of the watermarked image content.
Table 4: Bit number used in our experiments for different watermarking methods.
Watermark
DwiDeiSvd
RivGGAN
SSL
StableSigmature
StegaStamp
Bits
32
32
32
48
96
17

---

Published as a conference paper at ICLR 2025
B ALGORITHM OF PROPOSED METHODS
Algorithm CtrlRegen
Input: Watermarked image Tw..
1: &w f(xw)
2: - In)
3: for t- =T,TT 1,... 1 do
4: & + Iwo Bwos
Tws
5: 2t-1 6-1 2t. St
6: end for
7: I ((0))
Output: Cleaned image ă.
Algorithm CtrlRegen+
Input: Watermarked image Twe, Noising step t*.
1: fw f(x.); 20 E(xw)
2: - (N0 2x- =
var- + vI E
3: for 1- = ....... , do
4: fe + E0(2,, Two Twoy t)
5: 4 bt Et
6: end for
7: Ø ((2))
Output: Cleaned image ã.
18


Reviews:
[{"review_id": "aVF6EtAmgf", "reviewer": "Reviewer_6bHr", "summary": "1. The paper introduces CtrlRegen, a new method for removing image watermarks using controllable diffusion models. The method starts with clean Gaussian noise and regenerates the original image, using a semantic control adapter and spatial control network to guide the denoising process.\n2. The paper also introduces CtrlRegen+ which is another version of CtrlRegen that allows for adjustable watermark removal. This method  starts with noising the latent representation of a watermarked image instead.\n3. Compared to existing approaches, these methods achieve better visual consistency and are able to remove robust watermarks and handle both low and high perturbation watermarks.", "questions": "Please see weaknesses section above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality and Significance:\nThe paper has novel ideas and also creatively uses existing ideas\n1. Novel Idea: The method CtrlRegen is a new method for removing image watermarks using controllable diffusion models, starting from clean Gaussian noise. This enables it to handle high perturbation attacks which the previous methods ex: Regen were unable to.\n2. Creative combinations of existing ideas: ControlNet like spatial control network and semantic control adapter to guide the denoising process, ensuring consistency between the original and cleaned images. \n3. CtrlRegen+ enables an adjustable regeneration scheme - it allows for varying degrees of watermark destruction to counter different watermark strengths.\n\nClarity: The paper clearly explains the main ideas and method, provides context and presents quantitative results and comparisons in a clear manner.\n\nQuality:\n1. Results: The method outperforms previous regeneration approaches (Regen and Rinse) and demonstrates this through quantitative evaluation on both low and high perturbation watermarks. Multiple metrics are used to asses visual similarity and quality such as CLIP-FID, PSNR, Q-Align, LIQE beside evaluations related to watermark detection performance.\n2. Consideration for use cases: The introduction of CtrlRegen+ shows consideration for different use cases and watermark strengths.", "weaknesses": "1. The paper does not outline/show any failure cases, limitations or drawbacks of the method.\n2. The paper does not provide sufficient qualitative comparisons comparing the regeneration methods on diverse images  (ex: images containing text, scenes).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "majDHhxeB5", "reviewer": "Reviewer_sQK5", "summary": "This paper introduces a watermark removal method called CtrlRegen, which regenerates watermarked images starting from clean Gaussian noise using a controllable diffusion model. The method employs a semantic control adapter and a spatial control network to ensure image quality and consistency during the denoising process. Additionally, authors propose an adjustable regeneration scheme to balance watermark removal performance and image fidelity. Experimental results demonstrate that this method offers superior visual consistency and enhanced watermark removal performance compared to existing regeneration approaches.", "questions": "1.Why are the results for CtrlRegen+ not included in Table 1? How does its performance compare to the other methods across all watermarking techniques?\n\n\n2. Meanwhile. why Figure 5 exlude base version of CtrlRegen?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The idea of using a controllable diffusion model for watermark removal is conceptually interesting\n2. Incorporation of semantic and spatial controls is a way to maintain image quality during the process.", "weaknesses": "1. In Table 1, the evaluation is only conducted against two baselines, Regen and Rinse. However, there are other existing watermark removal methods, such as adversarial attacks, editing attacks, or general-purpose methods like Unmarker[1], that are not included in the comparisons. \n\n2. While the paper acknowledges that high-perturbation watermarks (e.g., StegaStamp and TreeRing) are more challenging to remove compared to low-perturbation watermarks, there is a lack of in-depth theoretical analysis or experimental results to explain why these watermarks are harder to remove. Providing more detailed insights into the mechanisms behind the difficulty of removing high-perturbation watermarks would strengthen the paper's argument.\n\n\n3. The paper introduces both semantic and spatial controls to guide the watermark removal process, but it does not provide sufficient analysis. \n\n\n[1] Kassis, Andre, and Urs Hengartner. \"UnMarker: A Universal Attack on Defensive Watermarking.\" arXiv preprint arXiv:2405.08363 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SYpAqxdESh", "reviewer": "Reviewer_EJxb", "summary": "This paper introduces CtrlRegen, a watermark removal method that leverages controllable image regeneration from clean Gaussian noise to effectively remove state-of-the-art image watermarks, even those with high perturbation.  \nCtrlRegen employs a trained semantic control adapter and spatial control network to guide the denoising process using extracted semantic and spatial features from the watermarked image.  \nCtrlRegen+, further enhances the method by adding adjustable noise steps to the latent representation before denoising, allowing for a smoother trade-off between watermark removal and image quality.  \nExperiments across various watermarking techniques demonstrate CtrlRegen's superiority in visual consistency and watermark removal performance compared to existing approaches , particularly for high-perturbation watermarks like StegaStamp and TreeRing.", "questions": "1. What is the model size and inferencing time for the proposed method CtrlRegen and CtrlRegen+? \n2. Compared with the common sophisticated attack on the watermarked image such as VAE or the recently proposed watermark removal schemes in Regen, what is the key advantage of the proposed method?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. CtrlRegen uses clean Gaussian noise as a starting point for denoising in a diffusion model, while existing regeneration attacks that add limited noise to the watermarked image's latent representation\n2. CtrlRegen+ adds an adjustable noising step before denoising, allowing for a trade-off between watermark removal and image quality.  \n3. Experiments show CtrlRegen outperforms existing regeneration methods (Regen and Rinse) in removing high-perturbation watermarks while maintaining better visual quality.\n4. The method is a \"no-box\" attack, requiring only a watermarked image without knowledge of the watermarking scheme.  Ablation studies confirm the importance of both semantic and spatial control in maintaining image consistency.", "weaknesses": "1. Though the method proposes a novel approach to remove the image watermark, the watermark's perturbation is vulnerable to sophisticated attacks such as VAE. Applying VAE can also easily remove the watermark. The difficulty of watermark removal is relatively less challenging, the main contribution lies in the semantic consistency part.\n2. The proposed method relies on regenerating a new clean from noise, while when there may exist large-scale watermarked image in 2K or 4K, marking the proposed method hard to regenerate a large-scale clean image while still keeping high semantic consistency.\n3. The selected baselines ReGen contains theoretical explanations on why the watermarks are provably removable. The proposed method lacks a theoretical analysis of why the regeneration scheme from noise can also be proved to remove the watermark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y5YQX2uG7C", "reviewer": "Reviewer_JmKA", "summary": "This paper introduces a watermark attack method capable of removing watermark embeddings through either low or high perturbations by regenerating the original image from clean gaussian noise. The approach preserves the original content by injecting features from the watermarked image into the regeneration process.", "questions": "Please refer to the Weaknesses section above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* This approach is novel; it begins with Gaussian noise and leverages a controllable removal process that injects the features of the watermarked image during the denoising process to achieve a clean, watermark-free image.\n\n* The experimental results effectively demonstrate this method’s capability in removing watermarks across various watermarking techniques.", "weaknesses": "* The paper lacks an ablation study for each of the modules in the proposed method, specifically the semantic control and spatial control modules. The contribution of each control module to overall performance should be investigated.\n\n* As shown in Table 1, the quality of the resulting images, measured by reference-based metrics, is low, indicating that the method significantly alters the images.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LlcRzSJpCq", "reviewer": "Reviewer_FvFj", "summary": "In this paper, the authors developed a method to remove watermarked embedded in images. They use clear noise, watermarked image embedding, and edges of watermarked images to regenerate the image, such that the image content is reminded but the watermark is removed. More precisely, they used attention mechanism to fuse the watermarked image embedding to the SD UNet and used Spatial-Net to extract edge features and add to the UNet. The authors conducted comprehensive experiments and demonstrated the proposed methods offer stronger removal effect, comparing with previous methods.", "questions": "See the weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1)\tThe presentation is clear, and the paper is easy to follow.\n2)\tThe use to clean noise is a good starting point\n3)\tThe experiments demonstrate that it outperforms the other methods. The scale of the experiments is reasonable.", "weaknesses": "1)\tAlthough the clean noise and edge images are unlikely contained watermark information, I feel that it is possible for it to go through the image encoder and finally, appear in the output image. The authors should discuss more why the watermark information cannot go through the semantic control network. \n2)\tThe authors use two mechanisms to fuse semantic information and spatial information. Why don’t use the same approach, such as attention? Any reason behind of this design. \n3)\tThe authors should revise the motivation. If the aim is to develop a stronger method to break all watermark algorithms, what is good the society and the community? \n4)\tI cannot find out how many bits are used in the watermarking experiments. \n5)\tCan the watermarking algorithms use more bits to increase the protection?\n6)\tAlthough in visual quality indexes, expect for PSNR, the proposed methods are stronger than the baselines. The images for visual comparisons are not enough. The authors should provide more visual comparison. \n7)\tThe authors should provide a scheme to defend the proposed methods. Otherwise, this paper will cause more harm than good to the society. \n8)\tDo the constraints e.g., L2 norm or L infinite norm using in watermark algo development affect the removal capacity?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)