DERO: Diffusion-Model-Erasure Robust Watermarking

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 09:20:46

Abstract: The powerful denoising capability of the latent diffusion model creates new demands on the robustness of image watermarking algorithms, as attackers can erase the watermark by performing a forward diffusion, followed by backward denoising. While such denoising might introduce large distortion in the pixel domain, the image semantics remain similar. Unfortunately, most existing robust watermarking methods fail to tackle such an erasure attack since they are primarily designed for traditional channel distortions. To address such issues, this paper proposed DERO, a diffusion-model-erasure robust watermarking framework. Based on the frequency domain analysis of the diffusion model's denoising process, we designed a destruction and compensation noise layer (DCNL) to approximate the distortion effects caused by latent diffusion model erasure (LDE). In detail, DCNL consists of a multi-scale low-pass filtering and a white noise compensation process, where the high-frequency components of the image are first obliterated, and then full-frequency components are enriched with white noise. Such a process broadly simulates the LDE distortions. Besides, on the extraction side, we cascaded a pre-trained variational autoencoder before the decoder to extract the watermark in the latent domain, which closely adapts to the operation domain of the LDE process. Meanwhile, to improve the robustness of the decoder, we also design a latent feature augmentation (LFA) operation on the latent feature. Throughout the end-to-end training with the DCNL and LFA, DERO can successfully achieve robustness against LDE. Our experimental results demonstrate the effectiveness and the generalizability of the proposed framework. The LDE robustness is significantly improved from 75% with SOTA methods to an impressive 96% with DERO.

Main paper: 
 introduction: 
 Deep learning-based image watermarking mechanisms [1, 5, 14, 15, 18, 19, 26, 33] have garnered significant attention in recent research. These mechanisms typically rely on an architecture known as "Encoder-Noise Layer-Decoder". In this setup, the encoder's purpose is to embed the watermark into the host image, the noise layer introduces deliberate distortions into the image, and the decoder tries to extract the embedded watermark signal from the distorted images. Among these components, the critical factor for achieving robustness lies in the effective design of the noise layer.Many methods have been proposed to address diverse attack scenarios. The majority of them focus on transmission distortions, for example, JPEG-Mask [33] and MBRS [10] tailored for the robustness of JPEG compression; Stegastamp [26] tailored for print-to-camera scenarios, and PIMoG [5] tailored for screen-to-camera robustness.However, beyond transmission distortions, watermarked images may also be subjected to deliberate erasure. Previously, designing a high-quality watermark erase mechanism needed massive "hostwatermarked" image pairs [7, 27] . However, the powerful denoising ability of the latent diffusion model provides feasibility for erasure mechanism design. One can easily leverage a black-box, openly accessible latent diffusion model API (e.g., Stable Diffusion) to eliminate the embedded watermark signal by first adding noise and then conducting the denoising process on the noised latent. Due to the powerful denoising capability of the latent diffusion model, the watermark signal can be efficiently removed from the watermarked image, while maintaining its semantic integrity and high visual quality, as shown in Fig. 1 .Our experimental results show the vulnerability of existing watermarking schemes to diffusion model erasing, including the stateof-the-art (SOTA) deep learning schemes and the classical watermarking schemes, as well as a well-known commercial watermarking system, Digimarc, as shown in Table 1 . We randomly sampled 50 images from COCO datasets [13] and reshaped them to size 512×512 and embedded the watermark with length 256 bits into these images. Then we perform no distortion, Gaussian noise and latent diffusion model erasure (LDE) distortion (with Stable Diffusion v1.5 and denoising strength 0.15) on the watermarked images and test Table 1 : The extraction accuracy with different watermarking mechanisms.HiDDeN [33] MBRS [10] DADW [14] FIN [6] CIN [15] StegaStamp [26] PIMoG [5] LGDR [16] DWTDFT [11] A trivial way to ensure the LDE robustness is by integrating the LDE process directly as a noise layer into the framework's training. However, due to the intricate nature of the diffusion model and the numerous sampling iterations involved, this approach imposes a substantial memory overhead for gradient computation. Consequently, it leads to time-consuming and inefficient training processes.To address such limitations, we propose an alternative approach by introducing an LDE distortion approximation scheme as a noise layer during training. Building upon insights from [29] and our own observations, we note that in the generation process of the latent diffusion model, low-frequency components of an image are initially recovered, followed by gradual refinement of high-frequency components. Leveraging this observation, we designed the destruction and compensation noise layer (DCNL) to emulate the LDE distortions. Specifically, DCNL incorporates multi-scale low-pass filtering and a white noise compensation process, wherein the highfrequency components of the image are initially removed, followed by the addition of white noise to the full-frequency components. Consequently, the distortion introduced by LDE can be effectively approximated.However, relying solely on DCNL may not suffice to guarantee robustness against LDE. Therefore, we introduce a complementary technique called latent feature augmentation (LFA) on the extraction side. Specifically, we utilize a pre-trained variational autoencoder (VAE) to encode the distorted images and apply LFA on the VAE-encoded features. LFA combines VAE-encoded features with Gaussian noise in a randomly weighted manner, which enables the decoder to be trained to extract watermarks from severely distorted latent representations, thereby enhancing the overall robustness of the system against LDE.Based on DCNL and LFA, we proposed DERO, a diffusion-modelerasure robust watermarking framework. Additionally, by configuring different noise layers, the framework demonstrates excellent adaptability to various types of distortions.The main contributions of this paper are summarized as follows:1) We post a potential threat of latent diffusion model-based erasure (LDE) on watermarking systems and underscore the vulnerability of existing watermarking systems to LDE distortions. 2) Based on our analysis of the LDE process, which involves the initial recovery of low-frequency components followed by the gradual refinement of high-frequency components,  
related works 2.1 deep learning-based watermarking: 
 The current deep learning-based watermarking algorithm adopts an "Encoder-Noise Layer-Decoder" architecture, initially proposed by Zhu et al. [33] . By introducing various distortions within the noise layer, targeted robustness can be achieved. An essential requirement for the noise layer is its differentiability, and significant research efforts are directed toward designing differentiable operations that can effectively replicate non-differentiable distortions. For example, to ensure robustness against JPEG compression, Zhu et al. [33] introduced a differentiable operation called JPEG-Mask. Building upon this work, Jia et al. [10] developed a mini-batch-based noise layer to enhance JPEG robustness. In addition to addressing image editing distortions, some noise layers are tailored to physical distortions. To ensure print-shooting robustness, Tancik et al. [26] proposed a noise layer comprising several differentiable operations to replicate printshooting distortions. Similarly, Fang et al. [5] introduced PIMoG, a noise layer designed for screen-shooting robustness. Furthermore, rather than utilizing specific operations, Luo et al. [14] suggested employing an adversarial network as the noise layer to generate the worst-case distorted image and iteratively train for improving robustness. As for the unknown distortions, a combined noise layer that contains multiple distortions is often employed [15] . However, the design of existing noise layers has predominantly concentrated on mitigating transmission distortions. In the case of LDE, none of these methods can be successfully applied.Diffusion models as well as the latent diffusion models [9, 20, 22, 24] have demonstrated remarkable performance in generative tasks, fitting applications across various computer vision domains, including inpainting, super-resolution, and even text-to-image generation [2, 4, 23] . Thanks to the exceptional sampling quality, diffusion models have emerged as the leading architecture for generation tasks nowadays. In general, diffusion models encompass two primary processes: (1) the forward diffusion process, which introduces Gaussian noise on the image to perturb the input into random noise, and (2) the reverse process, which progressively transforms the random noise back into high-quality images through a step-by-step denoising procedure. The latent diffusion model typically uses variational autoencoder VAE to encode the image into latent features, and the processes of forward-diffusion and backward-denoising occur in the latent domain. The effectiveness of high-quality denoising in the reverse process has led to a series of intriguing developments in adversarial purifications [21] . Moreover, some advancements in the certifiable robustness of adversarial attacks are built upon the analysis of diffusion models [30] . It should be noted that the purification process is similar to watermark erasure, where the diffusion model is adept at erasing unexpected perturbations effectively. Such a capability poses a potential threat to the watermarking systems.Recently, many diffusion model-based watermarking schemes have been proposed [17, 25, 28, 31] . These methods typically follow a common procedure: given an image and its watermark, they first extract the diffused latent features of the image and then embed the watermark either within the latent features or during the denoising process. However, these approaches often fail to meet the visual quality requirements in the pixel domain for image watermarking. This limitation arises from the fact that latent domain embedding tends to preserve only semantic consistency, potentially leading to significant visual distortion in the pixel domain. 
background 3.1 denoising diffusion models: 
 The denoising diffusion model (DDM) reverses a progressive noise process. Given the image sampled from the real distribution 𝑥 ∈ 𝑋 , it is noised with𝑇 steps by Gaussian noise 𝜖 ∼ 𝑁 (0, 1) with different schedules, shown as Eq. ( 1)EQUATIONwhere 𝛼 𝑡 is a set of monotonic increasing scheduled timesteps{𝛼 𝑡 } 𝑇 𝑡 =0 , 𝛼 𝑇 = 0, 𝛼 0 = 1.The DDM 𝜖 𝜃 is trained with the loss function 𝑀𝑆𝐸 (𝜖 𝜃 (𝑥 𝑡 , 𝑡), 𝜖) to predict the added noise on each timesteps. At the backward denoising process, given a noise vector 𝑥 𝑇 , the noise is progressively diminished through sequential predictions made by 𝜖 𝜃 over 𝑇 steps. The most common sampling scheme is DDIM [24] where intermediate steps are calculated as:EQUATIONIn the current text-guided diffusion models, DDM 𝜖 𝜃 also contains another input which is text condition 𝐶 to generate the image corresponding to the given conditioning prompt, i.e., 𝜖 𝜃 (𝑥 𝑡 , 𝑡, 𝐶).For the latent diffusion model [22] , the image is first encoded by a pre-trained VAE and all the diffusion and denoising operations are performed on the latent features 𝑧 𝑡 = E 𝑉 𝐴𝐸 (𝑥 𝑡 ). Here we described the latent diffusion model erasure process. For the watermarked images 𝑥 𝑤 , the attacker can first encode with the VAE encoder of the LDM to get the latent features 𝑧 𝑤 = E 𝑉 𝐴𝐸 (𝑥 𝑤 ).Then Gaussian noise of zero mean and certain variance ε is added to 𝑧 𝑤 to get the noisy latent 𝑧 𝑤 𝑁 = 𝑧 𝑤 + ε. Subsequentially, 𝑧 𝑤 𝑁 is treated as the starting point and is denoised by LDM 𝜖 𝜃 with T steps. The denoised latent 𝑧 w is then decoded by VAE decoder to get the erased image 𝑥 𝑒 = D 𝑉 𝐴𝐸 (𝑧 w ). Such an erasure process is easy to automatically deploy by choosing "img2img" functions in the "WebUI" of Stable Diffusion1 , details can be found in the supplementary materials. 
proposed framework 4.1 motivation and analysis: 
 The key to ensuring LDE robustness is designing a noise layer capable of accurately representing the LDE distortion. To achieve this objective, we begin by analyzing the denoising process of the latent diffusion model from a frequency domain perspective. We record the generation process of the latent diffusion model with different timesteps in both spatial domain and Fourier domain, as shown in Fig. 2 . It is clear that with the time steps, more details of the generated image are refined in the spatial domain. From the Fourier perspective, low-frequency components are first generated, then high-frequency components are gradually completed. Meanwhile, during the refinement of the high-frequency component, changes are also observed in the low-frequency part. Based on the observation, we design the destruction and compensation noise layer, which characterizes the LDE distortion through a model involving low-pass filtering followed by full-band component compensation. Nevertheless, such an approximation is not sufficient to model all the distortions generated by the LDE. To address this limitation, we additionally perform latent feature-level augmentation to introduce further distortions in training the decoder.The framework of the proposed DERO is shown in Fig. 3 . In the training stage, the host image 𝐼 ℎ ∈ R 𝐶 ×𝐻 ×𝑊 and the watermark 𝑤 ∈ {0, 1} are first fed into the encoder E which outputs the watermarked image 𝐼 𝑤 ∈ R 𝐶 ×𝐻 ×𝑊 . Then 𝐼 𝑤 is distorted by DCNL to get the distorted image 𝐼 𝑑 ∈ R 𝐶 ×𝐻 ×𝑊 . Subsequentially, 𝐼 𝑑 is first encoded by a pre-trained VAE encoder E 𝑉 𝐴𝐸 to get the latent feature 𝐿 𝑑 ∈ R 𝐶 ×𝐻 /8×𝑊 /8 , which is further augmented by LFA to The encoder E aims to embed the watermark into the host image. In this paper, we utilize a UNet-based architecture as the backbone of E. Specifically, four "double conv" blocks (the concatenation of two "Conv-BN-ReLU" blocks) are first applied to downsample 𝐼 ℎ to the feature with size of 𝐻 /32 × 𝑊 /32. At the same time, one linear layer is applied to the watermark message to generate the watermark feature with the same size of 𝐻 /32 ×𝑊 /32. Then four "up-conv" blocks (the concatenation of two "UpSampling-Conv-BN-ReLU" blocks) are applied to each hidden feature of E, which is further concatenated by the upsampled watermark features. The UNet-based architecture effectively facilitates the fusion of the watermark feature with the host image features across different scales, thereby enhancing the quality of the watermarked images.The purpose of DCNL is to introduce a similar distortion as LDE to the watermarked images. Drawing from our observations of LDM generation, we devised a mechanism for destroying high-frequency components and compensating for full-frequency components. In detail, after obtaining the watermarked image 𝐼 𝑤 , we first down-sample it to generate the images with different scales I, as shown in Eq. (3) .EQUATIONwhere D𝑜𝑤𝑛 indicates the down-sampling operation and 𝑠 is the scale of the down-sampling. I = {𝐼 0 𝑤 , 𝐼 1 𝑤 , 𝐼 2 𝑤 , ...}. For each 𝐼 𝑠 𝑤 , we apply discrete Fourier transformation (DFT) with Eq. ( 4).EQUATIONwhere 𝐼 𝑠 𝑤 (𝑢, 𝑣) is the pixel value at (𝑢, 𝑣) in spatial domain and 𝐹 𝑠 𝑤 (𝑈 , 𝑉 ) is the complex value of (𝑈 , 𝑉 ) in frequency domain. Then a low-pass filtering process is performed on each Fourier coefficients 𝐹 𝑠 𝑤 with Eq. ( 5).EQUATIONwhereEQUATION𝐷 (𝑢, 𝑣) = 𝑢 2 + 𝑣 2 , 𝐷 𝑠 indicates the cutoff frequency of scale 𝑠. So in total, we can get multiple low-pass filtered frequency components with different scales F 𝐷𝑒 = {𝐹 0 𝐷𝑒 , 𝐹 1 𝐷𝑒 , 𝐹 2 𝐷𝑒 ...}. After that, an upsampling operation with scale 𝑠 is first applied on F 𝐷𝑒 , then white Gaussian noises 𝜉 with variance 𝜎 are added into each upsampled frequency component to get the compensated Fourier coefficientsEQUATIONGiven that the spectral density of white noise remains constant, implying its energy presence across the full band, we can effectively simulate the full band changing by adding white noise in the frequency domain. Then, by applying the inverse DFT on F 𝐶 , we can get the compensated images I 𝐶 . pre-trained VAE encoder E 𝑉 𝐴𝐸 to get the latent representation 𝐿 𝑑 .It is important to note that DCNL only serves as a rough simulation of the distortion introduced by LDE, which may not be sufficient to ensure robustness against LDE. To further enhance robustness, we apply augmentations on 𝐿 𝑑 . Specifically, 𝐿 𝑑 is augmented through a weighted noising operation, as depicted in Eq. (8) .EQUATIONwhere 𝜂 ∼ 𝑁 (0, 1) and is with the same size of 𝐿 𝑑 . 𝛼 and 𝛽 indicates the weights. By performing LFA, the latent feature is further distorted, thereby presenting more stringent samples to enhance the extraction capability of the decoder. In this paper, 𝛼 sampled uniformly from [0.3,0.5] and 𝛽 sampled uniformly from [0.5,0.8].The decoder D is designed to extract the watermark 𝑤 𝑒𝑥 from the augmented latent features 𝐿 𝑎 , which consists of one "single-conv" ("Conv-BN-ReLU") block, three "Res-Block" [8] , one "Conv" block and one linear block. The downsampling operation is carried out in the "Res-Block".Two main loss functions are applied in training the whole framework, one is the image loss L 𝐼 which constrains the visual similarity of the watermarked image and the host image:L 𝐼 = ∥𝐼 ℎ -𝐼 𝑤 ∥ 2 = ∥𝐼 ℎ -E(𝐼 ℎ , 𝑤)∥ 2 (9)Another loss function is the message L 𝑀 which ensure the extraction accuracy of the watermark:EQUATIONSince all the procedures in DCNL and LFA are differentiable, the whole network can be trained end-to-end with L 𝐼 and L 𝑀 in the following manner:EQUATIONwhere 𝜆 𝐼 and 𝜆 𝑀 are the weight of the loss functions. The length of the watermark message is set to 256 bits. The pretrained VAE encoder E 𝑉 𝐴𝐸 employed in this paper is the default VAE provided with Stable Diffusion v1.5. The entire framework is implemented using PyTorch [3] and executed on a Tesla V100 GPU. Adam [12] is utilized for parameter optimization. 
baseline and benchmark.: 
 For the latent diffusion model erasure process, we utilize the "img2img" function in open-sourced Stable Diffusion web UI (SD-webUI). The specific procedure entails embedding the watermark message into the host image, followed by applying SD-webUI to erase the watermark. Upon erasure, we extract the watermark message from the erased image and record the extraction accuracy. The version of Stable Diffusion we use is v1.52 . The default image size is 512×512 with 30 sampling steps and a guidance scale of 7. All the comparison experiments are performed with these settings. To demonstrate the effectiveness of the proposed method, we compare its performance with seven state-ofthe-art (SOTA) methods: HiDDeN [33] , DADW [14] , StegaStamp [26] , MBRS [10] , CIN [15] , PIMoG [5] and FIN [6] . For HiDDeN, MBRS, FIN, and CIN, they were trained with the combined noise of JPEG compression, median filtering and Gaussian noise.We evaluated the performance in two main aspects: Visual Quality of Watermarked Images: Measured by the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) (where a larger value represents better visual quality), and the LPIPS [32] , which is a learned perceptual similarity metric (a lower value indicates better visual quality).Robustness Against LDE Distortion: Assessed by the extraction accuracy of the watermark message after LDE distortion. Higher accuracy indicates stronger robustness.We provide one example to subjectively evaluate the visual quality of the watermarked image generated with different methods, as shown in Fig. 4 . Additionally, we provide objective evaluations of visual quality through metrics (PSNR, SSIM, and LPIPS), as detailed in Table 2 . In Fig. 4 , each column represents a different method. The first row, second row, third row, and bottom row correspond to the host images, watermarked images, erased images, and residuals, respectively. From the objective assessment results in Table 2 , we can see that the proposed method maintains a high level of visual quality, where the PSNR value and SSIM value are higher than 40 dB and 0.98, respectively. Moreover, compared to the state-of-theart (SOTA) methods, the proposed scheme achieves the best visual quality. All subsequent experiments assessing LDE robustness are conducted based on these visual quality results.To comprehensively assess the robustness of LDE distortion, we utilized SD-webUI with varying denoising strengths (0.1/0.15/0.2) and different sampling methods (Eular, LMS, and DPM adaptive) for the diffusion erasure process. The robustness test results are presented in Table 3 .As shown in Table 3 , it is evident that for all the compared methods, the latent diffusion model can successfully erase the watermark signal, even at a denoising strength of 0.1. Among them, the highest extraction rate is achieved by StegaStamp, reaching 80.89%. However, the proposed method consistently achieves high extraction accuracy, surpassing 97% at a denoising strength of 0.1. This significant performance gap underscores the superiority of the proposed framework. In addition, with regard to denoising strength, it is observed that as the strength increases, the watermark extraction accuracy decreases. This is because larger denoising strengths will result in stronger noise added to the watermarked latent, and consequently, a more pronounced erasure of the watermarked signal. Different sampling methods yield varied removal results, leading to distinct effects on the watermarked images. Notably, the proposed method demonstrates applicability across all tested sampling methods. Irrespective of denoising strengths and sampling methods, the proposed method consistently exhibits the highest performance, with an extraction accuracy exceeding 94%, at least 17% higher than the compared methods. This outcome strongly underscores the effectiveness of the proposed noise layer and the robustness of the proposed method against LDE distortion.5.4.1 Different LDM Versions. The advancement of diffusion model technology has led to the development of numerous fine-tuned models. Therefore, it is crucial to ensure LDE robustness across different versions. In this section, we primarily assess the generalizability of LDE robustness concerning model versions. Specifically, we gather seven different and commonly used versions (Stable Diffusion v1.2, v1.4, v1.5, v2.1, Deliberate (Deli), Dreamshaper (DS), and Realis-ticVision (RV)) for testing purposes. Subsequently, we conduct the LDE process with each of these versions. The visual results of the erased images (with sampling method "Eular" and strength 0.15) are presented in Fig. 5 , and the corresponding PSNR and extraction accuracy are summarized in Table 4 .It is clear from Fig. 5 that the diffusion model with different versions will result in different erased images, primarily differing in details. Notably, when utilizing the "Dreamshaper" model, the color contrast becomes sharper, as observed in the zoom-in patches of the images. However, from the perspective of overall image quality, all models effectively maintain visual consistency with the watermarked image. This conclusion is further supported by Table 4 , where the PSNR values of erased images with different versions of LDM are found to be at similar levels. Regarding LDE robustness, the proposed scheme consistently achieves a high extraction accuracy of at least 96%, demonstrating its effectiveness and resilience in the face of LDE distortions across different versions of the diffusion model. This underscores the robustness and reliability of the proposed approach. 5.4.2 Different Sampling Settings. In the diffusion process, the denoising result is affected by two main parameters: the sampling steps and the denoising strength. Different sampling steps will lead to different schedules of timestep, which will further influence the sampling weight 𝛼 𝑡 in each step. Different denoising strength gives different starting points to the latent sampling. Higher denoising strengths result in deeper denoising. In this section, we evaluate LDE robustness under different conditions by varying sampling steps and denoising strengths. We fixed the denoising strength at 0.15 and varied the sampling steps from 20 to 50 to conduct the test. Additionally, we fixed the sampling steps at 30 and tested with 2024-04-13 01:13. Page 6 of 1-9. denoising strengths ranging from 0.05 to 0.35. The visual results of the erased images are presented in Fig. 6 . Furthermore, we showcase the corresponding PSNR of the erased images and extraction accuracy with different erasure settings in Fig. 7 . From Fig. 6 , it is apparent that the most influential parameter is the denoising strength. As the denoising strength increases, there is a significant alteration in the details of the images. On the other hand, variations in the sampling steps do not significantly affect the erasure performance of the images. Since sampling steps only impact the sampling schedules and not the starting point, they do not produce substantial differences in the final image. These observations are also reflected in the results of PSNR values and extraction accuracy, as shown in Fig. 7 . When fixing the denoising strength and increasing the sampling steps, the PSNR of the erased images remains consistent, as does the extraction accuracy. However, when fixing the sampling steps and increasing the denoising strength, the visual quality of the erased image deteriorates, evidenced by a decrease in PSNR. Similarly, the extraction accuracy also decreases with increasing denoising strength. It is worth noting that even when the denoising strength is set to 0.35, resulting in heavily altered erased images, the extraction accuracy remains higher than 91%. This indicates the strong robustness of the proposed framework against LDE distortions.In latent diffusion model generation, to produce an image that aligns with conditioning 𝐶 to the desired extent, the model must exhibit a stronger bias toward generating outputs that are in harmony with 𝐶. This bias is typically accomplished by adding weights to the unconditional prediction 𝜖 𝜃 (𝑥 𝑡 , 𝑡, ∅). Thus, the final generation process is expressed as:Φ 𝜃 (𝑥 𝑡 , 𝑡, 𝐶, 𝐺) = 𝜖 𝜃 (𝑥 𝑡 , 𝑡, ∅) + 𝐺 × (𝜖 𝜃 (𝑥 𝑡 , 𝑡, 𝐶) -𝜖 𝜃 (𝑥 𝑡 , 𝑡, ∅))where 𝐺 is the weight called the guidance scale. Different 𝐺 will result in different generation effects. In this section, we test the LDE robustness of the algorithm under different settings of 𝐺 = {1, 3, 5, 7, 9, 11, 13}. The test model version is v1.5, with fixed sampling steps at 30, denoising strength at 0.15, and the sampling method "Eular". The results are presented in Fig. 8 and Table 5 . It can be seen that the changing of the guidance scale will not greatly influence the visual quality of the erased images. Even with guidance 𝐺 = 11, the images are still similar to those with guidance 𝐺 = 1, where the PSNR value with different guidance scales is at the same level. This phenomenon occurs because the LDE process adds only a small amount of noise to the original latent as the starting point, rather than generating from random noise. Additionally, the corresponding timestep weights are not very large. Consequently, the guidance scale does not have a substantial impact on the generated image. Furthermore, from Table 5 , it is apparent that for all tested guidance scales, the proposed method achieves high extraction accuracy exceeding 98%. This demonstrates the robustness of the algorithm across different guidance scales.It is crucial to consider that in real-world scenarios, watermarked images may encounter various distortions beyond just LDE. Therefore, assessing the robustness compatibility of the proposed framework to these different distortions is essential. In this section, we conduct experiments to evaluate the framework's compatibility with various distortions. Following the common settings of stateof-the-art methods [6, 15] , we train the entire framework with a combined noise layer containing Gaussian noise, JPEG compression, Dropout and Gaussian blur, named DERO++. Subsequently, we test the corresponding robustness with DERO and DERO++. The results of these experiments are presented in Table 6 . When only with DERO, the robustness with certain distortions is not strong enough. Taking "Dropout" distortion as an example, the extraction accuracy of DERO against "Dropout-0.08" is only 84.06%. But with combined noise layer training, it achieves 94.18%. Remarkably, the extraction results under "Median Blur", "Salt & Pepper Noise" demonstrate the generalizability of the proposed method. These positive results underscore the effectiveness of the designed DCNL in cooperation with traditional noise layers.5.6.1 Importance of DCNL and LFA. The crucial component for the LDE robustness of the proposed method is the design of DCNL and LFA. To demonstrate the importance of each, we conduct ablation experiments. We train the whole model with only DCNL and LFA individually. Additionally, we attempt a trivial approach to simulate LDE distortion by generating massive "original-distortion" image pairs and training a neural network to mimic the distortion. This neural network can also serve as the noise layer for training. Specifically, we generate 5000 training pairs using MS COCO training datasets [13] (Stable Diffusion v1.5, sampling steps 30, denoising strength 0.15, guidance scale 7, sampling method "Eular"). Then, we utilize the network architecture proposed in [34] to train the surrogate model for LDE distortions and train it with the well-trained distortion simulation model. Finally, we test the LDE robustness with these three models. The results are shown in Table 7 . For a fair comparison, the PSNR of the watermarked images trained with different settings is set at the same level as 40.5 ± 0.2 dB. It can be seen that when training with the surrogate model, the extraction accuracy can only achieve 81%, which is 15% lower than the proposed methods. We believe the reason is that a simple network cannot well simulate the LDE distortion since the LDE process is complex. On the other hand, training only with DCNL and LFA ensures a certain level of robustness, with an extraction accuracy of up to 95%. Both DCNL and LFA simulate the LDE distortion to some extent, contributing to this level of robustness. However, training with both DCNL and LFA achieves the best extraction accuracy, surpassing the other settings by 2%. This indicates that the combination of DCNL and LFA synergistically enhances the model's ability to LDE robustness. 
conclusion: 
 This paper points out a potential vulnerability in watermarking systems, revealing their susceptibility to high-quality erasure using the latent diffusion model. While existing schemes mainly address image processing distortions, they often fall short in adapting to latent diffusion erasure distortion. To fortify robustness, we first analyze the denoising process of LDM in the frequency domain. Leveraging the observation of the generation in frequency components, we successfully provide a destruction and compensation noise layer, which gives a rough simulation of LDE distortions. Combined with a VAE-based decoder and a latent feature augmentation operation, the whole system can be trained to guarantee LDE robustness. Experimental results affirm the method's efficacy, demonstrating superior robustness against latent diffusion erasure compared to state-of-the-art methods.  
: 
 Figure 1 : The process and influence of LDE on watermarked images, evaluated with StegaStamp [26] . After LDE, the watermark cannot be extracted accurately. 
 
 Appendix: https://github.com/AUTOMATIC1111/stable-diffusion-webui2024-04-13 01:13. Page 3 of 1-9.https://huggingface.co/runwayml/stable-diffusion-v1-5 2024-04-13 01:13. Page 5 of 1-9. 


Decision: This manuscript is well-written and provides convincing evaluations.
The unclear parts also have been clarified during the rebuttal process.
ALL reviewers consistently gave positive comments on this paper.
Thus, the AC recommends to accept this manuscript.

Reviews: review: paper_summary: paper_summary: The paper proposes a robust watermarking method which grants resilience against diffusion model erasure.
strength_weakness: strength_weakness: The paper is well structured and presented.
It addresses the issue of robustness in deep learning-based watermarking.
Performances seems to improve the sota methods in terms both of visual quality and watermark extraction accuracy.
limitations: limitations: Some experimental details are missing and an analysis on the various issues of the procedure would be needed as evidenced by the major comments hereafter:

1) It is not so clear if the method can maintain the same performance when the watermarked image does not belong to the same dataset used for training (MS COCO in this case), could the authors discuss such an issue? This is crucial because it determines the generalization capability of the proposed procedure.

2) Figure 4, the row "residual": it is not understandable how residual image is computed. Is it computed as difference between the host image and the watermarked one or between the watermarked and erased one? 

3) There is nothing concerning the impact of the watermark message: which is the length of the watermark? How many bits? What about information capacity?

4) The sota methods: have they been retrained or taken as they were? What about the set-up for the comparison?
paper_summary: paper_summary: This paper proposes a watermarking framework to improve the robustness of watermarking to diffusion models, called DERO. DERO incorporates a Destruction and Compensation Noise Layer (DCNL) to simulate the distortion effects of the Latent Diffusion Model Erasure (LDE) and a Latent Feature Augmentation (LFA) operation to enhance the robustness of the watermark extraction process.
strength_weakness: strength_weakness: This manuscript proposes an interesting watermarking scheme to resist the diffusion-model-erasure attack. The paper is well-structured, and the proposed scheme is clearly described. In the experiments, the proposed method demonstrates superior visual quality and robustness under LDE distortions.
limitations: limitations: The experimental evaluation could be further improved.
1.  The article considers diffusion and denoising processes in LDE attack. As far as I know, this is not a common attack. However, image editing based on diffusion models is more commonly used, such as image inpainting and outpainting[1].
2. Although the proposal is mainly used to improve the robustness of LDE, common attacks should also be considered, such as geometric attacks (resize, crop, rotation, flip) [2] [3].
3. The method only compares the robustness with other methods on the LDE attack. In Table 6, the proposed method is not compared with previous methods in terms of more common distortion attacks.

[1] Fernandez, Pierre, et al. "The stable signature: Rooting watermarks in latent diffusion models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[2] Ma, Rui, et al. "Towards blind watermarking: Combining invertible and non-invertible mechanisms." Proceedings of the 30th ACM International Conference on Multimedia. 2022.

[3] Guo, Hengchang, et al. "Practical Deep Dispersed Watermarking with Synchronization and Fusion." Proceedings of the 31st ACM International Conference on Multimedia. 2023.
paper_summary: paper_summary: This paper addresses the challenge of image watermark robustness in the face of advanced denoising attacks using latent diffusion models (LDM). Traditional watermarking methods are ineffective against such attacks, where attackers can effectively remove watermarks through forward diffusion followed by backward denoising. This paper proposes a diffusion-model-erasure robust watermarking framework called DERO. DERO simulates the distortions caused by LDM erosion using frequency domain analysis to design a destruction and compensation noise layer (DCNL). Additionally, a pre-trained variational autoencoder is cascaded before the decoder to extract the watermark in the latent domain, enhancing the system's robustness. DERO significantly improves robustness against LDM erosion, with experimental results showing an increase in robustness from 75% to 96% on the MS COCO dataset compared to state-of-the-art methods.
strength_weakness: strength_weakness: **Strengths:**

1. First of all, this paper is well-written and structured clearly, making it easy for anyone to read and understand.
2. The design of the DCNL part is very interesting. It cleverly uses high- and low-frequency information to simulate LDM attacks.
3. Overall, this paper provides a high-quality watermark embedding method that resists diffusion erasure without compromising image quality (even slightly improving it), offering a new and effective way for copyright protection.
4. The overall experimental design is very reasonable. The visual quality experiments demonstrate that the method does not degrade the quality of the embedded watermark. The robustness against LDE experiments proves the method's effectiveness in countering diffusion erasure. The ablation experiments show the importance of the proposed DCNL and LFA methods.
5. The supplementary materials provide more visual results, further proving the superiority of the method.
6. In this paper, the authors just use binary messages as the watermark. Is there any other option? What are the challenges or benefits?
limitations: limitations: **Weaknesses:**

1. My biggest concern comes from the comparison with state-of-the-art methods. The paper [ZoDiac: Robust Image Watermarking using Stable Diffusion](https://arxiv.org/abs/2401.04247v1) undertakes almost the same task as the authors and claims to be the first work to counteract diffusion watermark removal, showing seemingly better results. I suggest the authors compare their method with ZoDiac under fair settings and update the results accordingly.

2. Another concern is that the authors conducted experiments only on the [MS-COCO](https://arxiv.org/abs/1405.0312) dataset. I recommend that the authors also perform the same experiments on other datasets such as [DiffusionDB](https://arxiv.org/abs/2210.14896) and [WikiArt](https://arxiv.org/abs/1505.00855v1).

3. The authors mentioned the GPU used in their experiments but did not provide details on computational complexity, training time, and inference time.

4. The diffusion models used in this paper are mainly applied to generation tasks. Have the authors tried adversarial experiments against diffusion models specifically designed for watermark attacks, such as [Evading Watermark-based Detection of AI-Generated Content](https://arxiv.org/abs/2305.03807) and [Invisible Image Watermarks Are Provably Removable Using Generative AI](https://arxiv.org/abs/2306.01953)?

5. Just curious, could the authors explain why the watermark image quality generated by DERO is higher than that of other watermarking methods? Which part of the process enhances the quality of the embedded watermark?

ZoDiac can be considered concurrent work to DERO, so it is understandable that a comparison was not made. Given the high quality of the authors' work, I am giving a score of Borderline Accept. If the authors can supplement their work with more experiments, I promise to increase the score. However, if my concerns are not addressed, I will lower the score.