FreqMark: Invisible Image Watermarking via Frequency Based Optimization in Latent Space
Authors: YiYang Guo
Ruizhe Li
Mude Hui
Hanzhong Allan Guo
Chen Zhang
Chuangjian Cai
Le Wan
Shangfei Wang
Venue: NeurIPS 2024 poster (2024)
Keywords: deep watermarking
latent frequency optimization
Primary area: privacy
TL;DR: We propose a novel method for invisible watermark by optimizing the latent frequency space of images, named FreqMark, providing remarkable robustness and flexibility.

Abstract: Invisible watermarking is essential for safeguarding digital content, enabling copyright protection and content authentication. 
However, existing watermarking methods fall short in robustness against regeneration attacks.
In this paper, we propose a novel method called FreqMark that involves unconstrained optimization of the image latent frequency space obtained after VAE encoding. Specifically, FreqMark embeds the watermark by optimizing the latent frequency space of the images and then extracts the watermark through a pre-trained image encoder. This optimization allows a flexible trade-off between image quality with watermark robustness and effectively resists regeneration attacks.
Experimental results demonstrate that FreqMark offers significant advantages in image quality and robustness, permits flexible selection of the encoding bit number, and achieves a bit accuracy exceeding 90\% when encoding a 48-bit hidden message under various attack scenarios.

Farq Invisible Image Watermarking via Frequency Based Optimization in Latent Space
Yiyang G,ot, Ruizhe Li Mude Hui Hanzhong Guo', Chen Zhang' Chuangjaan Cai Le Wan Shangfei Wang!!
University of Science and Technology of China FFudan University University California, Santa Cruz "The University of Hong Kong SIEG, Tencent (guoyiyang, zhangchenzc) @mail ustc. edu. en; 1irz228m. fudan, edu, en; muhui @ucsc. edu; hanzhong@connect. hku. hk; {herbertcai, vinowan)@tencentt com; sfwangdustc. edu. cn
Abstract
Invisible watermarking is essential for safeguarding digital content, enabling copy- right protection and content authentication. However, existing watermarking meth- ods fall short in robustness against regeneration attacks. In this paper, we propose novel method called FreqMark that involves unconstrained optimization of the image latent frequency space obtained after VAE encoding. Specifically, FreqMark embeds the watermark by optimizing the latent frequency space of the images and then extracts the watermark through pre- pre-trained image encoder. This optimiza- tion allows a flexible trade-off between image quality with watermark robustness and effectively resists regeneration attacks. Experimental results demonstrate that FreqMark offers significant advantages in image quality and robustness, permits flexible selection of the encoding bit number, and achieves a bit accuracy exceeding 90% when encoding a 48-bit hidden message under various attack scenarios.
Introduction
As the development of generative models [42, 32], distinguishing between Al Al-generated and real images becomes increasingly challenging, which brings new risks such as deepfakes and copyright infringement 31J. By adding invisible watermarks within images, the concealed message be tracked and used for purposes such as copyright verification, identity authentication, copy control, etc., thereby safeguarding against the misuse of image content.
Traditional methods [12, 15, 14, 36, 15] conceal hidden messages in the frequency space of images, providing resistance to Gaussian noise attacks but susceptibility to brightness, contrast, and regenera- tion attacks. The rapid development of deep learning has also promoted the iteration of watermarking techniques with enhanced robustness numerous attacks. A common approach is train water- mark embedding network through supervised learning to introduce imperceptible perturbations to images, and then to retrieve the hidden messages through decoding network [46, 10, 55, 3]. An alternative method sets the optimization objective as the image itself, utilizing pre-trained neural networks calculate the perturbations to be added, thereby bypassing the higher cost of training networks and providing a more flexible approach [29, 20). However, with the continued advancement of generative models, leveraging their generalization capabilities to denoise watermarked images through regeneration attacks has proven to be an effective method for watermark removal [58, 43]. logical step is to encode the watermark in the image latent space, allowing the watermark to
'Equal Contribution. Work is done during an internship IEG, Tencent. *Corresponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Afice!
Alice!
Alice!
Watemark Message Encoding in Frequeacy Space
Watemmrk Message Enceding in Image Space
Watermark Message Encoding in Latent Frequeacy Space
FFT
FFT
Gaussian
Regoneration Attack
Gaussian Noise Attack
Regemeration
Gaussian
Rugeneration Attack
Attuck
Attack
Attack
Alice!
1QaPck
Hiljo!
APacc3
Alice!
Alice!
Figure 1:
The robustness of different watermark encoding positions. Left: Encoding in image
frequency space resists Gaussian noise but is vulnerable regeneration attacks. Middle: Encoding in image latent space enhances resistance to regeneration attacks but introduces vulnerabilities to Gaussian noise. Right: FreqMark encodes latent frequency space in the image, achieving a strong defense against regeneration and traditional attacks.
be incorporated into the latent semantics of images. This approach improves robustness against regeneration attacks but increases susceptibility to Gaussian noise.
In this paper, we propose FreqMark, novel self superviue watermarking approach that amalgamate the benefits of frequency domain space and latent space, endowing it with dual-domain advantage. Specifically, it utilizes fixed pre-trained Variational Autoencoder (VAE) [28] to embed watermark messages into images by making subtle adjustments in the latent frequency space. These messages are then decoded using fixed pre-trained image encoder. Figure illustrates the primary motivation behind our method. Introducing perturbations watermark images in the latent and frequency domains offers distinct advantages. By optimizing the latent frequency domain of the image, combine both approaches to effectively leverage their strengths, achieving synergistic effect where the whole is greater than the sum of its parts.
We evaluate the performance of FreqMark on the DiffusionDB [49] and ImageNet [16] datasets, experimental results demonstrated that FreqMark achieves strong robustness while maintaining image quality, configurable payload capacity, and flexibility. With 48-bit encoding setting, the bit accuracy can exceed 90% under various attacks. This performance indicates significant advantages over baseline methods, particularly excelling during regeneration attacks [7, 13, 58].
Contributions: (1) We propose a novel invisible image watermarking method named Mark, which encodes hidden messages within the latent frequency space of images. FreqMark achieves watermark embedding through indirect optimization centered on the image itself without requiring network training. (2) Freq highly flexible, allowing for free trade-off between the bits number of the encoded message, image quality and watermark robustness to meet diverse requirements. (3) FreqMark demonstrates significant robustness advantages, particularly during regeneration attacks compared to baseline methods. Experimental results validate the superiority of our proposed method.
Related Work
Generative Models For long time, Generative Adversarial Networks (GANs) [27, 1, 26, 22, 41, 33] have dominated image generation. Recently, diffusion models have emerged as solid alternative GANs for image generation [25, 17, 34, 35]. These models show significant improvements and applications across various domains. DDIM sampling [45] and latent diffusion [42] further accelerate the generation progress, while ControlNet [56] provides powerful, controllable generation method. As high-quality image generation becomes more accessible, digital watermarking gains importance for protecting intellectual property rights and ensuring content authenticity.
Image Watermarking The research history image watermarking techniques is extensive. Early methods employ hand-crafted methods to hide messages within the spatial or frequency domain of images. Frequency-domain, segueenc--dmmainbbeeee based techniques typically exhibit better robustness and have been widely applied even before the rise of deep learning [12, 15, 14, 36, 15, 4]. The widely used open-source

---

model, Stable Diffusion [42], employs DwtDetSvd [14] as its default watermarking method. However, this method has been demonstrated to be relatively vulnerable to various attacks [58].
With the development of deep learning, methods utilizing encoder-decoder architectures have gained prominence. HiDDeN [60] and RivaGAN [55] train encoders embed watermark messages into images and decoders to extract them, thereby enhancing robustness against noise while preserving image quality. RedMark [3] and StegaStamp [46] improve robustness by integrating series of differentiable perturbations. RoSteALS [10] enhances the robustness of the watermark by fine-tuning the secret encoder and secret decoder in the latent space. In recent years, several innovative methods have been introduced. WatermarkDM [59] trains a diffusion model on watermarked images to create detectable watermarked images. Stable Signature [19] fine-tunes the decoder of a latent diffusion model to embed specific hidden messages. Tree-Ring [50] adopts unique approach by encoding a particular pattern shape in the initial noise frequency space during the diffusion process and utilizes
DDIM inversion [45] to detect watermarks. Differently, FNNS [29] and SSL. [20] achieve message encoding by optimizing the image itself. However, the aforementioned methods struggle to strike perfect balance between flexibility and robustness, In contrast, Freq offers higher degrees of freedom, allowing for a better trade-off among task requirements, and demonstrates robustness against regeneration attacks.
3 Background
FreqMark operates on the following scenario: A user embeds a k-bit watermark message me into an image I to get the watermarked image I for which they hold the copyright. Upon discovering unauthorized usage of the image Iw. the user decodes the infringed image to obtain the message ma, which serves as evidence to prove their ownership of the image's copyright.
Assuming that each bit of the decoded message from clean images independent and has an equal probability of being -1 or 1, this method allows us mathematically calculate the False Positive Rate (FPR) of decoding.
Let the encoded message be me E 13 and the decoded message be my E {-1,,1} The func- tion M (me; (m) measures the number of matching bits between m. and ma. Given the assumption regarding the image encoder output mentioned earlier, each bit of the decoded message from clean images is independent and follows Bernoulli random variables with a probability of 0.5 [19]. By set a decoding threshold т E {O..... ........,, k). once M(R., ma) 2 т. we consider that the image has encoded the message m.. Consequently, M (m., ma) follows a binomial distribution Bkk, 0.5). The final
message will be transformed {0, 1]* by applying function f(z) (x 1)/2 for easier processing of the binary format.
We could test the hypothesis Hy: the image has hidden watermark, and against the null hypothesis Ho: the image x has no hidden watermark. From this, we can obtain a closed-fore solution for the FPR under the threshold e(T) using the regularized incomplete beta function Ig (a; b):
k
Σ ((T) = (MM,me, ma) > T Ho) 2 i=
= + +1, T).
(I)
Based on the above formula, FPR ~ 1.65 x 10 when k = 48, 39; FPR R 5.04 x 10-6 when k 48, 7 41. The results of watermark detection True Positive Rate (TPR) under various FPR settings are shown in Figure 8 of Appendix A.3.
4 Method
4.1
Overview
Figure 2 presents an overview of FreqMark. FreqMark employs a strategic methodology to embed invisible watermarks images by adding perturbations in the latent frequency space. Specifically, we utilize a Variational Autoencoder (VAE) [28] to encode images into representations and then transform these image latents into the frequency domain. Only the perturbation within the latent frequency space of images is trained during watermark encoding and all the networks are fixed. A
3

---

+
8 m
Encoding
VAE Encoder
Fz
Vector
IFFT
Dot Product
Vector
Image Encoder
Vector
VAE Decoder
Iw
Pre-defined
E2
E1
011
Secret Message
Decoding
Figure 2: Overview of FrrqMark. Encoding: Freq employs pre-trained VAE model to encode watermarks within the latent frequency space of the image. and e2 are Gaussian noise perturbations added during training. All networks are fixed and only perturbation 6m is trained. Decoding: FreqMark utilizes a pre-trained image encoder to extract features from the image and extracts the watermark by comparing this feature against predefined directions.
pre-trained image encoder used for watermark extraction, the features of the watermarked image obtained from the encoder are compared with predefined directional vectors to reveal the hidden message.
In the optimization process, the image quality is maintained with minimal degradation by utilizing PSNR and LPIPS loss [57], while the watermark message constrained by using hinge loss. In addition, noises and augmentation are introduced for enhanced robustness. Figure 3 shows some watermark image examples from FrrqMark.
4.2 Message Embed in Latent Frequency Space
Some current methods embed hidden messages by directly optimizing image pixels [20, 29], which leads to poor robustness against regeneration attacks [58]. FreqMark involves strategically adding perturbations in the frequency domain of the image latent representation to embed watermark. These perturbations in the frequency domain are more concealed than those in the pixel space, making them harder to eliminate and thus offering superior robustness against regeneration attacks while minimizing the impact on image quality.
invisible
First, we use pre-trained VAE encoder E to encode the image into a latent image and then transform the latent image into the frequency domain using Fast Fourier Transform (FFT), represented as:
Fz = FF=FFD()()),
where the input image, E(I) is the latent representation encoded by the VAE encoder E, and Fz the frequency domain latent representation of the image.
Next, we embed the hidden message into the frequency domain latent representation Fz by adding slight perturbation, then decode the frequency domain latent representation back into the image using inverse Fast Fourier Transform (iFFT) FFT and the VAE decoder D, as follows:
T D(FFT-1(F) + 8mn)),
where y is the watermarked image, 6m is the perturbation added in the frequency domain of the latent image for embedding the watermark.

---

4.3 Decoding by Pre-trained Image Encoder
During the decoding process, similar to SSL[20), we predefine a set of N-dimensional vectors V {ui u2,,... UK K S N within the feature space of the pre-trained image encoder Esng- For images with embedded watermarks LuJ the feature vector 21... Eing (fw) is obtained after passing through Eimg- By calculating the signs of the dot product between the direction vectors V and I we can extract the hidden message m..
m sign(=1.. Uk) sign(Eing (w)) Uk), oo E VR
(4)
where m represents the k-th bit of ma and UR denotes the k-th direction vector V sign(x) = 1 when æ > O; sign(x)- = -1 when < O.
One sigmificant advantage of this decoding method is its flexibility setting the number of watermark bits. The number of direction vectors determines the encoding bits. Using the technique mentioned above of optimizing images within the frequency domain, we can ensure the robustness of the hidden messages while significantly minimizing the quality impact on the image.
4.4 Training Objective
Our goal is to embed an invisible watermark by optimizing the frequency map of the image latent Fz. where the perturbation 6n serves as the trainable parameter, with all pre-trained networks remaining fixed. The optimization via perturbations ensures both watermark robustness and the preservation of image quality, offering flexibility in encoding bits and embedding strength.
Image Quality We utilize PSNR loss to reduce discrepancies between the watermarked image and the original image and also incorporate LPIPS loss [57] to make alterations less perceptible.
Cp PSNRRL
(5)
C, LPIPS(
(6)
Watermark Message The optimization goal is to modify the image features =I., processed by the pre-trained image encoder Esng. aligning them on the K direction vectors V to correspond with the encoded message. We define message loss as the hinge loss with margin μ 2 on the projections:
L. (mn(Jw)
m0 (μ (=1, v.)) mx)), un K my E (-1,1)
K
k=1
4.5 Robustness Enhancement Strategy
During the training process. we employ an augmentation strategy by adding Gaussian noise with mean of and standard deviations of s1 and s2 to the latent space and pixel space, represented as el and e2, respectively. The result is the acquisition of the perturbed images 4p1 and 4p2:
4p1 (LFFT-1(F2 +m)) +el),
(8)
1p2 = D(FFT (Fz + +mm) + c2.
(9)
The final loss is defined as:
L= Cm Cm (4p1) + L. (1p2) + XpCp(tw: AeCr
(10)
where Xp and è the respective weights for each loss function.
By incorporating corresponding attacks during training, we optimize the solution space for hidden message embedding, enabling the image to withstand a broader range of attacks. Experimental results reveal that this straightforward approach can effectively enhance the model's robustness against regeneration attacks.
5

---

Figure 3: Examples watermarked images. The first three columns are from ImageNet [16], and the others are generated by the prompts from DiffusionDB [49]. These watermarked images have 48-bit messages and are robust to various attacks. Top: origin image. Middle: watermarked image. Bottom: pixel difference (amplified by factor of 10 to enhance the visual effect).
Experiments
In this section, we evaluate our method based on the image quality and robustness metries under various attacks, comparing it with the baseline methods. Additionally, ablation studies and analyses are carried out to explore the process deeply. In our experiments, KL auto-encoder [28] from Stable Diffusion [42] is used as the pre-trained VAF, and DINO v2 small [37] is used as the pre-trained image encoder. More implementation details are in the Appendix A.
5.1 Experimental Setup
Datasets A test dataset is compiled, consisting of 500 images randomly selected from the ImageNet validation set [16], in conjunction with 500 images generated using Stable Diffusion [42] based on prompts from the DiffusionDB [49] dataset. This diverse data collection enables a thorough assessment of the performance across various scenarios and perspectives.
Comparison Methods Three methodologies are selected for comparison. DwtDetSvd [14] is classic frequency-doman dooae-doomi---aban based approach and the default watermarking method in Stable Diffusion [42]. Stable Signature [19] embeds specific hidden messages by fine-tuning the VAE decoder of Stable Diffusion, enabling the watermarking process to integrated with the generation process. SSL [20] is the latest method that employs image pixel optimization to embed watermarks. We employ their default configurations, wherein the PSNR threshold of SSL is set to dB for convenient comparison of bit accuracy.
Following previous work [19, 20, 58], PSNR and SSIM [48] are used as the image quality benchmark. To evaluate robustness, we utilize bit accuracy metric to measure the degradation of hidden messages under diverse attacks. Various attack methods have been implemented, including brightness change of 0.5, contrast change of 0.5, 50% JPEG compression, Gaussian blur with kernel size of 5, and Gaussian noise with o = 0.05. Moreover, the experiments incorporate two types of VAE regeneration attacks [7, 13] from the CompressAl library [9] with compression factor of 3, and a diffusion regeneration attack carried out with 60 denoising steps [58].
Evaluation Metries
5.2 Benchmarking Watermark Accuracy and Image Quality
Table I displays the image quality and bit accuracy following watermark embedding on ImageNet [16] and DiffusionDB [49] datasets. The classic method DwtDetSvd [14] shows robustness against Gaussian attacks but poor performance against others, while the others show better robustness against

---

Table 1: Performance of different watermarking methods on ImageNet and DiffusionDB.
Bit Accuracy
Method
PSNR SSIM Nome Brightness
Contrust JPEG
Gau. blur
Gau, noise
VAE-B
VAE-C Diffusion
Avg
Image
DwtDerSvd[[4]
39.67
0.978
0.993
0.636
0.489
0.848
0.992
0,993
0,550
0.562
0.592
0.739
0.307
0.222
0,058
0.051
0.078
N/A
SSL Watermark 201
31.04
0.862
1,000
1,000
1,000
0.972
1,000
0.937
0,793
0.777
0.743
0.914
1std
0.029
0.000
0.000
0.000
0.096
0.077
N/A
Stable Signaturel [9]
28.74
0,838
0.978
0.971
0.937
0.832
0.859
0.892
0.630
0.645
0.54
0,809
=std
3,246
0.080
0.054
0.061
0.092
0.106
0.121
0.117
0.086
0.105
0,064
N/A
Freq (Our)
31.27
0.857
1,000
0.995
1,000
0.991
1,000
0.939
0,938
0.924
0.988
0.973
3,359
0.038
0.000
0.028
0.000
0.024
0.000
0.088
0.083
0.081
0.052
N/A
DiffusionDB
DedDeiSvd[14]
39,49
0.978
1,000
0.607
0.457
0.887
1.000
1.000
0.563
0.556
0.509
0.738
E
1.182.
0,006
0.000
0.308
0.194
0.109
0.000
0.000
0.053
0.059
0.085
N/A
SSL Watermark[00]
31.01
0.827
1.000
1.000
1.000
0.956
1.000
0.954
0,742
0.744
0.729
0720
0.903
0.064
0,000
0,000
0.000
0.102
0,081
N/A
Stable Signaturel [9]
28.31
0.844
0.996
0.996
0.990
0.896
0.858
0.967
0.668
0.733
0.527
0,848
1std
0.012
0.014
0.042
0.086
0,063
0.049
0.040
N/A
Freq (urr))
31.20
0.854
1,000
1,000
1,000
L,000
1,000
0.934
0.925
0.897
0,945
0.967
s
1,538
0.029
0.000
0.000
0.000
0.000
0.000
0.061
0.066
0.059
0.047
N/A
Table 2: Comparison of image quality between VAE and FreqMark.
Decidedd Heneage
Nendice Nemsige
Dataset
PSNR
SSIM
VAE 31.37 + 4.59
ImageNet [16]
0.868 + 0.085 0.879 + 0.032
DiffusionDB [49]
31.22 1.96
Freq
ImageNet [16]
31.27 + 3.36
0.857 + 0.038 0.854 + 0.029
Figure 4: The correlation matrix of each bit of the decoded message from the vanilla images and the random message.
DiffusionDB [49]
31.20 + 1.54
brightness, contrast, and JPEG attacks but still poor in regeneration attacks. Freq demonstrates exceptional robustness against regeneration attacks with acceptable image quality.
We also calculate the PSNR and SSIM of the images from two datasets after VAE reconstruction. The data in Table 2 demonstrates that the impact of FreqMark on image quality is limited. Moreover, the standard deviation of image quality after FreqMark processing is also reduced to some extent.
In order to verify the hypothesis proposed in Section 3, randomly select 2,000 clean images to perform message decoding in 128-bit and calculate the correlation coefficients between each bit, comparing them to random messages. Figure 4 shows that different bits with near-zero correlation coefficients can be considered as independent random variables.
Notably, FreqMark allows users to adjust image quality and encoding bits, showing remarkable robustness while maintaining acceptable image quality, particularly against VAE and diffusion regeneration attacks [9, 7, 13, 58]. We will further evaluate the robustness of FreqMark the DiffusionDB dataset [49] in Section 5.4. The ablation studies in Section 5.5 will explore the impact of parameter adjustments robustness.
5.3 Why the Frequency Domain of Image Latent Space
To further illustrate the advantages of optimizing in the frequency domain of the image latent space, We conducted experiments using the same settings to optimize the pixel, the image latent space, and the frequency domain of image pixels in the DiffusionDB dataset [49]. Simultaneously, enhancements in pixel and latent domains are retained (if the method involves the latent space). consistent average image quality range is maintained for comparison purposes.
As illustrated Figure and Table 3, optimizing the frequency domain of image pixels provides the watermarked image with robustness against all attacks except the regeneration attack. In contrast,
7

---

Origin
Pixel
Pixel Frequency
Latent
Latent Frequency
Figure 5: Watermarked images under different optimization locations. We compared four distinet optimization objectives for watermark embedding. including the image pixel domain, the frequency domain of the image pixel, the image latent space, and the frequency domain of the image latent space (ours). The difference after watermarking addition is amplified by a factor of 10.
Table 3: Performance of different optimization locations.
Bit Accuracy
Location
PSNR
SSIM
None
Brightness
Contrast
JPEG
Gan. blur
Gau. noise
VAE-B
Diffusion
Avg
Pixel
31.36
0.771
0.950
0.935
0.937
0.848
0.885
0.925
0.642
0.654
0.542
0.813
Pixel Frequency
31.31
Latent
31.35
0.886
0.994
0.993
0.981
0.906
0,979
0,804
0.796
0,833
0.675
0.885
Frequency
31.20
0.934
images obtained by optimizing the image latent space appear natural and smooth, with strong correlation to the semantic information of the image. This approach demonstrates robust potential against diffusion attacks, indicating that both optimization methods have unique features. Combining the advantages of both approaches, FreqMark optimizes the frequency space of the image latent. The watermark closely related local patterns while retaining the characteristics of frequency domain optimization. Combining these two aspects produces synergistic effect, making FreqMark robust against regeneration attacks.
5.4 Further Robustness Results
Diffusion Attack Steps We evaluate the impact of diffusion attacks of varying intensities bit accuracy and also calculate the PSNR between the attacked image and the original watermarked image. Table indicates that FreqMark maintains commendable robustness even under higher intensity attacks.
Table Performance under Different Diffusion Steps.
Diffusion Steps
60
80
100
120
140
160
180
200
Bit Acc PSNR
0.945
0.863
0.831
0.754
0.712
0.692
0.660
0.637
27.67
26.95
26.19
25,46
24.92
24.47
23.99
23.57
Vae Attack Strength We employ same VAE used watermarking process FreqMark to conduct perturbation attacks. Table 5 demonstrates that FreqMark exhibits exceptional robustness under such attacks. This is because the perturbation on the latent FFT changes the overall distribution of the image latent, making the watermark message affect the entire image globally. Therefore, it

---

Table Performance under VAE Attack in Latent FFT Domain and Gaussian Noise Disruption in Pixel FFT Domain.
VAE Attack (Latent FFT)
Gaussian Noise Attack (Pixel FFT)
PSNR
31.43
30.31
28.98
27.39
25.82
31.09
29.68
28.04
26.46
25.04
Bit Acc
1.000
1.000
0.998
0.990
0.975
1.000
1.000
1.000
1.000
1.000
is difficult for perturbations the latent of image to damage the watermark message. A similar phenomenon is observed the pixel FFT when facing the Gaussian Noise attack.
Adversarial Attack Following the settings in [5], apply adversarial attacks targeting the latent representations of the watermarked images.
maV [F(Jado) F...11, Tox Se
(11)
Table 6: Performance under Different Adversarial Attack Strength.
Attack Strength (eps)
Bit Acc
TPR
2/255
1.000
1.000
4/255
0.987
1.000
6/255
0.944
0.986
8/255
0.893
0.972
Table demonstrates that Freq exhibits strong robustness when facing adversarial attacks targeting latent representations. We believe that this can be attributed to the limited impact of attacks targeting latent representations on the latent FFT domain.
5.5
Ablation Studies
Image Quality As watermarking always requires modifications to the image pixels, a trade-off between image quality and embedded message robustness is inevitable. FreqMark allows users find satisfactory balance. We control watermarked image quality within a specific range by adjusting the weight of the PSNR loss function. As shown in Figure 6a, FreqMark can achieve a PSNR that is nearly 2dB higher compared to the reconstruction obtained using VAE with robustness against varying attacks, maintaining strong performance with over 80% accuracy.
Encoding Bits The number of bits in the encoding message significantly impacts robustness, with more bits embedded at a given image quality potentially reducing robustness. FreqMark allows users to adjust the watermark bit number based on their needs. Figure 6b shows bit accuracy for encoding bit numbers. We test robustness from 32 to 128 bits of watermark message 16-bit increments, setting the PSNR of the watermarked image to about 31 dB. FreqMark also demonstrates strong robustness, with the lowest bit accuracy remaining above 0.75 even under regeneration attacks on images watermarked with 128-bit messages.
Noising Scale We also study the impact of latent noise e and pixel noise e2 on robustness. Incorporating noise attacks during training can effectively improve robustness in Gaussian noise and regeneration attacks, but excessive noise may disrupt training and reduce performance. We analyze how noises latent and pixel space affect robustness by fixing either éj or eg and changing the other standard deviation. As shown in Figure 6c and Figure 6d, the results confirm that introducing both types of noise positively influences robustness, and adding an appropriate amount of noise can significantly enhance resistance against regeneration attacks. We finally set to 0.25 and to 0.06 based on the experimental results. The complete experimental data can be found in Table of the Appendix A.3.
Spacial Perturbations
The bit accuracy of FreqMark significantly declines under specific spatial transformation attacks, such rotation and cropping. By incorporating relevant augmentations like

---

0.95
0.95
0.90
0.90
Nore
0.80
PRG
PRG
0.75
WEE
0.75
INED
DMfuetee
Dffaatee
0.70
0.70
24
30
32
40
80
120
PSNR
Bit
(a) Image Quality
(b) Encoding Bits
1.00
0.95
0.95
0.90
0.90
Nore Conrest
Mord
Contrst
0.80
Pin
0.80
Bocosan ta
0.75
0.75
Officice
Offrcies
0.0
0.1
0.2
0.3
0.4
0.5
0.00
0.05
0.10
0.15
0.20
Latent Noise Stal
Pixcl Noise Stal
(c) Latent Noise Std
(d) Pixel Noise Std
Figure 6: Impact of different parameter settings on the robustness of the watermark. (a) Bit accuracy under different watermarked image quality (48 bits). (b) Bit accuracy under different encoding bits (31dB). (c) Bit accuracy under different latent noise E (E2 = 0.06). (d) Bit accuracy under different latent noise E2 (e, 0.25).
Table 7: Performance on Spatial Perturbations,
Bit Acc
Spatial Augmentations
Resize 0.3
Rotate 90
Crop 0.7
0.961
0.621
0.721
0.968
0.927
0.921
random rotation and random cropping during training, effectively bolster the robustness against these transformations, as evidenced in Table
Conclusion
In this paper, we introduce technique for embedding hidden messages in images by optimizing the frequency domain the latent space, named FreqMark, providing remarkable robustness due to deeper perturbations. FreqMark allows user-defined encoding bits and watermark strength, striking an optimal balance between image quality and robustness.
Limitations and Broader Impacts Farq exhibits notable flexibility and robustness, yet there is room for optimization, particularly in the degree naturalness the fusion of watermarks and images. Future research can focus on improving image quality and human perceptibility constrained by the limitations of VAE, as superior VAE would likely boost processing time and overall effectiveness. FreqMark presents innovative image watermarking approach for image provenance, copyright protection, etc. However, like other image watermarking methods, also necessitates the prevention of unauthorized misuse such copyright abuse [5].
10

---

References
[1] ABDAL, R., QIN, Y., AND WONKA, P. Image2stylegan: How to embed images into the stylegan latent space? In 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019), IEEE Computer Society, pp. 4431-4440.
[2] ABDELNABI, S., AND FRITZ, M. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privacy (SP) (2021),
IEEE, pp. 121-140.
[3] AIMMADI, M., NOROUZI, A., KARIMI, N., SAMAVI, S., AND EMAMI, A. Redmark: Frame- work for residual diffusion watermarking based on deep networks. Expert Systems with Appli- cations 146 (2020), 113157.
[4] AL-HAJ, A. Combined dwt-det digital image watermarking, Journal of computer science 3, (2007), 740-746.
[5] AN, B., DING, M., RABBANI, T., AGRAWAL, A., XU, Y., DENG, C., ZHU, S., MOHAMED, A., WEN, Y., GOLDSTEIN, T., ET AL. Benchmarking the robustness of image watermarks. arXiv preprini urXiv:2401.08573 (2024).
[6] ATHALYE, A., ENGSTROM, L., ILYAS, A., AND KwOk, K. Synthesizing robust adversarial examples. In International conference on machine learning (2018), PMLR, pp. 284-293.
[7] BALLÉ, J., MINNEN, D., SINGH, S., HwANG, S. J., AND JOHNSTON, N. Variational image compression with scale hyperprior. arXiv preprint
(2018).
[8] BARDES, A., PONCE, J., AND LECUN, Vicreg: Varianceiivariane--vvaaaae regulariza- tion for self-supervised learning. arXiv preprint 44X4v:2005.04406 (2021).
[9] BÉGAINT, J., RACAPÉ, F., FELTMAN, S., AND PUSHPARAJA, A. Compressai: py- torch library and evaluation platform for end-to-end compression research. arXiv preprint arXiv:20/ [tti:20//..00222 1.03029 (2020).
[10] BUI, T., AGARWAL,, S., YU, N., AND COLLOMOSSE, J. Rosteals: Robust steganography using autoencoder latent space. In Proc. CVPR WMF (2023).
[11] CHAL L., BAU, D., LIM, S.-N., AND ISOLA, P. What makes fake images detectable? understanding properties that generalize. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVI 16 (2020), Springer, pp. 103-120.
[12] CHANG, C.-C., TSAL, P., AND LIN, C.-C. Svd-based digital image watermarking scheme. Pattern Recognition Letters 26, 10 (2005), 1577-1586.
[13] CHENNG, Z., SUN, H., TAKEUCHII, M., AND KATTO, J. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2020), pp. 9939-7948.
[14] Cox, I., MILLER, M., BLOOM, J., FRIDRICH, J., AND KALKER, T. Digital watermarking and steganography. Morgan kaufmann, 2007.
[15] Cox, I. J., KILIAN, J., LEIGHTON, T., AND SHAMOON, T. Secure spread spectrum water- marking for images, audio and video. In Proceedings of 3rd IEEE international conference on image processing (1996), vol. IEEE, pp. 243-246.
[16] DENG, J., DONG, W., SOCHER, R., L.-, LI, K., AND FEI-FEI, L. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition (2009), leee, pp. 248-255.
[17] DHARIWAL, P., AND NICHOL,, Diffusion models beat gans on image synthesis. Advances in neural information processing systems 34 (2021), 8780-8794.
11

---

[18] FEI, J., XIA, Z.. TONDI, B., AND BARNI, M. Supervised gan watermarking for intellectual property protection. In 2022 IEEE International Workshop on Information Forensics and Security (WIFS) (2022), IEEE, pp. 1-6.
[19] FERNANDEZ, P., COUAIRON, G., JÉGOU, H., DOUZE, M., AND FURON, T. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 22466-22477.
[20] FERNANDEZ, P., SABLAYROLLES, A., FURON, T., JÉGOU, H., AND DOUZE, M. Water- marking images in self-supervised latent spaces. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2022), IEEE, pp.
[21] FRANK, J., EISENHOFER, T., SCHÖNHERR, L.., FISCHER, A., KOLOSSA, D., AND H0177, T. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning (2020), PMLR, pp. 3247-3258.
[22] GOODFELLOW, I., POUGET-ABADIE, J., MIRZA, M., XU, B., WARDE-FARLEY, D., OZAIR, S., COURVILLE, A., AND BENGIO, Y. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
[23] GRAGNANIELLO, D., COZZOLINO, D., MARRA, F., POGGI, G., AND VERDOLIVA, L. Are gan generated images easy detect? critical analysis of the tfate-of-the-arr In 2021 IEEE international conference on multimedia and expo (ICME) (2021), IEEE, pp. 1-6.
[24] GUARNERA, L., GIUDICE, O., AND BATTIATO, S. Deepfake detection by analyzing convo- lutional traces. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition workshops (2020), pp. 666-667.
[25] Ho, J., JAIN, A., AND ABBEEL, P. Denoising diffusion probabilistic models. Advances in neural information processing systems 33 (2020), 6840-6851.
[26] KARRAS, T., AILA, T., LAINE, S., AND LEHTINEN, J. Progressive growing of gans for im- proved quality, stability, and variation. International Conference on Learning Representations (2018).
[27] KARRAS, T., LAINE, S., AITTALA, M., HELLSTEN, J., LEHTINEN, J., AND AILA, T. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition (2020), pp. 8110-8119.
[28) KINGMA, D. P., AND WELLING, M. Auto-encoding variational bayes. arXiv preprint 1rXiv:1312.61 /4 (2013).
[29] KISHORE, V., CHEN, X., WANG, Y., LI, B., AND WEINBERGER, K. Q. Fixed neural network steganography: Train the images, not the network. In International Conference on Learning Representations (2021).
[30] LEE, J.-E., SEO, Y.-H., AND KIM, D.-W. Convolutional neural network-based digital image watermarking adaptive to the resolution of image and watermark. Applied Sciences 10, 19 (2020), 6854.
[31] LUKAS, N., DIAA, A., FENAUX, L., AND KERSCHBAUM, Leveraging optimization for adaptive attacks on image watermarks. In The Twelfth International Conference on Learning Representations (2023).
[32] MIDIOURNEY.COM Midjourney. Accessed 3 April 2024. http://www.middouuee..comm 2022.
[33] MIRZA, M., AND OSINDERO, S. Conditional generative adversarial nets. arXiv preprint 141. (2014).
[34] NICHOL, A., DHARIWAL, P., RAMESH, A., SHYAM, P., MISHKIN, P., McGREW, B., SUTSKEVER, I., AND CHEN, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.1074 10741 (2021).
[35] NICHOL, A. Q., AND DHARIWAL, Improved denoising diffusion probabilistic models. In International conference machine learning (2021), PMLR, pp. 8162-8171.
12

---

[36] Ó RUANAIDH. J.. DOWLING, W., AND BOLAND, F. Watermarking digital images for copyright protection, IEE PROCEEDINGS VISION IMAGE AND SIGNAL PROCESSING 143 (1996). 2500256.
[37] OQUAB, M., DARCET, T., MOUTAKANNI, T., Vo, H., SZAFRANIEC, M., KHALIDOV, V., FERNANDEZ, P., HAZIZA, D., MASSA, F., EL-NOUBY, A., ET AL. Dínov2: Learning robust visual features without supervision. arXiv preprint ar 231422304.07193 (2023).
[38] O'RUANAIDH, J. J., AND PUN, Rotation, scale and translation invariant digital image watermarking. In Proceedings of International Conference Image Processing (1997), vol. 1, IEEE, pp. 536-539.
[39] PARMAR, G., ZHANG, R., AND ZHU, J.-Y. On aliased resizing and surprising subtleties in gan evaluation. In CVPR (2022).
[40] QIAO, T., MA, Y., ZHENG, N., WU, H., CHEN, Y., Xu, M., AND LUO, X. A novel model watermarking for protecting generative adversarial network. Computers & Security 127 (2023), 103102.
[41] RADFORD, A., METZ, L., AND CHINTALA, S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv: 1511 1.06434 (2015).
[42] ROMBACII R., BLATTMANN, A., LORENZ, D., ESSER, P., AND OMMER, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2022), pp. 10684-1069)
[43] SABERI, M., SADASIVAN, V. S., REZAEL, K., KUMAR, A., CHEGINI, A., WANG, W., AND FEIZI, S. Robustness of ai-image detectors: Fundamental limits and practical attacks. In The Twelfth International Conference Learning Representations (2023).
[44] SHI, C., HOLTZ, C., AND MISHNE, G. Online adversarial purification based on self- supervision. arXiv preprint arXiv:2 (2021).
[45] SONG, MENG, C., AND ERMON, S. Denoising diffusion implicit models. In International Conference on Learning Representations (2020).
[46] TANCIK, M., MILDENHALL, B., AND NG. R. Stegastamp: Invisible hyperlinks in physical photographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2020), pp. 2117-2126.
[47] VUKOTIĆ, V., CHAPPELIER, V., AND FURON, T. Are deep neural networks good for blind image watermarking? In 2018 IEEE International Workshop on Information Forensics and Security (WIFS) (2018), IEEE, pp. 1-7.
[48] WANG, 7., BOVIK, A. C., SHEIKH, H. R., AND SIMONCELLI, E, P. Image quality assessment: from error visibility structural similarity. IEEE transactions image processing 13, 4 (2004), 600-612.
[49] WANG, 7. J., MONTOYA, E., MUNECHIKA, D., YANG, H., HOOVER, B., AND CHAU, D. H. Diffusiondb: A large-scale prompt gallery dataset for text-to-image generative models. arXiv preprint arXiv:2210.14890 (2022).
[50] WEN, Y., KIRCHENBAUER, J., GEIPING, J., AND GOLDSTEIN. T. Tree-rings watermarks: Invisible fingerprints for diffusion images. In Advances in Neural Information Processing Systems (2023), A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds.. vol. 36, Curran Associates, Inc., pp. 58047-58063.
[51] WU, H., LIU, G., YAO, Y., AND ZHANG, X. Watermarking neural networks with watermarked images. IEEE Transactions on Circuits and Systems for Video Technology 31, (2020), 2591- 2601.
[52] YOON, J., HWANG, J., AND LEE, J. Adversarial purification with score-based generative models. In International Conference on Machine Learning (2021), PMLR, pp. 12072.
13

---

[53] YU, N., SKRIPNIUK, V., CHEN, D., DAVIS, L., AND FRITZ, M. Responsible disclosure of generative models using scalable fingerprinting. arXiv preprint arXiv:2012.08726 (2020).
[54] ZIANGG, C., BENZ, P., KARJAUV, A., SUN, G., AND KWEON, I. S. Udh: Universal deep hiding for steganography, watermarking, and light field messaging. Advances in Neural Information Processing Systems 33 (2020), 10224-10234.
[55] ZHANG, K. A., XU, L., CUESTA-INFANTE, A., AND VEERAMACHANENI, K. Robust invisible video watermarking with attention. arXiv preprint (exXiv:/9099 01285 (2019).
[56] ZHANG, L., RAO, A., AND AGRAWALA, M. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 3836-3847.
[57] ZHANG, R., ISOLA, P., EFROS, A., SHECHTMAN, E., AND WANG, O. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition (2018), pp. 595.
[58] ZHAO, X., ZHANG, K., SU. Z., VASAN, S., GRISHCHENKO, I., KRUEGEL, C., VIGNA, G., WANG, Y., AND LI, L. Invisible image watermarks are provably removable using generative ai. Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li, "Invisible image watermarks are provably removable using generative ai," Aug (2023).
[59] ZHAO, Y., PANG, T., DU, C., YANG, X., CHEUNG, N.-M., AND LIN, M. A recipe for watermarking diffusion models. arXiv preprint arXiv:2303. 10137 (2023).
[60] ZHU, J., KAPLAN, R., JOHNSON, J., AND FEI-FEI, L. Hidden: Hiding data with deep networks. Proceedings of the European conference on computer vision (ECCV) (2018), pp. 657-672.
14

---

A Appendix
A.1 Implement details
The KL auto-encoder from Stable Diffusion 2-1 [42] is utilized. Due to the significant reconstruction loss associated with low-resolution images, the images upscaled to 512 x 512 for processing. In the watermark addition stage, the Adam optimizer is used with a learning rate of 2.0 and training for 400 steps. We set the PSNR loss weight X to 0.05 and the LPIPS loss weight A, to 0.25. To encode the watermark, the first 128 dimensions of the output feature generated by the Dino v2 small image encoder [37] are utilized. In the experiments, we set the directional vectors as set of 48 vectors, where the i-th vector has a value of in its i-th dimension and for the remaining dimensions. and pixel noise are selected with equal probability. For rotation augmentation, the rotation angle is randomly chosen in 90-degree increments. The crop augmentation set with crop scale range of [0.2, 1.0] and a crop ratio range of [3/4, 4/3).
Hyperparameters
addition, during the training phase, two types of spatial transformations
Compute Resources All experiments could be conducted on a single A-100 GPU with 40GB memory. Processing two images (512 x 512) in parallel takes about 5 to 6 minutes (400 steps, fp32). Increasing the batch size will improve overall efficiency. Figure illustrates the relationship between the number of training steps and performance. Experimental results indicate that FreqMark still exhibits satisfactory performance with fewer training steps.
License The assets and models used in this paper are all publicly available, including Stable Diffusion 2-1 (Open Rail++-M License) [42], DINO v2 (Apache-2.0 License) [37], DiffusionDB (CCO 1.0 License) [49], and ImageNet (Custom License, as viewed on https:/ htopp::/wwer //www. image-maeaeee net. org/dommoadd) [16].
A.2 Additional Comparison Results
We additionally compare the performance of our method with two watermarking approaches different from Farq: StegaStamp [46], classical encoder-decoder--aae
watermarking method, and TreeRing [50], watermarking technique that combines watermark embedding with image generation without requiring any training. Specifically, we evaluate the results under similar image quality conditions (PSNR of 28) and the same encoded bit number (100 bits) for StegaStamp. Due to TreeRing is I-bit encoding method, the true positive rate at 1% false positive rate (TPR 1%FPR) is specifically compared against TreeRing [50].
Table 8: Additional Comparison of StegaStamp [46] and Tree- Ring [50].
Bright
Contrast
.PPG
G.Noise
VAE-B
VAF-C
Diffission
Avg
StegaStamp[-1G]
0.999
0.999
0.998
0.994
0.997
0.991
0.981
0.984
0.857
0.978
FreqMark(100 bits)
1.000
1.000
0.999
0.989
0.976
0.933
TPR@ 1FPR
None
Bright
Contrast
JPEG
G.Blur
G.Noise
VAE-B
VAE-C
Diftusion
Avg
Tree-Ring[[0)
1.000
1.000
1,000
0.996
0,999
0.918
0.991
0.995
0.998
0.989
Compared StegaStamp [46], FreqMark offers strong robustness against diffusion attacks and greater flexibility due to its network independence. Users can adjust image quality, encoding bits, and customize the decoding vectors enhance the watermark security. As opposed to Tree-Ring [50], Frrq is capable of encoding significantly more information (48 bits vs. 1 bit) at a similar TPR @ FRF..
A.3 Additional Experimental Results
Training Steps Figure 7 shows that FreqMark achieves considerable robustness even with fewer training steps, and there is no significant change in image quality throughout the process. Users can adjust the number of training steps according to their requirements trade-off.
15

---

0.9
55IM
0.7-
Neng
Dedral
FCc
0.6
Training Step
Training Step
Robustness
Quality
Figure 7: The relationship between the training steps, watermark robustness, and image quality.
1.0
0.8
0.8
True Positive Rate
Rositive
0.6
0.6
Kona
Rone
Brightness
Brightnass
0.4
Concrust
0.4
Cortrast
JEEO
PBS
Gaussan Bur
Goussan
0.2
hote
0.2
Gousdan holee
0.0
10-13
10-10
10-" 10-4 10-4 10-7 10a
0.0
10-1>
10-10
10-n 10-6 10-1 10-7 10"
False Positive Rate
False Positive Rate
DiffusionDB [49]
ImageNet [16]
Figure 8: The TPR/FPR curve under various attacks two datasets.
True Positive Rate vs. False Positive Rate We utilize 5,000 watermarked images to plot the FPR- TPR curve. Figure shows that FreqMark exhibits remarkably high watermark detection accuracy in the range of FPR=10-6 to 10 7. It is observed that the results for the DiffusionDB dataset [49]
exhibit significantly lower TPR extremely low FPR compared to the ImageNet dataset [16]. This could be attributed to the higher standard deviation of bit accuracy in the ImageNet dataset [16]. which consequently leads superior TPR under extreme conditions.
Additional Quality Metrics We include CLIP-FID [39] and 1.2 as supplementary image quality
metrics to further evaluate FreqMark's image quality the DiffusionDB dataset [49]. Table 9 demon- strates that FreqMark exhibits outstanding robustness performance while maintaining acceptable image quality.
Table Additional Image Quality Comparison.
DwtDet [4]
SSL [20]
Stable Signature [19] StegaStamp [46]
FreqMark
CLIP-FID [39]
2.36
6.88
1.70
5.50
3.84
L2
7.71
52.06
63.74
85.24
52.95
TPR/FPR Results on Diffusion Attacks We present the average TPR/FPR results across varying diffusion steps on the two datasets. Table indicates that FreqMark maintains excellent performance in TPR @0.1% FPR, particularly at higher diffusion steps.
16

---

Table 10: The TPR results for different Diffusson Steps and FPR values.
Diffusion Steps / FPR
1.5e-2
1e-3
3e-5
3e-7
7e-10
1e-13
60
1.000
1.000
0.996
0.990
0.927
0.636
80
1.000
1.000
0.946
0.778
0.360
0.019
100
0.995
0.941
0.742
0.486
0.153
0.008
120
0,936
0.804
0,465
0.147
0.024
0.000
140
0.853
0.569
0.240
0.048
0.000
0.000
160
0.667
0.328
0.120
0.027
0.000
0.000
180
0.486
0.193
0.052
0.000
0.000
0.000
200
0.294
0.094
0.010
0.000
0.000
0.000
Table 11: Average Bit Accuracy Gaussian noise and regeneration attacks for different combinations of Latent Noise and Pixel Noise.
Pixel Noise Std
Latent Noise
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
0.18
0.20
0.00
0.706
0.749
0.797
0.803
0.751
0.758
0.715
0.706
0.678
0.694
0.701
0.05
0.718
0.759
0.800
0.804
0.804
0.767
0.781
0.723
0.730
0.744
0.704
0.10
0.768
0.793
0.870
0.865
0.858
0.820
0.816
0.817
0.794
0.801
0.780
0.15
0,841
0.850
0.897
0.908
0.891
0.907
0,889
0.890
0.877
0.861
0,832
0.20
0,850
0.878
0,934
0.910
0,910
0.878
0.867
0.868
0.869
0.870
0.840
0.25
0.889
0.915
0.924
0.926
0.896
0.885
0.885
0.852
0.866
0.860
0.820
0.30
0.820
0.853
0.886
0,912
0.894
0.885
0.857
0.847
0.840
0.863
0.830
0.35
0.791
0.838
0.867
0,866
0,865
0.872
0.805
0,811
0.785
0.804
0.814
0.40
0.762
0.802
0.865
0,868
0.814
0.835
0.802
0.807
0.768
0.763
0.754
0.45
0.725
0.777
0.846
0.822
0.824
0.767
0.764
0.782
0.767
0.742
0.738
0.50
0.719
0.763
0.814
0.838
0.809
0.787
0.763
0.731
0.727
0.733
0.707
Noising Scale Table 11 displays the average bit accuracy under Gaussian noise and three regener- ation attacks for different combinations of Latent Noise and Pixel Noise. The results substantiate that judiciously introducing moderate noise in both the latent and pixel dimensions can effectively enhance the robustness of our method.
A.4 Additional Qualitative Results
Origin
32 bits
48 bits
96 bits
128 bits
Figure 9: Examples of watermarked images with various encoding bits. FreqMark maintains image quality(aproxximatel 31 dB in terms of PSNR) without degradation as the number of encoding bits increases while the bit accuracy against regeneration and Gaussian noise attacks may be reduced. Encoding 48 bits is the default setting.
17

---

Origin
psnr 28
psar 30
psnr 31
psnr 32
33
Figure 10: Examples of watermarked images with various quality. Freq allows users to balance robustness and image quality based on their requirements. Due to the inherent robustness advantage of Mark, it still remains competitive at higher image quality settings.
Origin
Watermarked
Difference (1
Figure 11:
Two examples of poor image quality are presented: in the first image, a visible Moire-like pattern is introduced, while in the second image, subtle alterations to the semantic information the background region are observed. This situation typically arises from the significant quality loss caused by the VAE during the image reconstruction process. Although this phenomenon is frequent, future VAE models with improved performance are anticipated mitigate this issue effectively. Moreover, this is direction for further exploration and enhancement in future research.
18

---

Original
Watermarked
Difference( 10)
Original
Watermarked
Difference( 10)
TAULANER
PAULANED NER MONCHEY
MONCHES
BIST
Figure 12: Additional qualitative results with a 48 bit hidden message. The images on the left are the results from ImageNet [16], while those the right from DiffusionDB [49].
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We reflect the paper's contributions and scope in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
20

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have disclosed all the information needed to reproduce the main experi- mental results in Appendix A.I.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open Access Data and Code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [No]
Justification: We will declutter and release the code in the future. We have provided sufficient details for the replication of the paper in Appendix A.I.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
public/guides/ https//oooeSuuiisinnnnee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https: CCodeSubmiis
httppscccppublcc//eec /ttoe/uuudee ssi wsionnoliccyio for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have specified all the training and details section and Appendix A.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We have included the mean and standard deviation of quality metrics (PSNR, SSIM) and robustness metrics (Bit Acc) across all images Section 5.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
22

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have shown the computer resources information in Appendix A.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https https://neeete httpp/://////pp///eeeee httppbiiic/thhhhhiieeeeeeeeeeeee
Answer: [Yes]
Justification:
The research conducted in this paper complies with the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: have discussed the broader impacts of our work in Section
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We have not released any new models or datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for Existing Assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The assets utilized in the paper are publicly available, with detailed license information provided in Appendix A.I.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "8M0EygXYhY", "reviewer": "Reviewer_BZ7h", "summary": "The authors propose a new post-processing watermark for imagery, FreqMark. The FreqMark embeds a binary message into the frequency domain of a VAE-encoded image via a small perturbation, and then following IFFT + decompression, utilizes a pre-trained image encoder to extract the message. A PSNR + LPIPS metric is used to protect mage quality against the message embedding. The dual approach of VAE encoding + Gaussian noise makes the mark resilient against regeneration and Gaussian noise attacks. The method demonstrates high bit accuracy and good image quality when compared against DwtDct, SSL, Stable Signature on watermarked DiffusionDB & Imagenet images.", "questions": "1. See weaknesses.\n\n2. Which version of Stable Diffusion was used for the regeneration attack?", "limitations": "Adequately discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "-Easy-to-follow narrative and nice motivation in the age of generative imagery. \n\nThe intuition behind the FreqMark algorithm is sound, easy to follow, and presumably simple to implement. \n\n-Method demonstrates robustness against regeneration, which is a very potent attack.", "weaknesses": "This work is missing two critical competitor baselines which must be addressed. \n\n-In my opinion, the FreqMark is an incremental variant of the StegaStamp [1]. Surprisingly, the authors did not compare against this method even though it is well-known in existing literature that the StegaStamp is robust against many attacks, including regeneration [2]. Like FreqMark, the StegaStamp increases resilience against attacks by incorporating them into the training pipeline and uses a critic loss to preserve image quality (LPIPS + L2, versus LPIPS + PSNR for FreqMark). The resemblance of equation (10) in this manuscript to the loss function equation (2) in [1] begs the question of novelty. It is also incorrectly stated in line 71 that the StegaStamp only relies on differential image perturbations for the training pipelines -- in fact, any attack can be added, as the decoder is trained after image manipulation. \n\n-Again, if the spirit of the paper is to increase resilience against regenerations, the authors also needed to compare against the state-of-the-art Tree-Ring watermark [4], which was noted to be incredibly resilient against regenerations by the authors of the regeneration attack [2]. As the Tree-Ring is an in-processing technique that embeds a message within a diffusion process, one way to set up a comparison is to post-process a collection of Tree-Ring watermarked images via FreqMark, and then independently extract both watermarks. \n\n-As noted in [3], there is no single perceptual metric that is an objective measure of image quality, thus low PSNR or LPIPS distance does not necessarily indicate the method is not introducing artifacts. The authors need to add 1-2 more metrics (maybe L2 and FID, for example) for a more convincing argument. \n\n-500 images is too small a sample size for the tested FPR thresholds. Modern literature in this field such as [3,4,5] are using several thousand images. \n\n-VAE regenerations are far weaker compared to diffusion regenerations. Readers will want to see how the FreqMark holds up against longer, deeper regenerations (>= 100 steps) to see how the decoding accuracy is affected.\n\n-As observed in [3], the use of publicly available VAEs to encode/decode watermarks is easily defeated if the attacker uses a regeneration leveraging encoders/decoders with the same architecture. \n\n\n[1] Tancik, M., Mildenhall, B., & Ng, R. (2020). Stegastamp: Invisible hyperlinks in physical photographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 2117-2126).\n\n[2] Zhao, X., Zhang, K., Su, Z., Vasan, S., Grishchenko, I., Kruegel, C., ... & Li, L. (2023). Invisible image watermarks are provably removable using generative ai. arXiv preprint arXiv:2306.01953.\n\n[3] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., ... & Huang, F. (2024). Benchmarking the robustness of image watermarks. arXiv preprint arXiv:2401.08573.\n\n[4] Wen, Y., Kirchenbauer, J., Geiping, J., & Goldstein, T. (2023). Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030.\n\n[5] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., & Feizi, S. (2023). Robustness of ai-image detectors: Fundamental limits and practical attacks. arXiv preprint arXiv:2310.00076.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer BZ7h,\nThank you for your suggestions. We address the concerns as follows:\n\n> W1: In my opinion, the FreqMark is an incremental variant of the StegaStamp [1]. ... The resemblance of equation (10) in this manuscript to the loss function equation (2) in [1] begs the question of novelty. \n\n**A1:** In fact, there are significant differences between FreqMark and StegaStamp, and FreqMark is not an incremental variant of StegaStamp. We have discussed StegaStamp in the Related Work section and elaborated on the differences between the two kind of approaches in **lines 22 to 27 of the Introduction section**.\n- FreqMark leverages the representational capacity of pre-trained networks to optimize **the latent FFT of images**, without the need for training any additional networks, adopting a case-by-case watermarking  approach. In contrast, StegaStamp involves training an encoder and decoder under various perturbation augmentations.\n- FreqMark only incorporates two types of noise as augmentations during the training of image perturbations, it maintains excellent robustness under various attacks.\n- The loss functions of FreqMark and StegaStamp hold distinct meanings. FreqMark employs an image encoder to encode the image into a representative vector, which is then multiplied by a predefined vector group and the watermark image's representative vector. The encoding is determined by the positive or negative nature of the result, which is entirely different from StegaStamp. \n\nWe conducted a comparison under the same bit number(100) and essentially consistent image quality.\n\n|Bit Acc|PSNR|SSIM|None|Brightness|Contrast|Jpeg|Blur|Noise|Vae-b|Vae-c|Diffusion|Avg|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|StegaStamp|28.47 |0.91| 0.999 | 0.999 | 0.998 | 0.994 | 0.997 | 0.991 | 0.981 | 0.984 | 0.857 | 0.978 |\n|FreqMark  |28.83 |0.88|1.000  |1.000|0.999|0.999|0.998|0.989| 0.976|0.934 |0.933|0.981|\n\nThe findings indicate that FreqMark presents a balanced performance and exceptional robustness against diffusion attacks. Additionally, its network independence provides great flexibility and security. Users can dynamically choose the optimal image quality and encoding bit number, and customize their decoding vector group, thereby augmenting the security of the watermark decoding process.\n\n> W2: The authors also needed to compare against the state-of-the-art Tree-Ring watermark [4].\n\n**A2:** Tree-Ring integrates the watermarking process and generation process, encoding 1-bit message (with or without watermark) through its approach, which is an in-process method. FreqMark is a post-processing watermarking technique capable of encoding multi-bit message, thus offering a broader range of applicability and a higher amount of encoded message. Therefore, following the metric setup of Tree-Ring, we compared the TPR@1%FPR of FreqMark and Treering under various attacks.\n|TPR@1%FPR|None|Brightness|Contrast|Jpeg|Blur|Noise|Vae-b|Vae-c|Diffusion|Avg|\n|-|-|-|-|-|-|-|-|-|-|-|\n|Tree-Ring|1.000|1.000|1.000|0.996|0.999|0.918|0.991|0.995|0.998|0.989|\n|FreqMark|1.000|1.000|1.000|1.000|1.000|0.986|0.989|0.975|1.000|0.994|\n\n\nThe experimental results demonstrate that the robustness performance of both methods is very close. However, it is important to note that FreqMark encode 48-bit message, which is a significantly larger amount of information compared to Treering's 1-bit encoding.\n\n> W3: As noted in [3], there is no single perceptual metric that is an objective measure of image quality...\n\n**A3:** Thank you for your suggestion. We have added CLIP-FID and L2 as new image quality metrics, with the results as follows:\n\n||DwtDct|SSL|Stable Signature|StegaStamp|FreqMark|\n|-|-|-|-|-|-|\n|CLIP-FID|2.36|6.88|1.70|5.50|3.84|\n|L2|7.71|52.06|63.74|85.24|52.95|\n\nThe experimental conclusions regarding image quality are consistent with the results in the paper, demonstrating that FreqMark exhibits outstanding robustness performance while maintaining acceptable image quality.\n\n> W4: 500 images is too small a sample size for the tested FPR thresholds. \n\n**A4:** We utilized 5,000 watermarked images to plot the FPR-TPR curve, and some key results are as follows:\n\n||1e-3|3e-5|3e-7|7e-10|1e-13|\n|-|-|-|-|-|-|\n|Jpeg|1.000|0.999|0.992|0.989|0.978|\n|Noise|0.983|0.979|0.934|0.851|0.575|\n|Vae-b|0.987|0.967|0.935|0.819|0.484|\n|Vae-c|0.971|0.961|0.891|0.689|0.385|\n|Diffusion|1.000|0.996|0.990|0.927|0.636|\n\nThe experimental results are close to those in the main text. We will update the experimental results in the paper accordingly.\n\n> W5: VAE regenerations are far weaker compared to diffusion regenerations. \n\n**A5:**  We have gradually increased the diffusion steps. The results are as follows:\n\n|Steps|60|80|100|120|140|160|180|200|\n|-|-|-|-|-|-|-|-|-|\n|Bit Acc|0.935|0.863|0.831|0.754|0.712|0.692|0.660|0.637|\n\nIt can be seen that FreqMark maintains good robustness even at higher diffusion steps.\n\n> W6: As observed in [3], the use of publicly available VAEs to encode/decode watermarks is easily defeated...\n\n**A6:** The characteristic of FreqMark that trains the image itself rather than the network can ensure strong robustness when facing regeneration attacks using the same VAE.\n\n|PSNR after Vae Attack|31.43|30.31|28.98|27.39|25.82|\n|-|-|-|-|-|-|\n|Bit Acc|1.000|1.000|0.998|0.990|0.975|\n\nThis is because the perturbation on the latent FFT changes the overall distribution of the image latent, making the watermark message affect the entire image globally. Therefore, it is difficult for perturbations on the image's latent to damage the watermark message. A similar phenomenon is observed in the pixel FFT of the image.\n\n|PSNR after Gaussian Noise Disrupt in Pixel Domain|31.09|29.68|28.04|26.46|25.04|\n|-|-|-|-|-|-|\n|Bit Acc|1.000|1.000|1.000|1.000|1.000|\n\n> Q1: Which version of Stable Diffusion was used for the regeneration attack?\n\n**A7:** Following the experimental setup of [2], we employ stable diffusion 2-1 to conduct diffusion attacks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nQaE8Xok7P", "reviewer": "Reviewer_BgZJ", "summary": "This paper proposes a method called FreqMark that is able to prevent the invisible watermarks from the regeneration attack. By using the unconstrained optimization of the image latent frequency space obtained after VAE encoding, the proposed FreqMark achieves better robustness against the regeneration attacks and traditional attacks.", "questions": "I have shown my concerns in the weakness part. The authors are suggested to consider them during the rebuttal.", "limitations": "Please address my concerns within the weakness part.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The proposed method achieves better robustness against regeneration attacks. By using a newly proposed optimization strategy, the proposed method achieves a balance between the image quality and the envisioned robustness.", "weaknesses": "1. The major contribution of this paper is to introduce a kind of image watermark robust to regeneration attacks. However, this part is not highlighted in its title. The authors are suggested to consider this in the title. \n2.  The proposed methodology follows an established manner with incremental innovations. The whole approach does not demonstrate enough differences when compared with the previous approaches. \n3. For the second contribution, the authors claim that their proposed framework is flexible and then say such flexibility guarantees a trade-off between the bits number, image quality, and robustness. However, why can such properties be considered as flexibility?\n4. The whole pipeline presented in Figure 2 lacks novelty. This is just a very common encoder and decoder framework. The authors need to better consider their contributions for this part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer BgZJ,\n\nThank you for your insightful feedback and questions. We address the concerns as follows:\n\n> W1: The major contribution of this paper is to introduce a kind of image watermark robust to regeneration attacks. However, this part is not highlighted in its title. The authors are suggested to consider this in the title.\n\n**A1:** Thank you for your suggestion, we will consider it positively. Yes, FreqMark's ability to resist regeneration attacks is a significant advantage, and we believe the primary contribution of the paper lies in the innovative approach of embedding watermark in the latent frequency space, and the extensive experiments conducted to demonstrate its effectiveness.\n\n> W2: The proposed methodology follows an established manner with incremental innovations. The whole approach does not demonstrate enough differences when compared with the previous approaches.\n\n**A2:** The core idea of FreqMark lies in embedding watermarks in the latent frequency space of images. Frequency domain optimization subtly alters the overall image, enabling the watermark information to be hidden within the entire picture. The latent domain represents the compressed image, and during the process of restoring the image using the decoder, the added watermark information naturally integrates with the semantic information of the image. This results in outstanding robustness when facing regeneration attacks and various other types of attacks.\n\n*Bit Accuracy on both ImageNet and DiffusionDB datasets:*\n\n| Bit Acc | JPEG | Gauss noise | VAE-B | VAE-C | Diffusion |\n| - | - | - | - | - | - |\n| SSL | 0.964 | **0.946** | 0.768 | 0.761 | 0.736 |\n| Stable Signature | 0.864 | 0.930 | 0.6490 | 0.689 | 0.531 |\n| **FreqMark** | **0.996** | 0.937 | **0.932** | **0.911** | **0.957** |\n\nFurthermore, our experiments demonstrate the impact of different watermark embedding positions on robustness, and we delve into the advantages and limitations of FreqMark.\n\n*Bit Accuracy on DiffusionDB datasets:*\n\n| Bit Acc | JPEG | Gauss noise | VAE-B | VAE-C | Diffusion |\n| - | - | - | - | - | - |\n| Pixel Frequency | 0.950 | **1.000** | 0.797 | 0.775 | 0.596 |\n| Latent | 0.906 | 0.804 | 0.796 | 0.833 | 0.675 |\n| **Latent Frequency** | **1.000** | 0.934 | **0.925** | **0.897** | **0.945** |\n\nFreqMark also boasts several additional advantages. It is a plug-and-play solution that can be directly applied to existing pre-trained models without any network training. During the watermark embedding process, users can freely adjust parameters to suit their specific needs, offering considerable flexibility, which will be discussed in detail in the W3.\n\nIt is important to emphasize that FreqMark differs from previous methods. \n- FreqMark, as a post-generation method, is fundamentally distinct from merged-in-generation methods like Stable Signature, and it offers greater convenience and flexibility in use.\n- Unlike most works such as StegaStamp, FreqMark does not require the training of any network parameters.\n- Compared to other network-training free methods like SSL, FreqMark substantially enhances robustness through its unique watermark embedding approach.\n\n> W3: For the second contribution, the authors claim that their proposed framework is flexible and then say such flexibility guarantees a trade-off between the bits number, image quality, and robustness. However, why can such properties be considered as flexibility?\n\n**A3:** There are three crucial attributes for invisible image watermarking—bits number, image quality, and robustness. Based on information theory, these attributes form an impossible trinity, meaning it is impossible to achieve all three simultaneously. For instance, if a method achieves high bits number and image quality, it inevitably cannot achieve high robustness; similarly, if it aims for high bits number and robustness, it cannot maintain high image quality. Thus, users need to make trade-offs among these attributes based on different application scenarios.\n\nSpecifically, the flexibility of FreqMark is reflected in the following tow aspects:\n\n1. Merged-in-generation methods fix the model's capabilities during the training process, which prevent users from choosing between three attributes. FreqMark grants this choice to users, allowing them to freely adjust parameters during the watermark embedding process to achieve the desired balance.\n2. Users can independently choose the VAE model and image encoder. Additionally, the pre-defined vector can be customized by users, serving as a key that adds an extra layer of security to the watermark.\n\n> W4: The whole pipeline presented in Figure 2 lacks novelty. This is just a very common encoder and decoder framework. The authors need to better consider their contributions for this part.\n\n**A4:** Figure 2 illustrates the entire process of watermark embedding and extraction, while the main idea of the paper is depicted in Figure 1. We still believe that presenting the overall process is necessary, as it helps readers better understand our method. Although Figure 2 reflects the encoder-decoder framework, it differs from other methods in the following ways:\n\n1. FreqMark uses only pre-trained encoders and decoders without requiring any network training. In Figure 2, we use a \"snowflake\" icon to denote the frozen neural networks.\n2. FreqMark achieves watermark embedding by training perturbations in the latent frequency space. In Figure 2, we use a \"fire\" icon to indicate the trained perturbations.\n\nThank you for your suggestion. We will consider improving or adding image to better highlight our contributions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DyXkoDJetk", "reviewer": "Reviewer_kwvu", "summary": "This paper introduces FreqMark, a novel invisible watermarking method that enhances digital content protection through optimization in the image's latent frequency space. Experiments have been conducted to demonstrate the robustness against regeneration attacks like VAE and diffusion model.", "questions": "Please see the Weaknesses.", "limitations": "The authors have addressed the limitations in their paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and well-organized.\n\n2. Extensive experiments have been conducted to demonstrate the robustness against various attacks including VAE and diffusion models.", "weaknesses": "1. Lacks some theoretical analysis. In the experiments, do you need to use the same VAE model for the attack as you have used during watermarking training? If not, then an analysis may be needed as to why it is still robust against attacks with different VAE models.\n\n2. Since the proposed method requires case-by-case optimization, what is the watermarking time for each case, and how does it compare to other competing methods?\n\n3. Why do we need a set of pre-trained N-dimensional direction vectors, but not directly produce the message? \n\n4. In Equation (4), why $k \\in N$? $N$ is a number, not a set, so $\\in N$ is not appropriate. Besides, using $v^N$ to denote a vector set and $v^k$ to represent a vector can lead to confusion. In Figure 2, there are $K+1$ vectors in the set, I think $K+1$ is the bit length.\n\n4. Minors:\n- $\\epsilon_1$ and $\\epsilon_2$ (or $\\varepsilon_1$ and $\\varepsilon_2$) appear in Figure 2, so it is better to mention them in the caption of Figure 2.\n- In line 213, page 7, the word 'We' should be 'we'.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer kwvu,\n\nThank you very much for your careful reading and recognition of our work, we will address each one sequentially below:\n\n> W1: Lacks some theoretical analysis. In the experiments, do you need to use the same VAE model for the attack as you have used during watermarking training? If not, then an analysis may be needed as to why it is still robust against attacks with different VAE models.\n\n**A1:** In the experiment, it is not necessary to use the same VAE model for attacks as the one used for watermark training. \n\nEssentially, adding information will inevitably alter some aspects of the image, and our goal is to ensure that this information remains as intact as possible during attacks. As demonstrated in Section 5.3 and Table 3, different optimization methods exhibit distinct characteristics. \n- Directly embedding watermarks in the Pixel or Latent domains makes them highly susceptible to various traditional attacks, as specific bits of information may independently reside in one or more pixels. In contrast, embedding watermarks in the frequency domain effectively resists multiple traditional attacks because this method distributes the information across the entire image. \n- Similarly, watermarks embedded in the pixel space are weak against regeneration attacks, whereas embedding watermarks in the latent space significantly enhances robustness against such attacks. This is due to the stronger correlation between the optimized latent embeddings and the image semantics, making the watermark more resistant to disruption by regeneration attacks.\n\nFreqMark embeds watermarks in the latent frequency space, combining these two aspects to produce a synergistic effect, making FreqMark robust against traditional attacks and regeneration attacks.\n\nFurthermore, due to FreqMark's characteristic of training the image itself, it exhibits stronger robustness against regeneration attacks using the same VAE.\n\n*Bit Accuracy of regeneration attack using the same VAE*\n\n| PSNR after VAE attack | 31.43 | 30.31 | 28.98 | 27.39 | 25.82 |\n| - | - | - | - | - | - |\n| Bit Acc | 1.000 | 1.000 | 0.998 | 0.990 | 0.975 |\n\n> W2: Since the proposed method requires case-by-case optimization, what is the watermarking time for each case, and how does it compare to other competing methods?\n\n**A2:** We employed half-precision training to reduce watermarking time and GPU memory usage without compromising performance. In our current experiments, using a single A-100 GPU with 40GB memory, and process four 512x512 images in parallel for 400 steps takes about 0.75 minutes per image. Increasing the batch size will improve overall efficiency.\n\nAs shown in Figure 7 in the Appendix, FreqMark demonstrates considerable performance at just 200 steps, and even at 100 steps. This allows for the reduction of steps as needed to save time. According to our experiments, processing four 512x512 images in parallel for 100 steps can be accelerated to about 12 seconds per image.\n\n*Bit Accuracy with different number of steps*\n| Bit Acc | JPEG | Gauss noise | VAE-B | VAE-C | Diffusion |\n| - | - | - | - | - | - |\n| 400 steps |1.000 | 0.934 |0.925 |0.897 |0.945 |\n| 200 steps |1.000 | 0.930 |0.923 |0.885 |0.933 |\n| 100 steps |0.987 | 0.922 |0.921 |0.881 |0.888 |\n\nTo train the Stable Signature, one must first spend a day training the watermark extractor on 8 GPUs. Subsequently, for any specific hidden message, the stable signature requires approximately 1 minute to fine-tune the VAE decoder.\n\nSSL directly optimizes the image pixels, resulting in faster processing speed. Under the same experimental settings, it takes approximately 1 second to process per image. However, FreqMark demonstrates a significantly stronger robustness advantage than SSL.\n\nWe will further attempt to reduce the optimization time without compromising performance.\n\n\n> W3: Why do we need a set of pre-trained N-dimensional direction vectors, but not directly produce the message?\n\n**A3:** The method of extracting hidden watermark information by predefined vector directions is referenced from SSL. Overall, this approach is more suitable for post-generation self-supervised methods, providing stronger robustness for hidden watermarks while maintaining image quality. Additionally, this method effectively addresses the challenge of obtaining hidden watermark information without the need for additional training of the extraction network.\n\n> W4: In Equation (4), why $k∈N$? N is a number, not a set, so $∈N$ is not appropriate. Besides, using $v^N$ to denote a vector set and $v^k$ to represent a vector can lead to confusion. In Figure 2, there are $K+1$ vectors in the set, I think $K+1$ is the bit length.\n\nThank you for your careful reading and for pointing out the errors. This section of the paper indeed has some incorrect expressions that caused confusion, and we will make the necessary corrections.\n\nThe dimension of the vector is N-dimensional. For clarity and expression, the number of pre-defined vectors is defined as K (we will correct Vector 0 to Vector 1 in Figure 2). To avoid interference between different bits of information, the pre-defined vectors should be orthogonal to each other, thus K should be less than or equal to N. This means that a maximum of N bits of information can be embedded for an N-dimensional vector.\n\nTherefore, The correct expression should be that the pre-defined vectors are a set of K N-dimensional vectors $V^N = \\\\{v^1, v^2, ..., v^K \\mid K\\leq N\\\\}$.\n\n And Equation 4 should be corrected to: \n\n$m_d^k = sign(z_{I_{w}} \\cdot v^k) = sign(E_{img}(I_{w}) \\cdot v^k), v^k \\in V^N$\n\nWe will continue to carefully review and correct any mistaken and confused statements in the paper. \n\nThanks again for your suggestions and corrections!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cDdsIEOfBH", "reviewer": "Reviewer_TJux", "summary": "This paper considers the problem of watermarking of images. In particular, it introduces a method called FreqMark where watermark is embedded in the latent frequency space obtained after variation auto encoder (VAE) encoding. Numerical results are carried out on test datasets containing 500 randomly selected images from the ImageNet and 500 images from the DiffusionDB dataset.  Comparative results against DwtDctSvd, Stable Signature and SSL indicate that FreqMark exhibits superior quality-robustness tradeoff compared to the benchmark methods.", "questions": "1. Are the FFT outputs complex and if so, how are the watermark bits being embedded?\n\n2. What do FFT outputs represent when applied to the latent vectors? When applied to 1D time signals or 2D images, the FFT output describes the frequency content of the signals and images. What does the FFT of latent space represent and why is it useful?", "limitations": "Authors mention that there is room for optimization in the fusion of images and watermarks. It would have been useful to have a more detailed discussion of this aspect of the proposed method.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The main strengths of this paper are the novelty of the proposed FreqMark algorithm and the superior performance of FreqMark compared to the baseline image watermarking methods.\n\nThe main idea behind FreqMark is hiding the message bits in the FFT outputs of latent vectors coming from the VAE. In numerical experiments on two image datasets (ImageNet and Diffusion DB), authors show that FreqMark provides a bit accuracy of better than 90% for a 48-bit encoding setting. The Ablation Studies investigating image quality, number of encoding bits, noise levels and spatial perturbations are reasonably convincing of the benefits offered by FreqMark.\n\nThe Appendices provided contain significant additional information regarding FreqMark and is one of the strengths of this paper.", "weaknesses": "One of the main weaknesses of the work is that there is no convincing proof or theoretical justification of why hiding watermark bits in the FFT of latent space should be beneficial. The usual argument of using frequency domain for hiding watermark bits makes sense in that the FFT is applied to images (i.e., image pixels) and captures the spatial relationships between the image pixels. What kind of \"spatial\" relationship is captured by the FFT in the latent space and why is that beneficial for hiding watermark?\n\nFFT produces complex values, yet there is no discussion of how the watermark bits are embedded on these complex values. In Fig. 1, the outputs of FFT are displayed as magnitude images and if this is what is intended, what happened to the phase of the FFT outputs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer TJux, \n\nWe sincerely appreciate your thoughtful review and constructive suggestions on our paper. We have carefully considered your comments and have taken them into account.\n\n> W1: One of the main weaknesses of the work is that there is no convincing proof or theoretical justification of why hiding watermark bits in the FFT of latent space should be beneficial. The usual argument of using frequency domain for hiding watermark bits makes sense in that the FFT is applied to images (i.e., image pixels) and captures the spatial relationships between the image pixels. What kind of \"spatial\" relationship is captured by the FFT in the latent space and why is that beneficial for hiding watermark?\n\n**A1:** The primary distinction between the latent and pixel domains is that the latent domain is the result of information compression and downsampling of the pixel domain. Training on the pixel FFT alters the overall distribution of the image, making it more robust against common attacks. A similar effect occurs in the latent FFT domain. The key difference is that the compressed latent information is restored back to the pixel domain with the assistance of the model's prior knowledge through the VAE decoder. During this process, the watermark information added to the latent FFT interacts and fuses with the semantic information of the image. Consequently, FreqMark exhibits enhanced robustness against regeneration attacks, as detailed in Section 5.3. This also serves as the answer to question 2.\n\n*Bit Accuracy on DiffusionDB datasets:*\n\n| Bit Acc       | JPEG      | Gauss noise | VAE-B     | VAE-C     | Diffusion |\n| - | - | - | - | - | - |\n| Pixel Frequency      | 0.950     | **1.000**   | 0.797     | 0.775     | 0.596     |\n| Latent               | 0.906     | 0.804       | 0.796     | 0.833     | 0.675     |\n| **Latent Frequency** | **1.000** | 0.934       | **0.925** | **0.897** | **0.945** |\n\n> W2: FFT produces complex values, yet there is no discussion of how the watermark bits are embedded on these complex values. In Fig. 1, the outputs of FFT are displayed as magnitude images and if this is what is intended, what happened to the phase of the FFT outputs.\n\n**A2:** The output of latent FFT is complex, and we optimize both the real and imaginary parts during the optimization process. \n\n\nThe latent FFT is then restored to the pixel domain through IFFT and the VAE decoder. The perturbation in the latent FFT domain alters the feature vector obtained after passing the image through an image encoder (DINO v2 in this case), thus achieving message encoding. \n\nFor example, the output vector after perturbation for an image is [-2, 1, -0.5] . We predefine three vector groups as [1, 0, 0], [0, 1, 0], and [0, 0, 1] (assuming a 3-bit message encoding for simplicity). After performing element-wise multiplication, we obtain the results -2, 1, and -0.5. We use the sign of these results as the 0-1 bit encoding, ultimately yielding an encoded message of **010**. This also serves as the answer to question 1.\n\n> L1: Authors mention that there is room for optimization in the fusion of images and watermarks. It would have been useful to have a more detailed discussion of this aspect of the proposed method.\n\n**A3:** Experimental results demonstrate that the image quality post watermark addition via FreqMark does not significantly deteriorate compared to the image quality achieved solely through VAE encoding and decoding. Therefore, a VAE with enhanced performance may potentially further improve the quality of the watermarked images.\n\n| Method               | PSNR      | SSIM        |\n| -------------------- | --------- | ----------- |\n| FreqMark             | 31.20±1.54| 0.854±0.029 |\n| VAE Reconstruction   | 31.22±1.96| 0.879±0.032 |\n\nWe plan to further optimize our method from the following aspects:\n\n1. Exploring the necessity of optimizing all elements in the latent FFT could be a focus for further investigations. Achieving better integration might be possible by optimizing only a portion or a specific channel while maintaining performance.\n  \n2. The utilization of diffusion models' robust capabilities could potentially facilitate the implantation of watermarks, resulting in superior fusion effects.\n\n3. With the assurance of performance, a strategy could be developed to further blend the watermarked image post FreqMark processing with the original image, potentially leading to enhanced fusion effects.\n\nWe genuinely appreciate your valuable suggestions and will continue to optimize and explore our methods with enthusiasm.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper has received four reviews, with three recommendations weakly recommending to accept the paper (6,6,4->6) and one weakly recommending to reject it (4)

The more critical review recognizes the improved robustness of the approach to regeneration attacks.  They are critical of the level of novelty in the technical contribution, considering it to be yet another variant of an encoder-decoder architecture.  They also query the motivation for flexibility in trading off the capacity, quality and robustness.

As the authors note, it is a well-known tradeoff in watermarking approaches to trade between capacity, perceptibility and robustness and enabling control over this is useful given different requirements of different use cases.  Therefore I would consider this a reasonable feature to highlight in the work.  Regarding the novelty, the authors have responded on this point.  I think it is reasonable that most modern ML watermarking approaches are some form of encoder-decoder approach – it appears the contribution here is actually to work with the latent spatial representation (running an optimization to perturb the 2D FFT of that) to introduce a mark.  It seems novel to my knowledge although I would agree not a huge contribution, it is an interesting take on the problem provided the analysis is done well enough to show a utility in the study.

The more positive reviews weakly recommend acceptance but have a number of criticisms.

BZ7h is concerned that the method is similar to StegaStamp which is another encoder-decoder approach that many papers will baseline against.  I can see that view but I think it is the FFT perturbation aspect of this work in the latent that distinguishes it.  The more serious criticism of this review is that StegaStamp and TreeRing are not compared against (particularly given TreeRing is a frequency method) and that primarily VAE regeneration is being considered as an attack, given the growing use of diffusion / img2img approaches.  That said, the authors provide the comparison in the rebuttal which helps mitigate these important shortfallings. The reviewer has moved their score to 5 and then to 6 ultimately in later discussions.  I feel this reviews critcisms were well addressed in the end.

kwvu mainly raises points of clarification in the technical exposition which must be carefully addressed in any final version.  They ask clarification questions about the experimental design that are well answered in the rebutall and the initial score to weakly accept is maintained.

TJux would like to see deeper theoretical discussion in the paper and has some questions around how the secret is embedded in the complex (FFT) domain. I think it is a reasonable criticism – this is not well explained in the paper, but ultimately this is because there is not a direct encoding or theoretical basis but simply a backprop optimization to perturb the complex numbers.  The reviewer seems satisfied with the response.

Overall I think this is a borderline acceptable paper.  The method is somewhat novel in its approach to directly perturb the latent signal in FFT space.  It is not a massive innovation but the experimental work is complete enough that it will provide some knowledge to the community of how far such an approach can be pushed to add robustness to a watermark.  Based on the above summary I consider all the criticisms were addressed and some important experimental results added, all of which must be present in the camera ready copy.  My recommendation is to accept the paper.
Decision: Accept (poster)