Can Simple Averaging Defeat Modern Watermarks?

Authors: pei  yang
hai  ci
yiren  song
mike zheng shou
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 12:47:49

Abstract: Digital watermarking techniques are crucial for copyright protection and source identification of images, especially in the era of generative AI models. However, many existing watermarking methods, particularly content-agnostic approaches that embed fixed patterns regardless of image content, are vulnerable to steganalysis attacks that can extract and remove the watermark with minimal perceptual distortion. In this work, we categorise watermarking algorithms into content-adaptive and content-agnostic ones, and demonstrate how averaging a collection of watermarked images could reveal the underlying watermark pattern. We then leverage this extracted pattern for effective watermark removal under both greybox and blackbox settings, even when the collection of images contains multiple watermark patterns. For some algorithms like Tree-Ring watermarks, the extracted pattern can also forge convincing watermarks on clean images. Our quantitative and qualitative evaluations across twelve watermarking methods highlight the threat posed by steganalysis to content-agnostic watermarks and the importance of designing watermarking techniques resilient to such analytical attacks. We propose security guidelines calling for using content-adaptive watermarking strategies and performing security evaluation against steganalysis. We also suggest multi-key assignments as potential mitigations against steganalysis vulnerabilities. Github page: \url{https://github.com/showlab/watermark-steganalysis}.

Main paper: 
 introduction: 
 Digital watermarking hides information within digital media, facilitating copyright protection and source authentication [1] [2] [3] . With the advances in AI-based image generation and editing [4] [5] [6] [7] , robust and secure digital watermarking is crucial for preventing deepfake misuse or manipulations of created contents [8, 9] .We categorise digital watermarking methods into two types: content-adaptive and content-agnostic. Content-adaptive methods take images into the watermarking process, dynamically adjusting the watermark's placement and strength based on the image content, as seen in technologies like HiD-DeN [10] and RivaGAN [11] . Content-agnostic methods, however, use fixed, predefined watermark patterns independent of or weakly dependent on image content. Apart from traditional methods like DwtDctSvd [12] , this also includes RoSteALS [13] that adds image-independent additive perturbations, and Tree-Ring [14] that places a ring pattern to the initial noise of a diffusion generation process. Content-adaptive methods typically offer better robustness against image processing distortions, while content-agnostic methods are computationally lighter and easier to implement.A fundamental requirement for digital watermarks is robustness, ensuring watermarks cannot be easily removed or tampered with [2] . To meet the requirement, existing methods have been improving watermark robustness through design considerations [14] or data augmentation during training [10] , and have demonstrated strong robustness to various image distortions like noise perturbations or JPEG compression [15, 14, 16] . Works like Tree-Ring [14] even demonstrated robustness against strong attacks like VAE compression and image re-generation [17, 18] . In this paper, however, we reveal that content-agnostic watermarking techniques, including Tree-Ring, are vulnerable to steganalysis attacks, unmasking their hidden fragility.To the best of our knowledge, our steganalysis is the first successful blackbox attack against Tree-Ring watermarks [14] . We discovered a content-agnostic ripple pattern in Tree-Ring-watermarked images and identified that this component is essential for watermark detection. Subtracting this pattern allows evading watermark detection with minimal impact on image perceptual quality. This raises the question: Do diffusion model watermarking methods that modify initial noise [19, 20, 14, 16] truly add semantic watermarks, or do they merely propagate low-level content-agnostic patterns to generated images? Lastly, we propose new security guidelines for the watermarking community, emphasizing the importance of robustness against steganalysis. The guidelines call for performing evaluations against steganalysis when proposing new watermarking methods. It also encourages the development of content-adaptive watermarking methods to enhance resistance to steganalysis. For existing contentagnostic watermarking methods, we suggest assigning multiple watermarks per user as a mitigation strategy. In summary, the main contributions of this paper are:• We reveal the vulnerability of content-agnostic watermarking methods to steganalysis removal and forgery.• To the best of our knowledge, we are the first to successfully attack Tree-Ring watermarks in a blackbox setting, providing deeper insights into the essence of diffusion noise watermarking methods.• We propose new security guidelines for future watermarking methods to help defend against steganalysis attacks.2 Related works 
digital image watermarking: 
 The field of digital image watermarking has evolved from traditional rule-based approaches to more recent deep learning-based techniques, with a significant focus on watermarking diffusion-generated images [14, 21, 22] . We categorise digital watermarking technologies into content-agnostic, which craft modifications based solely on the watermark information, and content-adaptive, which tailor modifications based on both watermark information and the image content.Content-agnostic watermarking Traditional methods like DwtDctSvd [12] employ fixed watermark patterns in transform domains, while more recent approaches modify the initial noise for diffusion-based image generation. Tree-Ring watermarks [14] replace the low-frequency Fourierdomain pixels of Gaussian noise with a ring pattern before using it for diffusion denoising. Similarly, Gaussian Shading [19] preserves the distribution while sampling the initial noise. Other approaches [13, 23, 24] train encoders to generate additive watermark perturbations without conditioning on image features.Content-adaptive watermarking These techniques leverage image features to generate watermarks tailored to the input image content. Early encoder-decoder methods like HiDDeN [10] and StegaStamp [15] employed deep neural networks to imprint watermarks onto the images. SSL [25] leveraged selfsupervised networks as feature extractors, while RivaGAN [11] used attention mechanisms to look for appropriate local regions for watermark encoding. Recent approaches like Stable Signature [22] , WADiff [21] , and Zhao et al. [26] finetune the diffusion model to enable content-aware watermarking of diffusion-generated images. WMAdapter [27] designs a dedicated contextual adapter.Through the classification, we highlight the vulnerability of content-agnostic techniques to steganalysis attacks, as they employ fixed or weakly content-dependent watermark patterns.Traditional attacks applied distortions to disrupt watermarks, performing signal-level distortions like image compression, noise perturbation, blurring or colour adjustment, and geometric transformations like rotation or cropping [28-30, 10, 12] . In terms of videos, codecs may also distort invisible watermarks [31, 32] . These attacks fool watermark detectors at the cost of significant image quality degradation. To resist such attacks, training-based methods have been simulating the distortions via "attack layers" during training [10, [33] [34] [35] , while training-free methods have been employing design considerations such as watermarking only low-frequency components [14] .Recent attacks base on deep models: regeneration attacks with diffusion models [36] and VAEs can provably remove pixel-level invisible watermarks [37, 18] . But such attacks are shown to be ineffective [17] for Tree-Ring [14] that alters the image a lot. When attackers can access the watermarking algorithm, they may also perform adversarial attacks [38] . The downside is that both regeneration and adversarial attacks are computationally expensive. In contrast, we propose a new type of blackbox steganalysis attack, which is efficient and works for content-agnostic watermarks.Steganalysis can extract meaningful watermark patterns, thus promoting further applications like forgery or explainability.3 Watermark steganalysis 
notations: 
 Let x ∅ denote the original digital image, w the watermark information (e.g., bit sequence or geometric pattern), and E the watermark encoder that imprints w into x ∅ , yielding the watermarked image x w = E(x ∅ , w). The embedding constraint ensures that x ∅ and x w are perceptually indistinguishable. A watermark decoder D recovers the embedded information ŵ = D(x w ) for authentication purposes.The adversary aims to fool D by manipulating x w using a strategy denoted as T (•) such that D(T (x w )) ̸ = w (watermark removal) or manipulating x ∅ such that D(T (x ∅ )) = w (watermark forgery). Formally, the adversary solves:EQUATIONsubject to the constraint that the original image x and the manipulated image T (x) are perceptually indistinguishable. Rather than applying strong distortions as T , we demonstrate that the adversary can take a steganalysis approach to fool D.Figure 1 illustrates our watermark removal/forgery strategy T , which assumes that E perturbs an additive pattern δ w agnostic to image content, such that x w = x ∅ + δ w . This assumption can be refined based on a detailed understanding of specific watermarking algorithms (as will be showcased in Section 4.4.1). Under this additive assumption, to either remove or forge watermarks, we can approximate δw = x w -x ∅ . To improve approximation and reduce randomness, we propose averaging over n images during pattern extraction:EQUATIONWith the approximated δw , the adversary can perform greybox watermark removal (x ∅ = T (x w ) = x w -δw ) or forgery (x w = T (x ∅ ) = x ∅ + δw ) on a given image x. Even without paired x ∅ , the adversary can perform blackbox removal/forgery by approximating x ∅ through averaging any Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]Answer: [No] Justification: The code and data are not provided in this manuscript submission.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setup: 
 Image experiments We evaluate our proposed steganalysis on 12 existing image watermarking methods: Tree-Ring [14] , RingID [16] , RAWatermark [23] , DwtDctSvd [12] , RoSteALS [13] , Gaussian Shading [19] , Stable Signature [22] , WmAdapter [27] , RivaGAN [11] , SSL [25] , HiDDeN [10] , and DwtDct [12] . For the greybox setting, we use the COCO2017 [39] validation set for WmAdapter [27] and Stable Signature [22] , Stable Diffusion Prompts [40] for Tree-Ring [14] and RingID [16] prompts, and DiffusionDB [41] for the remaining methods as the non-watermarked images (x ∅ ). The corresponding watermarked images (x w ) are generated using the respective watermarking methods. In the blackbox setting with no access to paired images, we substitute x ∅ with ImageNet [42] test set. The selection of images within the datasets is random. The datasets are resized to 256×256 for RoSteALS, SSL, and HiDDeN, and 512×512 for other methods.We assess watermark removal under different n (number of images averaged) during watermark pattern extraction, and test on 100 images3 during watermark removal. We report detection AUC for Tree-Ring [14] and RAWatermark [23] , and watermark decoding bit accuracy for the other methods. Additionally, we evaluate the image quality between x w and its non-watermarked counterpart, reporting PSNR in the main text and SSIM, LPIPS [43] , and SIFID [44] in the appendix.We then extend the experiments to audio watermark removal on AudioSeal [45] and WavMark [46] , using the zh-CN subset of the Common Voice dataset [47] . Each audio segment is preprocessed to a 16 kHz mono format, with only the first two seconds retained. We use paired audio for greybox removal, and unpaired audio for blackbox removal. We report the watermark detection accuracy for AudioSeal [45] , and watermark decoding bit accuracy for WavMark [46] . To quantify the audio quality after watermark removal, we calculate the Scale-Invariant Signal-to-Noise Ratio (SI-SNR) between the watermark-removed audio and its non-watermarked counterpart.The experiments were conducted on an AMD EPYC 7413 24-Core Processor and an Nvidia RTX 3090 GPU, requiring around 200GB of disk space. The execution time for each experiment ranges from around 10 minutes (HiDDeN) to around 10 hours (Tree-Ring).As shown in Figure 2 (left column), our steganalysis-based watermark removal method effectively degrades the detection performance of RAWatermark (0.5744 AUC), DwtDctSvd (0.5722 accuracy), Tree-Ring (0.2407 AUC), RoSteALS (0.2444 bit accuracy), and Gaussian Shading (0.5615 bit accuracy). The results highlight two key findings: (1) the aforementioned methods embed contentagnostic watermarks, and (2) content-agnostic watermarks are susceptible to steganalysis-based removal.The effectiveness of our method increases as n decreases, albeit at the cost of increased image distortion (smaller PSNR). In contrast, content-adaptive watermarking methods (Figure 2 right column) demonstrate robust resistance to this attack, maintaining high detection accuracy (> 0.95) upon convergence. This resilience underscores the importance of content-adaptivity in watermark design to thwart steganalysis-based removal attacks.In this qualitative analysis, we first examine the patterns extracted from various watermarking methods, then discuss how the removal of these watermarks affects image quality.Extracted patterns Figure 9 displays patterns extracted from content-agnostic methods, while Figure 10 shows those from content-adaptive methods. Content-agnostic watermarks tend to exhibit unique patterns, like fingerprints. For example, RingID [16] patterns are concentric rings with bright spots in the centre, DwtDctSvd [12] patterns resemble vertical lines like barcodes, and RoSteALS [13] patterns appear as grid-like patches with non-uniform illumination. In contrast, patterns extracted from content-adaptive watermarks are less discernible. Notably, under the greybox setting, the HiDDeN-extracted [10] pattern converges to zero, indicating completely no discernible pattern. Furthermore, patterns extracted in the greybox setting contain fewer visual artifacts than those in the blackbox setting. As more images are averaged, the extracted watermark pattern becomes clearer and more precise, with fewer residual artefacts from image contents. A more detailed analysis is presented in Appendix A.3. 12 and 13 show the visual impact of removing content-agnostic and content-adaptive watermarks, respectively. For all methods but Gaussian Shading [19] , under the greybox setting, when more than 50 images are averaged, virtually no visual artefacts remain after watermark removal. In the blackbox setting, averaging over 100 images is necessary to eliminate most artifacts. The exception is Gaussian Shading [19] , which consistently produces visible artefacts because the pattern extracted has a large magnitude. Subtracting this pattern significantly distorts the image. Appendix A.4 provides a more detailed analysis.To further reveal how steganalysis can be a threat to content-agnostic watermarking algorithms, we conduct a case study on Tree-Ring watermarks [14] . Tree-Ring is a sophisticated diffusion-based watermarking algorithm that injects a frequency-domain ring pattern into a Gaussian noise signal before using this watermarked noise for diffusion-denoising image generation. During detection, it performs DDIM inversion to recover the injected ring pattern from the initial noise and compares it to a reference pattern. In the following experiments, we demonstrate that with minimal modifications, we can both remove and forge Tree-Ring watermarks under different scenarios.This section focuses on revealing the low-level content-agnostic component of Tree-Ring watermarks [14] . First, we curate a specific steganalysis for Tree-Ring's detection algorithm, demonstrating how tailored steganalysis more accurately extracts watermark patterns than generic averaging. We then compare the extracted watermarks with the ground truth to showcase this low-level component.Our steganalysis incorporates the DDIM inversion steps from Tree-Ring's detection process. By inverting watermarked images to the DDIM-inverted latent space and averaging them, we obtain patterns (Figure 3 , second row). As more images are averaged, these patterns closely resemble those extracted under greybox or blackbox settings, manifesting as ripples spreading from the corners and forming aliasing patterns in the centre, reminiscent of superpositioned 2D sinc functions.In the Fourier domain (Figure 3 , first row), these patterns display a clear ring structure nearly identical to the ground truth. The high similarity between the ground truth and the patterns extracted from both the image and DDIM-inverted latent domains indicates that Tree-Ring likely propagates a content-agnostic ripple pattern throughout the image generation process, slightly but directly revealing it in the generated images. This insight enables us to fool Tree-Ring's watermark detector by simply subtracting this ripple pattern, effectively removing the watermark information.To compare perceptual quality degradation between our method and distortion-based ones, in Figure 4 , we plot Tree-Ring's AUC versus qualitative metrics varying signal strengths during watermark subtraction. In all four plots, the steganalysis-based watermark removal curves clustered in the bottom-left corner, indicating that effective steganalysis can remove watermarks with significantly less image quality degradation compared to distortion-based methods. Note that although in Section 4.2, the excess performance degradation under small n is believed to be caused by excess distortions 1.00 0.08 0.09 0.13 0.13 0.13 0.14 0.14 0.14 0.14 Forgery 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 introduced due to imperfect pattern extraction, in this section, we demonstrate that distortions generally do not help remove watermarks. Appendix A.2 visualises images under these distortions.We demonstrate the ability to forge Tree-Ring watermarks (x w = x ∅ + δw ) on non-watermarked images, in addition to watermark removal (x ∅ = x w -δw ). Table 1 shows the forged watermarks completely deceive Tree-Ring's detection. Figure 5 shows forged watermarks exhibit slightly larger distances compared to authentic watermarked images. However, when n is large (500 images), the forged images overlap with true watermarked images in the histogram, precluding threshold-based separation, thereby demonstrating Tree-Ring's vulnerability to steganalysis-based watermark forgery.We study a heterogeneous scenario where the adversary's image collection contains multiple different watermark patterns. When there are more patterns in the adversary's image collection, the detection AUC rises while the PSNR drops, indicating decreased steganalysis removal efficacy. Mixing three different watermark patterns increases Tree-Ring's detection AUC from below 0.2 to above 0.7 in both greybox and blackbox settings with n = 5000, indicating that mixing watermarking keys could improve security against simple steganalysis-based watermark removals. Nevertheless, the remaining 0.3 gap to perfectness still demonstrates the vulnerability of content-agnostic watermarking. We highlight that assigning multiple watermarks serves as a mitigation and cannot fundamentally address the steganalysis vulnerability. (Appendix A.6 gives more cases).In this case study, we rooted Tree-Ring's security vulnerabilities in its use of low-level contentagnostic ripple patterns as watermarks, rather than solely from semantic watermarking. This enables us to successfully fool Tree-Ring watermark detection with minimal impact on perceptual quality.Although it exhibits strong robustness to distortions [14] and regeneration attacks [18, 17] , through steganalysis-based removal, we are the first to effectively remove Tree-Ring watermarks without access to the algorithm.The distinction between content-agnostic and content-adaptive watermarks extends beyond images, applying equally to other media like audio. To test the generality of our steganalysis approach, we extend the experiments to two audio watermarking methods: AudioSeal [45] and WavMark [46] .Following the methodology outlined in Section 3.3, we extract audio watermark patterns by averaging in the time domain. Figure 7 illustrates the efficacy of this approach on audio watermarks. Similar to content-agnostic image watermarks, our steganalysis-based removal significantly impairs the performance of AudioSeal, reducing its detection accuracy from a perfect 1.0 to around 0.75 in both greybox and blackbox settings. This decline underscores AudioSeals's vulnerability to our simple averaging-based steganalysis.Interestingly, for WavMark, subtracting the averaged pattern counterintuitively improves its bet accuracy from below 0.8 to a perfect 1.0 when n is large. While the complexity of WavMark's algorithm precludes definitive conclusions from this experiment, the pattern extracted under large n showcased the existence of the systematic bias and its correlation with the watermark information. Although our method does not directly "remove" WavMarks' watermark in the traditional sense, the observed behaviour raises questions about its resilience to more sophisticated steganalysis attacks.In both cases, smaller n values lead to lower watermark detection rates and lower SI-SNR values, demonstrating coarsely extracted patterns further degrading detection performance with additional audio quality distortions. This mirrors our finding in the image domain, highlighting the importance of sufficient large n for more accurate watermark pattern extraction across different media types. 
guidelines towards steganalysis-secure watermarking: 
 Our analysis demonstrates that content-agnostic watermarking methods like Tree-Ring [14] are vulnerable to steganalysis-based attacks. Although these methods claim robustness by demonstrating strong resistance against distortions (e.g., blurring or noise perturbations), adversaries may still remove the watermark through steganalysis, thus compromising their robustness. Our experiments reveal that even complex, highly nonlinear methods based on deep neural networks are susceptible to steganalysis-based watermark removal.To avoid such threat, future watermarking algorithms shall use content-adaptive watermarking methods. One approach is to incorporate image features while encoding watermark information. For example, HiDDeN [10] and RivaGAN [11] introduce image features into the watermark encoder through concatenation and attention, respectively. For existing content-agnostic methods, assigning multiple watermarks per user can mitigate but not fundamentally solve steganalysis threat.While we performed simple averaging steganalysis on RGB images and 16 kHz monophonic audios, watermarking methods should resist more complex techniques, such as steganalysis in various colour spaces or different transform domains. Evaluating security against diverse steganalysis models, analogous to robustness tests against distortions, is crucial for developing secure watermarking algorithms. These two aspects form our security guidelines:1. Ensure watermarks are content-adaptive to resist steganalysis attacks.2. Evaluate watermark against steganalysis to ensure robustness. 
conclusions: 
 This work has revealed the vulnerability of content-agnostic watermarking algorithms to steganalysis attacks. We have demonstrated effective watermark removal and forgery techniques under both greybox and blackbox settings across twelve watermarking methods, including recent deep-learning approaches. Our findings extend to audio watermarking methods as well. To address these threats, we propose security guidelines that encourage exploring content-adaptive watermarking methods and evaluating them against steganalysis attacks. We have also proposed temporary mitigations for existing content-agnostic methods. Only by addressing the vulnerability to steganalysis can we develop secure and robust digital watermarking systems capable of safeguarding the integrity of digital content in the era of generative AI.Limitations and ethical considerations Steganalysis-based watermark removal/forgery is only effective against content-agnostic watermarking, not content-adaptive techniques. Responsible development and deployment of steganalysis technologies are crucial, adhering to fairness, accountability, and transparency principles to prevent misuse for unwarranted surveillance or privacy violations.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Yes. The experimental setups are provided in Section 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: No. Error bars are not presented because it would be too computationally expensive.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: Yes. The broader impacts, including potential positive and negative societal impacts, are discussed in Section 6. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer:[NA]Justification: The paper does not release any data or models that have a high risk for misuse.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: Yes. The use of existing datasets and evaluation methods are all credited with citations, as requested by the property owners or licenses.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: The proposed steganalysis attack and security guidelines extend beyond image watermarking. They apply to watermarking other media like video [48] , audio [49] , 3D models [50] [51] [52] , and to other domains. Our proposed guidelines and mitigation strategies strengthen watermarking security, contributing to a safer digital environment.A.1 Watermark implementation details Tree-Ring watermarks [14] are implemented following the corrected version proposed by RingID [16] under the verification setting. In comparison, the original Tree-Ring watermarks exhibited an issue where the watermark pattern actually injected into the initial noise pattern was inconsistent with the pattern claimed by Tree-Ring [14] .Our implementation adopts the original ring pattern proposed by Tree-Ring [14] with radii from 0 to 10, but the ring pattern is centred at the origin in the Fourier domain, following the correction proposed by RingID [16] . To ensure lossless watermark injection, we discard the imaginary part of the sampled watermark pattern during watermark detection. This lossless injection ensures that the watermark pattern actually injected matches the reference pattern used during detection.RingID [16] is implemented using patterns with radii ranging from 3 to 14 in the identification setting.The distortions applied in plotting Figure 4 patterns according to our analysis in the main paper. This demonstrates that steganalysis-based watermark removal effectively captures the vulnerability of the watermarking method through the extracted pattern, even without knowledge about the watermarking algorithm. Subsequently, it highlights that Tree-Ring's robustness to distortions is a Maginot Line that can be breached with steganalysis attacks.In the main text, we have visualised the patterns extracted from content-agnostic watermarking methods. In this section, we present a more comprehensive visualisation of patterns extracted from content-agnostic methods in Figure 9 , along with visualisations of the patterns extracted from content-adaptive watermarking techniques, as depicted in Figure 10 .While subtracting these patterns from watermarked images may not effectively circumvent the watermark detection process, the extracted patterns still exhibit regularities. By averaging 5000 images, the patterns extracted from both Stable Signature [22] and WmAdapter [27] display mild repetitive grid-like structures, suggesting that images watermarked with this technique may share commonalities at specific frequency components. For such repetitive patterns, it would be worth exploring whether they are caused by the patching mechanism of the transformer architecture. Patterns extracted by RivaGAN [11] exhibit a greenish tint, indicating RivaGAN introduces a systematic bias during the watermarking process despite content-adaptive components. The patterns extracted by SSL [25] , similar to RAWatermark [23] , are biased towards a specific noise distribution. However, while RAWatermark [23] is vulnerable to steganalysis-based removal, SSL [25] demonstrates robustness against such attacks. As the parameter n increases, the noise components in the patterns extracted by HiDDeN [10] transition towards a greyish hue and eventually diminish. By the gray-world assumption principle, this observation suggests that HiDDeN best incorporates content-adaptive watermarks into the images. Under spatial domain averaging, DwtDct [12] reveals nothing but a tiny systematic bias, which aligns with the factWe also visualise audio patterns extracted from AudioSeal [45] and WavMark [46] in Figure 11 .When n = 5000, the greybox AudioSeal-extracted pattern is highly imperceptible, with a signal strength below -45dB relative to the cover media. However, directly subtracting this pattern in the time domain significantly reduces the watermark detection accuracy by 30%. In contrast, WavMarkextracted patterns have larger amplitudes, indicating that WavMark also introduces a systematic bias during the watermarking process. Moreover, greybox-extracted WavMark patterns concentrate within the first second, which aligns with WavMark's method of adding watermark patterns at one-second intervals. This exposes WavMark's watermarking locations, allowing an adversary to potentially remove the watermark by simply clipping out these segments. The spectrograms further show that WavMark mainly adds watermarks below 6 kHz, which could represent the robustness threshold it was trained to withstand against low-pass filtering. In summary, although both methods claim robustness against various distortion-based attacks, their vulnerabilities can be easily exposed through simple steganalysis techniques.In the main text, we analyzed the perceptual quality of Tree-Ring watermarked images after watermark removal. We now visualise content-agnostically watermarked images after watermark removal in Figure 12 , and content-adaptively watermarked images after watermark removal in Figure 13 .From these two figures, we can observe that except for Gaussian Shading [19] , images carrying different watermarks, after watermark removal, have nearly identical visual quality. The visual quality is predominantly related to n, the number of images averaged during pattern extraction.In the greybox setting, when n > 50, there are no visually apparent artefacts. In the blackbox setting, n > 100 is generally required for eliminating significant artefact patterns. For images with Gaussian Shading [19] watermarks, due to the large magnitude of the extracted watermark pattern itself, steganalysis-based removal inevitably causes significant changes to the image content, thereby reducing perceptual quality. This leads to two insights:1. For adversaries, averaging more images allows for a better approximation of an effective content-agnostic watermark pattern that can be used for watermark removal; 2. For watermark distributors, reducing the count of distributing the same watermark could lower the security risk posed by steganalysis.In Section 4.2, we primarily discussed the Peak Signal-to-Noise Ratio (PSNR) as a measure of image distortion resulting from our steganalysis-based watermark removal method. To provide a more comprehensive assessment of the impact on visual quality, we also evaluated three additional metrics: Structural Similarity Index (SSIM), Learned Perceptual Image Patch Similarity (LPIPS), and Single Image Fréchet Inception Distance (SIFID).Tables 2-13 present the performance of various watermarking methods under our steganalysis-based removal attack, along with the corresponding image quality metrics. The trends observed in PSNR are largely mirrored in the additional metrics:Content-Agnostic Methods For methods like Tree-Ring [14] , RingID [16] , RAWatermark [23] , DwtDctSvd [12] , and RoSteALS [13] , all metrics show a consistent trend: as the number of images averaged (n) decreases, watermark removal becomes more effective (lower detection rates), but at the cost of increased image distortion (lower PSNR, SSIM, and higher LPIPS, SIFID).For example, in the case of RAWatermark (Table 4 , Blackbox setting), as n decreases from 5000 to 5, the AUC drops from 0.574 to 0.133, indicating more effective watermark removal. However, this comes at the cost of image quality: PSNR drops from 27.98 to 17.92, SSIM from 0.964 to 0.528, while LPIPS increases from 0.020 to 0.424, and SIFID from 0.022 to 1.401, all indicating significant visual degradation. Content-Adaptive Methods In contrast, content-adaptive methods like Stable Signature [22] , WmAdapter [27] , RivaGAN [11] , and SSL [25] maintain good detection performance (bit accuracies > 0.95) even as n decreases, with the image quality metrics also show minimal degradation. For instance, SSL (Table 11 , greybox setting) maintains a high bit accuracy (0.895 at n = 5) while preserving image quality: PSNR (34.26), SSIM (0.920), LPIPS (0.048), and SIFID (0.048) are all close to the no-removal (NRmv) values. This could be due to that the systematic biases averaged out from content-adaptive methods are mild, so subtracting such mild patterns from the original images would not lead to significant perturbations.In Section 4.4.4, our case study on Tree-Ring [14] demonstrated that when an adversary's image collection contains a mix of several different watermark patterns, the effectiveness of watermark removal is significantly reduced. Based on this finding, we proposed assigning multiple keys as a mitigation strategy against steganalysis threats. However, in this section, we caution that this approach is not a universal solution and should be applied judiciously. The high complexity of watermarking algorithms means that this method cannot guarantee enhanced watermark security. To illustrate this, we replicate our experiment using the DwtDctSvd [12] algorithm.Figure 14 shows that in the greybox scenario, DwtDctSvd behaves similarly to Tree-Ring: as the adversary's image collection incorporates more diverse watermarks, watermark removal becomes less effective without significantly impacting image quality. Interestingly, this strategy fails in the blackbox setting. When the adversary's collection mixes more than 4 watermarks, DwtDctSvd's detection accuracy surprisingly drops below 0.1. While the reasons behind this phenomenon warrant further investigation, this observation underscores our main point: assigning multiple watermarks per user is merely a temporary workaround to bolster current content-agnostic watermarking algorithms. It does not address the fundamental vulnerabilities of these algorithms and may even be counterproductive in certain scenarios. Therefore, watermark distributors should employ this method cautiously and at Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Each configuration is tested on 100 images or audio segments to minimise computational cost during repeated ablation studies on watermark removal and forgery. 


Decision: The real average rating of this paper should be 5.75 since reviewer 85eL said that the score should be bumped up to 5 but still remained as 4 (no response after emailing).

The paper introduces a steganalysis-based attack targeting content-agnostic image watermarking methods. Reviewers acknowledge that the paper effectively demonstrates vulnerabilities in these watermarking techniques (aL14, Jv2m, VkHo). However, concerns are raised about the novelty of the method, as similar ideas have been explored in prior works (aL14). Questions arise regarding the generality of the assumptions made, particularly for methods like Tree-Ring that embed watermarks in the frequency domain (Jv2m, VkHo). Practical impact is questioned due to potential image quality degradation after watermark removal (85eL) and the requirement for a large number of pre-existing watermarked images (85eL, aL14). While the paper is well-written and organized (Jv2m, 85eL, VkHo), the contribution is considered limited due to existing similar research and practical challenges in real-world applications.

In the rebuttal the authors address concerns about the novelty of their steganalysis-based attack on content-agnostic watermarks. They emphasize that their method is the first black-box, training-free approach that unifies both watermark removal and forgery without needing access to the watermark encoder or decoder, differing fundamentally from prior works. Additional experiments were provided comparing their method with existing attacks, demonstrating effectiveness with fewer images and minimal perceptual degradation. They also clarify that the vulnerability is severe and not trivial, extending their findings to other modalities like audio. The AC recommends accept as most of the reviewers acknowledged their concerns were addressed and bumped up their ratings.

Reviews: review: paper_summary: paper_summary: This paper introduces a steganalysis-based attack aimed at image watermarking methods. The attack is effective in both gray-box and black-box scenarios and focuses on identifying repeating patterns present in watermarked images. These patterns can be exploited to either remove watermarks or add them to non-watermarked images without needing access to the original watermarking algorithm. Experimental results demonstrate that this attack can successfully compromise watermarks that use content-agnostic watermark patterns.
strength_weakness: strength_weakness: - Similar ideas (i.e., considering watermarked and non-watermarked samples to extract watermark patterns for removal and forgery) have been explored in existing works [1][2]. For instance, [1] discusses a spoofing attack using a noise image $X$, watermarking it to obtain $X_{wm}$, and then adding $(X_{wm}-X)$ to clean images to create watermarked versions. Both [1] and [2] also describe a surrogate model adversarial attack, which involves training a CNN to distinguish between watermarked and non-watermarked images. This CNN is then used to adversarially alter samples to change their watermark status. This adversarial attack is a more advanced version of the method suggested in this paper and can potentially target content-aware watermarks as well.

- The authors claim, "Notably, our steganalysis-based approach is the first to effectively remove Tree-Ring watermarks without access to the algorithm," which is inaccurate. The adversarial attack proposed in [1], though ML-based, requires only a set of watermarked and non-watermarked samples (similar to the black-box attack described in this paper) to break Tree-Ring.

- The method introduced in this paper lacks novelty, and the vulnerability of content-agnostic watermarks to this type of attack is trivial and not a significant discovery.

[1] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., and Feizi, S. Robustness of AI-image detectors: Fundamental limits and practical attacks, 2023.

[2] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., Zhu, S., Mohamed, A., Wen, Y., Goldstein, T., et al.: Benchmarking the robustness of image watermarks, 2024.
questions: questions: - I suggest that the authors compare their attack to existing watermark attacks (e.g., from this watermark benchmark [1]). This comparison is necessary to illustrate the advantages of their attacks.

- The results for the forgery attack are provided only for the Tree-Ring watermark. I recommend including results for other watermarks as well to provide a more comprehensive evaluation.


[1] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., Zhu, S., Mohamed, A., Wen, Y., Goldstein, T., et al.: Benchmarking the robustness of image watermarks, 2024.
limitations: limitations: - The attack only works on content-agnostic watermarks, which the authors adequately mentioned in several parts of the paper.
paper_summary: paper_summary: The paper addresses vulnerabilities in digital watermarking methods, especially those that are content-agnostic. These methods, which embed fixed watermark patterns regardless of image content, are susceptible to steganalysis attacks that can extract and manipulate these patterns, potentially removing or forging watermarks with minimal impact on perceptual quality.
strength_weakness: strength_weakness: 1. The steganalysis method proposed in this paper is fundamentally general and not restricted to images. However, the experiments are confined to analyzing image watermarks. Providing evidence of successful steganalysis in other modalities, such as audio, would further validate and broaden the method's impact.

2. The steganalysis method proposed in this paper can extract a unified low-level watermark pattern from tree-ring watermarked images. So why can the tree-ring change the layout of the generated image?

3. The quality of the figures could be further improved. For example, the font in Figure 2.3.4 is too small. Please enlarge it for easier reading.
questions: questions: Please see above.
limitations: limitations: Please see above.
paper_summary: paper_summary: The paper introduces steganalysis techniques targeting watermark removal and forgery. The authors demonstrate through experiments that existing content-agnostic watermarking methods are unable to resist steganalysis attacks, advocating for the adoption of content-adaptive strategies.
strength_weakness: strength_weakness: 1. Although the examples provided by the authors in Figure 7 show that the images after watermark removal exhibit good image quality compared to the watermarked images. However, the PSNR data shown in Figure 2 are not ideal. For instance, in the first example "Tree-Ring," the average PSNR of the removed watermark image is only 17. According to [1],  distortion of modified images with PSNRs lower than 36 dB is noticeable to human visual system. If the quality of the image is severely degraded after watermark removal, then the act of removing the watermark becomes meaningless.
2. The author said “In contrast, traditional distortion-based removal techniques, such as noise perturbations or blurring, typically result in substantial perceptual degradation (as visualized in  Figure 9 in Appendix A.2)”. But I think comparing perceived degradation like this isn't fair because it's difficult to objectively evaluate the severity of different attacks.
3. While the authors present an interesting steganalysis method for removing or forging content-agnostic watermarks, the contribution of this approach may be somewhat limited. Additionally, the method's requirement for a large number of pre-existing watermarked images poses practical challenges for real-world applications.

[1] S. Sarreshtedari and M. A. Akhaee, “A source-channel coding approach to digital image protection and self-recovery,” IEEE Trans. Image Process., vol. 24, no. 7, pp. 2266–2277, Jul. 2015.
questions: questions: See Weaknesses
limitations: limitations: 1. The premise of steganalysis for watermarks requires a large number of images embedded with watermarks, unlike attacks such as JPEG compression, which can directly affect the watermark itself. This limitation makes steganalysis attacks somewhat impractical in real-world applications because attackers may not always have access to a sufficient number of watermarked images.
2. This method is only effective against content-agnostic watermarking.
paper_summary: paper_summary: This paper introduces a study on the vulnerabilities of digital watermarking techniques to steganalysis attacks. Extensive experiments are conducted to demonstrate the effectiveness of steganalysis in detecting and removing watermarks from images, especially when targeting content-agnostic watermarking methods.
strength_weakness: strength_weakness: 1. This paper adopts a simple linear assumption that the watermarked image is obtained by adding the content-agnostic perturbation to the original image. The generality of this assumption needs to be clarified, as it forms the theoretical foundation for the watermark attacks discussed in this paper. For instance, the Tree-Ring method embeds information in the frequency domain of initial noise, but why does it conform to this model?

2. The definition of steganalysis needs to be clearly stated, as it is an important concept in this paper.

3. In section 3.1, the paper mentions that ''Instead of applying strong distortions (e.g., noise), the adversary could take a steganalysis approach, first **approximating w** and then crafting minimal distortions to fool D''. However, in the following method, the watermark is never approximated. Instead, the additive perturbation is directly predicted to fool D.

4. In Figure 2, why are the performance metrics inconsistent? Although the caption specifies the use of AUC, the figure mixes AUC with Bit Acc as metrics for different methods.

5. It is better to explain why there is a sudden drop when n = 5 for content-adaptive methods like Stable Signature.
questions: questions: Please see Weaknesses.
limitations: limitations: The authors have addressed the limitations in section 5.

Comments: title: Thank you for engaging in the discussions
Reminder on Updating the Score
comment: Dear Reviewers,

Thank you all for engaging so thoughtfully with our work during the discussion period. We greatly appreciate your constructive feedback. We're also grateful that most of you considered raising the scores. We wish you all the best in your research and future career!

Best regards,
#9663 authors
Just a gentle reminder as the discussion period ends soon -- the score hasn't been updated in the system yet. Thanks again for your consideration!
Thanks for your thoughtful feedback and for raising the score. We'll include the additional experiments and discussion, which we also believe could broaden the impact of our work.
Thank you for the authors' clarification and extra experiments. My concerns have been addressed. I will raise my score to 8. If possible, please incorporate the extra experiments and discussion into the next version of the paper.
Thank you for addressing our responses positively. We appreciate your recognition of our rebuttal and the contributions of our work. We understand the limitation you've mentioned, which we have repeatedly mentioned in the paper as well.

Additionally, we kindly wanted to bring to your attention that the score hasn't been updated yet. We appreciate your time and consideration in doing so.
Thanks for your feedback.

　

**Regarding the impact of $\epsilon_\theta$**:

We agree that including the impact of the $\epsilon_\theta$ term could potentially provide deeper insights into the diffusion generation process. However, we respectfully note that a full analysis incorporating the $\epsilon_\theta$ term presents significant challenges due to the high non-linearity of the diffusion denoising network. Such an analysis would likely require extensive additional research and could potentially constitute a separate study in its own right.

　

**Regarding the applicability**:

From the perspective of an attack method, this is indeed the limitation, as we have repeatedly addressed in the paper.

We believe that, however, the broader impact of our work extends beyond proposing the specific attack technique. The simplicity and effectiveness of stegnanalysis reveals a critical vulnerability in modern watermarking methods, including recent DNN-based approaches. The patterns visualized explains the nature of content-agnostic watermarks, guiding the community towards developing more robust, content-adaptive ones and adopting comprehensive evaluation practices. We believe the contributions have significant potential to advance the field of digital watermarking as a whole.

　

Once again, thank you very much for your insightful feedback.
Thank you very much for your detailed response. I appreciate that most of my questions have been addressed. I will increase my score; however, given the inherent limitations of this method,  which is ineffective for content-adaptive issues, I will raise the score to 5 (borderline accept).
Thanks for your response. I appreciate the authors' efforts. Such attempts are beneficial for explaining this phenomenon, although I personally cannot fully confirm the correctness of the derivation at the moment, especially regarding neglecting the impact of term $\epsilon_{\theta}$. 

Considering this method is only for removing content-agnostic watermarks, it is limited by its applicability. I will keep my original rating.
To address this question, we provide a brief theoretical analysis that offers insight into the underlying mechanisms. 

Tree-Ring's diffusion pipeline utilizes DDIM sampling. One DDIM denoising step from a noisy image $\mathbf{x}\_t$ to a less noisy one $\mathbf{x}\_{t-\Delta t}$ is described by Equation 13 in [1], which can be rearranged as:

$$
\mathbf{x}\_{t-\Delta t} = \frac{\sqrt{\alpha\_{t-\Delta t}}}{\sqrt{\alpha\_t}}\mathbf{x}\_t+c\cdot \epsilon\_\theta^{(t)}(\mathbf{x}\_t),
$$

where $c$ is a time-dependent constant. Tree-Ring can be viewed as adding a systematic bias $\mathbf{\mu}$ (the ripple-like pattern visualized in Figure 6) to an initial noise vector $\mathbf{x}\_T$. Consequently, the first sampling step can be rewritten as:

$$
\mathbf{x}\_{T-\Delta t} = \frac{\sqrt{\alpha\_{t-\Delta t}}}{\sqrt{\alpha\_t}}(\mathbf{x}\_t + \mathbf{\mu})+c\cdot \epsilon\_\theta^{(t)}(\mathbf{x}\_t + \mathbf{\mu}).
$$

Empirical evidence suggests that the output of $\epsilon\_\theta$ follows a zero-mean Gaussian distribution, regardless of the timestep and the presence of bias $\mathbf{\mu}$ in the input. Therefore, the term $c\cdot \epsilon\_\theta^{(t)}(\mathbf{x}\_t + \mathbf{\mu})$ does not affect the accumulation of influence from $\frac{\sqrt{\alpha\_{t-\Delta t}}}{\sqrt{\alpha\_t}}\mathbf{\mu}$ through sampling. The accumulated bias term at $\mathbf{x}\_0$ can be expressed as:

$$
\frac{\sqrt{\alpha\_{t\_1}}}{\sqrt{\alpha\_T}}\cdot
\frac{\sqrt{\alpha\_{t\_2}}}{\sqrt{\alpha\_{t\_1}}}\cdot
\frac{\sqrt{\alpha\_{t\_3}}}{\sqrt{\alpha\_{t\_2}}}\cdots
\frac{\sqrt{\alpha\_{0}}}{\sqrt{\alpha\_1}}
\mathbf{\mathbf{\mu}}
=\frac{\sqrt{\alpha\_0}}{\sqrt{\alpha\_T}}\mathbf{\mu}
=\frac{\sqrt{0.9991}}{\sqrt{0.0047}}\mathbf{\mu}
=14.5799\mathbf{\mu}
$$

in Tree-Ring's implementation. This calculation demonstrates that the bias $\mathbf{\mu}$ is significantly amplified throughout the generation process.

Assuming that the low-level pattern is largely preserved through VAE decoding, this amplification explains how a content-agnostic ripple-like pattern can propagate through the complex generation process and manifest in the final images. This analysis provides a theoretical foundation for why the additive simple linear assumption model is still valid with the presence of a diffusion generation pipeline.

We're happy to help with any more questions or concerns you might have. 

　

[1] Song et al., Denoising Diffusion Implicit Models, ICLR 2021.
Please feel free to raise any further concerns or questions. We would be glad to provide additional details.
Thank you for your follow-up comments and recognizing our work's strengths and contributions. We appreciate the increased score.

　

Regarding your concerns:

　

> The spoofing attack is very similar to the steganalysis attack from this paper, and while [1] does not discuss on generalization of the attack on watermark removal, I don't see a reason why it wouldn't work.

[1]'s spoofing method requires **(1)** access to the model with **(2)** the original watermark key (bit sequence/Tree-Ring pattern), which limits it to whitebox attacks. In contrast, our method is capable of performing blackbox attacks, as it extends beyond [1] by utilizing steganalysis to extract content-agnostic watermarks. This also offers better explainability by enabling the visualization of watermark patterns extracted.

We generalized [1] to watermark removal with the following setup:

$$
    \text{Original spoofing: \quad}  x_{\text{spoofed}} = \beta x_{\text{clean}} + (1-\beta) w, \quad \beta = 0.7;
$$
$$
    \text{Generalized removal: \quad}  x_{\text{removed}} = \beta x_{\text{w}} + (1-\beta) w, \quad \beta = 1.3.
$$

Our black-box method reduced Tree-Ring's AUC to 0.241 with a PSNR of 29.79dB (before and after removal), while [1]'s removal only achieved 0.929 AUC with 22.71dB PSNR. **This demonstrates that [1] is ineffective at removing Tree-Ring watermarks, whereas our method achieves superior removal with less quality degradation.**

　

> I still believe that the vulnerability of content-unaware watermarks to this attack is trivial (especially in case of the gray-box setting).

1. We're the first to identify and systematically evaluate the content-agnostic vulnerability in modern watermarking systems.
2. Our finding, though simple, has broad implications for various watermarking methods. Highlighting this blind spot is crucial for guiding future research towards steganalysis-secure approaches.
3. Our graybox setting provides explainability to the content-agnostic issue. It enriches the application scenario and serves as a complement of the attack system.

　

[1] M. Saberi et al., Robustness of AI-image detectors: Fundamental limits and practical attacks, 2023.
Thanks for your thoughful feedback. We now respond to your follow-up questions:

1. **Regarding the setup of audio experiments**: We randomly sampled audio clips from the zh-CN subset of the Common Voice dataset [1]. For black-box settings, we ensured that the unwatermarked and watermarked audio sets were disjoint. All audio clips were sampled at 16kHz and preprocessed by cropping to retain only the first two seconds. This setup ensures that all audio clips contain watermarked segments and are aligned [2, 3].

2. **On content-adaptive audio watermarks**: Yes, we observed that WavMark [2] adds content-adaptive watermarks to audio clips.

| Blackbox |   NRmv  |    5    |    10   |   20   |   50   |   100  |   200   |   500   |   1000  |   2000  |   5000  |
|----------|:-------:|:-------:|:-------:|:------:|:------:|:------:|:-------:|:-------:|:-------:|:-------:|:-------:|
| Bit Acc  | 0.82    | 0.0088  | 0       | 0      | 0.01   | 0.0256 | 0.0294  | 0.1062  | 0.25    | 0.3669  | 0.85    |
| SNR (dB) | 44.0328 | -6.7481 | -4.0659 | 0.0338 | 5.5133 | 9.1686 | 13.0371 | 17.1913 | 20.6909 | 24.5124 | 28.1818 |

When averaging 5000 audio samples, the bit accuracy after watermark removal still reached 0.85. This demonstrates that steganalysis is not effective in reducing the watermark's bit accuracy. This finding is consistent with our conclusions from image experiments, indicating that across various modalities, our method can remove content-agnostic watermarks but is ineffective against content-adaptive watermarks.

3. **Clarification on Tree-Ring**: Exactly. We are demonstrating that adding a Tree-Ring watermark affects both aspects. Tree-Ring inevitably changes the initial noise, which leads to changes in layout. This has been visualized in the Tree-Ring paper. Simultaneously, our experiments prove that Tree-Ring's watermarking process also adds a content-agnostic component. It is primarily this content-agnostic pattern that affects Tree-Ring's watermark detection.

---

[1] R. Ardila et al., Common voice: a massively-multilingual speech corpus, LREC 2020.

[2] G. Chen, WavMark: Watermarking for audio generation, Arxiv 2023.

[3] R. S. Roman, Proactie detection of voice cloning with localized watermarking, ICML 2024.
Thanks for your response. Most of my concerns are addressed. I still have questions about additive simple linear assumption model. Specifically, why such pattern added to the initial noise can be propagate through the diffusion pipeline to the final image, as such a generation pipeline is not a simple linear system? While experiments in Figure 6 support this the concept, a theoretical analysis would be beneficial to provide a deeper understanding of the underlying mechanisms.
> The spoofing attack described in [1] requires access to the watermark encoder (white-box) and does not generalize to removal, while our steganalysis-based approach unifies removal and forgery without accessing the decoder (black-box).

I understand the point of authors about their attack being black-box, in contrast to the spoofing attack from [1]. However, the spoofing attack is very similar to the steganalysis attack from this paper, and while [1] does not discuss on generalization of the attack on watermark removal, I don't see a reason why it wouldn't work.

> It requires significantly fewer images than adversarial attacks, further showcasing its advantage over existing approaches.

Reading the authors' response, I accept that their method, while having similarities to the adversarial attack, has its own advantages and disadvantages (only working on content-unaware watermarks).

I still believe that the vulnerability of content-unaware watermarks to this attack is trivial (especially in case of the gray-box setting). However, for the black-box setting, where non-paired samples for watermarked and non-watermarked images are used, there is more novelty and contribution. I appreciate the authors for adding new results for comparison to the attacks from [2], and the forgery results. I will increase my score to 5 (borderline accept).







[1] M. Saberi et al., Robustness of AI-image detectors: Fundamental limits and practical attacks, 2023.

[2] B. An et al., Benchmarking the robustness of image watermarks, 2024.
Thank you for your response. I would appreciate some further clarifications on the following points:

1. **Regarding the setup of audio experiments**

Could you specify which dataset you used and whether any preprocessing was applied?

2. **On content-adaptive audio watermarks**

Have you observed any content-adaptive audio watermarks? Are the conclusions consistent with those on images? Could you share some results?

3. **Clarification on Tree-Ring**

To confirm, are you suggesting that the tree-ring affects the generated images in both high-level aspects (evident in the layout changes of generated content) and low-level aspects (appearing as a content-agnostic pattern overlaid on the generated images)? If this is the case, I would recommend explicitly stating in the paper that "watermarks added in the latent domain of diffusion models may not always result in a semantic watermark and could remain content-agnostic." Highlighting this risk is crucial for future security of diffusion watermarking.