Title: Can Simple Averaging Defeat Modern Watermarks?
Venue: NeurIPS 2024
Accepted: yes
Keywords: Watermark

Security
Abstract: Digital watermarking techniques are crucial for copyright protection and source identification of images, especially in the era of generative AI models. However, many existing watermarking methods, particularly content-agnostic approaches that embed fixed patterns regardless of image content, are vulnerable to steganalysis attacks that can extract and remove the watermark with minimal perceptual distortion. In this work, we categorise watermarking algorithms into content-adaptive and content-agnostic ones, and demonstrate how averaging a collection of watermarked images could reveal the underlying watermark pattern. We then leverage this extracted pattern for effective watermark removal under both greybox and blackbox settings, even when the collection of images contains multiple watermark patterns. For some algorithms like Tree-Ring watermarks, the extracted pattern can also forge convincing watermarks on clean images. Our quantitative and qualitative evaluations across twelve watermarking methods highlight the threat posed by steganalysis to content-agnostic watermarks and the importance of designing watermarking techniques resilient to such analytical attacks. We propose security guidelines calling for using content-adaptive watermarking strategies and performing security evaluation against steganalysis. We also suggest multi-key assignments as potential mitigations against steganalysis vulnerabilities. Github page: \url{https://github.com/showlab/watermark-steganalysis}.

Can Simple Averaging Defeat Modern Watermarks?

Abstract

Digital watermarking techniques are crucial for copyright protection and source identification of images, especially in the era of generative AI models. However, many existing watermarking methods, particularly content-agnostic approaches that embed fixed patterns regardless of image content, are vulnerable to steganalysis attacks that can extract and remove the watermark with minimal perceptual distortion. In this work, we categorise watermarking algorithms into content-adaptive and content-agnostic ones, and demonstrate how averaging a collection of watermarked images could reveal the underlying watermark pattern. We then leverage this extracted pattern for effective watermark removal under both greybox and blackbox settings, even when the collection of images contains multiple watermark patterns. For some algorithms like Tree-Ring watermarks, the extracted pattern can also forge convincing watermarks on clean images. Our quantitative and qualitative evaluations across twelve watermarking methods highlight the threat posed by steganalysis to content-agnostic watermarks and the importance of designing watermarking techniques resilient to such analytical attacks. We propose security guidelines calling for using content-adaptive watermarking strategies and performing security evaluation against steganalysis. We also suggest multi-key assignments as potential mitigations against steganalysis vulnerabilities. Github page: https://github.com/showlab/watermark-steganalysis .

1 Introduction

Digital watermarking hides information within digital media, facilitating copyright protection and source authentication [1-3]. With the advances in AI-based image generation and editing [4-7], robust and secure digital watermarking is crucial for preventing deepfake misuse or manipulations of created contents [8, 9].

We categorise digital watermarking methods into two types: content-adaptive and content-agnostic. Content-adaptive methods take images into the watermarking process, dynamically adjusting the watermark's placement and strength based on the image content, as seen in technologies like HiDDeN [10] and RivaGAN [11]. Content-agnostic methods, however, use fixed, predefined watermark patterns independent of or weakly dependent on image content. Apart from traditional methods like DwtDctSvd [12], this also includes RoSteALS [13] that adds image-independent additive perturbations, and Tree-Ring [14] that places a ring pattern to the initial noise of a diffusion generation process. Content-adaptive methods typically offer better robustness against image processing distortions, while content-agnostic methods are computationally lighter and easier to implement.

A fundamental requirement for digital watermarks is robustness, ensuring watermarks cannot be easily removed or tampered with [2]. To meet the requirement, existing methods have been improving

watermark robustness through design considerations [14] or data augmentation during training [10], and have demonstrated strong robustness to various image distortions like noise perturbations or JPEG compression [15, 14, 16]. Works like Tree-Ring [14] even demonstrated robustness against strong attacks like V AE compression and image re-generation [17, 18]. In this paper, however, we reveal that content-agnostic watermarking techniques, including Tree-Ring, are vulnerable to steganalysis attacks, unmasking their hidden fragility.

To the best of our knowledge, our steganalysis is the first successful blackbox attack against Tree-Ring watermarks [14]. We discovered a content-agnostic ripple pattern in Tree-Ring-watermarked images and identified that this component is essential for watermark detection. Subtracting this pattern allows evading watermark detection with minimal impact on image perceptual quality. This raises the question: Do diffusion model watermarking methods that modify initial noise [19, 20, 14, 16] truly add semantic watermarks, or do they merely propagate low-level content-agnostic patterns to generated images?

Lastly, we propose new security guidelines for the watermarking community, emphasizing the importance of robustness against steganalysis. The guidelines call for performing evaluations against steganalysis when proposing new watermarking methods. It also encourages the development of content-adaptive watermarking methods to enhance resistance to steganalysis. For existing contentagnostic watermarking methods, we suggest assigning multiple watermarks per user as a mitigation strategy. In summary, the main contributions of this paper are:

We reveal the vulnerability of content-agnostic watermarking methods to steganalysis removal and forgery.

To the best of our knowledge, we are the first to successfully attack Tree-Ring watermarks in a blackbox setting, providing deeper insights into the essence of diffusion noise watermarking methods.

We propose new security guidelines for future watermarking methods to help defend against steganalysis attacks.

2 Related works

2.1 Digital image watermarking

The field of digital image watermarking has evolved from traditional rule-based approaches to more recent deep learning-based techniques, with a significant focus on watermarking diffusion-generated images [14, 21, 22]. We categorise digital watermarking technologies into content-agnostic, which craft modifications based solely on the watermark information, and content-adaptive, which tailor modifications based on both watermark information and the image content.

Content-agnostic watermarking Traditional methods like DwtDctSvd [12] employ fixed watermark patterns in transform domains, while more recent approaches modify the initial noise for diffusion-based image generation. Tree-Ring watermarks [14] replace the low-frequency Fourierdomain pixels of Gaussian noise with a ring pattern before using it for diffusion denoising. Similarly, Gaussian Shading [19] preserves the distribution while sampling the initial noise. Other approaches [13, 23, 24] train encoders to generate additive watermark perturbations without conditioning on image features.

Content-adaptive watermarking These techniques leverage image features to generate watermarks tailored to the input image content. Early encoder-decoder methods like HiDDeN [10] and StegaStamp [15] employed deep neural networks to imprint watermarks onto the images. SSL [25] leveraged selfsupervised networks as feature extractors, while RivaGAN [11] used attention mechanisms to look for appropriate local regions for watermark encoding. Recent approaches like Stable Signature [22], WADiff [21], and Zhao et al. [26] finetune the diffusion model to enable content-aware watermarking of diffusion-generated images. WMAdapter [27] designs a dedicated contextual adapter.

Through the classification, we highlight the vulnerability of content-agnostic techniques to steganalysis attacks, as they employ fixed or weakly content-dependent watermark patterns.

2.2 Attacks on watermarking

Traditional attacks applied distortions to disrupt watermarks, performing signal-level distortions like image compression, noise perturbation, blurring or colour adjustment, and geometric transformations like rotation or cropping [28-30, 10, 12]. In terms of videos, codecs may also distort invisible watermarks [31, 32]. These attacks fool watermark detectors at the cost of significant image quality degradation. To resist such attacks, training-based methods have been simulating the distortions via "attack layers" during training [10, 33-35], while training-free methods have been employing design considerations such as watermarking only low-frequency components [14].

Recent attacks base on deep models: regeneration attacks with diffusion models [36] and VAEs can provably remove pixel-level invisible watermarks [37, 18]. But such attacks are shown to be ineffective [17] for Tree-Ring [14] that alters the image a lot. When attackers can access the watermarking algorithm, they may also perform adversarial attacks [38]. The downside is that both regeneration and adversarial attacks are computationally expensive. In contrast, we propose a new type of blackbox steganalysis attack, which is efficient and works for content-agnostic watermarks. Steganalysis can extract meaningful watermark patterns, thus promoting further applications like forgery or explainability.

3 Watermark steganalysis

3.1 Notations

Let x ∅ denote the original digital image, w the watermark information (e.g., bit sequence or geometric pattern), and E the watermark encoder that imprints w into x ∅ , yielding the watermarked image x w = E ( x ∅ , w ) . The embedding constraint ensures that x ∅ and x w are perceptually indistinguishable. A watermark decoder D recovers the embedded information ˆ w = D ( x w ) for authentication purposes.

3.2 Threat model

̸

The adversary aims to fool D by manipulating x w using a strategy denoted as T ( · ) such that D ( T ( x w )) = w (watermark removal) or manipulating x ∅ such that D ( T ( x ∅ )) = w (watermark forgery). Formally, the adversary solves:

subject to the constraint that the original image x and the manipulated image T ( x ) are perceptually indistinguishable. Rather than applying strong distortions as T , we demonstrate that the adversary can take a steganalysis approach to fool D .

3.3 Steganalysis: watermark extraction, removal and forgery

Figure 1 illustrates our watermark removal/forgery strategy T , which assumes that E perturbs an additive pattern δ w agnostic to image content, such that x w = x ∅ + δ w . This assumption can be refined based on a detailed understanding of specific watermarking algorithms (as will be showcased in Section 4.4.1). Under this additive assumption, to either remove or forge watermarks, we can approximate ˆ δ w = x w -x ∅ . To improve approximation and reduce randomness, we propose averaging over n images during pattern extraction:

With the approximated ˆ δ w , the adversary can perform greybox watermark removal ( ˆ x ∅ = T ( x w ) = x w -ˆ δ w ) or forgery ( ˆ x w = T ( x ∅ ) = x ∅ + ˆ δ w ) on a given image x . Even without paired x ∅ , the adversary can perform blackbox removal/forgery by approximating x ∅ through averaging any

+

=

Forged

ො 𝑥𝑤

Extracted

𝛿𝑤

Unpaired clean

𝑥∅

Watermark Forgery

Greybox Pattern Extraction

1

𝑛 ෍

෍

-

=

Extracted

𝛿𝑤

Paired clean

𝑥∅

Watermarked

𝑥𝑤

Blackbox Pattern Extraction

1

𝑛 ෍

෍

-

=

Extracted

𝛿𝑤

Unpaired clean

𝑥∅

Watermarked

𝑥𝑤

Watermark Removal

-

=

Removed

ො

𝑥∅

Extracted

𝛿𝑤

Watermarked

𝑥𝑤

collection of clean images from the Internet. There is a practical scenario where the adversary's watermarked image collection contains multiple different watermarks, we show in Section 4.4.4 that they can still use Equation 2 for pattern extraction.

4 Experiments

4.1 Experimental setup

Image experiments We evaluate our proposed steganalysis on 12 existing image watermarking methods: Tree-Ring [14], RingID [16], RAWatermark [23], DwtDctSvd [12], RoSteALS [13], Gaussian Shading [19], Stable Signature [22], WmAdapter[27], RivaGAN [11], SSL [25], HiDDeN [10], and DwtDct [12]. For the greybox setting, we use the COCO2017 [39] validation set for WmAdapter [27] and Stable Signature [22], Stable Diffusion Prompts [40] for Tree-Ring [14] and RingID [16] prompts, and DiffusionDB [41] for the remaining methods as the non-watermarked images ( x ∅ ). The corresponding watermarked images ( x w ) are generated using the respective watermarking methods. In the blackbox setting with no access to paired images, we substitute x ∅ with ImageNet [42] test set. The selection of images within the datasets is random. The datasets are resized to 256 × 256 for RoSteALS, SSL, and HiDDeN, and 512 × 512 for other methods.

We assess watermark removal under different n (number of images averaged) during watermark pattern extraction, and test on 100 images 3 during watermark removal. We report detection AUC for Tree-Ring [14] and RAWatermark [23], and watermark decoding bit accuracy for the other methods. Additionally, we evaluate the image quality between x w and its non-watermarked counterpart, reporting PSNR in the main text and SSIM, LPIPS [43], and SIFID [44] in the appendix.

Audio experiments We then extend the experiments to audio watermark removal on AudioSeal [45] and WavMark [46], using the zh-CN subset of the Common Voice dataset [47]. Each audio segment is preprocessed to a 16 kHz mono format, with only the first two seconds retained. We use paired audio for greybox removal, and unpaired audio for blackbox removal. We report the watermark detection accuracy for AudioSeal [45], and watermark decoding bit accuracy for WavMark [46]. To quantify the audio quality after watermark removal, we calculate the Scale-Invariant Signal-to-Noise Ratio (SI-SNR) between the watermark-removed audio and its non-watermarked counterpart.

Computing resources The experiments were conducted on an AMD EPYC 7413 24-Core Processor and an Nvidia RTX 3090 GPU, requiring around 200GB of disk space. The execution time for each experiment ranges from around 10 minutes (HiDDeN) to around 10 hours (Tree-Ring).

4.2 Quantitative analysis on watermark removal

As shown in Figure 2 (left column), our steganalysis-based watermark removal method effectively degrades the detection performance of RAWatermark (0.5744 AUC), DwtDctSvd (0.5722 accuracy), Tree-Ring (0.2407 AUC), RoSteALS (0.2444 bit accuracy), and Gaussian Shading (0.5615 bit accuracy). The results highlight two key findings: (1) the aforementioned methods embed contentagnostic watermarks, and (2) content-agnostic watermarks are susceptible to steganalysis-based removal.

The effectiveness of our method increases as n decreases, albeit at the cost of increased image distortion (smaller PSNR). In contrast, content-adaptive watermarking methods (Figure 2 right column) demonstrate robust resistance to this attack, maintaining high detection accuracy ( > 0 . 95 ) upon convergence. This resilience underscores the importance of content-adaptivity in watermark design to thwart steganalysis-based removal attacks.

4.3 Qualitative analysis

In this qualitative analysis, we first examine the patterns extracted from various watermarking methods, then discuss how the removal of these watermarks affects image quality.

Extracted patterns Figure 9 displays patterns extracted from content-agnostic methods, while Figure 10 shows those from content-adaptive methods. Content-agnostic watermarks tend to exhibit unique patterns, like fingerprints. For example, RingID [16] patterns are concentric rings with bright spots in the centre, DwtDctSvd [12] patterns resemble vertical lines like barcodes, and RoSteALS [13] patterns appear as grid-like patches with non-uniform illumination. In contrast, patterns extracted from content-adaptive watermarks are less discernible. Notably, under the greybox setting, the HiDDeN-extracted [10] pattern converges to zero, indicating completely no discernible pattern. Furthermore, patterns extracted in the greybox setting contain fewer visual artifacts than those in the blackbox setting. As more images are averaged, the extracted watermark pattern becomes clearer and more precise, with fewer residual artefacts from image contents. A more detailed analysis is presented in Appendix A.3.

Visual quality degradation Figures 12 and 13 show the visual impact of removing content-agnostic and content-adaptive watermarks, respectively. For all methods but Gaussian Shading [19], under the greybox setting, when more than 50 images are averaged, virtually no visual artefacts remain after watermark removal. In the blackbox setting, averaging over 100 images is necessary to eliminate most artifacts. The exception is Gaussian Shading [19], which consistently produces visible artefacts because the pattern extracted has a large magnitude. Subtracting this pattern significantly distorts the image. Appendix A.4 provides a more detailed analysis.

4.4 Case study: Tree-Ring watermarks

To further reveal how steganalysis can be a threat to content-agnostic watermarking algorithms, we conduct a case study on Tree-Ring watermarks [14]. Tree-Ring is a sophisticated diffusion-based watermarking algorithm that injects a frequency-domain ring pattern into a Gaussian noise signal before using this watermarked noise for diffusion-denoising image generation. During detection, it performs DDIM inversion to recover the injected ring pattern from the initial noise and compares it to a reference pattern. In the following experiments, we demonstrate that with minimal modifications, we can both remove and forge Tree-Ring watermarks under different scenarios.

4.4.1 Low-level content-agnostic pattern in Tree-Ring

This section focuses on revealing the low-level content-agnostic component of Tree-Ring watermarks [14]. First, we curate a specific steganalysis for Tree-Ring's detection algorithm, demonstrating how tailored steganalysis more accurately extracts watermark patterns than generic averaging. We then compare the extracted watermarks with the ground truth to showcase this low-level component.

Our steganalysis incorporates the DDIM inversion steps from Tree-Ring's detection process. By inverting watermarked images to the DDIM-inverted latent space and averaging them, we obtain patterns (Figure 3, second row). As more images are averaged, these patterns closely resemble those

0.2

0

2

0

4

6

8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (Blackbox)

All horizontal axes:

n

(number of images averaged during pattern extraction)

0

5

10

15

20

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

AUC

Tree-Ring

0

10

20

30

40

50

60

70

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

AUC

RAWatermark

0

10

20

30

40

50

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

DwtDctSvd

0

5

10

15

20

25

30

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

RoSteALS

9

9.2

9.4

9.6

9.8

10

10.2

10.4

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

0

5

10

15

20

25

30

35

0.88

0.92

0.96

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

Stable Signature

0

10

20

30

40

50

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

RivaGAN

0

10

20

30

40

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

SSL

15

20

25

30

35

40

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

HiDDeN

0

10

20

30

40

0.98

0.99

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

DwtDct

0

10

20

30

40

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Bit Acc

WmAdapter

10.5

11

11.5

12

12.5

13

13.5

14

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

PSNR (dB)

Ground Truth

5

10

50

100

500

5000

Spatial

Blackbox

Avg of Inverted Latents

Avg of Images

Number of Watermarked Images Averaged

Steganalysis

Noise

Blur

Brightness

extracted under greybox or blackbox settings, manifesting as ripples spreading from the corners and forming aliasing patterns in the centre, reminiscent of superpositioned 2D sinc functions.

In the Fourier domain (Figure 3, first row), these patterns display a clear ring structure nearly identical to the ground truth. The high similarity between the ground truth and the patterns extracted from both the image and DDIM-inverted latent domains indicates that Tree-Ring likely propagates a content-agnostic ripple pattern throughout the image generation process, slightly but directly revealing it in the generated images . This insight enables us to fool Tree-Ring's watermark detector by simply subtracting this ripple pattern, effectively removing the watermark information.

4.4.2 Comparison with distortion-based removal techniques

To compare perceptual quality degradation between our method and distortion-based ones, in Figure 4, we plot Tree-Ring's AUC versus qualitative metrics varying signal strengths during watermark subtraction. In all four plots, the steganalysis-based watermark removal curves clustered in the bottom-left corner, indicating that effective steganalysis can remove watermarks with significantly less image quality degradation compared to distortion-based methods. Note that although in Section 4.2, the excess performance degradation under small n is believed to be caused by excess distortions

Tampered images (watermarked images after removal/clean images after forgery)

Tree-Ring watermarked images

Clean images

Tree-Ring TPR@1%FPR threshold

Forgery Removal

introduced due to imperfect pattern extraction, in this section, we demonstrate that distortions generally do not help remove watermarks. Appendix A.2 visualises images under these distortions.

4.4.3 Watermark forgery

We demonstrate the ability to forge Tree-Ring watermarks ( ˆ x w = x ∅ + ˆ δ w ) on non-watermarked images, in addition to watermark removal ( ˆ x ∅ = x w -ˆ δ w ). Table 1 shows the forged watermarks completely deceive Tree-Ring's detection. Figure 5 shows forged watermarks exhibit slightly larger distances compared to authentic watermarked images. However, when n is large (500 images), the forged images overlap with true watermarked images in the histogram, precluding threshold-based separation, thereby demonstrating Tree-Ring's vulnerability to steganalysis-based watermark forgery.

4.4.4 Effectiveness of removal under multiple watermarks

We study a heterogeneous scenario where the adversary's image collection contains multiple different watermark patterns. When there are more patterns in the adversary's image collection, the detection AUC rises while the PSNR drops, indicating decreased steganalysis removal efficacy. Mixing three different watermark patterns increases Tree-Ring's detection AUC from below 0.2 to above 0.7 in both greybox and blackbox settings with n = 5000 , indicating that mixing watermarking keys could

7

0

5

10

15

20

1

2

3

4

5

6

Legend

No Removal

5

10

20

50

100

200

500

1000

2000

5000

# Imgs Avged During Extraction:

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

7

AUC

12

13

14

15

16

1

2

3

4

5

6

7

PSNR (dB)

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

7

AUC

12

13

14

15

16

1

2

3

4

5

6

7

PSNR (dB)

Blackbox

Horizontal Axis: Number of Heterogeneous Tree-Ring Keys Mixed in the Adversary's Collection

0.2

0

2

0

4

6 8

1

NR

5

10

20

50

100

200

500

1k

2k

5k

SI-SNR (Blackbox)

All horizontal axes:

n

(number of audio segments averaged during pattern extraction)

-10

0

10

20

30

40

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50 100 200 500 1k

2k

5k

SI-SNR (dB)

AudioSeal

-20

0

20

40

60

0

0.2

0.4

0.6

0.8

1

NR

5

10

20

50 100 200 500 1k

2k

5k

SI-SNR (dB)

Bit Acc

WavMark

improve security against simple steganalysis-based watermark removals. Nevertheless, the remaining 0.3 gap to perfectness still demonstrates the vulnerability of content-agnostic watermarking. We highlight that assigning multiple watermarks serves as a mitigation and cannot fundamentally address the steganalysis vulnerability. (Appendix A.6 gives more cases).

4.4.5 Summary

In this case study, we rooted Tree-Ring's security vulnerabilities in its use of low-level contentagnostic ripple patterns as watermarks, rather than solely from semantic watermarking. This enables us to successfully fool Tree-Ring watermark detection with minimal impact on perceptual quality. Although it exhibits strong robustness to distortions [14] and regeneration attacks [18, 17], through steganalysis-based removal, we are the first to effectively remove Tree-Ring watermarks without access to the algorithm.

4.5 Audio watermark steganalysis

The distinction between content-agnostic and content-adaptive watermarks extends beyond images, applying equally to other media like audio. To test the generality of our steganalysis approach, we extend the experiments to two audio watermarking methods: AudioSeal [45] and WavMark [46].

Following the methodology outlined in Section 3.3, we extract audio watermark patterns by averaging in the time domain. Figure 7 illustrates the efficacy of this approach on audio watermarks. Similar to content-agnostic image watermarks, our steganalysis-based removal significantly impairs the performance of AudioSeal, reducing its detection accuracy from a perfect 1.0 to around 0.75 in both greybox and blackbox settings. This decline underscores AudioSeals's vulnerability to our simple averaging-based steganalysis.

Interestingly, for WavMark, subtracting the averaged pattern counterintuitively improves its bet accuracy from below 0.8 to a perfect 1.0 when n is large. While the complexity of WavMark's algorithm precludes definitive conclusions from this experiment, the pattern extracted under large n showcased the existence of the systematic bias and its correlation with the watermark information. Although our method does not directly "remove" WavMarks' watermark in the traditional sense, the observed behaviour raises questions about its resilience to more sophisticated steganalysis attacks.

In both cases, smaller n values lead to lower watermark detection rates and lower SI-SNR values, demonstrating coarsely extracted patterns further degrading detection performance with additional audio quality distortions. This mirrors our finding in the image domain, highlighting the importance of sufficient large n for more accurate watermark pattern extraction across different media types.

5 Guidelines towards steganalysis-secure watermarking

Our analysis demonstrates that content-agnostic watermarking methods like Tree-Ring [14] are vulnerable to steganalysis-based attacks. Although these methods claim robustness by demonstrating strong resistance against distortions (e.g., blurring or noise perturbations), adversaries may still

remove the watermark through steganalysis, thus compromising their robustness. Our experiments reveal that even complex, highly nonlinear methods based on deep neural networks are susceptible to steganalysis-based watermark removal.

To avoid such threat, future watermarking algorithms shall use content-adaptive watermarking methods. One approach is to incorporate image features while encoding watermark information. For example, HiDDeN [10] and RivaGAN [11] introduce image features into the watermark encoder through concatenation and attention, respectively. For existing content-agnostic methods, assigning multiple watermarks per user can mitigate but not fundamentally solve steganalysis threat.

While we performed simple averaging steganalysis on RGB images and 16 kHz monophonic audios, watermarking methods should resist more complex techniques, such as steganalysis in various colour spaces or different transform domains. Evaluating security against diverse steganalysis models, analogous to robustness tests against distortions, is crucial for developing secure watermarking algorithms. These two aspects form our security guidelines:

Ensure watermarks are content-adaptive to resist steganalysis attacks.

Evaluate watermark against steganalysis to ensure robustness.

6 Conclusions

This work has revealed the vulnerability of content-agnostic watermarking algorithms to steganalysis attacks. We have demonstrated effective watermark removal and forgery techniques under both greybox and blackbox settings across twelve watermarking methods, including recent deep-learning approaches. Our findings extend to audio watermarking methods as well. To address these threats, we propose security guidelines that encourage exploring content-adaptive watermarking methods and evaluating them against steganalysis attacks. We have also proposed temporary mitigations for existing content-agnostic methods. Only by addressing the vulnerability to steganalysis can we develop secure and robust digital watermarking systems capable of safeguarding the integrity of digital content in the era of generative AI.

Limitations and ethical considerations Steganalysis-based watermark removal/forgery is only effective against content-agnostic watermarking, not content-adaptive techniques. Responsible development and deployment of steganalysis technologies are crucial, adhering to fairness, accountability, and transparency principles to prevent misuse for unwarranted surveillance or privacy violations.

Broader impacts The proposed steganalysis attack and security guidelines extend beyond image watermarking. They apply to watermarking other media like video [48], audio [49], 3D models [5052], and to other domains. Our proposed guidelines and mitigation strategies strengthen watermarking security, contributing to a safer digital environment.

References

R. C. Gonzalez and R. E. Woods, Digital image processing , 3rd ed. Upper Saddle River, N.J;Harlow;: Pearson/Prentice Hall, 2008.

M. Asikuzzaman and M. R. Pickering, 'An overview of digital video watermarking,' IEEE Transactions on Circuits and Systems for Video Technology , vol. 28, no. 9, pp. 2131-2153, 2018.

I. Cox, M. Miller, J. Bloom, and C. Honsinger, 'Digital watermarking,' Journal of Electronic Imaging , vol. 11, no. 3, pp. 414-414, 2002.

J. Ho, A. Jain, and P. Abbeel, 'Denoising diffusion probabilistic models,' Advances in neural information processing systems , vol. 33, pp. 6840-6851, 2020.

Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, 'Score-based generative modeling through stochastic differential equations,' arXiv preprint arXiv:2011.13456 , 2020.

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, 'High-resolution image synthesis with latent diffusion models,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2022, pp. 10 684-10 695.

C. Meng, Y. He, Y. Song, J. Song, J. Wu, J.-Y. Zhu, and S. Ermon, 'SDEdit: Guided image synthesis and editing with stochastic differential equations,' in International Conference on Learning Representations , 2022.

Y. Mirsky and W. Lee, 'The creation and detection of deepfakes: A survey,' ACM computing surveys (CSUR) , vol. 54, no. 1, pp. 1-41, 2021.

L. Verdoliva, 'Media forensics and deepfakes: an overview,' IEEE Journal of Selected Topics in Signal Processing , vol. 14, no. 5, pp. 910-932, 2020.

J. Zhu, R. Kaplan, J. Johnson, and L. Fei-Fei, 'Hidden: Hiding data with deep networks,' in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 657-672.

K. A. Zhang, L. Xu, A. Cuesta-Infante, and K. Veeramachaneni, 'Robust invisible video watermarking with attention,' arXiv preprint arXiv:1909.01285 , 2019.

I. Cox, M. Miller, J. Bloom, J. Fridrich, and T. Kalker, Digital Watermarking and Steganography , 2nd ed. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2007.

T. Bui, S. Agarwal, N. Yu, and J. Collomosse, 'Rosteals: Robust steganography using autoencoder latent space,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 933-942.

Y. Wen, J. Kirchenbauer, J. Geiping, and T. Goldstein, 'Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust,' arXiv preprint arXiv:2305.20030 , 2023.

M. Tancik, B. Mildenhall, and R. Ng, 'Stegastamp: Invisible hyperlinks in physical photographs,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020.

H. Ci, P. Yang, Y. Song, and M. Z. Shou, 'Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification,' 2024.

B. An, M. Ding, T. Rabbani, A. Agrawal, Y. Xu, C. Deng, S. Zhu, A. Mohamed, Y. Wen, T. Goldstein et al. , 'Benchmarking the robustness of image watermarks,' arXiv preprint arXiv:2401.08573 , 2024.

W. Nie, B. Guo, Y. Huang, C. Xiao, A. Vahdat, and A. Anandkumar, 'Diffusion models for adversarial purification,' arXiv preprint arXiv:2205.07460 , 2022.

Z. Yang, K. Zeng, K. Chen, H. Fang, W. Zhang, and N. Yu, 'Gaussian shading: Provable performancelossless image watermarking for diffusion models,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2024.

L. Lei, K. Gai, J. Yu, and L. Zhu, 'Diffusetrace: A transparent and flexible watermarking scheme for latent diffusion model,' arXiv preprint arXiv:2405.02696 , 2024.

R. Min, S. Li, H. Chen, and M. Cheng, 'A watermark-conditioned diffusion model for ip protection,' arXiv preprint arXiv:2403.10893 , 2024.

P. Fernandez, G. Couairon, H. Jégou, M. Douze, and T. Furon, 'The stable signature: Rooting watermarks in latent diffusion models,' in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 22 466-22 477.

X. Xian, G. Wang, X. Bi, J. Srinivasa, A. Kundu, M. Hong, and J. Ding, 'Raw: A robust and agile plug-and-play watermark framework for ai-generated images with provable guarantees,' 2024.

R. Ma, M. Guo, L. Yuming, H. Zhang, C. Ma, Y. Li, X. Xie, and S. Zhang, 'Pigw: A plug-in generative watermarking framework,' arXiv preprint arXiv:2403.12053 , 2024.

P. Fernandez, A. Sablayrolles, T. Furon, H. Jégou, and M. Douze, 'Watermarking images in self-supervised latent spaces,' in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 3054-3058.

Y. Zhao, T. Pang, C. Du, X. Yang, N.-M. Cheung, and M. Lin, 'A recipe for watermarking diffusion models,' arXiv preprint arXiv:2303.10137 , 2023.

H. Ci, Y. Song, P. Yang, J. Xie, and M. Z. Shou, 'Wmadapter: Adding watermark control to latent diffusion models,' arXiv preprint arXiv:2406.08337 , 2024.

F. Hartung and M. Kutter, 'Multimedia watermarking techniques,' Proceedings of the IEEE , vol. 87, no. 7, pp. 1079-1107, 1999.

F. A. Petitcolas, R. J. Anderson, and M. G. Kuhn, 'Information hiding-a survey,' Proceedings of the IEEE , vol. 87, no. 7, pp. 1062-1078, 1999.

A. A. Ukasha, R. M. Alabid, I. A. Nasir, and F. A. Alwafi, 'A robust color image watermarking improvement scheme based on image normalization,' in 2021 IEEE 1st International Maghreb Meeting of the Conference on Sciences and Techniques of Automatic Control and Computer Engineering MI-STA , 2021, pp. 356-362.

M. Celik, J. Talstra, A. Lemma, and S. Katzenbeisser, 'Camcorder capture robust low-complexity watermarking of mpeg-2 bit-streams,' in 2007 IEEE International Conference on Image Processing , vol. 5. IEEE, 2007, pp. V-489.

T. Yu-min, Q. Yun-hui et al. , 'Adaptive video watermarking algorithm based on mpeg-4 streams,' in 2008 10th International Conference on Control, Automation, Robotics and Vision . IEEE, 2008, pp. 1084-1088.

M. Ahmadi, A. Norouzi, N. Karimi, S. Samavi, and A. Emami, 'Redmark: Framework for residual diffusion watermarking based on deep networks,' Expert Systems with Applications , vol. 146, p. 113157, 2020.

X. Luo, Y. Li, H. Chang, C. Liu, P. Milanfar, and F. Yang, 'Dvmark: a deep multiscale framework for video watermarking,' IEEE Transactions on Image Processing , 2023.

K. A. Zhang, L. Xu, A. Cuesta-Infante, and K. Veeramachaneni, 'Robust invisible video watermarking with attention,' arXiv preprint arXiv:1909.01285 , 2019.

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, 'High-resolution image synthesis with latent diffusion models,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 10 684-10 695.

X. Zhao, K. Zhang, Z. Su, S. Vasan, I. Grishchenko, C. Kruegel, G. Vigna, Y.-X. Wang, and L. Li, 'Invisible image watermarks are provably removable using generative ai,' arXiv preprint arXiv:2306.01953 , 2023.

N. Lukas, A. Diaa, L. Fenaux, and F. Kerschbaum, 'Leveraging optimization for adaptive attacks on image watermarks,' arXiv preprint arXiv:2309.16952 , 2023.

T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, 'Microsoft coco: Common objects in context,' in Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 . Springer, 2014, pp. 740-755.

Gustavosta, 'Stable-diffusion-prompts,' https://huggingface.co/datasets/Gustavosta/ Stable-Diffusion-Prompts, 2023.

Z. J. Wang, E. Montoya, D. Munechika, H. Yang, B. Hoover, and D. H. Chau, 'Diffusiondb: A large-scale prompt gallery dataset for text-to-image generative models,' arXiv preprint arXiv:2210.14896 , 2022.

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, 'Imagenet: A large-scale hierarchical image database,' in 2009 IEEE Conference on Computer Vision and Pattern Recognition , 2009, pp. 248-255.

R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, 'The unreasonable effectiveness of deep features as a perceptual metric,' in CVPR , 2018.

T. R. Shaham, T. Dekel, and T. Michaeli, 'Singan: Learning a generative model from a single natural image,' in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 4570-4580.

R. S. Roman, P. Fernandez, A. Défossez, T. Furon, T. Tran, and H. Elsahar, 'Proactive detection of voice cloning with localized watermarking,' 2024.

G. Chen, Y. Wu, S. Liu, T. Liu, X. Du, and F. Wei, 'Wavmark: Watermarking for audio generation,' 2024.

R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, 'Common voice: A massively-multilingual speech corpus,' 2020.

A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts et al. , 'Stable video diffusion: Scaling latent video diffusion models to large datasets,' arXiv preprint arXiv:2311.15127 , 2023.

Anonymous, 'Stable audio,' https://stability.ai/stable-audio, 2024, accessed: 2024-06-12.

B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, 'Dreamfusion: Text-to-3d using 2d diffusion,' arXiv , 2022.

M. Deitke, R. Liu, M. Wallingford, H. Ngo, O. Michel, A. Kusupati, A. Fan, C. Laforte, V. Voleti, S. Y. Gadre et al. , 'Objaverse-xl: A universe of 10m+ 3d objects,' Advances in Neural Information Processing Systems , vol. 36, 2024.

H. Ci, M. Wu, W. Zhu, X. Ma, H. Dong, F. Zhong, and Y. Wang, 'Gfpose: Learning 3d human pose prior with gradient fields,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 4800-4810.

A Supplementary Materials

A.1 Watermark implementation details

Tree-Ring watermarks [14] are implemented following the corrected version proposed by RingID [16] under the verification setting. In comparison, the original Tree-Ring watermarks exhibited an issue where the watermark pattern actually injected into the initial noise pattern was inconsistent with the pattern claimed by Tree-Ring [14].

Our implementation adopts the original ring pattern proposed by Tree-Ring [14] with radii from 0 to 10, but the ring pattern is centred at the origin in the Fourier domain, following the correction proposed by RingID [16]. To ensure lossless watermark injection, we discard the imaginary part of the sampled watermark pattern during watermark detection. This lossless injection ensures that the watermark pattern actually injected matches the reference pattern used during detection.

RingID [16] is implemented using patterns with radii ranging from 3 to 14 in the identification setting.

A.2 Perceptual quality of images under distortions

The distortions applied in plotting Figure 4 are violent, as visualised in Figure 8. Blurring with a radius of 13 renders the cat's eyes invisible. Perturbing with σ = 100 noise eliminates the cat's pattern. Upscaling the brightness to eight times overexposes the entire background. However, as shown in Figure 4, none of these distortions can defeat Tree-Ring's detector, but steganalysis-based watermark removal can. This is possibly due to Tree-Ring's large-scale content-agnostic ripple-like

No Removal

Ours

×

1

×

2

×

3

No Removal

r

= 1

r

= 5

r

= 9

No Removal

σ

= 10

σ

= 20

σ

= 50

No Removal

Brightness

×

2

×

4

×

6

×

4

r

= 13

σ

= 100

×

8

patterns according to our analysis in the main paper. This demonstrates that steganalysis-based watermark removal effectively captures the vulnerability of the watermarking method through the extracted pattern, even without knowledge about the watermarking algorithm. Subsequently, it highlights that Tree-Ring's robustness to distortions is a Maginot Line that can be breached with steganalysis attacks.

A.3 More visualisations of patterns extracted

In the main text, we have visualised the patterns extracted from content-agnostic watermarking methods. In this section, we present a more comprehensive visualisation of patterns extracted from content-agnostic methods in Figure 9, along with visualisations of the patterns extracted from content-adaptive watermarking techniques, as depicted in Figure 10.

While subtracting these patterns from watermarked images may not effectively circumvent the watermark detection process, the extracted patterns still exhibit regularities. By averaging 5000 images, the patterns extracted from both Stable Signature [22] and WmAdapter [27] display mild repetitive grid-like structures, suggesting that images watermarked with this technique may share commonalities at specific frequency components. For such repetitive patterns, it would be worth exploring whether they are caused by the patching mechanism of the transformer architecture. Patterns extracted by RivaGAN [11] exhibit a greenish tint, indicating RivaGAN introduces a systematic bias during the watermarking process despite content-adaptive components. The patterns extracted by SSL [25], similar to RAWatermark [23], are biased towards a specific noise distribution. However, while RAWatermark [23] is vulnerable to steganalysis-based removal, SSL [25] demonstrates robustness against such attacks. As the parameter n increases, the noise components in the patterns extracted by HiDDeN [10] transition towards a greyish hue and eventually diminish. By the gray-world assumption principle, this observation suggests that HiDDeN best incorporates content-adaptive watermarks into the images. Under spatial domain averaging, DwtDct [12] reveals nothing but a tiny systematic bias, which aligns with the fact

We also visualise audio patterns extracted from AudioSeal [45] and WavMark [46] in Figure 11. When n = 5000 , the greybox AudioSeal-extracted pattern is highly imperceptible, with a signal strength below -45dB relative to the cover media. However, directly subtracting this pattern in the time domain significantly reduces the watermark detection accuracy by 30%. In contrast, WavMarkextracted patterns have larger amplitudes, indicating that WavMark also introduces a systematic bias during the watermarking process. Moreover, greybox-extracted WavMark patterns concentrate within the first second, which aligns with WavMark's method of adding watermark patterns at one-second intervals. This exposes WavMark's watermarking locations, allowing an adversary to potentially remove the watermark by simply clipping out these segments. The spectrograms further show that WavMark mainly adds watermarks below 6 kHz, which could represent the robustness threshold it was trained to withstand against low-pass filtering. In summary, although both methods claim robustness against various distortion-based attacks, their vulnerabilities can be easily exposed through simple steganalysis techniques.

A.4 Perceptual quality of images after watermark removal

In the main text, we analyzed the perceptual quality of Tree-Ring watermarked images after watermark removal. We now visualise content-agnostically watermarked images after watermark removal in Figure 12, and content-adaptively watermarked images after watermark removal in Figure 13.

From these two figures, we can observe that except for Gaussian Shading [19], images carrying different watermarks, after watermark removal, have nearly identical visual quality. The visual quality is predominantly related to n , the number of images averaged during pattern extraction. In the greybox setting, when n > 50 , there are no visually apparent artefacts. In the blackbox setting, n > 100 is generally required for eliminating significant artefact patterns. For images with Gaussian Shading [19] watermarks, due to the large magnitude of the extracted watermark pattern itself, steganalysis-based removal inevitably causes significant changes to the image content, thereby reducing perceptual quality. This leads to two insights:

For adversaries, averaging more images allows for a better approximation of an effective content-agnostic watermark pattern that can be used for watermark removal;

Blackbox

Blackbox

Tree-Ring

Blackbox

Blackbox

RAWatermark

DwtDctSvd

5

10

50

100

500

1000

5000

Number of Watermarked Images Averaged

Blackbox

RoSteALS

Blackbox

Graybox

Blackbox

Graybox

Blackbox

Stable Signature

WmAdapter

Graybox

Blackbox

Graybox

Blackbox

RivaGAN

SSL

5

10

50

100

500

1000

5000

Number of Watermarked Images Averaged

Graybox

Blackbox

HiDDeN

Graybox

Blackbox

DwtDct

AudioSeal

Blackbox

AudioSeal

WavMark

Blackbox

WavMark

n

= 5

n

= 10

n

= 50

n

= 100

n

= 500

n

= 5000

2.5

8k

4k

0

0

0.5

1

1.5

2

Time(s)

-2

8k

4k

0

0.5

1

1.5

2

Time (s)

8k

[ZH)

1

8k

4k

4k

0

0

0.5

1

1.5

2

0

0

0.5

1

1.5

2

Time (s)

Time(s)

2.5

8k

4k

0

0

0.5

1

1.5

2

Time(s)

0.0

8k

4k

0

0

0.5

1

1.5

2

Time(s)

2.5

<0.0

8k

8k

4k

4k

O

0

0.5

1

1.5

2

0

0

0.5

1

1.5

2

Time(s)

Time(s)

-1

8k

4k

0

0.5

1

1.5

2

Time（s)

-5

8k

4k

0

0

0.5

1

1.5

2

Time(s)

-1

8k

(ZH)

8k

4k

4k

0

0

0.5

1

1.5

2

0

0.5

1

1.5

2

me(s)

Time(s)

-5

8k

4k

0

0

0.5

1

1.5

2

Time(s)

-5

8k

4k

0

0

0.5

1

1.5

2

Time(s)

-5

4

-5

8k

(H) ba

8k

4k

4k

0

0.5

1

1.5

2

0

0

0.5

1

1.5

2

Time(s)

Time（s)

2.5

8k

4k

0

0

0.5

1

1.5

2

Time(s)

-5

8k

4k

0

0

0.5

1.5

2

Time(s)

2.5

2.5

8k

(ZH)

8k

4k

4k

0

0.5

1

1.5

2

0

0

0.5

1

1.5

2

Time(s)

Time(s)

-1

8k

4k

0

0

0.5

1

1.5

2

Time(s)

2.5

8k

4k

0

0

0.5

1

1.5

2

Time(s)

2.5

8k

8k

4k

4k

0

0

0.5

1

1.5

2

0

0

0.5

1

1.5

2

Time(s)

Time(s)

For watermark distributors, reducing the count of distributing the same watermark could lower the security risk posed by steganalysis.

A.5 More results on quantitative analysis on watermark removal

In Section 4.2, we primarily discussed the Peak Signal-to-Noise Ratio (PSNR) as a measure of image distortion resulting from our steganalysis-based watermark removal method. To provide a more comprehensive assessment of the impact on visual quality, we also evaluated three additional metrics: Structural Similarity Index (SSIM), Learned Perceptual Image Patch Similarity (LPIPS), and Single Image Fréchet Inception Distance (SIFID).

Tables 2-13 present the performance of various watermarking methods under our steganalysis-based removal attack, along with the corresponding image quality metrics. The trends observed in PSNR are largely mirrored in the additional metrics:

Content-Agnostic Methods For methods like Tree-Ring [14], RingID [16], RAWatermark [23], DwtDctSvd [12], and RoSteALS [13], all metrics show a consistent trend: as the number of images averaged ( n ) decreases, watermark removal becomes more effective (lower detection rates), but at the cost of increased image distortion (lower PSNR, SSIM, and higher LPIPS, SIFID).

For example, in the case of RAWatermark (Table 4, Blackbox setting), as n decreases from 5000 to 5, the AUC drops from 0.574 to 0.133, indicating more effective watermark removal. However, this comes at the cost of image quality: PSNR drops from 27.98 to 17.92, SSIM from 0.964 to 0.528, while LPIPS increases from 0.020 to 0.424, and SIFID from 0.022 to 1.401, all indicating significant visual degradation.

Blackbox

Blackbox

Tree Ring

Blackbox

Blackbox

RAWatermark

DwtDctSvd

No Removal

5

10

50

100

500

5000

Number of Watermarked Images Averaged

Blackbox

RoSteALS

Blackbox

Blackbox

Blackbox

Stable Signature

WmAdapter

Blackbox

Blackbox

RivaGAN

SSL

No Removal

5

10

50

100

500

5000

Number of Watermarked Images Averaged

Blackbox

HiDDeN

Blackbox

DwtDct

7

0

5

10

15

20

1

2

3

4

5

6

Legend

No Removal

5

10

20

50

100

200

500

1000

2000

5000

# Imgs Avged During Extraction:

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

7

AUC

12

13

14

15

16

1

2

3

4

5

6

7

PSNR (dB)

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

7

AUC

12

13

14

15

16

1

2

3

4

5

6

7

PSNR (dB)

Blackbox

Horizontal Axis: Number of Heterogeneous Tree-Ring Keys Mixed in the Adversary's Collection

Content-Adaptive Methods In contrast, content-adaptive methods like Stable Signature [22], WmAdapter [27], RivaGAN [11], and SSL [25] maintain good detection performance (bit accuracies > 0 . 95 ) even as n decreases, with the image quality metrics also show minimal degradation. For instance, SSL (Table 11, greybox setting) maintains a high bit accuracy (0.895 at n = 5 ) while preserving image quality: PSNR (34.26), SSIM (0.920), LPIPS (0.048), and SIFID (0.048) are all close to the no-removal (NRmv) values. This could be due to that the systematic biases averaged out from content-adaptive methods are mild, so subtracting such mild patterns from the original images would not lead to significant perturbations.

A.6 Effectiveness of removal under multiple watermarks

In Section 4.4.4, our case study on Tree-Ring [14] demonstrated that when an adversary's image collection contains a mix of several different watermark patterns, the effectiveness of watermark removal is significantly reduced. Based on this finding, we proposed assigning multiple keys as a mitigation strategy against steganalysis threats. However, in this section, we caution that this approach is not a universal solution and should be applied judiciously. The high complexity of watermarking algorithms means that this method cannot guarantee enhanced watermark security. To illustrate this, we replicate our experiment using the DwtDctSvd [12] algorithm.

Figure 14 shows that in the greybox scenario, DwtDctSvd behaves similarly to Tree-Ring: as the adversary's image collection incorporates more diverse watermarks, watermark removal becomes less effective without significantly impacting image quality. Interestingly, this strategy fails in the blackbox setting. When the adversary's collection mixes more than 4 watermarks, DwtDctSvd's detection accuracy surprisingly drops below 0.1. While the reasons behind this phenomenon warrant further investigation, this observation underscores our main point: assigning multiple watermarks per user is merely a temporary workaround to bolster current content-agnostic watermarking algorithms. It does not address the fundamental vulnerabilities of these algorithms and may even be counterproductive in certain scenarios. Therefore, watermark distributors should employ this method cautiously and at

their own discretion, understanding that it is not a comprehensive solution to the inherent security challenges of content-agnostic watermarking algorithms.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Yes, the main claims are accurately reflected in the abstract and introduction. Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Yes. The limitations are discussed in Section 6 under the paragraph "limitations".

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include any theoretical results or proofs.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Yes. The experimental setups are provided in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: The code and data are not provided in this manuscript submission.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Yes. The experimental setups are provided in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: No. Error bars are not presented because it would be too computationally expensive.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Yes. These are included in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes. The research conforms to the NeurIPS Code of Ethics, as discussed in Section 6.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Yes. The broader impacts, including potential positive and negative societal impacts, are discussed in Section 6.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer:[NA]

Justification: The paper does not release any data or models that have a high risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Yes. The use of existing datasets and evaluation methods are all credited with citations, as requested by the property owners or licenses.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not introduce any new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a study on the vulnerabilities of digital watermarking techniques to steganalysis attacks. Extensive experiments are conducted to demonstrate the effectiveness of steganalysis in detecting and removing watermarks from images, especially when targeting content-agnostic watermarking methods.\n\nSTRENGTHS:\n1. The paper is well-written and well-organized. The watermark detection and removal/forgery methods are simple and clear.\n2. The method of estimating the watermark disturbance by subtracting the average of watermarked images from the average of original images is interesting.\n3. The proposed watermark attacks are effective for content-agnostic watermarking methods.\n\nWEAKNESSES:\n1. This paper adopts a simple linear assumption that the watermarked image is obtained by adding the content-agnostic perturbation to the original image. The generality of this assumption needs to be clarified, as it forms the theoretical foundation for the watermark attacks discussed in this paper. For instance, the Tree-Ring method embeds information in the frequency domain of initial noise, but why does it conform to this model?\n\n2. The definition of steganalysis needs to be clearly stated, as it is an important concept in this paper.\n\n3. In section 3.1, the paper mentions that ''Instead of applying strong distortions (e.g., noise), the adversary could take a steganalysis approach, first **approximating w** and then crafting minimal distortions to fool D''. However, in the following method, the watermark is never approximated. Instead, the additive perturbation is directly predicted to fool D.\n\n4. In Figure 2, why are the performance metrics inconsistent? Although the caption specifies the use of AUC, the figure mixes AUC with Bit Acc as metrics for different methods.\n\n5. It is better to explain why there is a sudden drop when n = 5 for content-adaptive methods like Stable Signature.\n\nQUESTIONS:\nPlease see Weaknesses.\n\nLIMITATIONS:\nThe authors have addressed the limitations in section 5.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces steganalysis techniques targeting watermark removal and forgery. The authors demonstrate through experiments that existing content-agnostic watermarking methods are unable to resist steganalysis attacks, advocating for the adoption of content-adaptive strategies.\n\nSTRENGTHS:\nThis paper is interesting and well-written. The author introduces steganalysis attack can remove and forge watermarks from watermark images using simple linear addition and subtraction operations in both gray-box and black-box modes.\n\nWEAKNESSES:\n1. Although the examples provided by the authors in Figure 7 show that the images after watermark removal exhibit good image quality compared to the watermarked images. However, the PSNR data shown in Figure 2 are not ideal. For instance, in the first example \"Tree-Ring,\" the average PSNR of the removed watermark image is only 17. According to [1],  distortion of modified images with PSNRs lower than 36 dB is noticeable to human visual system. If the quality of the image is severely degraded after watermark removal, then the act of removing the watermark becomes meaningless.\n2. The author said “In contrast, traditional distortion-based removal techniques, such as noise perturbations or blurring, typically result in substantial perceptual degradation (as visualized in  Figure 9 in Appendix A.2)”. But I think comparing perceived degradation like this isn't fair because it's difficult to objectively evaluate the severity of different attacks.\n3. While the authors present an interesting steganalysis method for removing or forging content-agnostic watermarks, the contribution of this approach may be somewhat limited. Additionally, the method's requirement for a large number of pre-existing watermarked images poses practical challenges for real-world applications.\n\n[1] S. Sarreshtedari and M. A. Akhaee, “A source-channel coding approach to digital image protection and self-recovery,” IEEE Trans. Image Process., vol. 24, no. 7, pp. 2266–2277, Jul. 2015.\n\nQUESTIONS:\nSee Weaknesses\n\nLIMITATIONS:\n1. The premise of steganalysis for watermarks requires a large number of images embedded with watermarks, unlike attacks such as JPEG compression, which can directly affect the watermark itself. This limitation makes steganalysis attacks somewhat impractical in real-world applications because attackers may not always have access to a sufficient number of watermarked images.\n2. This method is only effective against content-agnostic watermarking.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses vulnerabilities in digital watermarking methods, especially those that are content-agnostic. These methods, which embed fixed watermark patterns regardless of image content, are susceptible to steganalysis attacks that can extract and manipulate these patterns, potentially removing or forging watermarks with minimal impact on perceptual quality.\n\nSTRENGTHS:\nThe findings of this paper are interesting, and it is reasonably well-written and easy to follow.\n\n1. This paper introduces a novel method for steganalysis that effectively exposes vulnerabilities in content-agnostic digital watermarking techniques. Using simple averaging to extract watermark patterns, the study demonstrates a practical approach that can be applied in gray-box and black-box settings. \n\n2. This paper provides a thorough evaluation of various watermarking methods under different settings. It includes quantitative and qualitative analyses across eight different watermarking techniques, highlighting the particular susceptibility of content-agnostic methods to steganalysis attacks.\n\n3. Recognizing the limitations of current watermarking methods, the paper proposes actionable security guidelines and strategies for enhancing digital watermark security.\n\nWEAKNESSES:\n1. The steganalysis method proposed in this paper is fundamentally general and not restricted to images. However, the experiments are confined to analyzing image watermarks. Providing evidence of successful steganalysis in other modalities, such as audio, would further validate and broaden the method's impact.\n\n2. The steganalysis method proposed in this paper can extract a unified low-level watermark pattern from tree-ring watermarked images. So why can the tree-ring change the layout of the generated image?\n\n3. The quality of the figures could be further improved. For example, the font in Figure 2.3.4 is too small. Please enlarge it for easier reading.\n\nQUESTIONS:\nPlease see above.\n\nLIMITATIONS:\nPlease see above.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a steganalysis-based attack aimed at image watermarking methods. The attack is effective in both gray-box and black-box scenarios and focuses on identifying repeating patterns present in watermarked images. These patterns can be exploited to either remove watermarks or add them to non-watermarked images without needing access to the original watermarking algorithm. Experimental results demonstrate that this attack can successfully compromise watermarks that use content-agnostic watermark patterns.\n\nSTRENGTHS:\n- The authors illustrate the vulnerability of content-agnostic watermarks to simple attacks, serving as a good reminder for researchers to ensure their watermarks are content-aware.\n\n- The proposed attack is simple and compared to some existing watermark attacks, has a lower computational cost. However, it's worth noting that the computational cost of other attacks is usually not significant either, so the value of this simplicity and speed-up remains debatable.\n\nWEAKNESSES:\n- Similar ideas (i.e., considering watermarked and non-watermarked samples to extract watermark patterns for removal and forgery) have been explored in existing works [1][2]. For instance, [1] discusses a spoofing attack using a noise image $X$, watermarking it to obtain $X_{wm}$, and then adding $(X_{wm}-X)$ to clean images to create watermarked versions. Both [1] and [2] also describe a surrogate model adversarial attack, which involves training a CNN to distinguish between watermarked and non-watermarked images. This CNN is then used to adversarially alter samples to change their watermark status. This adversarial attack is a more advanced version of the method suggested in this paper and can potentially target content-aware watermarks as well.\n\n- The authors claim, \"Notably, our steganalysis-based approach is the first to effectively remove Tree-Ring watermarks without access to the algorithm,\" which is inaccurate. The adversarial attack proposed in [1], though ML-based, requires only a set of watermarked and non-watermarked samples (similar to the black-box attack described in this paper) to break Tree-Ring.\n\n- The method introduced in this paper lacks novelty, and the vulnerability of content-agnostic watermarks to this type of attack is trivial and not a significant discovery.\n\n[1] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., and Feizi, S. Robustness of AI-image detectors: Fundamental limits and practical attacks, 2023.\n\n[2] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., Zhu, S., Mohamed, A., Wen, Y., Goldstein, T., et al.: Benchmarking the robustness of image watermarks, 2024.\n\nQUESTIONS:\n- I suggest that the authors compare their attack to existing watermark attacks (e.g., from this watermark benchmark [1]). This comparison is necessary to illustrate the advantages of their attacks.\n\n- The results for the forgery attack are provided only for the Tree-Ring watermark. I recommend including results for other watermarks as well to provide a more comprehensive evaluation.\n\n\n[1] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., Zhu, S., Mohamed, A., Wen, Y., Goldstein, T., et al.: Benchmarking the robustness of image watermarks, 2024.\n\nLIMITATIONS:\n- The attack only works on content-agnostic watermarks, which the authors adequately mentioned in several parts of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-23"}