# Evaluating the security of public surrogate watermark detectors

Chloé Imadache, Eva Giboulot, Teddy Furon

► To cite this version:

Chloé Imadache, Eva Giboulot, Teddy Furon. Evaluating the security of public surrogate watermark detectors. ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, Apr 2025, Hyderabad, India. pp.1-5, 10.1109/ICASSP49660.2025.10889821. hal-05168353

HAL Id: hal-05168353

https://hal.science/hal-05168353v1

Submitted on 17 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Evaluating the security of public surrogate watermark detectors

Chloé Imadache

Univ. Rennes, Inria, CNRS, IRISA

Rennes, France

chloe.imadache@inria.fr

Eva Giboulot

Univ. Rennes, Inria, CNRS, IRISA

Rennes, France

eva.giboulot@inria.fr

Teddy Furon

Univ. Rennes, Inria, CNRS, IRISA

Rennes, France

teddy.furon@inria.fr

<figure><img src="image_2.png" /></figure>

**Abstract**—The omnipresence of generated content has led to an increasing need of multimedia content traceability. Watermarking techniques have been proven to provide both detection guarantees and robustness. However, widespread use of such methods would require disclosing the watermark detector to the public. Such access breaches the watermark security: end-users with unlimited access to the detector could easily craft adversarial examples, through white-box and black-box attacks. To circumvent this issue, we suggest providing to the public a surrogate, less accurate detector. Calls to the private detector would be reserved for important or anomalous cases. This paper studies the potential leakage of information from the surrogate detector. We first create a wide panel of images adversarial to the surrogate detector. The efficiency of the private detector is then assessed on this data. This allows us to introduce a metric of the transferability of these attacks from the surrogate to the private detector. Through this metric, we evaluate the security of different designs of surrogate detectors.

**Index Terms**—Watermarking, Adversarial examples

## I. INTRODUCTION

Digital watermarking is a mature technology that remains unknown from the general public because its first requirement is imperceptibility. Millions of people are daily exposed to watermarked content be it photos of the news industry (web or print) to identify the source photo agency; audio and video of Digital Cinemas or Video On Demand portals to combat piracy; or audio of TV broadcasts for audience measurement.

Digital watermarking will soon be under the spotlights as it recently finds a new application in the era of Generative Artificial Intelligence. Identifying content provenance becomes an important concern because Generative AI is the ideal tool to forge fakenews and deepfakes used for biasing public opinion [1], fraud, or impersonation at a higher scale and more convincingly than even authentic content [2].

Governments are tackling this issue through new regulations that impose watermarking as a technical means for transparency and traceability [3]–[5]. The EU AI Act requires that outputs of generative models are labeled and identifiable as such, through methods “robust as far as this is technically feasible [...] such as watermarks”. The Californian bill CA AB3211 requires model providers to apply provenance data means which include “embedding of watermarks in digital content” [6].

French National Research Agency ANR.

At first sight, the primary concern is robustness: malicious actors might attempt to remove the watermark. This issue is frequently cited as the main barrier to implementing watermarking for detection purposes [7]–[9]. Improving robustness is challenging, but it should not be a case of not seeing the forest for the trees.

If watermark embedding becomes mandatory, an important issue is who is charge of the watermark detection. The California bill imposes that “a generative AI provider shall create and make available to the public a provenance detection tool [which] allows a user to assess whether digital content was created or altered by a generative AI system”. The watermarking research community has identified this as a security threat a long time ago [10], [11]. A public access to the detector allows watermark removal by oracle attacks [12], nowadays called black-box attacks [13]. On the other hand, Governments seem to be aware of this threat. The California bill also states that the “provider may limit access to the decoder to ensure the robustness and security of their provenance data techniques.”

Defenses range from time delay to slow down the attack [14], banning the user if such an attack is detected [15], or randomizing the watermark detector output to jeopardize the attack [16], [17].

This paper explores another idea dedicated to this need of a widely available detector not endangering the watermarking scheme. The immense capabilities of Machine learning in Computer Vision allow to train a deep neural network mimicking the watermark detector. The latter is coined the private detector whose access is restricted to few trusted actors, whereas the former plays the role of a public surrogate of the detector. The contribution of the paper are as follows:

1) a methodology for training post-hoc the public surrogate of a watermark detector.

2) an investigation on the trade-off between the fidelity of the public surrogate and its misuse by an attacker to delude the ground-truth ‘private’ detector.

## II. LEARNING THE SURROGATE WATERMARK DETECTOR

A sound watermark detector is defined as a function $f: \mathcal{X} \times \mathcal{K} \rightarrow [0, 1]$ which takes as an input an image $\mathbf{x} \in \mathcal{X}$, a secret key $\mathbf{k} \in \mathcal{K}$ and outputs a p-value probability. It means that under hypothesis $H_0$ where the input is a random non-watermarked image $\mathbf{X} \in \mathcal{X}$, $f(\mathbf{X}, \mathbf{k})$ is uniformly
distributed over [0, 1]. On the contrary, under hypothesis $H_1$ for a watermarked and possibly attacked image, the probability $f(\mathbf{X}, \mathbf{k})$ is very low. It is interpreted as the probability that a non-watermarked input triggers such a confident detection. Starting from a sound private watermarked detector $f^*$, our goal is to train a sound surrogate model $f$.

a) **Private detector:** Most current watermarking methods are multi-bit [18]–[22]: a message of size $M$ is extracted from the input using a decoder $d^* : \mathcal{X} \to \{0, 1\}^M$. This message plays the role of the secret key so that $\mathcal{K} = \{0, 1\}^M$. Image $\mathbf{x}$ is deemed watermarked if the decoded message $d^*(\mathbf{x})$ is close enough to the secret key $\mathbf{k}$. This similarity is measured by the Hamming distance $H(\mathbf{x}, \mathbf{k}) := \sum_{i=1}^M 1[d^*(\mathbf{x})_i = \mathbf{k}_i]$. Assuming that under $\mathcal{H}_0$ the decoded bits are independent and uniformly distributed, this is mapped onto a p-value thanks to the regularized incomplete beta function:

$$f^*(\mathbf{x}, \mathbf{k}) := \mathbb{P}(H(\mathbf{X}, \mathbf{k}) > H(\mathbf{x}, \mathbf{k}) | \mathcal{H}_0) \\ = I_{\frac{1}{2}}(H(\mathbf{x}, \mathbf{k}) + 1, M - H(\mathbf{x}, \mathbf{k})). \quad (1)$$

b) **Surrogate detector:** The surrogate detector $f_\theta$ is a deep neural network learned by distilling the private detector. We frame its training as a regression problem of the log-p-values of the private detector. This is performed by training $f_\theta$ with a mean squared error loss:

$$\mathcal{L}(\theta) = \frac{1}{|\mathcal{T}|} \sum_{\mathbf{x}_i \in \mathcal{T}} (\log f_{\theta}(\mathbf{x}_i) - \log f^*(\mathbf{x}_i, \mathbf{k}))^2, \quad (2)$$

where $\mathcal{T}$ is a training data set.

c) **Image augmentations:** The surrogate detector should be robust to usual image processing operations – i.e denoising, cropping, downsampling, JPEG compression etc.... To that aim, watermark decoders are usually trained by applying varying degradation operators to the images as data augmentations – see the noise models in [18], [21]. Our setting uses the same process, but two strategies are available:

* **Full distillation:** The p-value provided to the surrogate detector on the perturbed image $\tilde{\mathbf{x}}$ is the one computed by the private detector $f^*(\tilde{\mathbf{x}}, \mathbf{k})$.

* **Original p-value:** The p-value provided to the surrogate detector for $\tilde{\mathbf{x}}$ is the one computed by the private detector on the original image $f^*(\mathbf{x}, \mathbf{k})$.

Even though full distillation is the method that looks correct, using the original p-value instead allows to embed less information from the private detector within the surrogate while still providing correct detection scores – i.e perturbed watermarked images still receive low p-values from the surrogate. Note that in practice, this might lead to a surrogate detector which is more robust than the private detector.

d) **Keys augmentations:** The watermark signal embedded is engineered to minimize its perceptual impact, but it may be statistically detectable. In other words, watermarking is not steganography. This means that a surrogate detector trained on images watermarked with a given key might be able to detect watermarks generated with other keys. However, since the public detector does not output a message and does not

rely on a key at the detection side, correct content attribution necessitates the detector to be key specific. To ensure this, we augment the dataset by adding images watermarked using random keys.

### III. TRANSFERRING ATTACKS FROM THE SURROGATE TO THE PRIVATE DETECTOR

The surrogate detectors are meant to be shared to the public and as such are vulnerable to adversarial examples. This work considers a realistic use case where the public detector is available in a white-box setting. Consequently, powerful white-box attacks [23], [24] can be used to find adversarial examples fooling them. However, it is not guaranteed that images adversarial to a public detector also deceive the private detector. This section proposes a simple method to transfer adversarial perturbations computed on the public detector to the private detector.

The attacker first defines a high level of p-value $\pi \in [0, 1]$. His goal is to mount an attack $adv(\cdot, f) : \mathcal{X} \to \mathcal{X}$ which modifies a watermarked image $\mathbf{x}_w$ to an attacked image $adv(\mathbf{x}_w, f)$ so that the targeted detector $f$ now outputs a much larger p-value: $f(adv(\mathbf{x}_w, f)) > \pi$.

When the targeted detector is the public surrogate DNN $f_\theta$, $adv(\cdot, f_\theta)$ is nothing more than a white-box attack forging adversarial example for the following binary classifier: $\delta(\cdot, f) : \mathcal{X} \to \{0, 1\}$ with $\delta(\mathbf{x}, f) = 1$ if $f(\mathbf{x}) > \pi$, and 0 otherwise. The well-known white box attacks, PGD [25], Deepfool [26], CW [23], or ID [24] are good candidates.

When the targeted detector is the private algorithm $f^*$, the only option is a black-box attack. Since its access might be limited, we choose a black-box attack by transferability [] using the source model $f_\theta$ to delude the target $f^*$. We consider an attack which simply amplifies the adversarial perturbation:

$$adv(\mathbf{x}_w, f^*) = \mathbf{x}_w + \alpha \frac{adv(\mathbf{x}_w, f_\theta) - \mathbf{x}_w}{||adv(\mathbf{x}_w, f_\theta) - \mathbf{x}_w||}. \quad (3)$$

The attacker hopes that the gain $\alpha$ is large enough so that the adversarial perturbation transfers to the private detector. The next section empirically investigates the trade-off between the Attack Success Rate and the attack distortion by varying $\alpha$.

### IV. EXPERIMENTS AND RESULTS

#### A. Experimental setup

a) **Watermark private detector:** We provide results for two watermarking schemes: Trustmark [21] and Stable-Signature [20].

Trustmark is a traditional technique embedding a watermark into an original image. It is used in the context of Generative AI as a post-hoc operation: an image is first generated and then watermarked. We use the $Q$ model for Trustmark without error-correcting codes ($M = 100$).

Stable Signature is a technique dedicated to Stable Diffusion. For a given secret key $\mathbf{k}$, it fine-tunes this model so that it generates images natively watermarked. We use the original $other_dec_48b_whit$ model<fnref n="1" /> where $M = 48$.

<footnote n="1">¹Available at https://github.com/facebookresearch/stable_signature</footnote>
We chose them as representing the current state-of-the-art for post-hoc watermarking and watermarking of generative images respectively. Note that Stega-Stamp is another post-hoc watermark method considered state-of-the-art in the benchmark [19] but no official implementation is currently available<fnref n="2" />.

b) Dataset: We train the surrogate detectors on a subset of the MIRFlickR 1M dataset [27]. Specifically we use 10,000 images for the training set as well as for the validation and testing sets. All images are pre-processed by resizing to $D = 3 \times 224 \times 224$.

For the surrogate detector of Stable-Signature, we also need a set of images outputted by Stable-Diffusion VAE's decoder. Indeed, the detector might otherwise learn to discriminate between "real" images and "generated" images instead of learning the watermarking signal. We build this set by encoding then decoding each image of the MIRFlickR dataset using Stable-Diffusion's VAE. These images can thus be considered "generated" but do not contain any watermark.

c) Augmentations: When using image augmentations ("Aug." strategy in the figures), we select within four transformations: center crop keeping 10% or 50% and downsampling to 70% or 30% of the original size. When using key augmentations ("Mult. Keys" strategy in the figures), we sample a random message for each image in Trustmark. This is not possible for Stable Signature as it requires a fine-tuning of Stable Diffusion per key. We sample a key among a key space of size 10, i.e. within 10 fine-tuned versions of Stable Diffusion. Except when specified otherwise, we use the original p-value strategy outlined in Section II.

d) Training details: The surrogate detector's architecture is fixed as a ResNet-18 with ImageNet pretrained weights [28]. The head classifier weights are randomly initialized with Gaussian samples with $\mu = 0, \sigma = 1/512$ and biases set to 0. The detector is trained for 10 epochs with AdamW optimizer at a base learning-rate of $5e^{-4}$, divided by 2 on loss plateau. The batch size is set to 128. All tested training configurations reach a classification accuracy between watermarked and cover images close to 100% on the testing set for an appropriately chosen threshold on the p-values (i.e $\pi = 1e^{-6}$). The exception is for the full distillation strategy for which we did not achieve convergence with Trustmark and reach low accuracy for Stable Signature. This shows that a surrogate detector can be effectively trained with few images in a few minutes.

e) Attacks: We select four white-box attacks to be performed on the public detector: 1) PGD [25] and 2) Deep-Fool [26] as baseline attacks, 3) Carlini-Wagner (CW) [23] which specifically finds the minimum perturbation deceiving a detector and 4) Input Diversity (ID) [24] specifically designed for transferability. We transfer the attack from the public to the private detector using (3).

The attacks are performed on 512 watermarked images from the test set.

f) Image processing as baselines: Our attack scenario does not provide access of the private detector to the attacker. This prevents any white-box or black-box attacks directly targeting this detector, except by transferability (3). As a baseline, we evaluate the performances of common image editing: JPEG compression with quality factor $q \in \{1...80\}$, Total Variation denoising and box blurring with kernel size $k_s \in \{5...11\}$.

g) Metrics: We evaluate the performances of an attack on the private detector through two metrics: the distortion and the Attack Success Rate. The experimental protocol is the following. For each watermarked image $x_{w,i}$ of the test set, we increase the strength of the attack until the attacked image is no longer deemed watermarked by the private detector. For the transfer attack (3), it means that we gradually increase $\alpha$. At the turning point, we measure and record the distortion $d_i$. These numbers are then sorted: $d_{(1)} \le d_{(2)} \le \dots \le d_{(n)}$. The sequel shows the operating curve of the attack defined as the ASR as a function of the distortion:

$$ASR(\bar{d}) = \frac{1}{n} \sum_{k=1}^{n} [d_{(k)} \le \bar{d}]. \quad (4)$$

In this paper, the distortion is measured as the $l_2$ norm of the perturbation divided by $\sqrt{D}$, i.e. its empirical standard deviation.

Adapting the strength of the attack requires an access to the private detector which is excluded in our scenario. Nevertheless, we still compute it purely for evaluation purpose to showcase the inherent power of an attack.

## B. Results

a) Impact of training strategies: We report the results of attacks for each training strategy in Fig. 1 for TrustMark. The conclusions being similar for Stable Signature, we don't report them for the sake of space. Overall, for a fixed attack, training strategies tend to perform approximately the same for a given attack. Of note, the Augmentations | Multiple keys strategy provides the best or close to best defense against every attack for TrustMark as well as for the two most powerful attacks against Stable Signature (PGD and ID).

b) Impact of the attack choice: We report the results of every attack and image processing operations in Fig. 2 for TrustMark and Stable signature for the Multiple Keys strategies. Each attack's performance varies widely for a fixed strategy as well as their ranking. ID performs consistently the best for Stable Signature and CW the worst. Interestingly these results are reversed for TrustMark. This would tend to show the higher vulnerability of the latter as CW is designed to find the perturbation with minimal distortion that deceives a classifier and should thus be quite specific to the attacked classifier. On the other hand ID is designed to avoid overfitting a classifier which might explain its better performance on Stable Signature which is far more robust to other attacks.

Note that all white-box attacks were also performed directly on the private detectors. All of them are able to reach an

<footnote n="2">²The checkpoints have been removed by Berkeley University from the original GitHub: https://github.com/tancik/StegaStamp</footnote>
<figure><img src="image_3.png" /><figcaption>(a) PGD</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>(b) DeepFool</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(c) ID</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(d) CW</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(a) Stable Signature
No Aug. | Mult. Keys</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(c) Stable Signature
Aug. | Mult. Keys</figcaption></figure>

<caption>(d) TrustMark
Aug. | Mult. Keys</caption>

<caption>Fig. 2: Attack Success Rate as a function of distortion for different attacks.</caption>

ASR of 1.0 for a distortion below quantization levels – i.e $1/255\sqrt{D}$.

c) Transferability and watermarking method: From the attacks alone, Stable Signature seems significantly more robust to attacks performed on the surrogate detectors than TrustMark. This is even more apparent by observing the performance of image processing attacks. Indeed, every attack strongly outperforms JPEG compression – the strongest image processing attack – in the case of TrustMark. This is not the case for Stable Signature where even ID fails to consistently reach the performance of JPEG compression.

d) Soundness of the public detector: Finally, we study the distribution of p-values of surrogate detectors, compared to the distribution of the private detectors in Fig 3. Recall that a sound detector should output p-values which are uniformly distributed under $H_0$. First of all, note that neither Trustmark nor Stable Signature are perfectly sound, we thus cannot expect *apriori* any surrogate detector to be sound either. Now, if we observe the empirical false-alarm rate of the surrogate detectors, we can see that they are far from uniform despite good overall performance and low mean squared error on the training set. A possible explanation is that our loss might favor reducing the error on the watermarked images, since the value of their log p-values higher than those of cover images. Another possibility is that the detector learns to discriminate between watermarked and cover images and is thus unable to associate too low of a p-value to covers which, by definition, do not contain any watermarking signal.

V. CONCLUSION

This paper introduced the idea of preventing deep neural-network based watermark detectors to be easily defeated by constructing surrogate public detectors through distillation and

<figure><img src="image_9.png" /><figcaption>(a) Stable Signature</figcaption></figure>

<caption>(b) TrustMark</caption>

<caption>Fig. 3: Empirical $P_{FA}$ as a function of the theoretical $P_{FA}$ of private and surrogate detectors, computed on 10,000 images from the test set. A sound detector should match the black graph.</caption>

making the original detector private. We proposed baseline methods to transfer attacks on the public detector to the private detector and measured transferability accordingly. By showing different susceptibility to attack transfer depending on the private detector architecture and surrogate distillation methods, we open the question of finding better methods to transfer attacks and distillation methods leaking less information to the attacker.

<table><caption>TABLE I: Accuracy of the public detector on the test set for different training strategies at $\pi = 1e^{-6}$.</caption><thead><tr><th></th><th>No aug.<br>1 key</th><th>Aug.<br>1 key</th><th>No aug.<br>1 key</th><th>Aug.<br>Mult. keys</th><th>Full distil.<br>Mult. keys</th></tr></thead><tbody><tr><td>Stable Sig.</td><td>0.994</td><td>0.996</td><td>0.996</td><td>0.997</td><td>0.78</td></tr><tr><td>TrustMark</td><td>0.97</td><td>0.92</td><td>0.99</td><td>0.97</td><td>-</td></tr></tbody></table>
REFERENCES

[1] J. A. Goldstein, G. Sastry, M. Musser, R. DiResta, M. Gentzel, and K. Sedova, "Generative language models and automated influence operations: Emerging threats and potential mitigations," *arXiv preprint arXiv:2301.04246*, 2023.

[2] G. Spitale, N. Biller-Andorno, and F. Germani, "AI model GPT-3 (dis) informs us better than humans," *Science Advances*, vol. 9, no. 26, p. eadh1850, 2023.

[3] USA, "Ensuring safe, secure, and trustworthy ai." https://www.whitehouse.gov/wp-content/uploads/2023/07/Ensuring-Safe-Secure-and-Trustworthy-AI.pdf, July 2023. Accessed: [july 2023].

[4] "Chinese AI governance rules," 2023. Accessed on August 29, 2023.

[5] "European AI Act," 2023. Accessed on August 29, 2023.

[6] California State Leg., "Amendment to california assembly bill 3211: California digital content provenance standards." California State Legislature, August 2024. Amended in Senate.

[7] M. Christodorescu, R. Craven, S. Feizi, N. Gong, M. Hoffmann, S. Jha, Z. Jiang, M. S. Kamarposhti, J. Mitchell, J. Newman, et al., "Securing the future of GenAI: Policy and technology," *Cryptology ePrint Archive*, 2024.

[8] K. Knibbs, "Researchers tested AI watermarks—and broke all of them," *Wired*, October 2023. Accessed on May 2, 2024.

[9] H. Zhang, B. L. Edelman, D. Francati, D. Venturi, G. Ateniese, and B. Barak, "Watermarks in the sand: Impossibility of strong watermarking for generative models," in *ICLR 2024 Workshop on Secure and Trustworthy Large Language Models*.

[10] I. J. Cox and J.-P. M. Linnartz, "Public watermarks and resistance to tampering," in *International Conference on Image Processing (ICIP'97)*, pp. 26–29, Citeseer, 1997.

[11] J. P. M. Linnartz and M. Van Dijk, "Analysis of the sensitivity attack against electronic watermarks in images," in *Information Hiding: Second International Workshop, IH'98 Portland, Oregon, USA, April 14–17, 1998 Proceedings* 2, pp. 258–272, Springer, 1998.

[12] P. Comesana, L. Pérez-Freire, and F. Pérez-González, "Blind newton sensitivity attack," *IEE Proceedings-Information Security*, vol. 153, no. 3, pp. 115–125, 2006.

[13] T. Maho, T. Furon, and E. Le Merrer, "Surfree: a fast surrogate-free black-box attack," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 10430–10439, 2021.

[14] I. Venturini, "Oracle attacks and covert channels," in *Digital Watermarking* (M. Barni, I. Cox, T. Kalker, and H.-J. Kim, eds.), (Berlin, Heidelberg), pp. 171–185, Springer Berlin Heidelberg, 2005.

[15] B. Tondi, P. Comesaña-Alfaro, F. Pérez-González, and M. Barni, "On the effectiveness of meta-detection for countering oracle attacks in watermarking," in *2015 IEEE International Workshop on Information Forensics and Security (WIFS)*, pp. 1–6, 2015.

[16] M. El Choubassi and P. Moulin, "On reliability and security of randomized detectors against sensitivity analysis attacks," *IEEE Trans. on Information Forensics and Security*, vol. 4, p. 273–283, sep 2009.

[17] T. Maho, T. Furon, and E. Le Merrer, "Randomized Smoothing under Attack: How Good is it in Pratice?," in *ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal Processing*, (Singapore, Singapore), pp. 1–5, IEEE, May 2022.

[18] J. Zhu, "Hidden: hiding data with deep networks," *arXiv preprint arXiv:1807.09937*, 2018.

[19] M. Tancik, B. Mildenhall, and R. Ng, "Stegastamp: Invisible hyperlinks in physical photographs," in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 2117–2126, 2020.

[20] P. Fernandez, G. Couairon, H. Jégou, M. Douze, and T. Furon, "The stable signature: Rooting watermarks in latent diffusion models," in *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pp. 22466–22477, 2023.

[21] T. Bui, S. Agarwal, and J. Collomosse, "Trustmark: Universal watermarking for arbitrary resolution images," *arXiv preprint arXiv:2311.18297*, 2023.

[22] Y. Wen, J. Kirchenbauer, J. Geiping, and T. Goldstein, "Tree-rings watermarks: Invisible fingerprints for diffusion images," *Advances in Neural Information Processing Systems*, vol. 36, 2024.

[23] N. Carlini and D. Wagner, "Towards evaluating the robustness of neural networks," in *2017 IEEE Symposium on Security and Privacy (SP)*, pp. 39–57, IEEE, 2017.

[24] C. Xie, Z. Zhang, Y. Zhou, S. Bai, J. Wang, Z. Ren, and A. L. Yuille, "Improving transferability of adversarial examples with input diversity," in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 2730–2739, 2019.

[25] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, "Towards deep learning models resistant to adversarial attacks," *stat*, vol. 1050, no. 9, 2017.

[26] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, "Deepfool: a simple and accurate method to fool deep neural networks," in *Proceedings of the IEEE conference on computer vision and pattern recognition*, pp. 2574–2582, 2016.

[27] M. J. Huiskes and M. S. Lew, "The mir flickr retrieval evaluation," in *Proceedings of the 1st ACM international conference on Multimedia information retrieval*, pp. 39–43, 2008.

[28] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in *Proceedings of the IEEE conference on computer vision and pattern recognition*, pp. 770–778, 2016.