A Transfer Attack to Image Watermarks

## Abstract

Abstract Watermark has been widely deployed by industry to detect AI-generated images. The robustness of such watermark-based detector against evasion attacks in the white-box and black-box settings is well understood in the literature. However, the robustness in the no-box setting is much less understood. In this work, we propose a new transfer evasion attack to image watermark in the no-box setting. Our transfer attack adds a perturbation to a watermarked image to evade multiple surrogate watermarking models trained by the attacker itself, and the perturbed watermarked image also evades the target watermarking model. Our major contribution is to show that, both theoretically and empirically , watermark-based AI-generated image detector is not robust to evasion attacks even if the attacker does not have access to the watermarking model nor the detection API.

## 1 Introduction

Generative AI (GenAI) can synthesize extremely realistic-looking images,
posing growing challenges to information authenticity on the Internet.
Watermarking [bi2007robust, HiDDeN, 2019stegastamp, UDH, al2007combined, abdelnabi2021adversarial, pmlr-v202-kirchenbauer23a] was suggested as a key technology to distinguish AI-generated and non-AI-generated content in the Executive Order on AI security issued by the White House in October 2023. In watermark-based detection, a watermark is embedded into an AI-generated image before releasing it; and an image is detected as AI-generated if the same watermark can be decoded from it. Watermarking AI-generated images has been widely deployed in industry. For instance, Google’s SynthID watermarks images generated by Imagen [saharia2022photorealistic]; OpenAI embeds a watermark into images generated by DALL-E [ramesh2021zero]; and Stable Diffusion enables users to embed a watermark into the generated images [stable-diffusion].

An attacker can use *evasion attacks* [Evading-Watermark] to remove the watermark in a watermarked image to evade detection. Specifically, an evasion attack strategically adds a perturbation into a watermarked image such that the target watermark-based detector falsely detects the perturbed image as non-AI-generated. The literature has well understood the robustness of watermark-based detector against evasion attacks in the *white-box setting* (i.e., the attacker has access to the target watermarking model) and *black-box setting* (i.e., the attacker has access to the detection API) [Evading-Watermark]. Specifically, in the white-box setting, an attacker can find a small perturbation for a given watermarked image such that the perturbed image evades detection while maintaining the image’s visual quality; and in the black-box setting, an attacker can find such a perturbation once the attacker can query the detection API for enough times.

However, robustness of watermark-based detection in the *no-box setting* (i.e., the attacker does not even have access to the detection API) is much less understood. In such setting, an attacker may use *common post-processing* or
*transfer evasion attacks* (or *transfer attacks* for simplicity) [Evading-Watermark, an2024benchmarking]. Common post-processing refers to common image editing operations (e.g., JPEG compression), which can edit a watermarked image without any information about the watermark-based detector.
Transfer attacks perturb a watermarked image based on surrogate models. Depending on the surrogate models, transfer attacks can be *classifier-based* or *watermark-based*. Classifier-based transfer attacks [an2024benchmarking] (*concurrent* to our work) treat a watermark-based detector as a binary classifier and apply standard transfer adversarial examples to classifiers [liu2016delving, chen2023rethinking]. In particular, an attacker trains surrogate classifiers to distinguish watermarked and non-watermarked images. Given a watermarked image, the attacker uses conventional adversarial example technique to perturb it such that the surrogate classifiers predict the perturbed image as non-watermarked. Jiang et al. [Evading-Watermark] propose a watermark-based transfer attack that trains *one* *surrogate watermarking model* and uses the white-box attack [Evading-Watermark] to generate a perturbation for a given watermarked image based on it. A common limitation of these methods is that they achieve limited success rates at evading advanced watermarking methods, as confirmed in our experiments, which leads to a misleading conclusion in prior studies that watermarking is robust in the no-box setting.

Figure: Figure 1: Watermarked images generated by Stable Diffusion (first row) and their perturbed versions in our transfer attack that successfully evade detection (second row). The target watermarking model uses ResNet architecture. Our transfer attack uses 100 surrogate watermarking models, each of which uses CNN architecture.
Refer to caption: extracted/5851246/figures/example_images/wm_image_3.png

Our work:
In this work, we propose a new watermark-based transfer attack in the no-box setting to evade watermark-based detection of AI-generated images. Unlike classifier-based transfer attacks [an2024benchmarking], our attack directly leverages surrogate watermarking models and thus is better tailored to watermark-based detection. Unlike existing watermark-based transfer attack [Evading-Watermark], our attack leverages multiple surrogate watermarking models instead of one. Specifically, an attacker trains multiple surrogate watermarking models using a *surrogate dataset*. The surrogate watermarking models may have different neural network architectures and watermark lengths, compared to the target watermarking model; and the surrogate dataset may have a different distribution from the one used to train the target watermarking model.

A key challenge for our attack is how to aggregate the multiple surrogate watermarking models to find a small perturbation for a watermarked image such that the perturbed image evades the target watermarking model. For instance, we show that straightforward methods to aggregate surrogate watermarking models achieve suboptimal attack effectiveness. We propose a two-step approach to address the challenge. In the first step, the attacker picks a watermark (called *target watermark*) for each surrogate watermarking model to guide the perturbation search process. Specifically, the perturbation aims to make each surrogate watermarking model decode the target watermark from the perturbed image. For instance, one way is to flip each bit of the watermark decoded by a surrogate watermarking model for the watermarked image and treat the flipped watermark as the corresponding target watermark. Our intuition is that if the perturbation makes multiple surrogate watermarking models decode flipped watermarks from the perturbed image, the target watermarking model is also likely to decode a flipped watermark from the perturbed image, evading detection.

In the second step, given the target watermark for each surrogate watermarking model, we generate a perturbation by aggregating multiple surrogate watermarking models. For instance, one way is to leverage an existing white-box attack [Evading-Watermark] to find a perturbation based on each surrogate watermarking model and its target watermark; and then aggregate the multiple perturbations (e.g., take their mean) as the final perturbation. However, such strategy achieves limited success because the aggregation breaks the perturbation patterns, as shown in our experiments. To address the challenge, we find a perturbation by ensembling multiple surrogate watermarking models. Specifically, we formulate an optimization problem, whose objective function is to find a minimum perturbation and constraints are that each surrogate watermarking model decodes its target watermark from the perturbed image. However, the optimization problem is challenging to solve due to the hard constraints. We propose several strategies to reformulate the optimization problem and approximately solve it to find a perturbation.

We theoretically analyze the transferability of our transfer attack.
We first formally quantify the correlation between the target and surrogate watermarking models. Based on such quantified correlation, we derive the probability that the watermark decoded by the target watermarking model from the watermarked image is flipped after adding the perturbation found by our transfer attack to it. Moreover, based on this probability, we further derive both an upper bound and a lower bound for the probability that the watermark decoded by the target watermarking model from the perturbed image matches with the ground-truth watermark in the watermarked image. Such upper and lower bounds quantify the transferability of our attack.

We empirically evaluate our transfer attack on image datasets generated by Stable Diffusion and Midjourney. We consider multiple watermarking methods [HiDDeN, 2019stegastamp, fernandez2023stable, jiang2024certifiably]. Our results show that, with dozens of surrogate watermarking models, our attack successfully evades a watermark-based detector while maintaining image quality (Figure 1 shows several examples), even if the surrogate watermarking models use different algorithms, neural network architectures, and watermark lengths from the target one, and are trained using datasets with different distributions. Moreover, our attack substantially outperforms common post-processing,
existing transfer attacks [Evading-Watermark, an2024benchmarking], and the state-of-the-art purification method [nie2022DiffPure] for adversarial examples. Our results show that image watermarks are broken even in the no-box setting.

To summarize, our contributions are as follows:

- •
We propose a transfer attack based on multiple surrogate watermarking models to watermark-based AI-generated image detector.
- •
We theoretically analyze the effectiveness of our attack.
- •
We empirically evaluate our transfer attack and compare it with existing ones in different scenarios.

## 2 Related Work

### 2.1 Image Watermarks

Three components:
An image watermarking method consists of three components: a *watermark* that is a bitstring, an *encoder* that embeds a watermark into an image, and a *decoder* that decodes a watermark from an image. In *non-learning-based methods* [al2007combined, bi2007robust, affine-resistant, Log-Polar-Mapping, pramila2018increasing] that have been studied for decades, the encoder and decoder are handcrafted. These non-learning-based methods are even not robust to common post-processing such as JPEG compression or Gaussian noise [Evading-Watermark], which we confirm in Section 7.
Unlike these traditional, non-learning-based methods, *learning-based methods* [HiDDeN, UDH, 2019stegastamp] leverage deep learning. In these methods, both the encoder and decoder are neural networks and trained in an end-to-end manner using an image dataset. The encoder transforms a watermark and an image into feature vectors and combines them to generate a watermarked image; while the decoder outputs a watermark when taking an (unwatermarked or watermarked) image as input. When jointly training the encoder and decoder, the objectives are to minimize the visual difference between an image and its watermarked version while ensuring accurate decoding of the watermark.
Compared to non-learning-based ones, learning-based methods are more robust because they can leverage adversarial training [HiDDeN].
Therefore, we focus on learning-based methods.

Adversarial training:
Adversarial training [madry2018towards, goodfellow2014explaining] is a standard method to train robust classifiers and has been extended to train robust watermarking models [HiDDeN]. The key idea is to add a *post-processing layer* between the watermark encoder and decoder. The post-processing layer aims to mimic post-processing that a watermarked image may undergo in practice. Specifically, the post-processing layer post-processes a watermarked image before sending it to the decoder. After jointly training the encoder and decoder using adversarial training, the decoder can still decode the watermark in a watermarked image even if it undergoes some post-processing. Thus, we use adversarial training in our experiments to train encoders and decoders.

### 2.2 Evasion Attacks

White-box:
Jiang et al. [Evading-Watermark] proposed a white-box attack which assumes the attacker has access to the target watermark decoder. Given the target watermark decoder and a watermarked image, an attacker finds a small perturbation such that the watermark decoded from the perturbed image is close to a random watermark.
In other words, the perturbation removes the watermark from the watermarked image and thus the perturbed image evades watermark-based detection.
Specifically, the attacker finds the perturbation by solving an optimization problem via gradient descent.

Black-box:  In black-box setting, an attacker has access to the API of a target watermark-based detector. Given an image, the detection API returns a binary classification result, i.e., AI-generated (watermarked) or non-AI-generated (non-watermarked). A black-box attack [Evading-Watermark] leverages the detection API to perturb a watermarked image to remove its watermark. In particular, given a watermarked image, the attacker starts from an initial perturbed image that is detected as non-AI-generated by the API. The initial perturbed image may have a large perturbation compared to the watermarked image. Then, the attacker repeatedly queries the API and gradually moves the perturbed image towards the watermarked image to reduce the perturbation based on the query results.
When an attacker can query the API for enough times, the attacker can find a perturbed image with a small perturbation that evades detection of the API.

No-box:  In this setting, an attacker does not even have access to the detection API. In such setting, *common post-processing* or *transfer attacks* can be used to remove watermark from a watermarked image. Specifically, common post-processing refers to common image editing operations such as JPEG compression, Gaussian noise, and Gaussian blur. Watermarks embedded by non-learning-based methods can be removed by common post-processing, but learning-based methods have good robustness against common post-processing due to adversarial training [HiDDeN].

Existing transfer attacks leverage conventional classifier-based transfer adversarial examples [an2024benchmarking] or only one surrogate watermarking model [Evading-Watermark].
As shown in prior works [Evading-Watermark, an2024benchmarking] and also confirmed in our experiments, these transfer attacks achieve limited success at evading learning-based watermarking methods, even if we leverage state-of-the-art multiple surrogate classifiers based transfer adversarial examples [chen2023rethinking].
Note that compared to the existing watermark-based transfer attack [Evading-Watermark] that leverages only one surrogate watermarking model, our transfer attack leverages multiple surrogate watermarking models. One technical contribution of our work is to develop methods to aggregate multiple surrogate watermarking models when perturbing a watermarked image. Another technical contribution is that we theoretically analyze the effectiveness of watermark-based transfer attacks.

## 3 Problem Formulation

### 3.1 Watermark-Based Detection

A GenAI service provider trains a watermark encoder (called *target encoder*) and decoder (called *target decoder* and denoted as $T$), and uses them for AI-generated image detection. During generation, a watermark $w$ is embedded into each AI-generated image using the target encoder. During detection, a watermark is decoded from a given image $x$ by the target decoder $T$.
We denote by $T(x)$ the decoded watermark. We represent the *bitwise accuracy* between two watermarks $w_{1}$ and $w_{2}$ as $BA(w_{1},w_{2})$, which is the proportion of bits that are identical in $w_{1}$ and $w_{2}$. Watermark-based detection determines whether the image $x$ is AI-generated based on the bitwise accuracy $BA(T(x),w)$ between the decoded watermark $T(x)$ and the ground-truth watermark $w$. Specifically,
the image $x$ is detected as AI-generated if the bitwise accuracy $BA(T(x),w)$ either exceeds a certain threshold $\tau$ or falls below $1-\tau$, i.e., $BA(T(x),w)>\tau$ or $BA(T(x),w)<1-\tau$.
$\tau$ is commonly set to a value such that the false detection rate, i.e., the probability of falsely detecting a non-AI-generated (i.e., non-watermarked) image as AI-generated (i.e., watermarked), does not exceed a desired small value $\eta$ [Evading-Watermark]. For instance, when the watermark has 30 bits and $\eta=10^{-4}$, $\tau\approx 0.83$.

The detector discussed above is known as *double-tail detector* [Evading-Watermark]. In a *single-tail detector*, the image $x$ is detected as AI-generated if the bitwise accuracy $BA(T(x),w)$ exceeds a threshold $\tau$, i.e., $BA(T(x),w)>\tau$. Double-tail detector is more robust than single-tail detector, e.g., if a perturbed watermarked image evades double-tail detector, then it also evades single-tail detector, but not vice versa. Therefore, we focus on double-tail detector in this work.

### 3.2 Threat Model

Attacker’s goal:
Suppose an attacker uses a GenAI service to produce a watermarked image $x_{w}$. The attacker’s goal is to introduce a minimal perturbation $\delta$ to the watermarked image $x_{w}$, aiming to evade watermark-based detection while preserving image quality.
Consequently, the attacker can engage in illicit activities using this image, such as boosting disinformation and propaganda campaigns as well as claiming ownership of the image.

Attacker’s knowledge:
A GenAI service provider’s watermark-based detector includes a target encoder, a target decoder, a ground-truth watermark $w$, and a detection threshold $\tau$. We assume that the attacker has *no-box* access to the watermark-based detector. Specifically, the attacker does not have access to the target encoder, target decoder, ground-truth watermark $w$, and detection threshold $\tau$. Moreover, the attacker does not know the neural network architecture of the target encoder and decoder, the length of the ground-truth watermark $w$, nor the image dataset used to train the target encoder/decoder. Such threat model arises when the GenAI service sets its watermark-based detector private and restricts access of the detection API to trusted customers.

Attacker’s capability:
We assume an attacker can add a perturbation to an AI-generated, watermarked image. Furthermore, we assume that the attacker has sufficient computational resources to train multiple surrogate watermarking models, where each surrogate watermarking model includes a surrogate encoder and a surrogate decoder.

## 4 Our Transfer Attack

### 4.1 Overview

We propose a transfer attack to evade watermark-based AI-generated image detection. Our transfer attack trains multiple surrogate watermarking models, and then generates a perturbation for a given watermarked image based on them. We train a set of diverse surrogate watermarking models independently using a dataset that may have a different distribution from the one used by the target watermarking model. For a given watermarked image $x_{w}$, we generate a perturbation $\delta$ based on the surrogate decoders such that the watermark decoded by each surrogate decoder for the perturbed image $x_{w}+\delta$ substantially differs from the watermark decoded for the original watermarked image $x_{w}$. Our intuition is that if multiple surrogate decoders decode a substantially different watermark for the perturbed image $x_{w}+\delta$, it is likely that the target watermarking decoder would decode a substantially different watermark, evading detection. Note that the same surrogate decoders are used to generate perturbations for all watermarked images.

Next, we describe the details of training surrogate models and generating perturbations based on them.

### 4.2 Train Surrogate Watermarking Models

Transfer attacks require a set of surrogate watermarking models to produce perturbation for a watermarked image. To improve the transferability of the perturbation produced in our transfer attack, a crucial challenge is to encourage diversity among the surrogate watermarking models. The attacker first collects an image dataset (called *surrogate dataset*) to train the surrogate watermarking models. The surrogate dataset may have a different distribution from the dataset used to train the target watermarking model. To encourage diversity, we propose to adopt *bootstrapping* [efron1994introduction], which is a widely used technique to train diverse models in statistics. Specifically, the attacker resamples $m$ subsets from the surrogate dataset, each of which is used to train one of the $m$ surrogate watermarking models. Moreover, the attacker can use different neural network architectures and different watermark lengths for the $m$ surrogate watermarking models.

### 4.3 Formulate an Optimization Problem

To evade watermark-based detection, we generate a perturbation $\delta$ for a given watermarked image $x_{w}$ based on the $m$ surrogate decoders since the detection process only involves decoders. Specifically, our goal is to add a perturbation $\delta$ to the watermarked image $x_{w}$ such that, for each surrogate decoder, the decoded watermark from the perturbed image $x_{w}+\delta$ matches an attacker-chosen watermark, which we call *target watermark*. Our transfer attack faces two key challenges: 1) how to select a target watermark for a surrogate decoder, and 2) how to generate a perturbation $\delta$ based on the $m$ target watermarks and surrogate decoders. We discuss how to address the two challenges in the following.

Select a target watermark for a surrogate decoder:
We use $w_{i}^{t}$ to denote the target watermark for the $i$th surrogate decoder, where $i=1,2,\cdots,m$. We consider the following approaches to select $w_{i}^{t}$.

Random-Different (RD). This method involves randomly generating different target watermarks for the $m$ surrogate decoders. For the $i$th surrogate decoder, each bit of $w_{i}^{t}$ is sampled from $\{0,1\}$ uniformly at random. The intuition of this method is that, if the $m$ surrogate decoders decode random watermarks from the perturbed image, then the target decoder is also likely to decode a random watermark from the perturbed image. As a result, the bitwise accuracy between the watermark $T(x_{w}+\delta)$ decoded by the target decoder for the perturbed image and the ground-truth watermark $w$ is expected to approach 0.5, thereby evading detection.

Random-Same (RS). This method randomly generates one random target watermark $w^{t}$ for all $m$ surrogate decoders, i.e., $w_{i}^{t}=w^{t},\forall i=1,2,\cdots,m$. The intuition is that it is more likely to find a perturbation $\delta$ such that the $m$ surrogate decoders decode the same target watermark from the perturbed image, and thus it is more likely for the target decoder to decode $w^{t}$ from the perturbed image. Since $w^{t}$ is picked uniformly at random, the bitwise accuracy between $w^{t}$ and $w$ is expected to approach 0.5 and detection is evaded.

Inverse-Decode (ID).
However, generating random watermarks as target watermarks for surrogate decoders may have an issue. Specifically, for each surrogate decoder, approximately 50% of the bits require to be flipped while optimizing the perturbation, given that each bit of the target watermark is uniformly sampled from $\{0,1\}$. Among these bits to be flipped, some may be very robust to pixel changes and thus large perturbation is required to alter them.

To address this issue, we propose to use the inverse of the watermark decoded by each surrogate decoder as the corresponding target watermark. Specifically, we use $S_{i}$ to denote the $i$th surrogate decoder. For a watermarked image $x_{w}$, $S_{i}$ decodes a watermark from it, denoted as $S_{i}(x_{w})$.
We use the inverse of the decoded watermark, i.e., $1-S_{i}(x_{w})$, as the target watermark $w_{i}^{t}$ for the $i$th surrogate decoder $S_{i}$, where $1-S_{i}(x_{w})$ means flipping each bit of $S_{i}(x_{w})$. The intuition of this method is that the watermark decoded by each surrogate decoder from the perturbed image is reversed, and thus it is likely that the watermark decoded by the target decoder from the perturbed image is also reversed, i.e., $T(x_{w}+\delta)$ is close to $1-T(x_{w})$. As a result, the bitwise accuracy between $T(x_{w}+\delta)$ and $w$ is very small since $T(x_{w})$ is close to $w$. We note that such bitwise accuracy may be even smaller than $1-\tau$, which means that the double-tail detector can still detect the perturbed image as AI-generated, watermarked image. We address this challenge by early stopping the process of finding the perturbation $\delta$, which we discuss in Section 4.4.

With the inverse watermark as target watermark, it may result in those bits that are less robust to pixel changes being flipped first during perturbation optimization process. As a result, it can lead to a smaller perturbation compared to random target watermark when the same portion of bits are flipped, as shown in our experiments.

Aggregate $m$ surrogate decoders:
Given a target watermark $w_{i}^{t}$ for the $i$th surrogate decoder, the attacker then finds a perturbation $\delta$ for the watermarked image $x_{w}$ by aggregating the $m$ surrogate decoders. We consider the following two ways to aggregate the $m$ surrogate decoders.

Post-Aggregate (PA).
A straightforward way is to apply an existing white-box attack to find a perturbation $\delta_{i}$ based on the $i$th surrogate decoder and target watermark. For instance, given the watermarked image $x_{w}$, $i$th surrogate decoder $S_{i}$, and target watermark $w_{i}^{t}$, the attacker can use the state-of-the-art white-box attack in Jiang et al. [Evading-Watermark] to find the perturbation $\delta_{i}$. Formally, the attacker gets $m$ perturbations $\{\delta_{i}\}_{i=1}^{m}$. Then, the attacker can aggregate them as the final perturbation $\delta$, i.e., $\delta=aggr(\delta_{1},\cdots,\delta_{m})$, where $aggr$ is an aggregation method.
For instance, $aggr$ can be $Mean$ or $Median$, which takes the mean or median of the $m$ perturbations as the final perturbation $\delta$. However, aggregating the perturbations generated by the surrogate decoders may ruin the patterns in them which are effective to mislead the target decoder before aggregation. As a result, the obtained perturbation achieves limited transferability to the target decoder, as shown in our experiments.

Ensemble-Optimization (EO).
To address the challenge of Post-Aggregate, Ensemble-Optimization considers the $m$ surrogate decoders when optimizing the perturbation $\delta$.
Specifically, Ensemble-Optimization aims to find a minimum perturbation such that the watermark $S_{i}(x_{w}+\delta)$ decoded by each surrogate decoder for the perturbed image $x_{w}+\delta$ is the same as its corresponding target watermark $w_{i}^{t}$. Formally, we formulate an optimization problem as follows:

$$ $\displaystyle\min_{\delta}\ \|\delta\|_{\infty}$ (1) $\displaystyle s.t.\ S_{i}(x_{w}+\delta)=w_{i}^{t},\forall i=1,2,\cdots,m$ $$

Ensemble-Optimization finds a shared perturbation $\delta$ such that all the $m$ surrogate decoders output their target watermarks for the perturbed image $x_{w}+\delta$. The generalization ability of the shared perturbation for all surrogate decoders could enhance the transferability to the target decoder, as shown in our experiments. Note that we use $\ell_{\infty}$-norm to measure the magnitude of the perturbation. Our method is also applicable to other metrics such as $\ell_{2}$ or SSIM, as shown in our experiments.

### 4.4 Solve the Optimization Problem

Solving the optimization problem in Equation 1 gets the perturbation $\delta$ for the watermarked image $x_{w}$.
However, since the constraints of the optimization problem are extremely strict, it is difficult to solve the optimization problem. To address this challenge, we relax the constraints and reformulate the optimization problem as follows:

$$ $\displaystyle\min_{\delta}\ \|\delta\|_{\infty}$ (2) $\displaystyle s.t.\ l(S_{i}(x_{w}+\delta),w_{i}^{t})<\epsilon^{\prime},\forall i =1,2,\cdots,m,$ $$

where $l(\cdot,\cdot)$ denotes a metric to measure the distance between two watermarks. For instance, $l(\cdot,\cdot)$ is the mean square error in our experiments.

The reformulated optimization problem is still challenging to solve due to the high non-linearity of the relaxed constraints. To address this challenge, we further reformulate the optimization problem as follows:

$$ $\displaystyle\min_{\delta}\ \frac{1}{m}\sum_{i=1}^{m}l(S_{i}(x_{w}+\delta),w_{ i}^{t})$ (3) $\displaystyle s.t.\ \|\delta\|_{\infty}<r,$ $\displaystyle\quad\frac{1}{m}\sum_{i=1}^{m}BA(S_{i}(x_{w}+\delta),w_{i}^{t})>1 -\epsilon,$ $$

where $r$ is a perturbation budget and $\epsilon$ (called *sensitivity*) is used to ensure that the bitwise accuracy between the watermark decoded by each surrogate decoder for the perturbed image and the corresponding target watermark is high enough to produce an effective perturbation.

We use projected gradient descent (PGD) [madry2018towards] to solve the optimization problem in Equation 3. To better control the strength of our transfer attack to evade double-tail detector, we consider the second constraint in the optimization problem in Equation 3 as a condition for early stopping. Our detailed algorithm to find the perturbation $\delta$ is shown in Algorithm 1 in Appendix. It first initializes $\delta$ to zero and then compute the gradient of the objective function as defined in Equation 3 with respect to $\delta$. The gradient is then used to update $\delta$ according to gradient descend. If the $\ell_{\infty}$-norm of the updated $\delta$ exceeds $r$, it is then projected onto the $\ell_{\infty}$-norm ball with radius $r$. The algorithm terminates and outputs the final perturbation $\delta$ once the number of optimization iterations or the average bitwise accuracy exceeds a predefined threshold.

## 5 Theoretical Analysis

Given the watermarks decoded by the $m$ surrogate decoders for the perturbed image, we derive both an upper bound and a lower bound for the probability that the watermark decoded by the target decoder for the perturbed image matches with the ground-truth watermark. We derive both an upper bound and a lower bound of the probability because we consider double-tail detectors. Such upper bound and lower bound quantify the transferability of our transfer attack. Specifically, they enable us to compute the upper bound and lower bound of the expected bitwise accuracy between the watermark decoded by the target decoder for the perturbed image and the ground-truth watermark. If such upper bound and lower bound of the expected bitwise accuracy fall between $\tau$ and $1-\tau$, the perturbed image produced by our transfer attack is expected to evade the target watermark-based detector. All our proofs are shown in Appendix.

### 5.1 Notations

We adopt $T$ to denote the target decoder and $\left\{S_{i}\right\}_{i=1}^{m}$ to represent the $m$ surrogate decoders. $w$ denotes the ground-truth watermark and $x_{w}$ denotes a watermarked image embedded with $w$. $\delta$ denotes a perturbation that satisfies the constraints of the optimization problem in Equation 3.
We use $n_{s}$ and $n_{t}$ to denote the watermark length of surrogate decoders and target decoder respectively, while we define $n=\mathop{min}(n_{s},n_{t})$.
$T(\cdot)_{j}$ and ${S_{i}}(\cdot)_{j}$ denote the $j$th bit of the watermarks decoded by $T$ and $S_{i}$ for an image, respectively. Moreover, we consider inverse watermarks as the target watermarks in our transfer attack.

### 5.2 Formal Definitions

We formally define *unperturbed similarity*, *positive transferring similarity*, *negative transferring similarity*, *$q$-attacking strength*, *$\beta$-accurate watermarking*, *Bit-level dependency*, *Independency*, and *Conditional independency*. Details of them are shown in Section F in Appendix.

### 5.3 Deriving P ⁢ r ⁢ ( T ⁢ ( x w + δ ) j = 1 − T ⁢ ( x w ) j ) 𝑃 𝑟 𝑇 subscript subscript 𝑥 𝑤 𝛿 𝑗 1 𝑇 subscript subscript 𝑥 𝑤 𝑗 Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}) italic_P italic_r ( italic_T ( italic_x start_POSTSUBSCRIPT italic_w end_POSTSUBSCRIPT + italic_δ ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT = 1 - italic_T ( italic_x start_POSTSUBSCRIPT italic_w end_POSTSUBSCRIPT ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT )

We first derive the probability that the watermark decoded by the target decoder is flipped after adding the perturbation found by our transfer attack to the watermark image, conditioned on that the watermark decoded by a surrogate decoder is flipped or not after adding the perturbation. Formally, we have the following lemma:

###### Lemma 1 .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $T$ for $x_{w}+\delta$ is the inverse of the $j$th bit of the watermark decoded by $T$ for $x_{w}$ with probability $a_{ij}k_{ij}+(1-a^{\prime}_{ij})(1-k^{\prime}_{ij})$, given that the $j$th bits of the watermarks for the $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are inverse. Conversely, it occurs with probability $(1-b^{\prime}_{ij})k^{\prime}_{ij}+b_{ij}(1-k^{\prime}_{ij})$ given that the $j$th bits of the watermarks for $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are identical. Formally, for $j=1,2,\cdots,n$, we have:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j }=1-S_{i}(x_{w})_{j})$ (4) $\displaystyle=c_{ij},$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j }=S_{i}(x_{w})_{j})$ $\displaystyle=c^{\prime}_{ij},$ $$

where $c_{ij}=a_{ij}k_{ij}+(1-a^{\prime}_{ij})(1-k_{ij})$ and $c^{\prime}_{ij}=(1-b^{\prime}_{ij})k^{\prime}_{ij}+b_{ij}(1-k^{\prime}_{ij})$.

Then, we derive the unconditional probability that the watermark decoded by the target decoder is flipped after adding the perturbation to the watermarked image as follows:

###### Theorem 1 .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $T$ for the perturbed image is the inverse of the $j$th bit of the watermark decoded by $T$ for the watermarked image with probability $c_{ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij})$. Formally, for $j=1,2,\cdots,n$, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})=e_{j},$ (5) $$

where $e_{j}=c_{ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij}),\forall i$.

Then we derive the probability that the watermarks decoded by the surrogate decoders are flipped when the watermark decoded by the target decoder is flipped after adding the perturbation as follows:

###### Lemma 2 .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $S_{i}$ for the perturbed image is the inverse of the $j$th bit of the watermark decoded by $S_{i}$ for the watermarked image with probability $\frac{c_{ij}q_{ij}}{c_{ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij})}$, given that the watermarks for the perturbed image and the watermarked image decoded by $T$ are inverse. Formally, for $j=1,2,\cdots,n$, we have the following:

$$ $\displaystyle Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j}\mid T(x_{w}+\delta )_{j}=1-T(x_{w})_{j})$ (6) $\displaystyle=\frac{c_{ij}q_{ij}}{c_{ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij})}.$ $$

Finally, we derive the probability that the watermark decoded by the target decoder is flipped after adding the perturbation to the watermarked image, given the watermarks decoded by the $m$ surrogate decoders for the perturbed image. Formally, we have the following:

###### Theorem 2 .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $T$ for the perturbed image is the inverse of the $j$th bit of the watermark decoded by $T$ for the watermarked image with probability $p_{j}$, when the watermarks for the perturbed image decoded by the $m$ surrogate decoders are given. Formally, for $j=1,2,\cdots,n$, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{1}(x_{w}+\delta)_{j },\cdots,S_{m}(x_{w}+\delta)_{j})$ (7) $\displaystyle=p_{j},$ $$

where $p_{j}=\mathop{min}(e_{j}\prod_{i\in M_{j1}}\frac{c_{ij}}{c_{ij}q_{ij}+c^{
\prime}_{ij}(1-q_{ij})}\prod_{i\in M_{j2}}\frac{c^{\prime}_{ij}}{c_{ij}q_{ij}+
c^{\prime}_{ij}(1-q_{ij})},\\
1)$, $M_{j1}=\{i\mid S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j}\}$, and $M_{j2}=\{i\mid S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j}\}$. $M_{j1}$ (or $M_{j2}$) is the set of surrogate decoders whose $j$th bits of the decoded watermarks flip (or not flip) after adding the perturbation to the watermarked image.

###### Lemma 1 .

###### Theorem 1 .

###### Lemma 2 .

###### Theorem 2 .

### 5.4 Deriving P ⁢ r ⁢ ( T ⁢ ( x w + δ ) j = w j ) 𝑃 𝑟 𝑇 subscript subscript 𝑥 𝑤 𝛿 𝑗 subscript 𝑤 𝑗 Pr(T(x_{w}+\delta)_{j}=w_{j}) italic_P italic_r ( italic_T ( italic_x start_POSTSUBSCRIPT italic_w end_POSTSUBSCRIPT + italic_δ ) start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT = italic_w start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT )

The bitwise accuracy between the decoded watermark $T(x_{w}+\delta)$ and the ground-truth watermark $w$ is used to detect whether the perturbed image $x_{w}+\delta$ is AI-generated. Therefore, given the watermarks decoded by the $m$ surrogate decoders for the perturbed image $x_{w}+\delta$, we derive an upper bound and a lower bound of the probability that the watermark decoded by the target decoder matches with the ground-truth watermark after adding the perturbation. Note that we consider $1\leq j\leq n_{t}$ in this section. Formally, based on the theorems in Section 5.3, we derive the following theorem:

###### Theorem 3 .

Given any watermarked image $x_{w}$, $\delta$ satisfying the constraints of the optimization problem in Equation 3, and the watermarks decoded by the $m$ surrogate decoders for the perturbed image, we have an upper bound and a lower bound of the probability that the $j$th bit of the watermark decoded by $T$ for the perturbed image matches with the $j$th bit of the ground-truth watermark $w$ as follows:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots, S_{m}(x_{w}+\delta)_{j})$ (8) $\displaystyle\geq\left\{\begin{aligned} &\mathop{max}(\beta_{j}-p_{j},0),&1 \leq j\leq n,\\ &0,&n<j\leq n_{t}.\end{aligned}\right.$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots, S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\leq\left\{\begin{aligned} &1-|p_{j}+\beta_{j}-1|,&1\leq j\leq n, \\ &1,&n<j\leq n_{t}.\end{aligned}\right.$ $$

Our theoretical analysis demonstrates that the probability of the $j$th bit of the watermark decoded by $T$ for the perturbed image matching with the $j$th bit of the ground-truth watermark can be bounded, which can be used to compute the upper bound and lower bound of the expected bitwise accuracy between $T(x_{w}+\delta)$ and $w$. If such upper bound and lower bound fall between $\tau$ and $1-\tau$, the perturbed image is expected to evade the target watermark-based detector.

###### Theorem 3 .

## 6 Experiments

### 6.1 Experimental Setup

Datasets:  In our experiments, we utilize AI-generated images for watermarking model training and testing. For the target watermarking models, we use two public datasets [wang2022diffusiondb, midjourney-database] generated by Stable Diffusion and Midjourney respectively. Following HiDDeN [HiDDeN], we randomly select 10,000 images from each dataset to train the target watermarking encoders and decoders.
For testing, we randomly sample 1,000 images from the testing set of each dataset, embed the ground-truth watermark into each of them using a target encoder, and then find the perturbation to each watermarked image using different methods. To train the surrogate watermarking models, we sample 10,000 images from another public dataset [dalle2-database] generated by DALL-E 2, i.e., the surrogate dataset consists of these 10,000 images. The input image size of the watermarking models is $128\times 128$ pixels.

Surrogate watermarking models and watermark lengths:
Unless otherwise mentioned, we employ HiDDeN [HiDDeN] and the convolution neural network architecture described in HiDDeN with the watermark length of 30 bits to train surrogate watermarking models. We use the same architecture and watermark length for all surrogate watermarking models for two main purposes. Firstly, it can give us a clearer understanding of our attack’s transferability, especially when the target watermarking model and surrogate watermarking models have different architectures and watermark lengths. Secondly, it allows for a better analysis of the impact of the number of surrogate watermarking models on our attack’s transferability, without the confounding effects of variations in architecture and watermark length of the surrogate watermarking models.

Target watermarking models and watermark lengths:
For the target watermarking model, we use different choices of watermarking methods, architectures, and watermark lengths to analyze the transferability of our transfer attack in different scenarios. Specifically, we consider HiDDeN [HiDDeN], StegaStamp [2019stegastamp], Stable Signature [fernandez2023stable], Smoothed HiDDeN [jiang2024certifiably], and Smoothed StegaStamp [jiang2024certifiably] to train target watermarking models. For HiDDeN, we consider two architectures for the target watermarking model: CNN and ResNet. For the CNN architecture, the encoder consists of 4 convolutional blocks, while the decoder consists of 7 convolutional blocks. Each block integrates a Convolution layer, Batch Normalization, and ReLU activation. For the ResNet architecture,
the encoder consists of 7 convolutional blocks and the decoder is the ResNet-18.

For StegaStamp, we use model architecture introduced in the original paper [2019stegastamp]. For Stable Signature, we use the public watermarking model provided by Meta [fernandez2023stable] as the target watermarking model. Smoothed HiDDeN and Smoothed StegaStamp are certifiably robust against bounded adversarial perturbations. For them, we adopt the regression smoothing [jiang2024certifiably] to obtain the smoothed versions of the corresponding HiDDeN and StegaStamp watermarking models.

Additionally, we also employ different watermark lengths for the target watermarking models. Specifically, we evaluate our transfer attack on the target watermarking models with watermark lengths of 20 bits, 30 bits, and 64 bits. These variations include watermark lengths that are shorter than, equal to, and longer than those used by the surrogate watermarking models, offering a thorough analysis of our transfer attack’s performance across different watermark lengths in the target watermarking model.

Common post-processing methods:
Specifically, we consider the following common post-processing methods.

JPEG.
It is a commonly used compression method in digital imaging, which can reduce image file sizes while preserving a reasonable level of image quality. The quality of images processed through JPEG is governed by a quality factor $Q$. As the quality factor decreases, detecting the watermark in the post-processed images becomes more challenging, although this also results in a lower image quality.

Gaussian noise.
This method involves adding statistical noise that follows a Gaussian distribution with a zero mean and a standard deviation of $\sigma$, which effectively simulates various environmental noise effects encountered in real-world scenarios. A larger $\sigma$ value leads to increased difficulty in watermark detection, but also results in lower image quality.

Gaussian blur.
This method smooths the image by averaging pixel values with their neighbors. It applies a Gaussian filter with kernel size of $k\times k$ to an image, characterized by a bell-shaped curve with a zero mean and a standard deviation of $\sigma$. A larger $\sigma$ causes more pronounced blurring, which results in lower watermark detection rate and image quality. Following Jiang et al. [Evading-Watermark], we set $k=5$ and vary $\sigma$.

Brightness/Contrast.
This method modifies the brightness and contrast of an image by adjusting pixel values throughout the image. Specifically, it operates by either increasing or decreasing these values to make the image brighter or darker. The process is regulated by two factors: a brightness factor $B$ and a contrast factor $C$. Formally, for each pixel value $v$, the method transforms it to $Cv+B$. Following Jiang et al. [Evading-Watermark], we set $B=0.2$ and vary $C$.

Figure: (a)
Refer to caption: x1.png

Figure: (a)
Refer to caption: x7.png

Figure: (a)
Refer to caption: x13.png

Transfer attacks:
In our experiments, we compare with 5 existing transfer attacks and 1 state-of-the-art purification method for adversarial examples. They are WEvade-B-S [Evading-Watermark], AdvEmb-RN18 [an2024benchmarking], AdvCls-Real&WM [an2024benchmarking], AdvCls-Enc-WM1&WM2, MI-CWA [chen2023rethinking], and DiffPure [nie2022DiffPure]. For the former 5 transfer attacks, WEvade-B-S is watermark-based, while the other four are classifier-based. In particular, MI-CWA leverages the state-of-the-art multiple surrogate classifiers based transfer adversarial examples.

WEvade-B-S [Evading-Watermark].
This method trains one surrogate watermarking model.
A watermarked image is perturbed such that the watermark decoded by the surrogate decoder for the perturbed image matches with a preset random watermark. In our experiments, we train the surrogate watermarking model on 10,000 images generated by DALL-E 2. To give advantages to this transfer attack, we assume the attacker knows the architecture
of the target watermarking model and uses it for the surrogate watermarking model.

AdvEmb-RN18 [an2024benchmarking].
This method uses a ResNet-18 pretrained on ImageNet to generate a feature embedding for a watermarked image. Then it perturbs the image such that its embedding lies far from the one of the watermarked image.

AdvCls-Real&WM [an2024benchmarking].
This method trains a surrogate classifier using watermarked and non-watermarked images. The watermarked images are generated by the target GenAI and are watermarked by the target watermarking model, and the non-watermarked images are drawn from a distribution different from the one of the images generated by the target GenAI. A watermarked image is perturbed such that it is misclassified by the surrogate classifier as a non-watermarked image. In our experiments, we utilize images generated by DALL-E 2 as the non-watermarked images. More specifically, the training set for the surrogate classifier comprises 8,000 images generated by Stable Diffusion (or Midjourney) and watermarked by the target watermarking model, and 8,000 non-watermarked images generated by DALL-E 2. The surrogate classifier is based on the ResNet-18 architecture.

AdvCls-Enc-WM1&WM2.
This method is the same as AdvCls-Real&WM except for the training data used for the surrogate classifier.
It assumes that the attacker has access to the target watermarking encoder and can use it to embed any watermark into an image. The surrogate classifier is trained to distinguish the images embedded with the ground-truth watermark and those embedded with an attacker-chosen watermark.
In our experiments, the training set for the surrogate classifier comprises 8,000 images generated by the target GenAI and watermarked by the target watermarking model with one watermark, and 8,000 images generated by DALL-E 2 and watermarked by the target watermarking model with another watermark, where both watermarks are randomly picked.

MI-CWA [chen2023rethinking]. Existing classifier-based transfer attacks (i.e., the above three) only leverage one surrogate classifier. We extend state-of-the-art multiple surrogate classifiers based transfer adversarial examples [chen2023rethinking] to watermarks. Given the same training dataset as AdvCls-Real&WM, we train 100 surrogate classifiers, each of which has the ResNet-18 architecture. Given a watermarked image, this transfer attack perturbs it such that it is misclassified by the surrogate classifiers as non-watermarked.

DiffPure [nie2022DiffPure].
This method uses diffusion model to purify adversarial examples. We extend it to remove watermark. Specifically, DiffPure first adds Gaussian noise gradually to turn a watermarked image into a noised image. Then diffusion model is used to predict the noise step by step to get the denoised image. We use its public code.

Evaluation metrics:
We mainly use two metrics: *evasion rate* and *average perturbation*.
Evasion rate is the proportion of the perturbed images that evade the target watermark-based detector. Average perturbation is the perturbation added to a watermarked image measured by $\ell_{\infty}$-norm averaged across 1,000 watermarked images in the testing set.
Following Jiang et al. [Evading-Watermark], given that the pixel value range of [0, 255] is normalized to [-1, 1], we adjust the perturbation by dividing it by 2. This adjustment ensures that the perturbation is expressed as a fraction of the full pixel value range [0, 255].

Parameter settings:
We use adversarial training to train watermarking models since it enhances robustness. Specifically, for common post-processing methods, we consider the following range of parameters during adversarial training for a target watermarking model: $Q\in[10,99]$ for JPEG, $\sigma\in[0,0.1]$ for Gaussian noise, $\sigma\in[0,2.0]$ for Gaussian blur, and $C\in[1,3]$ for Brightness/Contrast. For surrogate watermarking models, we adopt a smaller range of parameters for some common post-processing methods during adversarial training to achieve weaker robustness than the target watermarking model
as follows: $Q\in[50,99]$ for JPEG, $\sigma\in[0,0.1]$ for Gaussian noise, $\sigma\in[0,1.0]$ for Gaussian blur, and $C\in[1,3]$ for Brightness/Contrast.

By default, we set maximum number of iterations $max_iter=5,000$, perturbation budget $r=0.25$, sensitivity $\epsilon=0.2$, and learning rate $\alpha=0.1$ for our transfer attack. Unless otherwise mentioned, we use Inverse-Decode to select a target watermark for a surrogate decoder, and Ensemble-Optimization to find the perturbation.
$\alpha$ is increased when the number of surrogate watermarking models increases in order to satisfy the constraints of our optimization problem within $5,000$ iterations.
The detailed settings for $\alpha$ for different number of surrogate models are shown in Table 1 in Appendix.
Moreover, we use $\ell_{2}$-distance as the distance metric $l(\cdot,\cdot)$ for two watermarks.

For the detection threshold $\tau$, we set it based on the watermark length of the target watermarking model. Specifically, we set $\tau$ to be a value such that the false positive rate of the watermark-based detector is no larger than $10^{-4}$ when the double-tail detector is employed. Specifically, $\tau$ is set to be 0.9, 0.83, and 0.73 for the target watermarking models with watermark lengths of 20 bits, 30 bits, and 64 bits, respectively.

### 6.2 Experimental Results

Our transfer attack is successful:  Figure 2 and Figure 3 respectively show the evasion rate and average perturbation of our transfer attack as the number of surrogate models increases when the target model uses different neural network architectures and watermark lengths on the two datasets.
First, our transfer attack is successful at evading watermark-based detection.
The evasion rate is higher than 85% and the average perturbation is no larger than 0.25 when 100 surrogate models are used, no matter what architectures and watermark lengths are used by the target model.

Second, when the target model’s architecture is more complex than the surrogate models’, our attack requires more surrogate models and larger average perturbation to succeed. For example, our attack uses 40 surrogate models and average perturbation 0.09 to achieve an evasion rate 100% when the target model is CNN and uses 30-bit watermarks on Stable Diffusion, while our attack uses 60 surrogate models and average perturbation 0.14 to achieve a similar evasion rate when the target model is ResNet and uses 30-bit watermarks. Third, when the target model uses a longer/shorter watermark than the surrogate models, our attack requires more surrogate models and larger average perturbation to succeed. For example, our attack uses 40 surrogate models and average perturbation 0.08 to achieve an evasion rate 85% when the target model is ResNet and uses 30-bit watermarks, while our attack uses 100 surrogate models and average perturbation 0.15 to achieve a similar evasion rate when the target model uses 64-bit watermarks.

Figure: Figure 5: Comparing evasion rates of existing and our transfer attacks. The target model is ResNet and uses watermarks with different lengths. Dataset is Stable Diffusion. Similar results for Midjourney are shown in Figure 14 in Appendix.
Refer to caption: x16.png

Our transfer attack outperforms common post-processing:  Figure 4 compares the average perturbations of common post-processing methods and our transfer attack when they achieve the same evasion rates for different target models on the two datasets. Given an evasion rate achieved by our transfer attack using a certain number of surrogate models, we set the parameter of a common post-processing method such that it achieves very close evasion rate. We observe that our transfer attack adds much smaller perturbations than common post-processing methods when achieving the same evasion rate.
Note that for some target models, Brightness/Contrast can only achieve about 75% and 50% evasion rate at most. Thus, the curves for Brightness/Contrast are shorter in the corresponding graphs.

To get a more comprehensive comparison with common post-processing methods, Figure 12 and Figure 13 in Appendix show the comparison results using average perturbation measured by $\ell_{2}$-norm and average SSIM, respectively. SSIM measures the similarity between a perturbed image and the corresponding watermarked image by considering changes in texture, luminance, and contrast. Average SSIM is the SSIM averaged across 1,000 watermarked images and their perturbed versions in the testing set.
We still observe that our attack adds much smaller perturbations and better maintains the images’ visual quality than common post-processing when $\ell_{2}$-norm or SSIM is used to measure the perturbation.

Figure: (a)
Refer to caption: x17.png

Our transfer attack outperforms existing ones:  Figure 5 compares the evasion rates of existing and our transfer attacks when the target model is ResNet and uses watermarks with different lengths. We note that since AdvCls-Real&Wm and AdvCls-Enc-WM1&WM2 can only achieve average perturbation around 0.1 no matter how large the perturbation budget $r$ we set, we add one extra variant of our method with perturbation budget $r=0.1$ for comparison. Additionally, for fair comparison, we constrain the perturbation for MI-CWA and DiffPure such that they achieve an average SSIM of about 0.9, which is comparable to our transfer attack with 100 surrogate models. For other transfer attacks, the perturbation budget $r$ is set to be 0.25. We observe that our transfer attack is much more effective than existing ones. Specifically, existing transfer attacks achieve almost 0 evasion rates except that AdvCls-Enc-WM1&WM2 achieves around 10% evasion rates on the Stable Diffusion dataset when the target model’s watermark length is 20 or 30 bits. Note that AdvCls-Enc-WM1&WM2 requires access to the target encoder, while other transfer attacks do not.
The very small evasion rates of state-of-the-art multiple surrogate classifiers based transfer adversarial example MI-CWA demonstrate the inherent limitation of applying classifier-based transfer attacks to watermarking models.
Additionally, DiffPure also achieves very small evasion rates. Since it is designed for adversarial examples’ purification, it changes the watermarked images a lot to remove the watermark and achieves limited evasion rates when the norm of the perturbation is constrained.
Our transfer attack achieves much higher evasion rates. We note that the average SSIM of our transfer attack is higher than 0.97 for $r=0.1$ and higher than 0.92 for $r=0.25$, which means that the visual quality of the perturbed images is maintained well.

Figure: (a)
Refer to caption: x19.png

Our transfer attack transfers to other target watermarking methods:
Figure 6 shows the evasion rate and average $\ell_{\infty}$-norm perturbation of our transfer attack to different target watermarking methods. To enhance the transferability of our attack, we utilize both HiDDeN and StegaStamp as surrogate watermarking models in equal numbers.
The results indicate that our transfer attack remains effective even when the surrogate and target watermarking methods differ, achieving an evasion rate exceeding 77% across all five target watermarking methods when using 100 surrogate models.
Note that Smoothed HiDDeN and Smoothed StegaStamp are certifiably robust. However, we observe that they are still vulnerable to our attack. Although the evasion rates are slightly lower for Smoothed HiDDeN with 20 surrogate models and for Smoothed StegaStamp with more than 60 surrogate models, our attack still achieves evasion rates exceeding 77% when the number of surrogate models is at most 100. This is because our added perturbations are larger than their certified bound and still maintain image quality.

Impact of sensitivity $\epsilon$:  Figure 7 shows the evasion rate and average $\ell_{\infty}$-norm perturbation when our transfer attack uses different sensitivity $\epsilon$ on Stable Diffusion dataset, where the target model is CNN and uses 30-bit watermarks. Figure 15 in Appendix shows the results on average $\ell_{2}$-norm perturbation and average SSIM. We observe that our attack requires less surrogate models to achieve the same evasion rate when $\epsilon$ is smaller. However, when $\epsilon$ is too small (e.g., 0.1), the evasion rate first increases and then decreases as the number of surrogate models increases. This is because the bitwise accuracy between the decoded watermark of the perturbed image and the ground-truth watermark becomes too small when $\epsilon$ is too small, and thus the perturbed image is still detected as AI-generated by the double-tail detector. Moreover, for different $\epsilon$, the average perturbation (measured by $\ell_{\infty}$-norm, $\ell_{2}$-norm, or SSIM) keeps almost the same when achieving the same evasion rate. This is because the average perturbation and the evasion rate have strong correlations in our attack.

Figure: (a)
Refer to caption: x21.png

Different variants of our transfer attack:  Our attack has two steps, each of which has multiple design choices. We use RD, RS, and ID to respectively denote Random-Different, Random-Same, and Inverse-Decode in the first step; and PA and EO to respectively denote Post-Aggregate and Ensemble-Optimization in the second step. We concatenate the symbols to represent a variant, e.g., RD-PA means using RD and PA as the two steps. Moreover, when PA is used, we further use suffix ”Mean” or ”Median” to indicate the adopted aggregation rule. For instance, RD-PA-Mean indicates that Mean is used as the aggregation rule.

Figure 8 shows the evasion rate and average $\ell_{\infty}$-norm perturbation when different variants of our transfer attack are used on Stable Diffusion dataset, where the target model is CNN and uses 30-bit watermarks. Figure 15 in Appendix shows the results on average $\ell_{2}$-norm perturbation and average SSIM. We observe that all the three variants using EO can evade detection successfully while none of the variants using PA can evade detection, which indicates that EO is much more effective at aggregating surrogate decoders than PA. Additionally, within the three variants using EO, RD and RS have similar results for all metrics and are less effective than ID. Specifically, ID-EO achieves 100% evasion rate with less number of surrogate models compared to RD-EO and RS-EO. For the same 100% evasion rate, ID-EO achieves a much lower average $\ell_{\infty}$-norm and $\ell_{2}$-norm perturbation and a much higher average SSIM than RD-EO and RS-EO.

Figure: (a)
Refer to caption: x23.png

Theoretical vs. empirical results:
Figure 9 shows the empirical average bitwise accuracy between the watermark decoded by the target decoder for a perturbed image and the ground-truth one, and our theoretical upper and lower bounds
on Stable Diffusion dataset.
The theoretical bounds are estimated as follows: we first measure the unperturbed similarity, transferring similarity, and attacking strength using the 1,000 testing images; then we compute the probability $p_{j}$ for each $j$th bit based on Theorem 2, following which we compute the upper and lower bounds of the probability for each bit of the watermark decoded by the target decoder to match with that of the ground-truth watermark; and finally we average these bounds across the bits.
We observe that our theoretical upper and lower bounds match well with the empirical bitwise accuracy. Specifically, the empirical results fall within our theoretical bounds except a few cases. Such exception cases happen because the theoretical bounds are estimated using only 1,000 images, resulting in inaccurate estimations.

## 7 Discussion and Limitations

Figure: (a)
Refer to caption: x25.png

We focus on transfer attacks to learning-based target watermarking models because learning-based models are more robust than non-learning-based ones. For instance, common post-processing like JPEG compression or Gaussian noise can already remove watermarks embedded by a non-learning-based model, but learning-based models have good robustness against common post-processing [Evading-Watermark]. For instance, Figure 10 shows the evasion rate of JPEG compression to DWT-DCT [al2007combined] which is the watermark used by Stable Diffusion [invisible-watermark] and Gaussian noise to Tree-Ring [wen2024tree]. Our results show that common post-processing methods can achieve evasion rates higher than 50% when the average $\ell_{\infty}$-norm perturbation is around $0.2$, while the evasion rates of learning-based watermarking methods are still lower than 5% with the same average $\ell_{\infty}$-norm perturbation.

However, our attack can also be applied to non-learning-based models. For instance, Figure 17 in Appendix shows the evasion rate of our transfer attack when the target model is DWT-DCT as the number of surrogate models increases on the two datasets, where the surrogate models are the same as those in Section 6. Our results show that our transfer attack can still achieve high evasion rates when the target model is non-learning-based. Compared to learning-based target model, our transfer attack requires more surrogate models to achieve high (e.g., 100%) evasion rates. This is because non-learning-based watermarking model is substantially different from learning-based, and our attack needs more surrogate models to increase diversity to enhance transferability.

## 8 Conclusion and Future Work

In this work, we find that watermark-based detection of AI-generated images is not robust to transfer attacks in the no-box setting. Given a watermarked image, an attacker can remove the watermark by adding a perturbation to it, where the perturbation can be found by ensembling multiple surrogate watermarking models. Our results show that transfer attack based on surrogate watermarking models outperforms those based on surrogate classifiers that treat a watermark-based detector as a conventional classifier. Moreover, leveraging surrogate watermarking models enables us to perform a rigorous analysis on the transferability of our attack. Interesting future work includes extending our transfer attack to text and audio watermarks, as well as designing more robust watermarks.

**Table 1: The detailed settings of $\alpha$ for different number of surrogate models.**
| # of Surrogate Models | 1 | 5 | 10 | 20 | 30 | 40 | 60 | 80 | 100 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| $\alpha$ | 0.1 | 1 | 2 | 4 |  |  |  |  |  |

Figure: (a)
Refer to caption: x26.png

Figure: (a)
Refer to caption: x29.png

Figure: (a)
Refer to caption: x35.png

Figure: Figure 14: Comparing the evasion rates of existing and our transfer attacks when the target model is ResNet and uses watermarks with different lengths. Dataset is Midjourney.
Refer to caption: x41.png

Figure: (a)
Refer to caption: x42.png

Figure: (a)
Refer to caption: x44.png

Figure: (a)
Refer to caption: x46.png

Figure: Algorithm 1 Find the Perturbation $\delta$

## Appendix A Proof of Lemma 1

Based on Definition 13,  14, and  15, we have the followings:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j }=1-S_{i}(x_{w})_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j},T(x_{w})_{j}=S_{i}(x_{w})_ {j}\mid$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad+Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j},T(x_{w})_{j}=1-S_{i}( x_{w})_{j}\mid$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid T(x_{w})_{j}=S_{i}(x_{ w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad+Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid T(x_{w})_{j}=1-S_ {i}(x_{w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=1-S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid T(x_{w})_{j}=S_{i}(x_{ w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad+(1-Pr(T(x_{w}+\delta)_{j}=T(x_{w})_{j}\mid T(x_{w})_{j}=1-S _{i}(x_{w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j}))$ $\displaystyle\quad\times(1-Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=1-S_{i}(x_{w})_{j}))$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=S_{i}(x_{w}+\delta)_{j}\mid T(x_{w})_{j}= S_{i}(x_{w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad+(1-Pr(T(x_{w}+\delta)_{j}=S_{i}(x_{w}+\delta)_{j}\mid$ $\displaystyle\quad T(x_{w})_{j}=1-S_{i}(x_{w})_{j},S_{i}(x_{w}+\delta)_{j}=1-S _{i}(x_{w})_{j}))$ $\displaystyle\quad\times(1-Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=1-S_{i}(x_{w})_{j}))$ $\displaystyle=a_{ij}k_{ij}+(1-a^{\prime}_{ij})(1-k_{ij})$ $\displaystyle=c_{ij}.$ $$

Similarly, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j }=S_{i}(x_{w})_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j},T(x_{w})_{j}=S_{i}(x_{w})_ {j}\mid$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j})+Pr(T(x_{w}+\delta )_{j}=1-T(x_{w})_{j},$ $\displaystyle\quad T(x_{w})_{j}=1-S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j} =S_{i}(x_{w})_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid T(x_{w})_{j}=S_{i}(x_{ w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle\quad+Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid T(x_{w})_{j}=1-S_ {i}(x_{w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=1-S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle=(1-Pr(T(x_{w}+\delta)_{j}=T(x_{w})_{j}\mid T(x_{w})_{j}=S_{i}(x_ {w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j}))$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle\quad+Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid T(x_{w})_{j}=1-S_ {i}(x_{w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle\quad\times(1-Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=S_{i}(x_{w})_{j}))$ $\displaystyle=(1-Pr(T(x_{w}+\delta)_{j}=S_{i}(x_{w}+\delta)_{j}\mid T(x_{w})_{ j}=S_{i}(x_{w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j}))$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle\quad+Pr(T(x_{w}+\delta)_{j}=S_{i}(x_{w}+\delta)_{j}\mid T(x_{w}) _{j}=1-S_{i}(x_{w})_{j},$ $\displaystyle\quad S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j})$ $\displaystyle\quad\times(1-Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)_{j}=S_{i}(x_{w})_{j}))$ $\displaystyle=(1-b^{\prime}_{ij})k^{\prime}_{ij}+b_{ij}(1-k^{\prime}_{ij})$ $\displaystyle=c^{\prime}_{ij}.$ $$

## Appendix B Proof of Theorem 1

Based on Lemma 1 and Definition 16, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j},S_{i}(x_{w}+\delta)=S_{i}( x_{w}))$ $\displaystyle\quad+Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j},S_{i}(x_{w}+\delta)=1 -S_{i}(x_{w}))$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+\delta)=S_ {i}(x_{w}))$ $\displaystyle\quad\times Pr(S_{i}(x_{w}+\delta)=S_{i}(x_{w}))$ $\displaystyle\quad+Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+ \delta)=1-S_{i}(x_{w}))$ $\displaystyle\quad\times Pr(S_{i}(x_{w}+\delta)=1-S_{i}(x_{w}))$ $\displaystyle=c^{\prime}_{ij}(1-q_{ij})+c_{ij}q_{ij}$ $\displaystyle=e_{j}.$ $$

## Appendix C Proof of Lemma 6

$$ $\displaystyle Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j}\mid T(x_{w}+\delta )_{j}=1-T(x_{w})_{j})$ $\displaystyle=\frac{Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j},T(x_{w}+ \delta)_{j}=1-T(x_{w})_{j})}{Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})}$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j }=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad\times\frac{Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})}{ Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})}.$ $$

Based on Lemma 1 and Theorem 1, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j }=1-S_{i}(x_{w})_{j})$ $\displaystyle\quad\times\frac{Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})}{ Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})}$ $\displaystyle=\frac{c_{ij}q_{ij}}{c_{ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij})}.$ $$

## Appendix D Proof of Theorem 2

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{1}(x_{w}+\delta)_{j },\cdots,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle=Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j}\mid$ $\displaystyle\quad\quad\quad\quad\quad T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle\quad\times\frac{Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{1} (x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j})}.$ $$

Based on Assumption 18 and 19, we have the following:

$$ $\displaystyle Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j}\mid T( x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle\quad\times\frac{Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{1} (x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j})}$ $\displaystyle=Pr(S_{1}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle\quad\cdots Pr(S_{m}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j}=1- T(x_{w})_{j})$ $\displaystyle\quad\times\frac{Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{1} (x_{w}+\delta)_{j})\cdots Pr(S_{m}(x_{w}+\delta)_{j})}.$ (9) $$

Given that $M_{j1}=\{i\mid S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j}\}$, we have the following according to Lemma 6:

$$ $\displaystyle Pr(S_{i}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle=Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})\mid T(x_{w}+\delta)_{j }=1-T(x_{w})_{j})$ $\displaystyle=\frac{c_{ij}q_{ij}}{c_{ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij})}, \forall i\in M_{j1}.$ (10) $$

Given that $M_{j2}=\{i\mid S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j}\}$, we have:

$$ $\displaystyle Pr(S_{i}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle=1-Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})\mid$ $\displaystyle\quad\quad\quad\quad\quad T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle=1-\frac{c_{ij}q_{ij}}{c_{ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij})}$ $\displaystyle=\frac{c^{\prime}_{ij}(1-q_{ij})}{c_{ij}q_{ij}+c^{\prime}_{ij}(1- q_{ij})},\forall i\in M_{j2}.$ (11) $$

Then Equation 9 can be reformulated as follows:

$$ $\displaystyle Pr(S_{1}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle\quad\cdots Pr(S_{m}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j}=1- T(x_{w})_{j})$ $\displaystyle\quad\times\frac{Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{1} (x_{w}+\delta)_{j})\cdots Pr(S_{m}(x_{w}+\delta)_{j})}$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle\quad\prod_{i\in M_{j1}}\frac{Pr(S_{i}(x_{w}+\delta)_{j}\mid T(x_ {w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{i}(x_{w}+\delta)_{j})}$ $\displaystyle\quad\prod_{i\in M_{j2}}\frac{Pr(S_{i}(x_{w}+\delta)_{j}\mid T(x_ {w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{i}(x_{w}+\delta)_{j})}.$ $$

Then, based on Definition 16, Theorem 1, Equation 10, and Equation 11, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j})$ $\displaystyle\quad\prod_{i\in M_{j1}}\frac{Pr(S_{i}(x_{w}+\delta)_{j}\mid T(x_ {w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{i}(x_{w}+\delta)_{j})}$ $\displaystyle\quad\prod_{i\in M_{j2}}\frac{Pr(S_{i}(x_{w}+\delta)_{j}\mid T(x_ {w}+\delta)_{j}=1-T(x_{w})_{j})}{Pr(S_{i}(x_{w}+\delta)_{j})}$ $\displaystyle=e_{j}\prod_{i\in M_{j1}}\frac{c_{ij}q_{ij}}{c_{ij}q^{2}_{ij}+c^{ \prime}_{ij}(1-q_{ij})q_{ij}}$ $\displaystyle\quad\quad\prod_{i\in M_{j2}}\frac{c^{\prime}_{ij}(1-q_{ij})}{c_{ ij}q_{ij}(1-q_{ij})+c^{\prime}_{ij}(1-q_{ij})^{2}}$ $\displaystyle=\mathop{min}(e_{j}\prod_{i\in M_{j1}}\frac{c_{ij}}{c_{ij}q_{ij}+ c^{\prime}_{ij}(1-q_{ij})}$ $\displaystyle\quad\quad\quad\quad\prod_{i\in M_{j2}}\frac{c^{\prime}_{ij}}{c_{ ij}q_{ij}+c^{\prime}_{ij}(1-q_{ij})},1)$ $\displaystyle=p_{j}.$ $$

## Appendix E Proof of Theorem 8

When $1\leq j\leq n$, the conditional expectation of $|T(x_{w}+\delta)_{j}-T(x_{w})_{j}|$ can be represented as:

$$ $\displaystyle E(|T(x_{w}+\delta)_{j}-T(x_{w})_{j}|\mid S_{1}(x_{w}+\delta)_{j} ,\cdots,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle=0\times Pr(T(x_{w}+\delta)_{j}=T(x_{w})_{j}\mid$ $\displaystyle\quad\quad\quad\quad\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_ {w}+\delta)_{j})$ $\displaystyle\quad+1\times Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid$ $\displaystyle\quad\quad\quad\quad\quad\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{ m}(x_{w}+\delta)_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid$ $\displaystyle\quad\quad\quad\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+ \delta)_{j}).$ $$

According to Theorem 2, we have:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=1-T(x_{w})_{j}\mid S_{1}(x_{w}+\delta)_{j },\cdots,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle=p_{j}.$ $$

According to Assumption 19, when $1\leq j\leq n$, the conditional expectation of $|T(x_{w})_{j}-w_{j}|$ can be represented as:

$$ $\displaystyle E(|T(x_{w})_{j}-w_{j}|\mid S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}( x_{w}+\delta)_{j})$ $\displaystyle=0\times Pr(T(x_{w})_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots ,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\quad+1\times Pr(T(x_{w})_{j}=1-w_{j}\mid$ $\displaystyle\quad\quad\quad\quad\quad\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{ m}(x_{w}+\delta)_{j})$ $\displaystyle=Pr(T(x_{w})_{j}=1-w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots,S_{m} (x_{w}+\delta)_{j})$ $\displaystyle=\frac{Pr(T(x_{w})_{j}=1-w_{j},S_{1}(x_{w}+\delta)_{j},\cdots,S_{ m}(x_{w}+\delta)_{j})}{Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{ j})}$ $\displaystyle=\frac{Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j} \mid T(x_{w})_{j}=1-w_{j})}{Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+ \delta)_{j})}$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=1-w_{j})$ $\displaystyle=\frac{Pr(S_{1}(x_{w}+\delta)_{j}\mid T(x_{w})_{j}=1-w_{j})}{Pr(S _{1}(x_{w}+\delta)_{j})}$ $\displaystyle\quad\cdots\frac{Pr(S_{m}(x_{w}+\delta)_{j}\mid T(x_{w})_{j}=1-w_ {j})}{Pr(S_{m}(x_{w}+\delta)_{j})}$ $\displaystyle\quad\times Pr(T(x_{w})_{j}=1-w_{j}).$ $$

Since the flipping behavior of surrogate models is irrelevant to Definition 5, then we have the following:

$$ $\displaystyle E(|T(x_{w})_{j}-w_{j}|\mid S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}( x_{w}+\delta)_{j})$ $\displaystyle=Pr(T(x_{w})_{j}=1-w_{j}).$ $$

Based on Definition 5, we have:

$$ $\displaystyle E(|T(x_{w})_{j}-w_{j}|\mid S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}( x_{w}+\delta)_{j})$ $\displaystyle=1-\beta_{j}.$ $$

Furthermore, the conditional expectation of $|T(x_{w}+\delta)_{j}-w_{j}|$ can be represented as:

$$ $\displaystyle E(|T(x_{w}+\delta)_{j}-w_{j}|\mid S_{1}(x_{w}+\delta)_{j},\cdots ,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle=0\times Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j} ,\cdots,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\quad+1\times Pr(T(x_{w}+\delta)_{j}=1-w_{j}\mid$ $\displaystyle\quad\quad\quad\quad\quad\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{ m}(x_{w}+\delta)_{j})$ $\displaystyle=Pr(T(x_{w}+\delta)_{j}=1-w_{j}\mid S_{1}(x_{w}+\delta)_{j}, \cdots,S_{m}(x_{w}+\delta)_{j}).$ (12) $$

Based on the triangle inequality, we have the following:

$$ $\displaystyle E(|T(x_{w}+\delta)_{j}-w_{j}|\mid S_{1}(x_{w}+\delta)_{j},\cdots ,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\leq\mathop{min}(E(|T(x_{w}+\delta)_{j}-T(x_{w})_{j}|+|T(x_{w})_{ j}-w_{j}|\mid$ $\displaystyle\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j}),1)$ $\displaystyle=\mathop{min}(1-\beta_{j}+p_{j},1),$ $$

and the following:

$$ $\displaystyle E(|T(x_{w}+\delta)_{j}-w_{j}|\mid S_{1}(x_{w}+\delta)_{j},\cdots ,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\geq E(||T(x_{w}+\delta)_{j}-T(x_{w})_{j}|-|T(x_{w})_{j}-w_{j}||\mid$ $\displaystyle\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j})$ $\displaystyle=|p_{j}+\beta_{j}-1|.$ $$

Therefore, based on Equation 12, when $1\leq j\leq n$, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots, S_{m}(x_{w}+\delta)_{j})$ $\displaystyle=1-Pr(T(x_{w}+\delta)_{j}=1-w_{j}\mid$ $\displaystyle\quad\quad\quad\quad\quad\quad S_{1}(x_{w}+\delta)_{j},\cdots,S_{ m}(x_{w}+\delta)_{j})$ $\displaystyle\geq 1-\mathop{min}(1-\beta_{j}+p_{j},1)$ $\displaystyle=\mathop{max}(\beta_{j}-p_{j},0),$ $$

and the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots, S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\leq 1-|p_{j}+\beta_{j}-1|.$ $$

If $n_{t}>n$, when $n<j\leq n_{t}$, we have the following:

$$ $\displaystyle 0\leq Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j}, \cdots,S_{m}(x_{w}+\delta)_{j})\leq 1.$ $$

Therefore, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots, S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\geq\left\{\begin{aligned} &\mathop{max}(\beta_{j}-p_{j},0),&1 \leq j\leq n,\\ &0,&n<j\leq n_{t}.\end{aligned}\right.$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=w_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots, S_{m}(x_{w}+\delta)_{j})$ $\displaystyle\leq\left\{\begin{aligned} &1-|p_{j}+\beta_{j}-1|,&1\leq j\leq n, \\ &1,&n<j\leq n_{t}.\end{aligned}\right.$ $$

## Appendix F Details of Formal Definitions

In this section, we will introduce the details of our formal definitions.
Specifically, unperturbed similarity measures the probability that the watermarks decoded by the target decoder and surrogate decoders are identical for a watermarked image $x_{w}$ without perturbation, which is formally defined as follows:

###### Definition 1 (Unperturbed similarity) .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $T$ matches with the $j$th bit of the watermark decoded by $S_{i}$ for $x_{w}$ with probability $k_{ij}$, given that the $j$th bits of the watermarks for $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are inverse. Conversely, it occurs with probability $k^{\prime}_{ij}$ given that the $j$th bits of the watermarks for $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are identical. Formally, for $i=1,2,\cdots,m$ and $j=1,2,\cdots,n$, we have:

$$ $\displaystyle Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j}=1-S _{i}(x_{w})_{j})=k_{ij},$ (13) $\displaystyle Pr(T(x_{w})_{j}=S_{i}(x_{w})_{j}\mid S_{i}(x_{w}+\delta)_{j}=S_{ i}(x_{w})_{j})=k^{\prime}_{ij}.$ $$

Note that both $k_{ij}$ and $k^{\prime}_{ij}$ can be estimated in experiments. Positive transferring similarity measures the probability that the watermarks decoded by target decoder and surrogate decoders for a perturbed image $x_{w}+\delta$ are identical when the watermarks decoded by the surrogate decoders are flipped after adding the perturbation, which is defined as follows:

###### Definition 2 (Positive transferring similarity) .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $T$ matches with the $j$th bit of the watermark decoded by $S_{i}$ for $x_{w}+\delta$ with probability $a_{ij}$, given that the $j$th bits of the watermarks decoded by $T$ and $S_{i}$ are identical for $x_{w}$, and the $j$th bits of the watermarks for $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are inverse. Conversely, it occurs with probability $a^{\prime}_{ij}$ given that the $j$th bits of the watermarks decoded by $T$ and $S_{i}$ for $x_{w}$ are inverse, and the $j$th bits of the watermarks for $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are also inverse. Formally, for $i=1,2,\cdots,m$ and $j=1,2,\cdots,n$, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=$ $\displaystyle S_{i}(x_{w}+\delta)_{j}\mid T(x_{w})_{j}=S_{i}(x_{w})_{j},$ (14) $\displaystyle S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})=a_{ij},$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=$ $\displaystyle S_{i}(x_{w}+\delta)_{j}\mid T(x_{w})_{j}=1-S_{i}(x_{w})_{j},$ $\displaystyle S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})=a^{\prime}_{ij}.$ $$

Both $a_{ij}$ and $a^{\prime}_{ij}$ can be estimated in experiments. Similarly, we use negative transferring similarity to measure the probability that the watermarks decoded by target decoder and surrogate decoders for a perturbed image $x_{w}+\delta$ are identical when the outputs of surrogate decoders remain unchanged after adding the perturbation, which is defined as follows:

###### Definition 3 (Negative transferring similarity) .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $T$ matches with the $j$th bit of the watermark decoded by $S_{i}$ for $x_{w}+\delta$ with probability $b_{ij}$, given that the $j$th bits of the watermarks decoded by $T$ and $S_{i}$ are inverse for $x_{w}$, and the $j$th bits of the watermarks for $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are identical. Conversely, it occurs with probability $b^{\prime}_{ij}$ given that the $j$th bits of the watermarks decoded by $T$ and $S_{i}$ for $x_{w}$ are identical, and the $j$th bits of the watermarks for $x_{w}+\delta$ and $x_{w}$ decoded by $S_{i}$ are also identical. Formally, for $i=1,2,\cdots,m$ and $j=1,2,\cdots,n$, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=S_{i}(x_{w}$ $\displaystyle+\delta)_{j}\mid T(x_{w})_{j}=1-S_{i}(x_{w})_{j},$ (15) $\displaystyle S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j})=b_{ij},$ $\displaystyle Pr(T(x_{w}+\delta)_{j}=S_{i}(x_{w}$ $\displaystyle+\delta)_{j}\mid T(x_{w})_{j}=S_{i}(x_{w})_{j},$ $\displaystyle S_{i}(x_{w}+\delta)_{j}=S_{i}(x_{w})_{j})=b^{\prime}_{ij}.$ $$

Both $b_{ij}$ and $b^{\prime}_{ij}$ can be estimated in experiments. To quantify the magnitude of our transfer attack, we formally define the $q$-attacking strength which measures the probability that the watermarks for the perturbed image and the watermarked image decoded by surrogate decoders are inverse as follows:

###### Definition 4 ( $q$ -attacking strength) .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bit of the watermark decoded by $S_{i}$ for the perturbed image is the inverse of the $j$th bit of the watermark decoded by $S_{i}$ for the watermarked image with probability $q_{ij}$. Formally, for $i=1,2,\cdots,m$ and $j=1,2,\cdots,n_{s}$, we have the following:

$$ $\displaystyle Pr(S_{i}(x_{w}+\delta)_{j}=1-S_{i}(x_{w})_{j})=q_{ij}.$ (16) $$

$q_{ij}$ can also be estimated in experiments. To quantify the performance of the target watermarking model when no perturbations are added to watermarked images, we introduce the following definition on $\beta$-accurate watermarking: assume the bitwise accuracy between the watermark decoded by the target decoder for the watermarked image and the ground-truth watermark following Poisson binomial distribution. Specifically, we have the following assumption:

###### Definition 5 ( $\beta$ -accurate watermarking) .

For any watermarked image $x_{w}$, the bits of the watermark decoded by $T$ are mutually independent and the probability that the $j$th bit of the decoded watermark matches with that of the ground-truth watermark $w$ is $\beta_{j}$, where $1\leq j\leq n_{t}$.

$\beta_{1},\beta_{2},\cdots,\beta_{n_{t}}$ can be estimated in experiments. Next, we introduce three assumptions regarding the surrogate decoders and the target decoder.
For bit-level dependency, it is assumed that each bit of the watermark decoded by the target decoder only depends on the corresponding bits of the watermarks decoded by the surrogate decoders. Formally, we have the assumption as follows:

###### Assumption 1 (Bit-level dependency) .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, each bit of the watermark decoded by $T$ for the perturbed image only depends on the corresponding bits of the watermarks decoded by the surrogate decoders for the perturbed image. Formally, for $j=1,2,\cdots,n$, we have the following:

$$ $\displaystyle Pr(T(x_{w}+\delta)_{j}\mid S_{1}(x_{w}+\delta),\cdots,S_{m}(x_{w }+\delta))$ (17) $\displaystyle=Pr(T(x_{w}+\delta)_{j}\mid S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}( x_{w}+\delta)_{j}).$ $$

Additionally, since all the surrogate decoders are trained independently with different subsets of data, we further assume that the watermarks decoded by the surrogate decoders are independent and conditionally independent given the watermark decoded by the target decoder for the watermarked or perturbed image.

###### Assumption 2 (Independency) .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bits of the watermarks decoded by the $m$ surrogate decoders for the perturbed image are independent. Formally, for $j=1,2,\cdots,n_{s}$, we have the following:

$$ $\displaystyle Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j})$ (18) $\displaystyle=Pr(S_{1}(x_{w}+\delta)_{j})\cdots Pr(S_{m}(x_{w}+\delta)_{j}).$ $$

###### Assumption 3 (Conditional independency) .

For any watermarked image $x_{w}$ and $\delta$ satisfying the constraints of the optimization problem in Equation 3, the $j$th bits of the watermarks decoded by the $m$ surrogate decoders for $x_{w}+\delta$ are independent when the $j$th bit of the watermark decoded by the target decoder for $x_{w}$ or $x_{w}+\delta$ is given. Formally, for $j=1,2,\cdots,n$, we have:

$$ $\displaystyle Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j}\mid T( x_{w})_{j})$ (19) $\displaystyle=Pr(S_{1}(x_{w}+\delta)_{j}\mid T(x_{w})_{j})\cdots$ $\displaystyle\quad Pr(S_{m}(x_{w}+\delta)_{j}\mid T(x_{w})_{j}),\forall j,$ $\displaystyle Pr(S_{1}(x_{w}+\delta)_{j},\cdots,S_{m}(x_{w}+\delta)_{j}\mid T( x_{w}+\delta)_{j})$ $\displaystyle=Pr(S_{1}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j})\cdots$ $\displaystyle\quad Pr(S_{m}(x_{w}+\delta)_{j}\mid T(x_{w}+\delta)_{j}),\forall j.$ $$

###### Definition 1 (Unperturbed similarity) .

###### Definition 2 (Positive transferring similarity) .

###### Definition 3 (Negative transferring similarity) .

###### Definition 4 ( q 𝑞 q italic_q -attacking strength) .

###### Definition 5 ( β 𝛽 \beta italic_β -accurate watermarking) .

###### Assumption 1 (Bit-level dependency) .

###### Assumption 2 (Independency) .

###### Assumption 3 (Conditional independency) .

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: Please see Weaknesses for my primary concerns, which also include some of my suggestions/questions for the authors.
soundness: 3
strengths: The robustness of image watermarking in no-box setting (i.e. with no access to watermarking models & with no access to detection API) is indeed a relatively overlooked area. Given that there are empirical ways to at least restrict the access to models/APIs, I consider this angle meaningful and worth investigating.

The presentation is clear enough overall and originality of this paper is not an issue.
summary: The paper proposes a transfer attack to (learned/learning-based) image watermarking that first trains a lot of surrogate watermarking models (encoders + decoders), then uses PGD to find a perturbation to the watermarked images that would change the extracted watermarks by the surrogate watermarking models.

It is argued that such approach requires smaller perturbations to break learning-based watermarking when being compared to common post-processing such as JPEG, Gaussian noise, Gaussian blur, and Brightness/Contrast; and it is more effective than existing transfer attacks.
weaknesses: My primary concerns regarding this paper are:
1. **No analysis of the computational cost in comparison with existing transfer attacks, which should be discussed clearly**. I think while the proposed method definitely incur a much larger overhead compared to previous attacks, it could still be ok to highlight the risks, providing that this limitation is presented upfront in the paper. Some analysis + some empirical numbers of computation time (with the corresponding compute resources) could be helpful for illustration.

2. **The comparison of the proposed attack with existing transfer attacks is somewhat inconclusive.**
This is because, as the authors must already know, there is a trade-off between evasion rate and the visibility/budget of the attacks (perturbation in L2/perturbation in Linf/SSIM/...). Thus a good comparison should at least compare the evasion rates with **all** baselines in representative budgets (if the other baselines work in such budgets). However, the comparison in the paper claims the proposed method is better than all others because it beats some baselines when r<= 0.1, and it beats MI-CWA and DiffPure assuming a SSIM>0.9. This is a rather misleading way of comparison, as some attacks will work better under certain (metrics of) budgets but not the others.

3. **The effectiveness of the proposed method as a "no-box" attack remain unclear, specifically regarding the transferability to different watermarking methods.**
While there are experiments involving a different target watermarking method from the surrogate ones, it is worth noting that these methods are somewhat similar (or as the authors called, they are learning-based watermarking). It is claimed that the proposed method also work for non-learning based methods (figure 13), but details regarding this set of experiments, especially what are the target watermarking methods, are missing. To my understanding, there is no rule preventing the victim using a watermarking scheme that is very different or even designed to be different from the ones used by the attacker as surrogates. This is not a deal breaker but such discussion seems to be missing.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 4
questions: Please see the weaknesses.
soundness: 3
strengths: 1. This paper is well-written with clear motivation. The no-box setting is practical.
2. The experiments and theoretical results support their claims and method design.
3. The transfer results are effective against several learning-based watermarking systems.
summary: This paper proposed an evasion attack to interfere with the detection of watermarked images under the no-box setting, i.e. the attacker has no access to the watermark detector. They approach the objective through the transfer attack, where they train surrogate watermark models and find adversarial perturbation to fool the detection of local surrogate models. Both theoretical analysis and empirical evidence demonstrate that this transfer attack method can successfully undermine the detection capabilities of learning-based watermark systems across various watermarking scenarios.
weaknesses: 1. While I acknowledge the authors have assumed that the attackers should have sufficient computational resources, the main concern of this paper is the computational cost. The costs associated with both surrogate models and perturbation discovery are large which prevents the practical usage of this method.  
2. For the transfer attack to a specific architecture such as ResNet-18, did you use a similar architecture in your surrogate models, for instance, ResNet-X? Besides, have you considered attacks with totally different architectures, such as transformer-based watermark models?
3. Another weakness of this paper is that the transfer attack primarily focuses on learning-based watermarks, neglecting non-learning-based watermark variants that may exhibit greater robustness than the baselines discussed here, such as variants of Tree-Ring [1][2], etc. Besides, can the proposed method attack the 0-bit watermark? 
The cost of attacks escalates significantly with complex architectures a
4. The attack cost is significantly increased with complex architectures and longer watermarks. This poses a hindrance to the practical implementation of transfer attacks involving high-capacity watermarks [3].

[1] RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identification

[2] Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models

[3] A Robust Image Watermarking System Based on Deep Neural Networks

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: See weakness
soundness: 3
strengths: This paper presents a practical attack effective across multiple watermarking methods
The issue of watermarking the outputs of generative models is timely and interesting.
summary: This paper demonstrates that watermark-based detection methods for AI-generated images are vulnerable to transfer attacks in a no-box setting. Specifically, an attacker can effectively remove the watermark from a given watermarked image by applying a perturbation, which can be identified through ensembling multiple surrogate watermarking models.
weaknesses: To ensure high image quality, the L infinity bound on perturbations should be kept low, as higher values (e.g., 0.2 as shown in Figures 3 and 4) can noticeably degrade image quality, reducing practicality. For a more realistic assessment, the paper should include comparisons using a set of small enough  L infinity bound (such as 0.03, 0.05) bounds to demonstrate the proposed method’s effectiveness under conditions that minimize visible impact.
If Section 5 is one of the paper’s main contributions, it would be better placed in the main text rather than in the appendix.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Please see the weakness section above.
soundness: 4
strengths: 1. The paper is well-written and easy to follow.
2. This paper proposed an intuitive yet attractive method that seems work very well in the experimental studies. 
3.  The experimental studies are comprehensive.
summary: This paper develops an attack that evades a target image watermarking detector, without acquiring the access to this detector's model or API. The proposed method adds a perturbation trained to mislead multiple surrogate detector models, and hopes that this perturbation can transfer to the target detector and therefore succeed. The experiment results on various datasets demonstrate the effectiveness of the proposed method, which outperforms many existing attacks and post-processing methods.
weaknesses: **Major Concerns:**

1. Theoretical contributions. While authors provide some theoretical analysis, I think the current results are not sufficient to be claimed as signficant. The reasons are as follows:
(1) It is in doubt whether those independence assumptions will hold. As $\delta$ is obtained by solving Eq. (3), which optimizes the chances that $m$ decoders will extract carefully designed watermarks from an image perturbed by $\delta$, it is likely that coordinates of decoders' output is correlated. The authors may want to provide additional envidence to validate their assumptions.
(2) The analysis only applies to inverse-decode. 
(3) The bound given by the theorems are not available in practice. Recall that this paper considers the setting where $T$ and its output is unknown to the attacker. In the meanwhile, the parameters including $a, b,\beta, p$ involves the unknown target decoder $T$ and unknown data distribution.

2. Empirical results. The empirical results are impressive, and it raises my question why it significantly outperforms SOTA methods? Could the author provide any justification and insight for the superior performance of the proposed method?


**Minor Concerns:**

1. Since the proposed method has to train a bunch of surrogate models, will it be computationally expensive? Also, every time this algorithm has to solve Eq. (3), which may also be expensive. The authors may want to add a table comparing the FLOPs and time used by different methods.

Meta-review: 2x accept, 2x borderline accept. This paper introduces a no-box scenario attack framework that trains multiple surrogate watermarking models and ensembles them to generate perturbations that remove or alter watermarks in AI-generated images without access to the target model. The reviewers agree on the (1) strong empirical performance on diverse watermarking methods, (2) comprehensive experiments and theoretical discussions, and (3) clear, well-structured presentation of the attack pipeline. However, they note (1) considerable computational overhead, (2) limited coverage of completely different watermark schemes (especially non-learning-based), and (3) partial reliance on theoretical assumptions requiring further clarifications. The authors have followed up with detailed responses and additional experiments addressing these concerns—showing acceptable runtime through parallelization, expanding coverage of non-learning-based methods, and refining theoretical justifications—so the AC leans to accept this submission.

Decision: Accept (Poster)