Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks
Authors: Mehrdad Saberi
Vinu Sankar Sadasivan
Keivan Rezaei
Aounon Kumar
Atoosa Chegini
Wenxiao Wang
Soheil Feizi
Venue: ICLR 2024 poster (2024)
Keywords: ai-image detection
image watermark
deepfake detection
watermark attack
generative models
Primary area: generative models

Abstract: In light of recent advancements in generative AI models, it has become essential to distinguish genuine content from AI-generated one to prevent the malicious usage of fake materials as authentic ones and vice versa. Various techniques have been introduced for identifying AI-generated images, with watermarking emerging as a promising approach. In this paper, we analyze the robustness of various AI-image detectors including watermarking and classifier-based deepfake detectors. For watermarking methods that introduce subtle image perturbations (i.e., low perturbation budget methods), we reveal a fundamental trade-off between the evasion error rate (i.e., the fraction of watermarked images detected as non-watermarked ones) and the spoofing error rate (i.e., the fraction of non-watermarked images detected as watermarked ones) upon an application of a diffusion purification attack. In this regime, we also empirically show that diffusion purification effectively removes watermarks with minimal changes to images. For high perturbation watermarking methods where notable changes are applied to images, the diffusion purification attack is not effective. In this case, we develop a model substitution adversarial attack that can successfully remove watermarks. Moreover, we show that watermarking methods are vulnerable to spoofing attacks where the attacker aims to have real images (potentially obscene) identified as watermarked ones, damaging the reputation of the developers. In particular, by just having black-box access to the watermarking method, we show that one can generate a watermarked noise image which can be added to the real images to have them falsely flagged as watermarked ones. Finally, we extend our theory to characterize a fundamental trade-off between the robustness and reliability of classifier-based deep fake detectors and demonstrate it through experiments. Code is available at https://github.com/mehrdadsaberi/watermark_robustness.

Published as a conference paper at ICLR 2024
ROBUSTNESS OF AI-IMAGE DETECTORS: FUNDAMENTAL LIMITS AND PRACTICAL ATTACKS
Mehrdad Saberi', Vinu Sankar Sadasivan', Keivan Rezaei', Aounon Kumar', Atoosa Chegini', Wenxiao Wang', Soheil Feizi'
Department of Computer Science. University of Maryland
(msaberi, , vi vinu, krezaei, aounon, atoocheg, wax, s fei j@und. edu
ABSTRACT
In light of recent advancements in generative Al models, it has become essential distinguish genuine content from Al-generated one to prevent the malicious usage of fake materials as authentic ones and vice versa. Various techniques have been introduced for identifying Al-generated images, with watermarking emerging promising approach. In this paper, we analyze the robustness of various Al-image detectors including watermarking and classifier-based deepfake detectors, For watermarking methods that introduce subtle image perturbations (i.e., low perturbation budget methods), we reveal a fundamental trade-off be- tween the evasion error rate (i.e., the fraction of watermarked images detected as non-watermarked ones) and the spoofing error rate (i.e., the fraction non- watermarked images detected as watermarked ones) upon application of dif- fusion purification attack. To validate our theoretical findings, we also provide empirical evidence demonstrating that diffusion purification effectively removes low perturbation budget watermarks by applying minimal changes to images. For high perturbation watermarking methods where notable changes images, the diffusion purification attack not effective. In this case, we develop model substitution adversarial attack that can successfully remove watermarks. Moreover, we show that watermarking methods are vulnerable to spoofing attacks where the attacker aims to have real images (potentially obscene) identified as watermarked ones, damaging the reputation of the developers. In particular, by just having black-box access the watermarking method, we show that one can generate watermarked noise image, which can be added to the real images, lead- ing to their incorrect classification as watermarked. Finally, we extend our theory characterize fundamental trade-off between the robustness and reliability of classifier-basee deep fake detectors and demonstrate it through experiments. Code is available at
applied
robustness) robuutrsterrrr..rsssesss.s
INTRODUCTION
As generative AI systems advance in sophistication and accessibility, the production of persuasive fabricated digital content becomes more accessible. These systems have the ability to craft hyper- realistic media forms such as images, videos, and audio (referred as deepfakes), capable of deceiv- ing viewers and listeners (Helmus, 2022). This misapplication of AI introduces potential hazards related to misinformation. fraud, and even national security issues like election manipulation (Blauth) et al. 2022] Chesney & Citron] 2019). Moreover, deepfakes can result in personal harm, spanning from character defamation to emotional distress, impacting both individuals and broader society [Ice] 2019). Consequently, the identification of Al-generated content and, importantly, tracing its sources, emerges as a crucial challenge to address.
Over the years, numerous techniques for recognizing Al-generated images have emerged. Among these, Image watermarking stands out as a promising approach (Honsinger, 2002 Swanson et al., 1998). Watermarking techniques, along with their many other applications (Potdar al. 2005 Zhao 2023c. Cui et al.) 2023), can be integrated with image generation models (Rombach et al. to inject watermarks to Al-generated images, which enables them be differentiated from
et al.
2022

---

Published as a conference paper at ICLR 2024
Low Perturbation Budget
F
2y
FF
(II) Denotsing Process
(1) Adding Noise
#: ~ NCV (1 Ge)l)
xt
Indirtnnguthaable
Zo
Riy
agg
R
non non--alermarrke watermarked
High Perturbation Budget
Training Classifter
(1I) Applying PGD
(III) Attack on Black-Box Detector
watermarked
Dedtector
non-walermarked
ap
-oo- non-watermarkeel
-watermarkee
Figure 1: Illustration of attacks against image watermarking methods. Upper panel demon- strates the diffusion purification attack for low perturbation budget (imperceptible) watermarks. adds Gaussian noise images, creating indistinguishable region, which results in certified lower bound on the error of watermark detectors. Noisy images are then denoised using diffusion models. See Section 3.1 for the definition of the used terms (e.g.. R, F). Lower panel depicts our model substitute adversarial attack against high-perturbatiin volves training substitute classifier, conducting PGD attack on the substitute model, and using these manipulated images to deceive the black-box watermark detector.
budget watermarks. Our attack in-
real images later. These techniques also allow for tracing the source of generation for images. Given the continuous enhancement in deepfake image quality and the growing challenge of distinguishing them from real ones, the adoption of image watermarking over classifier-based detection techniques is becoming more sensible choice.
In this paper, we demonstrate fundamental constraint the robustness of image watermarking methods. We leverage a technique called diffusion purification (Nie et al. (2022), originally proposed defense against adversarial examples. This approach involves the introduction of Gaussian noise to images and utilizing the denoising process of diffusion models (Ho et al., 2020) to eliminate the added noise. We offer both theoretical and empirical evidence that this attack amplifies the error rates of watermarking methods that have a low Wasserstein distance between the distributions of their watermarked and waneraaerrr images, which we refer to as "low perturbation budget" watermarking methods; i.c., watermarks with subtle image perturbations.
To elaborate, if and represent the distributions non-watermarked and watermarked images, and R' and J denote the distributions of these images after the application of the diffusion purifi- cation attack, we demonstrate that:
co(FtD))+ CR' D) - erf( Va, W(R. F
2 2(1 σc) )
where eo and é1 correspond to the evasion (type I) and spoofing (type II) errors of detector (i.e., formally defined in Definition 1. W(...) stands for the Wasserstein distance function, erf(.) is the Gauss error function, and 0e represents the cumulative alpha of the diffusion model at step complete theoretical findings, empirically show that diffusion purification attack reduce the AUROC (Area Under the Receiver Operating Characteristic) of some existing now-perturbation watermarks (Zhang et al.) 2019c Cox et al. 2007 Zhao et al. 2023b) to values less than 0.65 by applying minimal changes to images.
If the Wasserstein distance between the distributions of watermarked and non-wateen watermarked images large (i.e., high perturbation budget watermarking), our theoretical bound based on diffusion pu- rification attack becomes vacuous. In fact, also empirically observe that this attack does not
2

---

Published as a conference paper at ICLR 2024
compromise existing high perturbation budget watermarking methods where notable changes applied to the images such as TreeRing (Wen et al. 2023) (Figure 4.. In this regime, we develop method that trains substitute classifier capable of distinguishing between watermarked and non- watermarked images. Subsequently, we execute an adversarial attack (Madry et al. 2017) on images using this substitute classifier. Intriguingly, these attacks appear to transfer successfully to the thentic watermark detector. Our adversarial attack manages to decrease the AUROC of the TreeRing method to 0.14 by employing an fo attack with e 2/ 2/255. A distinguishing feature of our attack, in contrast to previously proposed white-box and black-box attacks (Jiang et al. (023), is that it does necessitate real-time access to the watermark detector. Instead, operates by collecting images watermarked by specific watermark model from the internet.
We note that some watermarking methods such as StegaStamp (Tancik et al. 2020g impose large perturbations in the latent (feature) space but relatively smaller perturbations in the image space (Table [] We show that both the diffusion purification attack (in the image space) as well as our model substitution adversarial attack are successful in breaking the StegaStamp watermark, espe- cially using larger diffusion steps or adversarial perturbation budgets.
In addition to the previously mentioned attacks, we introduce spoofing attack designed to target the spoofing error in watermarking methods. These attacks have the potential to erroneously categorize explicit inappropriate content watermarked, which could have adverse implications for the developers associated with a watermarked generative model, including loss of trust, financial loss, and negative publicity. Our attack functions by instructing watermarking models to watermark white noise image and then blending this noisy watermarked image with non non-watermarked ones to deceive the detector into flagging them watermarked.
Finally, we extend our theory originally established for watermarking methods, to offer a corre- sponding theoretical insight for classifier-based Al-image detectors. Our analysis demonstrates fundamental trade-off between the robustness and reliability of these detectors. As the distributions of real and fake images grow more alike, this trade-off becomes more pronounced. This implies that a detector could only achieve good performance or high robustness, but not both, simultaneously. We further present empirical evidence for this trade-off on some real-world detectors.
Summary of Contributions. In this paper, we make the following contributions:
1. We characterize a fundamental trade-off between evasion and spoofing error rates of image watermarking upon the application of a diffusion purification attack. Empirically, we show that diffusion purification attack can break whole range of watermarking methods that introduce subtle image perturbations (i.e., low perturbation budget image watermarking).
2. For high perturbation image watermarking that leaves notable changes on the original im- ages, we show that the diffusion purification attack is not effective. Instead, we develop model substitution adversarial attack that can successfully remove the watermarks.
3. We introduce spoofing attack against watermarking by adding watermarked noise image to clean images, in order to deceive the detector into flagging them as watermarked.
4. We develop fundamental trade-off between the robustness and reliability of deepfake detectors and substantiate this concept through experiments.
2
PRIOR WORK
Image Watermarking, Image watermarking is versatile technology with applications in copyright protection, content authenticity, data authentication, privacy preservation, and branding. Its evolu- tion began with manual methods like LSB (Wolfgang Delp 1996), and later techniques involved altering spatial or frequency domains (Ghazanfari et 2011 1, Holub & Fridrich, 2012, Pevný et al.)
2010 Boland et al. 1995 Cox et al. 1996 O O'Rannaidh Pun 1997). Various transformations such as DCT, DWT. SVD-decomposition (Chang et al. 2005). and Radon transformations (Seo) et al. 2004) were explored. Recent advancements incorporate deep learning and generative models like SteganoGAN (Zhang et al., 2019a). StegaStamp (Tancik et al. 2020) RivaGAN (Zhang al. 2019c), WatermarkDM (Zhao et al.) (0235), MBRS (Jia et al., 2021). and Tree Ring (Wen et al.
2023). each employing different methods embed watermarks into images.
3

---

Published as a conference paper at ICLR 2024
1.0
1.0
0.8
True Positive Rate
0.8
63
0.6
0.6
5-0.
WotermarKDM, a.0-0.05 MBRS, suc
0.4
W-10
0..4
R-20
RivaCan, auc-0.54
0.2
W-30
0.2
DwiDDeSvvd, auc-0.52
DwtDet, auc-0.56 No Artack
W-50
0.0
0.0
0.4
1.0
0.0
0.2
0.4
0.6
0.3
1.0
Diffusion Step (t)
False Positive Rate
Figure 2: Lower bound on the sum of eva- sion and spoofing errors of image watermarks against diffusion purification attack from The- orem D The beta schedule for the diffusion model is linear in the range [0.0008, 0.0120].
Figure 3: ROC curves for empirical robustness of image watermark methods against diffusion purification attack with t = 0.2. The dashed lines show the ROC curves of methods without attacking them.
There have been several works trying attack watermarking methods (Jiang et al. 2023 Wang et al. 2022a). Notably recent concurrent work (Zhao et al.] (023a) also proves that the diffusion purification attack successful against invisible (low perturbation budget) watermarking. However, (Zhao et al. (2023a) is unable to attack high perturbation budget watermarking methods such as Tree Ring or StegaStamp and argues that they are more reliable watermarking alternatives. In contrast, we show that our model substitution adversarial attack can effectively break those watermarking methods. Additionally, we show that several watermarking approaches are vulnerable to spoofing attacks and characterize robustness-rcliabiliy trade-off for classification- based deepfake detector.
Classifier-based Detectors. Several machine-learning approaches focusing on detecting artifacts in Al-generated content have been studied. For instance, Matern et al. (2019) target irregularities
in face editing algorithms, while Ciftci & Demir (2019) exploit biological signals. Li et al. (2020) introduce technique for identifying partially manipulated videos, and Guarnera al. (2020) har-
ness the traces from convolutional layers of generative adversarial networks in fake image detection.
Bonomi et al. [202T) analyze spatiotemporal texture dynamics of video signals for Deepfake detec- tion. A plethora of works focus on facial forgery or Deepfake detection using convolution net bes-based classifiers (Cozzolino al. [0117] Bayar & Stamm/ [016 Rahmouni al. 2017 Raja al. 2017 Zhou et al. 2017 Dogoulis et al. 2023). Rössler et al. (2019) proposed a face forensics dataset and train ResNet (He al.) (2155) 3 and XceptionNet (Chollet) (016)) based classifiers using However, as noted in Haliassos et al. (2021), machine learning-basee detectors are often vulnerable to novel in- put perturbations. Such limitations challenge the practical utility of these methods, as any real-world detector should achieve good performance while being robust to small perturbations in the input.
3 ROBUSTNESS OF IMAGE WATERMARKING FOR AI-IMAGE DETECTION
In this section, we first present our theoretical results on fundamental constraints for watermarking methods followed by our practical attacks. Proofs are presented Appendix B
3.1 FUNDAMENTAL CONSTRAINTS FOR WATERMARKING METHODS
Consider FF to represent the distribution of images that have been watermarked using particular key string k while R represents the distribution wateraarrdr images.
Definition 1 (Evasion and Spoofing Errors). Consider watermark detector D that predicts values of 0 and 1, for non non-watermarked and watermarked images, respectively. We define evasion error (co) and spoofing error (ex) of D distributions R and F as follows:
co PP = O] and e1 (R, PERR =1]
We measure distance between the distributions R and F using the Wasserstein metric defined as: W(RFF)=
inf (e [1|=] x2 []]
(2)
(EI(R.F)
4

---

Published as a conference paper at ICLR 2024
where FR. is the set of all joint probability distributions of R and F..
Definition 2. (Diffusion Purification) Diffiusion purification using a denoising diffusion probabilistic model consists of two steps. In the first step, image Zo is received and Da is calculated
It. NN ( (1-10-1)))) 1.)
where Oi is an increasing fuccttoingfuunctin of t that spans from 1
progresses from to Afterward,
Ie is denoised using denoising model to output an image r The denoising model is trained to minimize |1 zoll. We represent diffusion purification as DP)(.) where z6 putt (xo(
This technique was previously used in some other applications. For instance, in a prior study NNie al. 2022). was employed eliminate adversarial perturbations from images as defense strat- egy against adversarial attacks. In the following theorem, we claim that applying diffusion purifica- tion on images establishes lower bound on the sum of evasion and spoofing errors of watermark detectors. Luo| (2022) presents comprehensive details denoising diffusion models and their asso- ciated parameters, including ät.
Let R' be the distribution of xguut DPo( where R. Similarly, define F. Below, provide a lower bound on the detector's ' error after performing diffusion purification on R and F.
Theorem 1. The of evasion and spoofing errors of watermark detector D distributions R' and F is lower bounded as follows:
0h W(R,F)
ea(F D) + ei (R', D) 1
2 2(1 ai)))
where (r) is the Gauss error function, and the Wasserstein distance is measured w.r.t the lg norm.
In Appendix A.2 we elaborate how this theorem can be extended to apply diffusion purification in the latent space rather than the pixel space. Theorem implies that when the Wasserstein distance between the watermarked and non watermarked distributions is low (either in pixel or latent spaces), i.e. watermarking with low perturbation budget, diffusion purification is effective in compromising the watermark. The lower bound presented in Theorem] | practical diffusion model, is illustrated in Figure demonstrating the applicability of the theoretical findings in practical scenarios (i.c., the value of error lower bound is considerable, for real-world values of Wasserstein distance).
employing real-world configurations of
We note that, even though Theorem is stated w.r.t using diffusion models as the method to denoise images after adding Gaussian noise to them, our theoretical bound can be attained with any arbitrary denoising technique (Elad et al.) [023] Wang et al. (022b) (refer to Appendix [ for more information). stronger denoising technique permits the of higher magnitude of Gaussian noise, resulting in a more significant lower bound on the error according to Theorem
In the next section, we provide empirical evidence supporting theoretical result.
3.2 Low PERTURBATION BUDGET WATERMARKS: EMPIRICAL ATTACKS
We first categorize certain established watermarking methods into two groups: "low" and "high" perturbation budget watermarks. This categorization relies on the image space l2 distance between corresponding watermarked and non-watermarked samples for these methods, as detailed in Table We opt for the l distance as surrogate for the actual Wasserstein distance, as it offers an upper bound on the Wasserstein distance, and computing the exact Wasserstein distance is expensive.
In this section, we leverage Theorem to attack watermarking techniques with low perturbation budgets (i.e., known as imperceptible or invisible watermarks in prior work) utilizing the diffusion purification attack as outlined in Definition 2 We will discuss attacks on "high" perturbation budget watermarks in the next subsection.
We use 64-bit binary keys for watermarking techniques. Our evaluation is conducted on a set of 100 images drawn from the ImageNet dataset (Russakovsky et al.) 2015), and their watermarked counterparts using each method. For the WatermarkDM method, which necessitates pre-training of its models, we undertake training of the injector and detector models for 20 epochs the ImageNet dataset. Watermark detectors, when given an input image and an encryption key, produce a con- fidence score that corresponds to the likelihood of the image being watermarked with that specific
5

---

Published as a conference paper at ICLR 2024
1.0
1.0
0.8
AUROC
0.8
AUROC
0.6
leellnng /Adx
WopemankDDM
Tiweingg Radd
0.7
StngaStomp (Adv
ReaGan
01.4
StapeStamp (Rrd
DesDeted
0.2
0.5
Trafing
StegStomp
0.0
0.05
0.10
2
4
8
10
12
14
Diffusion Purification Step (f)
Epsilon
Figure 4: AUROC of watermarking methods against diffusion purification attack for range of t values. As expected, the robustness of methods against this attack has correlation with the average image & distance from Table
Figure 5: AUROC of high-perturbation water- marking methods against &oo adversarial attack w.r.t adversarial perturbation size c. The col- ored dashed lines measure robustness against uniform random noise in the range [-2r, 2e).
Original
DwiDet
DwiDeiSVD RivaGAN WatermarkDM
StegaStamp
TreeRing
0.95
1.0
1.0
1.0
1.0
0.42
0.5
0.45
0.59
0.59
0.67
0.11
Diffpure,
Figure 6: Illustrations of images subjected the image diffusion purification attack and our ad- versarial model substitution attack. The value represents the confidence score assigned by the watermark detector to the images where higher score indicates greater likelihood of the image being watermarked. These attacks are able to significantly reduce the AUROC of the detectors (de- tails can found in Figures 3 and Figure 5)
key. Subsequently, these images are categorized as watermarked the confidence score exceeds a predefined threshold, which may either be constant value threshold that varies. In exper- iments, we specifically adopt a variable threshold for the process of watermark detection, and use AUROC (Area Under the Receiver Operating Characteristic) measure
evaluation metric.
The diffusion purification attack, as defined in Definition 2 involves a two-step process: adding noise images and then denoising them using denoising model. In a diffusion model with N steps, diffusion purification attack with parameter f E [0, 1] on image Io creates a noisy image Ie (N (arzo (1) and denoises it with trained neural network over N x steps. Based on Theorem I the diffusion purification attack is expected to lower the performance of watermarking methods, particularly when there is a low Wasserstein distance between the distributions of water- marked and non-watermarked images.
We make use of the image diffusion models presented in Nie et al. 22022. particularly a 256 256 unconditional guided diffusion model designed for ImageNet images. As illustrated in Figure 3 becomes evident that all the examined watermarking methods can be compromised through carry out latent diffusion purification
diffusion purification attack with 0.2. Additionally, attack, the results of which are detailed in Appendix A.2 Choosing a higher value of t results in better attack success rate. however, it might degrade the quality of the generated images. Some examples of attacked images using different values are shown in Figure 6 and the quality of output images is measured in Table 2 using image quality metric. The diffusion purification

---

Published as a conference paper at ICLR 2024
dw1Det
dwvietSdd
TreaRing
WatermarkDM
1.0
Clean
0.8
TPR
0.6
detos figeelech
Swcact Mjjc 5.750
Nolse
0.4
Swcouse beeehet AUnOC 7533
mea AUROC 9983
0.2
Witomanth oppurer sunoc 0.3422
tiaeking AUROC Galls
Spooted
Rar dardem derection
0.0
0.2
0.4
0.6
0.8
1.0
FPR
Figure 7: Left: The figure demonstrates the spoofing of watermarking techniques, comprising clean ImageNet dataset images (top row), noisy watermarked images (middle row), and spoofed water- marked images (bottom row). Spoofed images blend clean and noisy ones, enabling detection as watermarked, Right: Spoofing attack on various watermarking methods.
method lowers AUROC by reducing the detector's confidence watermarked images and does not consistently boost the confidence of non-watermarked ones. This is understandable the space of watermarked images is typically much smaller than won-maatrrrrked images due to the key string size. Since diffusion purification is a no-box attack, it cannot apply specific watermark patterns to non-watermarked images without prior knowledge of methods or key strings.
3.3 HIGH PERTURBATION BUDGET WATERMARKS: EMPIRICAL ATTACKS
For the watermarking methods that impose high perturbations to the inputs (i.e., TreeRing (Wen iet al. (202) and StegaStamp (Tancik et al. 2020)), our bound in Theorem since the Wasserstein distance between watermarked and non-watermarked distributions becomes large. In fact, Figure 4 shows empirical evidence that as the perturbation budget of watermarking methods increases, the diffusion purification attack becomes less effective, e.g., TreeRing shows strong robustness against that.
[ becomes vacuous
The StegaStamp watermarking (Tancik et al. 2020) imposes large perturbations in the feature space. While its & perturbation in the image space is larger than that of other invisible watermarking meth- ods (Table D. it is much smaller than that of the TreeRing. That is partially the reason that the diffusion purification attack in the image space is relatively successful against StegaStamp, espe- cially at higher values such as 0.3 which might leave some artifacts on images. Nevertheless, categorize StegaStamp high perturbation budget watermarking and study additional alterna- tive attacks against in this section.
For the high perturbation budget watermarking schemes, develop model substitution adversarial attack that can successfully alter the watermark detector's decision. To do this, we first train a ResNet-18 18 (He al. 20155) classifier on the train split distinguish between watermarked and non- watermarked images. Then, we target the watermark detector by executing PGD adversarial attacks on test split using the substitute classifier that have trained. Interestingly, this attack transfers well to the original watermark detector which we assume we do not have a white box access to.
Figure 5 displays the AUROC of the methods following adversarial attacks using various adver- sarial perturbation budgets denoted as e. StegaStamp demonstrates greater resilience to our attack, requiring an e value of 12/255 before its performance degrades to the level of a random detector. We note that this level of adversarial noise may leave perceptible artifacts on the images. However, TreeRing is found to be more vulnerable, as a perturbation budget as low as 2/255 can render it completely ineffective. Note that the transferability of the adversarial attacks is reliant the substitute classifier's architecture and training procedure. In our case, we employ a basic ResNet- model with standard training procedures. Opting for more suitable model configuration may lead to a substantial increase in the attack's 's success rate on the watermark detector. More details about the adversarial attack can be found in Appendix A.3

---

Published as a conference paper at ICLR 2024
3.4 SPOOFING ATTACKS ON WATERMARKING METHODS
An effective watermarking method should minimize both spoofing and evasion errors. High spoof- ing errors enable adversaries manipulate natural images, leading "spoofing attack". Such attacks can falsely identify obscene images as watermarked, potentially harming the reputation of the developers of a watermarked generative model. In this section, we evaluate various watermark- ing techniques in the presence of adversarial spoofing attempts.
We use simple strategy spoof various watermarking techniques by blending watermarked noisy images with clean images (see Algorithm D A detailed explanation of the attack is available in Appendix A.4 Figure 7 shows examples of spoofed images for various watermarking methods. While evaluating the AUROC metric, we also augment the images in our dataset using two differ- ent techniques: random cropping to x00 200-dimensional images and resizing back to 256x256- dimensional images, and random rotations between -30 and 30 degrees. Figure shows ROC curves for our spoofing attack. As seen here, the AUROC and TPR at low FPR metrics of all the water- marking methods considered here drop after spoofing attack. RivaGAN seems to be the most robust to our spoofing attack. However, at low FPR regimes, some of the RivaGAN images can be spoofed as well.
4
TRADE-OFF OF DEEPFAKE DETECTORS
A reliable deepfake detector should exhibit the following two properties: (i) Robustness: Minor input image perturbations should not influence performance. (ii) Reliability: The detector should accurately identify fake images while minimizing false positives. In this section, we extend the techniques used in proving Theorem show a fundamental trade-off between these two properties.
Let R and F denote the distributions of real and fake images. Consider a detector D that maps an input image x I to a latent representation p(x) E RR that encodes the perceptual features of the image and uses this representation for detection. We define the robustness of D) as its ability to correctly classify a noisy version of the image in this latent space. Let (()(z) a) denote the distribution of noisy versions of the image x in the latent space, where a is parameter representing the size of the noise distribution. Here, represents a general noise distribution with size parameter For (xample.N(á((t))) ((((x), or) could represent an isometric Gaussian distribution with variance o² or a uniform distribution with width a centered at p(z).
Definition 3 (Robust Detector). We say detector D is (a, o-)-rbbstt on R and F under noise distribution N if. for an image x drawn from either R or F, its prediction is consistent on latent representations from N ((6z)), σ) with probability at least (1 - for some a 2 0, iee.
Vk E (0, 10, E FR,F). PZwp, (n-PP,pnN(((())..o
( = ()D((()) = k 2 21-a. (3)
This indicates a robust detector's prediction should remain largely unchanged for noisy inputs.
To measure the distance between two distributions R and F we use the Wasserstein metric, follow- ing similar formulation as Equation 2 However, here, we define the distance with respect to norm || in the latent space RR. as follows:
W(R.
inf rET(R.F)
Be [11((11) d(x2)]]]-
(4)
Consider images 21 and 22- Let v-)(-) denote a concave upper bound the total variation between the corresponding noise distributions (((()) a) and N ((((z2), a) as a function of the distance [})(;;) - d(x2))l between the corresponding images in the latent space, i.e.,
(N) o)) S vo (1)0(1)) - ((x2)
(5)
Note that a concave upper bound like this always exists for any noise distribution N. This is because the total variation between the noise distributions for two images goes from zero to one as the distance between them in the latent space increases. Thus, a trivial bound could be obtained by simply considering the convex hull of the region under the curve of the total variation with respect to the distance. In the case where A is an isometric Gaussian and the distance is measured using the l2 norm, this bound takes the form of the Gauss error function, more precisely:
Vo (1l0(11) - ((22)12) l2) = erf [[o(z)) ) 0(22))()))))))))
2/20
8

---

Published as a conference paper at ICLR 2024
Performance (AUROC)
0.8
0.6
AUROC
0.4
4,0
WIR. F) 2.0
W0R, F) 1.0
Deeptakes
Random
Face5wap
12
14
20
7.5
20.0
12.5
15.0
27.5
200
Robustness (o)
1%
Figure 8: Detection performance w.r. ro- bustness parameter for different values of the Wasserstein distance between real R and fake F distributions.
Figure AUROC vs. o for a robust deepfake detector with ResNet-18 backbone on DeepFakes (deeptakes) and FaceSwap (MarekKowalski) datasets.
Theorem 2. (a, a-obuutsst detector's AUROC upper bounded as follows:
AUROC(D) & 1 do (W, 1h (W,
1 2a -2a²
+
2
2(1 - -a)
For example, when the Wasserstein distance is measured using E in the latent space and the noise is isometric Gaussian with variance o², vo takes the form of the Gauss error function: (=(2) = erf(2 erf(z/(2v2o) We set to some small positive value (i.e., behavior of the bound for different values of a. Figure 8 shows the behavior of the bound with re- spect to the robustness parameter for different values of the Wasserstein distance while Figure I6 shows the behavior of the bound with respect to the Wasserstein distance for different values of a. The detection performance bound has negative relationship with the amount of noise that can be tolerated.
1%) and analyze the
Experiments. We perform experiments the images from the FaceForensicst* dataset hosted by Rössler et al. (2019) to verify our theoretical insights empirically. We use ImageNet pretrained ResNet- 18 (He al. 2015) (based on popular DeepFake detectors (Rössler et al. 2019) Dessa 2019)) and VGG-I6-BN (Simonyan & Zisserman 2014). More details on dataset preprocessing and experiments are provided in Appendix F We train the models classify between real and synthetic facial images. The initial trained layers of the models are fixed to be the latent representation ó given in Equation 3 The remaining layers of the models represent detector D. For both ResNet-18 and VGG-16-BN, we choose every layer except the last two convolution layer blocks to represent Φb. De- tectors with varying robustness random noise are trained using noisy latent space feature vectors output from 6. We train different detectors with the standard deviation of noise varied from O to 20. For different detectors, we compute the inference ar on the test dataset at which they achieve an a of 0.01 using Equation In Appendix [ (Figure 18), we show that the detector's 's robustness (infer- ence a at a = 1%) to random noise increases as the training sigma increases. We use ten randomly sampled Gaussian noises for each sample ((x) for this evaluation. After five independent trials, plot AUROC vs. a for various (a, a = 0.01 -robust detectors in Figure using a ResNet- 18 back- bone for the detector (see plot using VGG-16-BN in Figure 17). Our empirical results show that as the robustness or a at fixed a increases, the AUROC or the performance of the detectors drops.
5 CONCLUSION
In this work, we studied the robustness of Al-image detection methods. We proposed diffusion purification as certified attack against low-perturbation watermarks, and a model substitution ad- versarial attack against sesturerturbatoon watermarks. Furthermore, showed fundamental reli- ability vs. robustness trade-off for classifier-basee deepfake detectors. Based on our results, design- ing robust watermark is challenging, but not necessarily impossible task. An effective method should possess specific attributes, including a substantial enough watermark perturbation, resistance naive classification, and resilience noise transferred from other watermarked images.

---

Published as a conference paper at ICLR 2024
ETHICS STATEMENT
In our research, we follow academic integrity and responsible AI practices. We aim to contribute to discussions the security of detecting Al-generated content. We prioritize ethical considerations and focus on the societal impact of our findings, Our commitment is to transparency and aware- ness in the evolving field of generative Al technologies, with the goal of preventing misuse while encouraging progress.
REFERENCES
Belhassen Bayar and Matthew C Stamm. A deep learning approach to universal image manipu- lation detection using a new convolutional layer. In Proceedings of the 4th ACM workshop on information hiding and multimedia security, pp. 5-10, 2016.
Taís Fernanda Blauth, Oskar Josef Gstrein, and Andrej Zwitter. Artificial intelligence crime: An overview of malicious use and abuse of ai. IEEE Access, 10:77110-77122, 2022. doi: 10.1109/
FRANCIS MORGAN Boland, Joseph JK O' O'Ruanaidh, and C Dautzenberg. Watermarking digital images for copyright protection. 1995.
Mattia Bonomi, Cecilia Pasquini, and Giulia Boato. Dynamic texture analysis for detecting fake faces in video sequences. J. Vis. Commun. Image Represent. 79: 79:103239, 2021. doi: 10. 10.1016/j.
jvcir.2021.103239, httpssss::///oi http:///ee/aaaa..oo/.....///////
Chin-Chen Chang, Piyu Tsai, and Chia-Chen Lin. Svd-based digital image watermarking scheme. Pattern Recognition Letters, 26 26(10):1577-1588 2005.
Robert Chesney and Danielle Citron. Deepfakes and the new disinformation war: The coming age of post-truth geopolitics. Foreign Affairs, 98(1):147, Jan 2019. utt.ttts::/ //wp::////wwww///// proquest.
war coming age pottppos// docvieww 21 61 21e/e2111111//// se-2 Copyright Copyright Council on Foreign Relations NY
commmaagazines http////aaa.ieeeeeeeeeeeeeeeeeee deepfakes new www.ddiiiiaiiiiooooooooooi
Jan/Feb 2019; Last updated 2022-1 11-09.
François Chollet. xception: Deep learning with depthwise separable convolutions", arxiv preprint.
arXiv preprint arXiv:1610.02355 2016.
Umur Aybars Ciftci and llke Demir. Fakecatcher: Detection synthetic portrait videos using biological signals. CoRR, abs/1901.02212, 2019. URL httt:://axxv..or//ab..
02212
Ingemarr Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. Digital Watermark- ing and Steganography. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2 edition, 2007. ISBN 9780080555805.
Ingemar Cox, Joe Kilian, Tom Leighton. and Talal Shamoon. Secure spread spectrum watermark- ing for images, audio and video. Proceedings of 3rd IEEE international conference on image processing, volume pp. 243-246. IEEE, 1996.
Davide Cozzolino, Giovanni Poggi, and Luisa Verdoliva. Recasting residual-based local descriptors as convolutional neural networks: an application to image forgery detection. In Proceedings of the 5th ACM workshop on information hiding and multimedia security, pp. 159-164, 2017.
Yingqian Cui, Jie Ren, Han Xu, Pengfei Hc, Hui Liu, Lichao Sun, and Jiliang Tang. Diffusion- shield: A watermark for copyright protection against generative diffusion models. arXiv preprint arXiv:2306.0466 04642, 2023.
deepfakes. Deepfakes. URL http:////...tccocccc///eeefe//es
Dessa. ber
Towards
deepfake
detection
that
actually works.
Novem-
2019.
URL b com wew.d
10

---

Published as a conference paper at ICLR 2024
Pantelis Dogoulis, Giorgos Kordopatis-Zilon loannis Kompatsiaris, and Symeon Papadopoulos. Im- proving synthetically generated image detection in cross- concett settings. In Proceedings of the 2nd ACM Internatinnal Workshop on Multimedia Al against Disinformation, pp. 28 28-35, 2023.
Michael Elad, Bahjat Kawar, and Gregory Vaksman. Image denoising: The deep learning revolution and beyond-a a survey paper. SIAM Journal on Imaging Sciences, 16(3): (133:1194--6542 1654, 2023.
Patrick Esser, Robin Rombach, and Bjom Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference computer vision and pattern recogni- tion, pp. 12873-1288 12883, 2021.
Kazem Ghazanfari, Shahrokh Ghaemmaghami, and Saeed R Khosravi. Lsb++: An improvement to Ish+ Steganography. In TENCON 2011-2011 IEEE Region 10 Conference, pp. 364-368. IEEE, 2011.
Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. Fighting deepfake by exposing the convo- lutional traces images. IEEE Access, 8:165085-665098, 2020. doi: 10. 109/ACCSSS.2000. 3023037.
2020. 3023037
Alexandros Haliassos, Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. Lips don't t lie: A generalisable and robust approach face forgery detection. In IEEE Conference on
Computer Vision and Pattern Recognilion, CVPR 2021, virtual, June 19-25, 2021, pp. 5039- 5049. Computer Vision Foundation / IEEE, 2021. doi: 10.1 (009//CYYYA4477.222.00000
URL
https: http://///peccccsssss thecvf com/ http:///c....o.ee///////////////
Lips Dont_Lie, A Geeerelliabllleeeoooaaeeaa
Robustt_Appoaccee Topproe _VPR
2021 1 paper.pp..... html
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. corr abs/1512.00336 (2015), 2015.
Todd C. Helmus. Artificial Intelligence, Deepfakes, and Disinformation: Primer. RAND Corpo- ration, Santa Monica, CA, 2022. doi: 10.7249/PEA1043-1.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances Neu- ral Information Processing Systems, volume 33, pp. 6840-6851. Curran Associates, Inc.,
fttperrrriieee//e////00... 2020. URL http:////peeeeeee.eee.c.ccse/ees httl://cbccccbbaaa000offaa111113
Vojtëch Holub and Jessica Fridrich. Designing steganographic distortion using directional filters. In 2012 IEEE International workshop on information forensics and security (WIFS), pp. 234-239. IEEE, 2012.
Chris Honsinger. Digital watermarking. Journal of Electronic Imaging, 1133)4144444,2000
Jessica L. Ice. Defamatory political deepfakes and the first amendment. Case Western Reserve law review, 70:417, 2019.
Zhaoyang Jia, Han Fang, and Weiming Zhang. Mbrs: Enhancing robustness of dnn-based water- marking by mini-batch of real and simulated jpeg compression. In Proceedings of the 29th ACM international conference on multimedia, pp. 41-49, 2021.
Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai-generated content. arXiv preprint arXiv:2305.0388 2023.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation, 2022.
Xiaodan Li, Yining Lang, Yuefeng Chen, Xiaofeng Mao, Yuan He, Shuhui Wang, Hui Xue, and Quan Lu. Sharp multiple instance learning for deepfake video detection. In Chang Wen Chen, Rita Cucchiara.
Hua, Guo-Jun Qi, Elisa Ricci, Zhengyou Zhang, and Roger Zimmermann Multimedia, Virtual Event / Seattle, WA, USA, October 12- 16. 2020, pp. 1864-18722 1872. ACM, 2020. doi: 011144//394111.3444009
(eds.), MM '20: The 28th ACM International Conference
URL https: //ooi htt.b/o.oorr111111553333355
.41 3034034
11

---

Published as a conference paper at ICLR 2024
Calvin Luo. Understanding diffusion models: A unified perspective.
arXiv preprint
arXiv:2208. 112008.11990 11970, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks, arXiv preprint arXiv:1706.0608. 2017.
MarekKowalski. Faceswap. URL
Falko Matern, Christian Riess, and Mare Stamminger. Exploiting visual artifacts to expose deep- fakes and face manipulations. 20/9 IEEE Winter Applications of Computer Vision Workshops (WACVW), pp. 83-92, 2019. doi: 10.1 N0.100//AAC..221990000
Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. arXiv preprint arXiv:2205.07440 2022.
Joseph JK O' O'Ruanaidh and Thierry Pun. Rotation, scale and translation invariant digital image watermarking. In Proceedings of International Conference on Image Processing, volume 1, pp. 536-539. IEEE, 1997.
Tomáš Pevný, Tomás Filler, and Patrick Bas. Using high- figh-dimensional image models to perform highly undetectable steganography. In Information Hiding: 12th International Conference, III 2010, Calgary, AB, Canada, June 28-30, 2010, Revised Selected Pupers 12, pp. 161-177. Springer, 2010.
V.M. Potdar, S. Han, and E. Chang. A survey of digital image watermarking techniques. In IN- DIN '05. 2005 3rd IEEE International Conference Industrial Informatics, 2005., pp. 709-716, 2005. doi: 10. 10.11000 2000//
Nicolas Rahmouni, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. Distinguishing computer graphics from natural images using convolution neural networks. In 2017 IEEE workshop on information forensics and security (WIFS), pp. 1-6. IEEE, 2017.
Kiran Raja, Sushma Venkatesh, RB Christoph Busch, et al. Transferable deep-cnn features for detecting digital and print-scanned morphed face images. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition Workshops, pp. 10-18, 2017.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10693 2022.
Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nicßner. FaceForensics+::: Learning to detect manipulated facial images. In International Con- ference on Computer Vision (ICCV), 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-22 1-52,, 2015.
Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs, 2021.
Jin S Seo, Jaap Haitsma, Ton Kalker, and Chang D Yoo. A robust image fingerprinting system using the radon transform. Signal Processing: Image Communication, 19(4):325-339 2004.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free!, 2019.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1rXiv:1409.1533 2014.
M.D. Swanson, M. Kobayashi. and A.H. Tewfik. Multimedia daba-embedding and watermarking 1998. doi: 10. (099//68783/000
technologies. Proceedings of the IEEE,
12

---

Published as a conference paper at ICLR 2024
Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical pho- tographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pp. 2117-2126, 2020.
Chunpeng Wang, Qixian Hao, Shujiang Xu, Bin Ma, Zhiqiu Xia, Qi Li, Jian Li, and Yun-Qing Shi. Rd-iwan: Residual dense based imperceptible watermark attack network. IEEE Transactions on Circuits and Systems for Video Technology. 32(11):4400-743) 2022a. doi: 10.1109/TCSVT. 2022.3188524.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint wrxiv:2212.00490, 2022b.
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fin- gerprints for diffusion images that are invisible and robust. arXiv preprint arX0v:2305.20030, 2023.
Raymond B Wolfgang and Edward Delp. A watermark for digital images. In Proceedings of 3rd IEEE International Conference on Image Processing, volume 3, pp. 222. IEEE, 1996.
Kevin Alex Zhang, Alfredo Infant-, Lei Xu, and Kalyan Veeramachaneni. Steganogan: High capacity image steganography with gans, arXiv preprint arXiv:1901.03892 2019a.
Kevin Alex Zhang, Lei Xu, Alfredo Cuesta Cuesta-Infante, and Kalyan Veeramachaneni. Robust invisible video watermarking with attention. CoRR, hts/1909.01288 2019b. URL http:://rxiv..
org/abs 1909. 01285
Kevin Alex Zhang, Lei Xu, Alfredo Cuesta -eeetteIn and Kalyan Veeramachameni. Robust invisible video watermarking with attention. arXiv preprint arXiv:1909.0128;, 2019c.
Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai, 2023a.
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, and Min Lin. A recipe for watermarking diffusion models. arXiv preprint arXiv:2303. 2023b.
Zhengyue Zhao, Jinhao Duan, Xing Hu, Kaidi Xu, Chenan Wang, Rui Zhang, Zidong Du, Qi Guo, and Yunji Chen. Unlearnable examples for diffusion models: Protect data from unauthorized exploitation. arXiv preprint arXiv:2306.0199 0/00,, 2023c.
Peng Zhou, Xintong Han, Vlad I Morariu, and Larry S Davis. Two-stream neural networks for tampered face detection. In 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 1831-1839. IEEE, 2017.
13

---

Published as a conference paper at ICLR 2024
Image E distance
Latent & distance
Method
RivaGAN Zhang et al.
2019b)
4.19
8.47
DwtDet (Cox et al. 2007)
5.59
5.47
DwtDetSvd (oox al. 2007
5.54
6.67
WMtermarkDM
iZhao et al.
2023b)
7.26
13.84
StegaStamp
Tancik et al. 2020
17.40
118. 17
TreeRing
Wen et al.
2023)
117.58
52.81
Table I: Average & distance between corresponding watermarked and non-watermarked images for each method. The latent representations were obtained using a VQGAN model (Esser et al.) 2021), commonly used for latent diffusion models. We consider the first four methods as low perturbation, and the last two as high perturbation ones.
t=0.1
0.2
1.00
1.00
True Positive Rate
True Positive Rate
0.75
0.75
wetermar ADM, auc-0.88
WatermanDDM auc-0.65
0.50
0.30
rvoGan, auc-0.79
riveGan suc-0
dwiDetSvd, auc-0.64
dwtDctSvd, auc-0.52
0:25
dwtDet, auc-
0.25
dwtDet. suc-0.56
stagaStammp
stagaStamp, treeRing auc 0.98
treefling, auc=
0.00
0.00
0.0
1.0
0.0
False Positive Rate
False Positive Rate
t- 0.3
True Positive Rate
0.75
Ma wir 0.57
rivaaar auc-0.45
desiasSed auc-0.50
0.25
dwrDet, suc-0.54
stagsStamp, auc-0.79
treeRing, auc-0.52
0.00
False Positive Rate
Figure 10: ROC curves for watermarking methods against diffusion purification attack with different
values of t.
A COMPLEMENTARY RESULTS FOR WATERMARKING METHODS
A.1 DIFFUSION PURIFICATION ATTACK
Figure showcases images that have undergone the diffusion purification attack with varying values, while Figure TO displays ROC curves for watermarked techniques under these attacks. In the low FPR regime, the TPR of all methods declines at some value of Table [ numerically measures the quality of watermarked images that attacked using diffusion purification w.r.t. the non-attacked images. The quality of images is measured using image quality metrics such as PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure).
Note that the quality of images for the TreeRing watermark depends on the captions that are pro- vided for the images, and in case, Therefore, the images watermarked by TreeRing might exhibit dissimilarity compared to their non- watermarked counterparts. However, this does not influence the results when attacking the TreeRing
using simple captions based on ImageNet classes.
14

---

Published as a conference paper at ICLR 2024
Original
DwiDet
DwiDeiSVD
RivaGAN
DM
StegaStamp
TreeRing
t 0.1
f 0.2
Original
DwtDet
DwtDetSVD
RivaGAN
WatermarkDM
StegaStamp
TreeRing
1= 0.2
1=
Figure 11: Watermarked images subjected to the image diffusion purification attack are shown with varying values of the parameter For 0.3, the attack may excessively alter images, making unsuitable for some applications.
15

---

Published as a conference paper at ICLR 2024
PSNR
SSIM
Method
0.1
t 0.2
== 0.3
t 0.1
== 0.2
= 0.3
RivaGAN
29.77
26.10
23.61
0.83
0.72
0.63
DwtDet
29.64
26.03
23.70
0.83
0.72
0.63
DwtDetSvd
29.69
26.08
23.60
0.83
0.72
0.63
WMtermarkDM
30.33
26.41
23.87
0.86
0.75
0.66
MBRS
29.96
26.23
23.76
0.83
0.73
0.64
StegaStamp TreeRing
30.35
26.52
24.08
0.84
0.73
0.64
32.45
28.27
25.49
0.92
0.86
0.81
Table 2: Analysis of the quality of images after being attacked using diffusion purification.
1.0
Essilon 4, AUC:
0.9
True Positive Rate
0.8
Fpellon 8, AIC 0.06
AUROC
0.8
Treffin
0.4
AstmmaUM
0.6
Rudfas
0.5
0.0
0.2
0.3
0.4
0.5
0.0
0.2
0.1
Diffusion Purification Step (t)
False Positive Rate
Figure 12: AUROC of watermarking meth- ods against latent diffusion purification at- tack w.r.t the value of
Figure 13: ROC curves for attacking wa- termarked and non-watermarked images that are generated with text from LAION- captions with the TreeRing method.
we demonstrate that our adversarial attacks on TreeRing also extend successfully captions from LAION-captions data.
watermark. Nevertheless, in Fig 13
A.2 LATENT DIFFUSION PURIFICATION ATTACK
similar bound from Theorem I can be proven for latent diffusion models. The diffusion process for latent diffusion model consists of: mapping Zn to the latent space, i.e., =n <(xa); calculating .out (20) using a latent diffusion model; and mapping zot back to image space, i.e., xg -0
1=gutt In this case, since the noise is applied to latent space 6, the Wasserstein distance in Theorem will be replaced by the Wasserstein distance the latent distributions, i.c., W(R,, Fo) with R being the distribution of images 2n 6(x0) where To R, and Fg defined similarly.
In practice, we perform latent diffusion purification attack by employing a Text-Guided Image- to Im-le Stable Diffusion model (Rombach et al.) (022), and using BLIP model (Li et al.) (022)) to generate image captions, as guidance for diffusion models. Figure 12 includes the AUROC of watermarking methods against this attack, and Figure [4] contains samples output images for this attack.
A.3 ADVERSARIAL ATTACK
We conduct adversarial attacks involving model substitution on nigh-perturbation budget water- marks, specifically StegaStamp and TreeRing. Our training dataset comprises 7, 500 watermarked and 7, 500 non-watermarked images. For StegaStamp, we use images sourced from ImageNet, along with their watermarked versions, for both training and testing. contrast, for TreeRing, the non- watermarked images either be sourced from ImageNet generated using process similar to TreeRing's 's watermarking method, but employing random noise instead of TreeRing's ' key string. We have observed through empirical testing that the effectiveness of our adversarial attack remains consistent, regardless of the choice between these two types of non- watermaarrrk training data. As result, opted for the latter approach.
16

---

Published as a conference paper at ICLR 2024
Original
DwtDet
DwtDetSVD
RivaGAN
WatermarkDM
StegaStamp
TreeRing
f 0.2
f
f=
1 0.5
Figure 14: Watermarked Images subjected to the latent diffusion purification attack are shown with varying values of the parameter t. For 0.5, the attack drastically changes the images in most cases (except for TreeRing).
17

---

Published as a conference paper at ICLR 2024
Watermarked
é 4/255
8/255
12/255
Perturbation Pattern
TreeRing
StegaStamp
Figure 15: Watermarked images subjected to the model substitution adversarial attack are shown with varying values of adversarial perturbation budget Attacks images watermarked with the same method show similar perturbation patterns.
For StegaStamp, we employ 100-bit binary keys, mirroring the key length described in their report. In the case of TreeRing, stick the ring-type key employed in the original implementation. TreeRing necessitates captions for generating watermark images, and for our ImageNet data, we utilize captions structured "a photo of a (imagenet-class). Nevertheless, in Figure 13 we demon- strate that our attacks on TreeRing also extend successfully to LAION-captions data (Schuhmann)
al. 202I).
Our substitute classifiers are trained for 10 epochs and receive higher than 99.8% accuracy on val- idation data. For StegaStamp, observed that augmenting the training data with Gaussian noise improves the transferability of the attacks on the watermark detector.
To launch adversarial attacks on images using substitute classifiers, we employ a PGD attack with 300 iterations and step size denoted as 0.05e. Our observations indicate that adversarial per- turbations for particular watermarking method exhibit roughly consistent pattern. Consequently, initiate our adversarial attacks each image from the perturbation discovered for the previous image, technique akin to the one employed in Shafahi et al.) (2019). To ensure the accurate identi- fication of the perturbation pattern, execute series of preliminary warm-up attacks at the outset. Some sample adversarial images can be seen in Figure 15
In Figure 13 we present ROC curves for attacking TreeRing images that are generated with text from LAION-Can -captions data (Schuhmann et al.) (2021). This shows that our adversarial attack which is performed on the classifier trained on ImageNet data, generalizes to any images watermarked using the TreeRing method.
A.4 SPOOFING
To perform the spoofing attack, we first generate random noisy images where pixels are drawn from different Gaussian distributions with varying standard deviations. The noisy images are normalized
18

---

Published as a conference paper at ICLR 2024
Method
Base
Blur (h: = 5)
JPEG
DiffPure (t 0.2)
t 0.1
0.655
0.692
0.679
0.638
RivaGAN
= 0.2
0.623
0.607
0.604
0.593
t 0.3
0.579
0.568
0.562
0.555
0.1
0.548
0.546
0,544
0.542
DwtDet
t 0.2
0.542
0.540
0.539
0.539
0.3
0.539
0.538
0.538
0.537
0.1
0.560
0.566
0.567
0.567
DwtDetSvd
t 0.2
0.564
0.560
0.561
0.558
0.3
0.555
0.553
0,551
0.549
t 0.1
0.876
0.885
0,805
0.597
WatermarkDM
0.2
0.644
0.630
0.604
0.518
t 0.3
0.568
0.604
0.565
0.564
t 0.1
0.914
0.828
0.874
0.597
MBRS
0.2
0.614
0.636
0.634
0.545
t 0.3
0.536
0.493
0.444
0.547
I 0.1
1.000
1.000
0.998
0.920
StegaStamp
t 0.2
0.966
0.960
0.971
0.832
0.3
0.781
0.802
0.767
0.659
0.996
0.989
0.947
0.935
TreeRing
t 0.2
0.976
0.956
0.923
0.912
t 0.3
0.928
0.907
0.871
0.876
Table 3: The AUROC of watermarking methods against diffusion purification attack, after applying post-attack mitigations to the attacked images.
to have values between and 1. For every watermarking method that we evaluate, we apply their watermarks these noisy images obtain corresponding watermarked noisy images.
We use an input prompt, "a noisy image", along with the noisy images to generate noisy water- marked TreeRing (Wen al. 2023) images. Once obtain the watermarked noisy images, do mixup (or image blending) by adding noisy images to the clean images to spoof them. We observe that the watermark signatures the noisy images help detect the resulting blended images as watermarked.
We provide the pseudocode for spoofing watermarks in Algorithm
Algorithm Watermark Spoofing
Require: clean image x, watermarking model W, mixup parameter & generate random noise with shape of image normalize
random(z.shor shash)
2.min()
z/2.max()
(=))=
I- watermark the noise; for TreeRing, condition with text "a noisy image"
m.max()
7 max)
D z + v can now only have a value of maximum >> spoofed image
return +
A.5 ROBUSTNESS OF ATTACKS AGAINST MITIGATIONS
In this section, measure the robustness of the diffusion purification and the model substitution adversarial attacks on image watermarking techniques. This robustness is measured by applying post-attack mitigations such Gaussian Blur and JPEG Compression to the attacked images. A robust attack is expected to result in a low AUROC on the watermark detector, even after the post- attack mitigations are applied.
19

---

Published as a conference paper at ICLR 2024
Method
Base
Blur (k 5)
Blur (k = 15)
JPEG
DiffPure (t = 0.2)
4
1.000
1.000
0.999
0.991
0.879
StegaStamp
c = 8
0.923
0.838
0.791
0.864
0.703
12
0.492
0.424
0.341
0.496
0.566
4
0.035
0.025
0.023
0.046
0.891
TreeRing
c =
8
0.002
0.001
0.001
0.006
0.531
12
0.001
0.0002
0.0002
0.001
0.074
Table 4: The AUROC of watermarking methods against model substitution adversarial attack, after applying post-attack mitigations to the attacked images.
Showeassee the AUROC of watermarking methods against diffusion purification attacks, after applying post-attack mitigations. The application of post-attack mitigations is not causing significant increases in the AUROC. This anticipated since the primary aim of the diffusion purification attack is the removal of watermarks from the watermarked images (i.e., to achieve a bit-accuracy close to 0.5 for both watermarked and non-wwatrmarred images). Therefore, is reasonable to expect that basic no-box post-attack mitigations will encounter challenges in recovering the watermark.
On the other hand, our proposed adversarial attack has black-box information about the watermark, and therefore, is able to target both won-watermarked and watermarked images for its attack, in order increase reduce their watermark bit-accuracy, respectively. Table @allee@ssoccasee the AUROC of watermarking methods against the adversarial attack, after applying post-attack mitigations. While post-attack mitigations, specifically DiffPure, are able to increase the AUROC some cases, they fail to negate the effect of the attack for higher attack budgets such = 8/255.
B PROOF OF THEOREM 1
Statement. The sum of evasion and spoofing errors of watermark detector on distributions R and F is lower bounded as follows:
erf( v a W(R.F) co(F",D) + e1 (R',
2 ki)
Proof. Let (-) denote concave upper bound the total variation between noise distribu- tions N (x) and N (x2,, a) as function of the distance || - æ2|| between the corresponding images, i.c.,
(1)(z1,)), o)) vo (1|=1 x2ll).
(6)
where TV is the total variation of two distributions.
Note that concave upper bound like this always exists for any noise distribution. N. This is be- cause the total variation of the noise distributions for two images goes from zero to one as the distance between them in the latent space increases. Thus trivial bound could be obtained by sim- ply considering the convex hull of the region under the curve of the total variation with respect to the distance. In the case where A is isometric Gaussian and the distance is measured using the 2g-norm, this bound takes the form of the Gauss error function, more precisely:
||-1| -x2ll
No (1 x2ll) erf
2/20
Now, consider the distribution of images under the noise distribution N.. Let Ry be the distribution of images ā. ~ (N(x,, r) where x ~ R. Similarly, define FN- The same equality as Equation 7 can be written for the Wasserstein distance of and J defined with respect to l2 norm, when D and 222 are sampled from R and F, respectively.
dh (W(RFF)) erf (W(R.F))
(8)
2v20
20

---

Published as a conference paper at ICLR 2024
We bound the total variation of the noisy distributions RN and Fx in terms of the Wasserstein distance between the original distributions R and F. The reason why this bound holds that as R and FF get closer to each other, RN and FN start to overlap due to the noise distribution / around them.
Lemma 1. The total variation of RN and Fx, and hence, the success rate of any detector D on these distributions, is upper bounded by a function of the Wasserstein distance of the original distributions R and F follows:
1 (cu(NN, D) + (RN, D)) I TV(RN, vo (W(R.F))
Proof. For simplicity the proof, assume D to be deterministic, however, the proof can be gener- alized for randomized detectors too. Define ED = fx D(x) 1}. Based on the definition of total variation,
(V(RN,FN) = sup Pai (81 E F) - Pag=FN (82 E E]|
E
2 PaINRN [1] E Ep] Pit==Ex (ī2 E Epl 1 D) - (1 -oooFNF D))1
(Definition 1
2 1 - (c, D) + te (RN, D)).
Furthermore, the inequality TV( < the (W(R,))
be derived from the proof pre- by substituting the latent function & with the identity function.
sented for Lemma 3 in Appendix
In Lemma we have shown that after applying Gaussian noise to R and F, they become more in- distinguishable. However, using Gaussian noise as an attack against image watermarks will degrade the quality of images. Therefore, we utilize denoising diffusion models to remove the added noise. Since the bound in Lemma Tis is on total variation, by applying a denoising function on the noisy distributions Rx and FAv. the bound still holds. Note that our theoretical results do not rely on the utilization of denoising diffusion models, and any arbitrary denoising technique (Elad et al. 2023 Wang et al. 2022b), can be used to achieve similar bounds.
Let R be the distribution of ot ~ (Vaato at) where To ~ R, and define Fj similarly. Additionally, define G.). (.) as the function that performs denoising process Ryy and Fi (i.e., samples of R' come from 26 G*;z) where x, ~ Ryy).
We use Lemma to get an upper bound on the total variation of R2y and F with o = v (1-aa) a.).
based on the defniiion of R2 and Fj-:
(V(R S do (W(R,F))
= =erf( Var W(R, F)
(Fquation g
2/2(11 (at)
Next, we use the fact that after applying the function (.).) on samples from Ry and Fjv- the total variation does not increase, i.e.
TV(R', & TV(R),, F(t)
Now, the theorem's statement be proven follows:
a0 W(R,F)
TV(R', TV(R',, F&) erf(
2 2 - -
01, F(
1 1- (eo D) + en (R!, , D)) < erf(
(Lemma 1
2 2 2(1
- -(6)
Vaa W(R.F) 2 2(111 (4,)
co(F", + (R', D) 2 erf(
21

---

Published as a conference paper at ICLR 2024
We note that inequality 9 can be written for any arbitrary denoising function that receives noisy images of Ryx and F as inputs, and outputs denoised images with acceptable image quality.
C PROOF OF THEOREM 2
Statement. The performance of a (a, rorbbust detector measured using its AUROC upper bounded as follows:
AUROC(D) S 1-a (W,,W the ( ( +
+ 2a 20
2
2(1 a)
Proof. We quantify the dissimilarity between the distributions and F Fusing the Wasserstein metric defined with respect to norm || I| |n the latent space R as follows:
inf (R, TEI(RR.F)
((=,,0))~>)106 [l(())) (0(00)((2))))) ((x2)]]]
(10)
where F), is the set of all joint probability distributions of R and F, ie.,
I(R,F) y: Rd IR + Rg0 / 7(11,
7(11, x2 )r22 = pdf (x1)
and / (211 22)dr1 = pdf (12)
where R and p represent the probability density functions of R and F. For the sake of simplicity, we assume that there exists an element 7* E I that achieves the infimum. Otherwise, one can derive our results for some y that achieves an expected distance of (R, + 5 for an arbitrarily small O.
We use the notation vio to represent concave upper bound on the total variation between two noise distributions, specifically (x(())) o)) and (((22)), σr.. This upper bound is expressed as function of the distance (1x(z1) (xx))] between the respective images in the latent space, i.e.,
()) .)) s via (1lo(=1) (xx2)
(11)
In the case where N is an isometric Gaussian and the distance measured using the f--norm, this bound takes the form of the Gauss error function, more precisely:
Vo (11((1)) - (2) erf (10((1) 6(x2)
2v20
Now, consider the distribution of noisy real images in the latent space under the noise distribution N. Let R2 be the distribution of latent representations ò ~ (ő(x), o) where R. Similarly, define F The following lemma relates the performance of (a, a) -robust detector D under the original and noisy versions of the two distributions.
The AUROC of (a. ()rouus detector D on the original distributions R and F is bounded by that for the noisy versions of the distributions R2y and Fx as follows:
Lemma 2.
AUROCx(D)
AUROC(D) < 1 1 +a.
Proof is available in Appendix D
Next, we bound the total variation between the noisy distributions R² and Fy in terms of the Wasserstein distance between the original distributions and F. The reason why this bound holds that as R and get closer to each other in the latent space, Ry and F start to overlap due to the noise distribution A around them.
22

---

Published as a conference paper at ICLR 2024
Lemma 3.
The total variation between the noisy distributions Ry and F is bounded by the Wasserstein distance of the original distributions R and F as follows:
TV S vo (W,
Proof is available in Appendix
Now, we use the above two lemmas put bound on the performance of the detector and We first show that the performance on the noisy distributions R and F bounded by the total variation between these distributions. We then use Lemma to convert this total variation distance to the Wasserstein distance between the original distributions R and F. Finally, we relate the bound to the detector's performance the original distributions using Lemma
The true positive rate TPRw and the false positive rate FPRN of the detector on the noisy distribu- tions Ry and F& can be bounded by the total variation between these distributions as follows:
TPPR> FPRwl | -O--F---NN00)) (D()) 1] - P. (-(()))))) (D(Ø) 1]]
TV(R
Since the true positive rate is also bounded by one, have:
TPR min(FPRx F 1).
Denoting FPRv, TPRv and TV(R)) F&) with x, y, and tu, respectively, for brevity, bound the A as follows:
AUROCy(D) = ydx S
min(x + Lu, 1 1)da
(x+ to)do + f dæ
1-t0
x22
1-tr
+ lux
+ | 1-t0
2
(1
tv)²
+ tu(l - -t) + to
2
2o
tu tv tu +
2
2
+ tu
2
Thus,
AUROCN(D) TV(R) Nr Fx)
then (R)
< Si (W (W,
2
(from Lemma and since 1/2 + x x²/2 increasing in [0, 1))
Finally, from Lemma 2
we have:
AUROC^(D) +a
AUROC(D) <
1-a
+ the (Wo(R,F) vo (Wo(R, F))²
<
+ a
(from above)
a
2
2
1
(v<(W(RFF)
2a (W(,
+ 2a 20
+
2(1 a))
23

---

TPR dFPRN Published as a conference paper at ICLR 2024
D PROOF OF LEMMA
2
Statement. The AUROC of a (a, ai) rrousst detector D on the original distributions R. and F is bounded by that for the noisy versions of the distributions R2 and F follows:
AUROCx(D)
AUROC(D) - 1 1 +
Proof. Let TPR, FPR, TPRv and FPRv denote the true and false positive rates of the detector on the original and noisy distributions, respectively, assuming the fake distribution the positive class. Then, by definition,
AUROCN(D) = TPRN dFPRy-
Now, relate this to AUROC, we lower bound TPRw and upper bound FPRM in terms of TPR and FPR.
TPR = P. P2~
(D()) =1]
= Pr INN (NN()())) (D(Ø) = 1|D(ó(x)) -]Pr->>(((((x) = 1]
+ PP = (D(Ø) = 1|D(o(=)) = =0]Fm==(D(d(n)) = =]
(law of total probability) (from Equation 3
2 (1 - InF 1]
= a1-a))PR.
FPRw = P R- (D() =1]
= = - (D()) = 1ID()(>)) = (|P=-R(((((()) =
+P + (D(Ø) = 1|D()(=) = ( = 0]
(law of total probability)
< P P 1]
(1 P ( (D() (()()))) RRR D()(x)) 0]
FPR + at (D(=R((((())) =
(from Equation 3
FPR a
Therefore,
(D) = TPRN dFPRN
/ (1 TPR dFPRN
(TPRw 2 (1- TPPR)
00
(1 a)
21 a)
TPR dFPR
(FPR S FPR + a)
2 (1 a) ( TPR dFPR - a
= (1 (AUROC - -a).
Hence,
AUROC(D) < AUROCA(D)
+ a.
24

---

Published as a conference paper at ICLR 2024
0.8
AUROC
AUROC
0.6
0.4
sigma 2.0
62
Random detector
sigma 4.0
DeepFakes
random dotactor
FaceSwap
0.0
Wasserstein Distance
O at = 1%
Figure 16: Deepfake detection performance bound w.n. Wasserstein distance between real R and fake F distributions for different values of a. A more robust detector (higher σ) has lower performance.
Figure AUROC vs. a plot for (a,a 0.01 -oobust deep fake detector with VGG- 16-BN backbone on DeepFakes (deepfakes) and FaceSwap (MarekKowalski) datasets. Consis- tent with Theorem [ AUROC drops as the ro- bustness of the detector increases.
E PROOF OF LEMMA 3
Statement. The total variation between the noisy distributions Ry and F& bounded by the Wasserstein distance of the original distributions R and F as follows:
TV(RS, 4o (W,
Proof. By definition of total variation, we have:
TV(R Fx)
sup P S R
T E El
P Panpelle 10 E]
sup RR ~ (id() E)
102 E]
(definition of R and FX)
sup (o (2) ( E
(e (n 10 E]]
(since >" has marginals R and
sup B (o P 1o
10 E E]
P 2 ((o0)))))))) E Fi]
< sup E =2) (
El
Sy (1( 10 E E]
(since |a + bl S |a| + [b]) < (B () (by definition of total variation) (from Equation I1
< (a)
vo [1(Ø(z)) ((1))ll)) (since th is concave and Jensen's 's inequality) (from definition of of and Equation 10p
(W),RFF)
25

---

Published as a conference paper at ICLR 2024
20.0
Inference d 1%
47.5
Inference a aat a 1%
17.5
12.3
12.3
10.0
7.3
7.3
500
5.0
2.5
DeepFakes
2.5
DeepFakes
FaceSwap
FaceSwap
0.3
0.0
5.0
23
5.0
7.5
10.0
12.3
15.0
17.5
10.0
0.0
233
5.0
2.3
10.0
12.5
25.0
17.5
20.0
Training
Training
(a) ResNet-18
(b) VGG-I6-BN
Detector robustness (inference a at a = 1%) to random noise in the ø latent space increases as the standard deviation of noise used for training increases. Various robust detectors are trained by adding Gaussian noise of standard deviation between 0 and 20 to the ø latent space. Y-axes represent the standard deviation of the noise at inference time on the test dataset for which the detector achieves a 0.01 as per Equation
Figure 18:
F MORE DETAILS ON DEEPFAKE DETECTOR EXPERIMENTS
Theorem 2 provides robustiess-reliability trade-off for deepfake detectors. Figure 8 shows how the AUROC reduces with robustness for different Wasserstein distances based on our bound. Fig- ure [[6] shows how the AUROC reduces with Wasserstein distance for various noise values a. We perform experiments on the FaceForensics++ dataset hosted by Rössler et al., (209)) to empirically verify our theoretical insights. FaceForensics++ (Rössler et al. 2019) is forensic dataset that con- sists of 1000 video sequences that are manipulated using different automated face manipulation techniqued"] For our experiments, we use frames from videos that are manipulated using FaceSwap MMarekKowalste and Deepfakes (deepfakes) FaceSwap manipulations are based on classical com- puter graphics- based methods, while DeepFakes relies set of preprocessing steps to extract aligned 228 x 228 face images from the videos using the Deep- Fakes software We randomly sample 5 frames from each video. We ensure that final image datasets have no overlap of identities hetween the training and test splits. After preprocessing, our FaceSwap image dataset contains 4316 (1059, respectively) original and 3529 (1857, respectively) manipulated images in the training (test, respectively) dataset. Similarly, our DeepFakes image dataset contains 4316 (1059, respectively) original and 3522 (1843, respectively) manipulated im- ages in the training (test, respectively) dataset.
learning- based approach. We perform
We train different detectors with the standard deviation of noise a varied from to 20 with the following objective
N
min 0
((D(((:))+ + n.), v.) N
where é is the cross-entropy loss, n, ~ NN 02), and represent the parameters that defines D. For different detectors, we compute the inference a on the test dataset at which they achieve an of 0.01 using Equation 3 Figure 18 shows that the detector robustness (inference ar at a 1%) to random noise increases as the training sigma increases. We use ten randomly sampled Gaussian noises for each sample ((x:) for this evaluation. Figures 9 and 17 plots the empirical trade-off between AUROC and robustness (a at 1%) for detectors with ResNet-18 and VGG-16-BN backbones, respectively, on the DeepFakes and FaceSwap datasets.
26

---

Published as a conference paper at ICLR 2024
Pece
Original
Poce
E=5 E=1
LA
POEC
1
POCO
BARAP
E 20
Figure 19: We use ResNet-18 and the FaceSwap dataset to visualize images that correspond to noisy latent space features. We optimize Equation 12 find additive noises in the image space that cause large fa perturbations in the latent space Φ. In row, we show the original images from the FaceSwap dataset. The rest of the rows show noisy images that produce perturbations corresponding in the latent space. Here, we show that small additive noises in the image space can lead to large perturbations the á space.
We also visualize how the noisy latent space vectors would look in the image space (see Figure 19 We optimize the following objective to find such images:
min (e - (()())))))))))))))()))))))))(( p(x + 8)112)
(12)
In the above optimization problem, we find an additive noise 5 when added to clean image x leads to & perturbation of in the latent space. As shown in Figure 19 FaceSwap images with small perturbations in the input space can cause large perturbations in the latent space & of ResNet-18.
27


Reviews:
[{"review_id": "XgiPAH8B6N", "reviewer": "Reviewer_pjit", "summary": "In this paper, the authors introduce a diffusion purification attack to break AI-image detection methods using watermarking with a low perturbation budget. For high perturbation image watermarking, they develop a model substitution adversarial attack. Besides, they successfully implemented a spoofing attack by adding a watermarked noise image with non-watermarked ones. Furthermore, they use comprehensive experiments to substantiate the trade-off between robustness and reliability of deepfake detectors.", "questions": "1. In Fig. 6, authors adopt four low perturbation budget watermarking, including DWTDCT, DWTDCTSVD, RivaGAN, WatermarkDM, to validate theorem 1. More watermarking methods are required to be considered, like reference [1]. Many cutting-edge watermarking methods are absent in Fig. 6.\n2. The experimental setup of section 4 is impractical. I suggest authors add noise to spatial images directly instead of interior feature maps. \n3. The title of the paper is “AI-image detectors”. The authors only consider fake facial images in section 4. AI-generated facial detectors are only a small part of AI-image detectors. Therefore, more AI-image detectors for arbitrary contexts require consideration.\n\n[1] MBRS : Enhancing Robustness of DNN-based Watermarking by Mini-Batch of Real and Simulated JPEG Compression", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "A novel watermarking erasing method is proposed to break high perturbation budget watermarking like Tree Ring or StegaStamp.\nThis paper is well-written and easy to understand. The presentation and organization of the paper are good.", "weaknesses": "1.\tThe crux of this paper is theorem 1, which gives a lower bound for the sum of evasion and spoofing errors with regard to the Wasserstein distance between diffusion purification processed images. However, the empirical studies are not sufficient.\n2.\tThe empirical studies for theorem 2 are unpractical. It is almost impossible to add noise to the interior feature maps directly in practical cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eR7KISznZC", "reviewer": "Reviewer_h5mn", "summary": "This work investigates the resilience of AI-image detection methods, focusing on watermarking and classifier-based deepfake detectors. The authors highlight the crucial need to distinguish between authentic and AI-generated content due to the rising threat of fake materials being used as genuine ones. They reveal a fundamental trade-off in the effectiveness of watermarking techniques, showcasing the limitations of low-perturbation and high-perturbation watermarking methods. Specifically, they propose diffusion purification as a certified attack against low-perturbation watermarks and a model substitution adversarial attack against high-perturbation watermarks. Additionally, the paper emphasizes the vulnerability of watermarking methods to spoofing attacks, which can lead to the misidentification of authentic images as watermarked ones. Finally, the authors extend their analysis to classifier-based deepfake detectors, demonstrating a trade-off between reliability and robustness.", "questions": "Please refer to the weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The strengths of this work are as follows:\n1. Comprehensive analysis: The paper provides a comprehensive analysis of the robustness of AI-image detection methods, focusing on both watermarking and classifier-based deepfake detectors. This thorough investigation helps in understanding the limitations and vulnerabilities of these methods.\n2. Practical attacks: The paper introduces practical attacks, such as diffusion purification and model substitution adversarial attacks, to illustrate the vulnerabilities of different watermarking methods. \n3. Clarity in trade-offs: The paper effectively highlights the trade-offs between various aspects of AI-image detection methods, such as the trade-off between evasion error rate and spoofing error rate in the case of watermarking methods. This clarity helps in understanding the challenges associated with designing robust AI-image detection systems.\n4. Sound theoretical study and guidelines for designing robust watermarks: The paper offers insights into the attributes that a robust watermark should possess, including significant perturbation, resistance to naive classification, and resilience to noise from other watermarked images. These guidelines can serve as a valuable reference for researchers and developers working on improving the security and reliability of AI-image detection methods.", "weaknesses": "The experimental results are missing a key element, specifically the PSNR, SSIM, or other image quality metrics comparing the diffusion-purified or adversarially attacked images with the original images. This result is crucial as adversaries aim to eliminate watermarks while preserving high-quality images simultaneously.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PhVze7B7Hf", "reviewer": "Reviewer_1Mes", "summary": "This paper analyzes the robustness of various AI image detectors, including watermarking and classifier-based deepfake detectors, wherein watermarking is considered a promising method for identifying AI-generated images. \n\nThe paper also evaluates the trade-off between evasion error rate and spoofing error rate in watermarking methods, introducing subtle image perturbations. Besides, it is demonstrated that a diffusion purification attack that amplifies the error rates of low perturbation budget watermarking methods, thereby revealing the fundamental limits of the robustness of image watermarking methods. For large-perturbation watermarking methods, the diffusion purification attack is ineffective. Therefore, the authors propose a model substitution adversarial attack to successfully remove watermarks.\n\nOverall, this paper makes significant contributions to the robustness of AI image detection methods, supported by detailed theoretical proofs of the viewpoints presented. Some of the theoretical analyses and attack methods in the article are relatively complex and may require readers to have a certain level of background knowledge to fully understand.", "questions": "1) Problem Description: The experiments conducted in the article primarily utilize the ImageNet dataset, potentially limiting the generalizability of the results. \nRecommendation: In future work, conduct experiments using multiple datasets from various fields and sources to validate the effectiveness and stability of the method.\n\n2) Problem Description: The theoretical analysis presented in the article is quite complex, which may be challenging for all readers to comprehend. \nRecommendation: Provide more intuitive explanations and examples, and simplify some of the theoretical derivations to make them more accessible and easier to understand.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) Originality: The paper challenges the limitations of existing watermarking techniques by proposing new attack methods, driving progress in the field.\n\n2) Quality: The quality of the paper is very high, with in-depth and rigorous theoretical analysis, reasonable experimental design, and results that fully validate the theoretical analysis.\n\n3) Clarity: The structure of the paper is clear, the logic is tight, and the discussion is detailed and easy to understand. However, some of the complex theoretical analyses and attack methods may require readers to have a certain level of background knowledge.\nImportance: By revealing the vulnerabilities of existing watermarking techniques, the paper lays the groundwork for further research and development in the field. Additionally, by proposing new attack methods, the paper challenges the limitations of existing watermarking technologies and promotes progress in the field.", "weaknesses": "1) Complexity of Theoretical Analysis: \nWhile the paper provides an in-depth theoretical analysis, the complexity of these analyses might pose challenges for some readers, especially those who are not familiar with diffusion models and the theoretical underpinnings of adversarial attacks. Certain sections of the paper may appear somewhat opaque to these readers. It would be beneficial if the authors could simplify the explanations or provide additional resources to aid understanding.\n\n\n2) Complexity of Theoretical Analysis: \nWhile the paper provides an in-depth theoretical analysis, the complexity of these analyses might pose challenges for some readers, especially those who are not familiar with diffusion models and the theoretical underpinnings of adversarial attacks. Certain sections of the paper may appear somewhat opaque to these readers. It would be beneficial if the authors could simplify the explanations or provide additional resources to aid understanding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QvOnxAP7cl", "reviewer": "Reviewer_APfc", "summary": "The paper analyzes the robustness of two common AI-generated image detection approaches: watermarking and classification. For watermarking, it points out that diffusion purification can effectively remove low-perturbation budget watermarks but fails to work on high-perturbation budget ones. For the latter one, the paper proposes using a model-substitution adversarial attack to remove the watermarks. It also proposes a spoofing attack against watermarking by blending a watermarked noise image with the non-watermarked target image. Finally, it demonstrates a trade-off between the robustness and reliability of classification-based deepfake detectors.", "questions": "Please refer to the comments in the weaknesses section.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper lacks a clearly identifiable strength.", "weaknesses": "The paper is poorly written and not well-organized. For example, acronyms were defined far after their first usage. Figures are hard to follow and understand.\n \nThe contribution of the paper is unclear. It is common sense pointed out by several previous research that the stronger the perturbations, the more difficult for purification. Using random noise or adversarial perturbations to compromise the machine-learning-based forensics models (watermarking and deepfake detection) has been studied in the past, which is also mentioned in the related work section.\n \nThe experiment settings are insufficient to demonstrate the claimed problems. Besides DiffPure (Nie et al., 2022), there are other diffusion-based approaches, such as DDNM (Wang et al., 2022), and non-diffusion-based ones, which are not investigated.\n \nThe robustness of the proposed attack methods, which is an important property, was not evaluated. DiffPure (Nie et al., 2022), JPEG compression, or Gaussian blur can be used to mitigate such attacks, although it may slightly degrade the clean accuracy.\n\nThe paper should include a paragraph of ethics statement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)