Towards the Detection of Diffusion Model Deepfakes
Authors: Jonas Ricker
Simon Damm
Thorsten Holz
Asja Fischer
Venue: Submitted to ICLR 2023 (2023)
Keywords: Diffusion Model
Generative Adversarial Network
GAN
Deepfakes
Detection
Frequency Artifact
Frequency Analysis
Spectrum Discrepancies
Synthetic Images
Disinformation
Social Media

Abstract: Diffusion models (DMs) have recently emerged as a promising method in image synthesis. They have surpassed generative adversarial networks (GANs) in both diversity and quality, and have achieved impressive results in text-to-image modeling. However, to date, only little attention has been paid to the detection of DM-generated images, which is critical to prevent adverse impacts on our society. While prior works have shown that GAN-generated images can be reliably detected using automated methods, it is unclear whether the same methods are effective against DMs. In this work, we address this challenge and take a first look at detecting DM-generated images. We approach the problem from two different angles: First, we evaluate the performance of state-of-the-art detectors on a variety of DMs. Second, we analyze DM-generated images in the frequency domain and study different factors that influence the spectral properties of these images. Most importantly, we demonstrate that GANs and DMs produce images with different characteristics, which requires adaptation of existing classifiers to ensure reliable detection. We believe this work provides the foundation and starting point for further research to detect DM deepfakes effectively.

Under review as a conference paper at ICLR 2023
TOWARDS THE DETECTION OF DIFFUSION MODEL DEEPFAKES
Anonymous authors Paper under double-bling review
ABSTRACT
Diffusion models (DMs) have recently emerged promising method in image synthesis. They have surpassed generative adversarial networks (GANs) in both diversity and quality, and have achieved impressive results in text-to-image and image-to-image modeling. However, to date, only little attention has been paid to the detection of DM--enerated images, which critical to prevent adverse im- pacts on our society. Although prior work has shown that GAN-generated images can be reliably detected using automated methods, it is unclear whether the same methods are effective against DMs. In this work, we address this challenge and take a first look at detecting DM-generated images. We approach the problem from two different angles: First, evaluate the performance of state-of-the-art detectors on variety of DMs. Second, we analyze DM-generated images in the frequency domain and study different factors that influence the spectral properties of these images. Most importantly, we demonstrate that GANs and DMs produce images with different characteristics, which requires adaptation of existing clas- sifiers to ensure reliable detection. We believe this work provides the foundation and starting point for further research detect DM deepfakes effectively.
INTRODUCTION
In the recent past, diffusion models (DMs) have shown a lot of promise as a method for synthesizing images. Such models provide better (or at least similar) performance compared to generative adver- sarial networks (GANs) and allow powerful text-to-image models such as DALL-E 2 (Ramesh et al.) 2022]. Imagen (Saharia et al. 2022), and Stable Diffusion (Rombach et al. 2022). Advances in im- age synthesis have resulted in very high-quality images being generated, and humans can hardly tell if given picture is an actual or artificially generated image (so-called deepfake) (Nightingale) & Farid 2022). This progress has many implications in practice and poses danger to our digital society: Deepfakes can be used for disinformation campaigns, as such images appear particularly credible due their sensory comprehensibility. Disinformation aims to discredit opponents in pub- lie perception, to create sentiment for or against certain social groups and thus influence public opinion. In their effect, deepfakes lead conspiracy theories, and promote a fundamental political camp formation. Despite the importance of this topic, there is only limited amount of research on effective deepfake detection. Previous work on the detection of GAN-generated images (c.g., Wang et al. (2020). Gragnanicllo et al. (2021). and Mandelli et al. (202aa) showed promising results, but it remains unclear if any these methods can be applied to DM- Regeerrrteed images.
erosion of trust in institutions individuals, support
In this paper. we present the first look at detection methods for DM-generated media. We tackle the problem from two different angles. On the one hand, we investigate whether DM-generated images can be effectively detected by existing methods that claim to be universal. We study ten models in total, five GANs and five DMs. We find that existing detection methods suffer from severe performance degradation when applied on DM-generated images, with the area under the receiver operating characteristic curve (AUROC) metric dropping by 15.2% on average compared to GANs. These results hint at a structural difference between synthetic images generated by GANs and DMs. We show that existing detection methods can be improved by fine-tuning, which makes detection almost perfect. However, our results also suggest that recognizing DM- -enerrete images more difficult task than recognizing GAN images.

---

Under review as a conference paper at ICLR 2023
On the other hand, we analyze DM-generated images in the frequency domain and compare them to -enggreee images. Although DMs do not exhibit strong frequency artifacts compared to GANs, their spectrum deviates from real images. We hypothesize that discrepancies in spectral properties are a possible reason for our identified differences. Therefore, we analyze the spectral properties of DM- and GAN-generated images in detail and find that high frequencies atically mismatched. Further analysis suggests that too little weight is given to these frequencies during training due the choice of the training objectíve. We believe that our results provide the foundation for further research on the effective detection of deepfakes generated by DMs.
system-
2 RELATED WORK
Universal Fake Image Detection While in recent years a variety of successful methods to detect artificially generated images has been proposed (Verdoliva 2020), generalization to unseen data remains a challenging task (Cozzolino et al. (219)). Constructing effective classifier for specific generator is considered straightforward. which is why more research effort is put into designing universal detectors (Xuan al. 2019 Chai et al. 2020; Wang et al. 2020 Cozzolino et al. 2021 Gragnaniello et al. 2021 Girish et al., 2021] Mandelli et al. 2022a). This is especially important in the context of deceptive media, since new generative models emerge
frequent basis and manually updating detectors is too slow to stop the propagation of harmful contents.
Frequency-Rand -aased Deepfake Detection Zhang al. (2019) were the first to demonstrate that the spectrum of GAN-generated images contains visible artifacts in the form of a periodic, grid-like pattern due to transposed convolution operations. These findings were later reproduced by |aang et al. (020)) and extended to the discrete cosine transform (DCT) by Frank et al. (2220). Another characteristic was discovered Syyaal et al. (2020). who showed that GANs are unable to correctly reproduce the spectral distribution of the training data. In particular, generated images contain in- creased magnitudes at high frequencies. While several works attribute these spectral discrepancies to transposed convolutions (Zhang al. 2019 Durall al. 2020) or, more general, up-sampling operations (Frank et al., 2020: Chandrasegaran et al. 2021 no consensus on their origin has yet been reached. Some works explain them by the spectral bias of convolution layers due linear dependencies (Dzanic et al. 2020. Khayatkhoei & Elgammal (022). while others suggest the dis- criminator not able to provide an accurate training signal (Chen et al. 2021 Schwwrz al., (02I).
Detection of DM-Generated Images Despite the massive attention from the scientific community and beyond, DMs have not yet been studied from the perspective of image forensics. A very specific use case is considered by Mandelli al. (2022b), where the authors evaluate methods for detecting western blot images synthesized by different models, including DDPM (Ho et al. 2020 More related to our analysis is the work of Wolter et al. (2022). which proposes detect generated images based on their sevelet-packke representation, combining features from pixel- and frequency space. While the focus lies on GAN-generater images, they demonstrate that the images generated by ADM (Dhariwal & Nichol, 2021) can be detected using their approach. They also report that the classifier "appears to focus on the highest frequency packet", which consistent with our findings.
Frequency Considerations in DMs Both]Kingnn al. 2021 and al. (2022b) experiment with adding Fourier features to improve learning of high-frequency content, the former reporting leads much better likelihoods. Another interesting observation made by Rissanen et al. (2022) who analyze the generative process of diffusion models in the frequency domain. They state that diffusion models have an inductive bias according to which, during the reverse process, higher frequencies are added to existing lower frequencies.
3 BACKGROUND ON DMS
DMs were first proposed by Sohl-Dickstein et al. (2015) and later advanced by Ho al. 2020))
who also pointed out the connections between DMs and score-based generative models (Song &
Ernon 2019: 2020 Song et al. 2022b). Since then, numerous modifications and improvements
have been proposed, leading to higher perceptual quality (Nichol & Dhariwal 2021 Dhariwal &
Nichol 2021 Choi al. 2022 Rombach al. 2022) and increased sampling speed (Song al.
2

---

Under review as a conference paper at ICLR 2023
2022a Liu et al. 2022 Salimans & Ho 2022] Xiao et al. 2022). In short, a DM models data distribution by gradually disturbing a sample from this distribution and then learning to reverse this diffusion process. In the diffusion (or forward) process, sample Xo (an image in most applications) is repeatedly corrupted by adding Gaussian noise in sequential steps = T.
q( g(x1|xx-1) (xxx v Bex, 1, 1,,
where Br defines a noise schedule. With oz == 1 - f3 and d+ == IT 1 Be), we can directly create a corrupted sample ze via
-
(xx/|x0) = (xx: vā,xo, (1-a,)]
(2)
During the denoising (or reverse) process, aim sample from g (xx,) to obtain clean image given Xr. However, since q (x)) is intractable as it depends on the entire underlying data dis- tribution, it is approximated by deep neural network. More formally, (xx=) is approximated by
(x xx) = (i (X,, t). Eg
(3)
where mean jág and covariance Eg are given by the output of the model (or the latter is set
constant proposed by Ho et al. (2020)) Predicting the mean of the denoised sample [x,
conceptually equivalent to predicting the noise that should be removed, denoted by eg Xx. l).
Predominantly, the latter approach is implemented (c.g., by Ho et al. (2020); Dhariwal & Nichol (202I) such that training a DM boils down to minimizing a (weighted) mean-squared error (MSE)
[E Eo (x,, t) between the true and predicted noise.
GENERATIVE MODELS UNDER EVALUATION
In contrast previous work, we evaluate all generative models trained on the same dataset. This avoids any biases due to the complexity of the dataset when estimating the accuracy of deep fake detection methods, e.g., high detection accuracy for model trained on complex dataset that cannot be modeled as well as simpler dataset. It also provides fair conditions for evaluating fine-tuned detector's 's ability to generalize to unseen data. addition, having only single dataset beneficial for frequency analysis. Although natural images have been shown to have similar power spectra across different domains (Burton & Moorhead 1987 Field 1987; 1999; [Tolhurst et al.) 1992 Torralba & Oliva 2003). model may be able to reproduce the frequencies of one dataset better than another.
We evaluate models trained on LSUN Bed- room (Yu et al., (016)) due to the availabil- ity of pre-trained models and/or samples for many state-of-the-art GANs and DMs. This dataset also has the advantage that no resizing operations (besides cropping I are required, which are potential source of error due to faulty implementations (Parmar et al.) 2022) or mismatch between pre-pree cee-processin for real and generated images (Chai et al., 2020). To demonstrate that our findings are not dataset specific, we show results on additional data in Appendix(D)
Table 1: Models evaluated in this work. Fréchet inception distances (FIDs) (Heusel et al.) 2017) on LSUN Bedroom 256x 256 are taken from the original publications and from Dhariwal & Nichol (202I) in the case of IDDPM. A lower FID corre- sponds to higher image quality.
Madel Class
Method
Publication
FID
PraGAN
Karrs
2015
8.34
StyleGiAN
GAN
ProjecteGAAN
Siaer 21
A191
1.52
Diff-Syl:GIAN2
Waner 21 m1
71777
3.65
Diff ProjecueGANN
4124
1.43
DOPM
Ho
2000
6.36
IDDM
Dinntwal
2021
4.24
An overview of the models under evaluation shown in Table I. and example images are given in Appendix A We consider data from ten models in total, five GANs and five DMs. This includes the seminal models ProGAN and StyleGAN, well as the more recent ProjectedGAN. Note that the recently proposed methods, SafftussionnSyyllGAAAl
DM
ADM
1.90
PNDM
Lib 21
5.68
LDM
Kombazz
2022
2.95
(the current state of the art on LSUN Bed- room) use a forward diffusion process optimize GAN training. but this does not change the GAN model architecture. From the class of DMs, we consider the original DDPM, its successor IDDPM,
and
All images have the smaller dimension equal 256 pixels.
3

---

Under review as a conference paper at ICLR 2023
and ADM, the latter outperforming several GANs with FID of 1.90 LSUN Bedroom. PNDM optimizes the sampling process by the factor of 20 using pseudo numerical methods, which can be applied to existing pre-trained DMs. Lastly, LDM uses adversarially trained autoencoder that transforms an image from the pixel space to a latent space. Training the DM in this more suitable latent space reduces the computational complexity and therefore enables training on higher resolu- tions. The success of this approach is underpinned by the recent publication of Stable Diffusion powerful and publicly available text-to-image model based on LDM. An interesting detail is that the autoencoder used to transform images is very similar to VQGAN (Esser et al. 2021
All samples have dimensions x 256 and are either directly downloaded or generated using code and pre- pre-trained models provided by the original publications. More detailed descriptions are given in Appendix [A] For each model, we collect 50k samples total from which 39k are used for training, 1k for validation, and 10k for testing and frequency analysis, if not stated otherwise.
5 DETECTION OF DM-GENERATED IMAGES
We now investigate the effectiveness of state-of-the-arr fake image detectors on ages. First, we analyze the performance of off-the-shelf pre-trained models which are highly capable of detecting images generated by GANs, and then perform experiments with models fine-tuned on DM-generated images. We also show example images considered more or less "fake" based on the detector's output in Appendix B.5
im-
In this work, we evaluate three state-of- weffthhe-n art detection methods by Wang et al. (2020). Grag naniello et al. (202T), and Mandelli et al. (2022a). A description of these detectors provided in Appendix B. All three claim to perform well on unseen data, but it is unclear whether this holds for DM-ggnerated images as well.
The performance of the analyzed classifiers is estimated in terms of the widely used AUROC. As pointed out by Cozzolino al. (2021), however, the AUROC tends to be overly optimistic since captures merely the potential of a classifier, but the optimal threshold is usually unknown. There- fore, we adopt the use of the probability of detection additional metric, which is given as the true positive rate at a fixed false positive rate. Intuitively, this corresponds to picking the y-coordinate of the ROC curve given an x-coordinate. This metric is valid choice for realistic scenarios such large-scale content filtering on social media, where only a certain amount of false positives is tolerable. We consider FARs of 5% and 1%.
fixed false alarm rate (Pd@FAR) as an
5.1 CAN USE STATE-OF-THE-AS DETECTORS FOR DM-GENERATED IMAGES?
Our first analysis aims to answer the question whether DM-generated images are sufficiently similar, with respect to detectable traces, to those generated by GANs.
Table 2: Detection performance of pre-trained universal detectors. For Wang et al. (020)) and Gragnaniello et al. (021). we consider two different variants, respectively. The best score (deter- mined by the highest Pd @ 1%) for each generator is highlighted in bold. We also report average scores per detector and model class in gray.
Wang 2020
Omgranicllo
Mindelli
2027a
AUROC// 5% PS//
Blur+JPEG
Blur-IPEG
ProGAN
StyleGAN2
PoNAAN
100.0/ 100.0/ 180J)
100.0/ 100.0 100,0
100.0/ 100,0/ 100,0
100.0/ 100,0/ 100.0
91.2/
54.6/
27.5
StyleGAN
93.7/ 81.4
95.57 84.4
100.0/ 100.0/ 100.0
100.0
43.67
ProjecteiCAAN
94.87 23.8/ 49.1
90.97
61.57 34.5
100.0/
99.3
99.91
99.67 97.8
59.4
2.4
Dif-Styk:GAN2
9307
99.6/ 97.9
1000//
99.9 99.3
100.0/ 100.0/ 100.0
100.0/ 100,0/ 100.0
100.//
180.0
99.9
Duff-PajecteaiiAN
95.87 69.57 43.3
88.8
54.67 27.2
99.9 99.9/ 99.1
99.8 99.57 96.6
62.1/
10.5
2.8
Average
100.0/
45.47
DDPM
85.2/
37.8/
14.2
80.8/
29.67
9.3
96.51
79.4/
39.1
95.1/
69.51
30,7
57.47
3.8/
0.6
IDDPM
1.3
ADM
68.3/
1
3.4
68.8/
/
4,0
77.81
20.7/
5.2
70.6
13.01
2.5
60.5
8.2
1.8
PNDM
79.07
27.5/
9.2
75.5/
22.61
6.3
91.61
52.07
16.6
91.51
$3.97
22.2
71.67
15.4/
4.0
97.0/
Average
78.67
26.87
61.17
2.0
http:::/ggtthb..comm////o
on
4

---

Under review as a conference paper at ICLR 2023
Using the dataset introduced in Section analyze the performance of the three state-of-the- art detectors. We use pre-trained models provided by the authors without any modifications and compute the evaluation metrics using 10k real and 10k generated images. The results are given in T For GeA-ggenerate images, both classifiers provided by Gragnaniello et al. (2021) achieve almost perfect detection results across all five generators, even w.r.t. the challenging Pd @ % metric. For images generated by ProjectedGAN and Diff-ProjectedGANN the variant trained on ProGAN images performs slightly better. The reduced scores for these two models could in part be explained by the high image quality indicated by low FIDs (see Table I. In comparison, the performance of the classifier provided by Wang et al. (2020) is reduced (except for ProGAN, on which the model was trained), which confirms the findings of Gragnaniello et al. (202T). Similar to Gragnaniello et al.) 2222T, ProjectedGAN and Diff-ProjectdGGAN achieve the lowest scores, with Pd@ 1% dropping below 50%. Except for Diff-StyleGii Mandellll Mandelli et
achieves significantly worse results than its contestants. A possible explanation could be that StyleGAN2 images make up a large portion of the training data for this classifier.
(2022a)
For images generated by DMs, we observe a sharp deterioration for all detectors, with AUROC dropping by 15.2% on average in comparison to GANs. The difference becomes even more severe when looking at the more realistic Pd@ FAR metrics. While the best model by Gragnaniello et al.) (202)) (trained on ProGAN images) achieves a Pd@ 5% of 100.0% and Pd@ 1 1% of 99.7% on GAN- generated images, the scores drop by 40.7% and 74.0% for DMs, respectively. Given a setting where a low FAR is required, we find that none of the detectors under evaluation are usable. Among the five DMs, ADM generated the images most difficult to detect, which corresponds to low FID. However, images from LDM, which achieves the second lowest FID, can be detected relatively effectively. A potential reason for this could be the adversarial training procedure of LDM (see Section 4 which could result in properties similar to those of GAN -egerrtee images.
Overall, we observe that pre-trained universal detectors suffer from severe performance drop when applied to DMMgenerated images (compared to GANs). We conclude that there exists structural difference between GAN- and DM-generated images that requires the adaptation of existing methods or the development of novel methods to ensure the reliable detection of synthesized images. In Appendix B.4 we also experiment with common image perturbations and show that these have stronger effect on DM-generated images.
5.2 CAN WE IMPROVE DETECTION BY FINE- TUNING?
Given the findings presented above, the question remains whether DMs evade detection in principle, or whether the detection performance can be increased by fine-tuning P We select Wang et al. (2020) as a baseline since the original training code is available which guarantees using the same train- ing procedure. Furthermore, we choose the configuration Blur+JPEG (0.5) as it yields slightly better scores on average, and use the validation set for adapting the learning rate and performing early stop- ping. Besides fine-tuning a model on data from each generator, we also consider three aggregated settings in which we fine-tuned on all images generated by GANs, DMs, and both, respectively.
The results are depicted in Figure I A model which is fine-tuned on images from a specific DM achieves near-perfect scores, with PD @ 1 % ranging from 97.7% for ADM to 100.0% for PNDM and LDM. To a limited extent, models are also capable of generalizing to data from other DMs, indicat- ing common, detectable characteristics. Interestingly, detectors fine-tuned on images from DMs are significantly more successful in detecting GAN-generan for models fine-tuned using images from multiple GANs and DMs, respectively. We hypothesize that detectors trained solely on GAN-generated images focus mostly on GAN-specific (arguably more prominent) patterns, while detectors trained on DM-generated images focus on common pat- terns. This is supported by frequency (see Section 6 and feature space analyses (see Appendix B.3
images than vice versa. The same holds
In addition, the results demonstrate that training a detector which is capable of correctly classifying both GAN- and DM-generated images is clearly possible (see rightmost column of Figure Ta]. While this finding is promising, is unclear whether similar performance can be achieved in the model- agnostic setting addressed by Wang et al. (2020).
In addition, we carry out the same evaluation for detectors trained from scratch, which leads to very similar results, as shown in Appendix B.2 https: / com
5

---

Under review as a conference paper at ICLR 2023
Fine-tuned on
Fing-tuned on
M
Loff Plgees pppeenatii
Proernatint
Exii bertie N42
Exiti Pegge Bedtie
ProciAy
Speecit
DAPS
AnM
GAN
ProCAA
ADM
PhT
Lan
ProGAN 1000 590
10CD
1000
102L
19.5
1000
55.
1000
PoGAAN
92.3
59.4
120.0
29.0
$5.0
N6.3
$1.6
$5.00
1000
90.4
1000
SkyleDAN
PagjonoGiAN
93.
61X
100.0
94
103
90.
91.
56.2
90.5
36.5
1000
1000
ProjcesSEAN
462
36
1000
1000
43.-
51
06
1000
SqyeeAAN2
1000
59.1
Styl-S
Tested 0B
Part Paj.aajjaNNGG
41.
100.0
09.5
Toseed
100
DDPX
924
51.5
85.0
1000)
55.5
90.7
59.1
18:2
12.4
1000
DDPM
100
43
190
TOCOO
ADM
555
+1.3
13.0
59.
96.
9399
95.0
1905
1000
ADM
95.
09.7
PNDM
LDM
c8.1
77.6
17.6
59.2
99.5
546
1020
17.9
1000
100
LDM
92.
110 2010 1000 1000
(a) AUROC
(b) PD@1%.
Figure 1: Detection performance for fine-tuned detectors. The columns GAN, DM, and All cor- respond to models fine-tuned on samples from all GANs, DMs, and both, respectively.
ANALYSIS OF DM-GENERATED IMAGES IN THE FREQUENCY DOMAIN
The visual quality of synthesized images has reached a level that makes them practically indis- tinguishable from real images for humans (Nightingale & Farid, 2022). However, in the case of GANs, the same does not hold when switching the frequency domain (Zhang al. 2019). this section, we analyze the spectral properties of DM-generated images and compare them to those of GAN-generated images. We use three frequency transforms that have been used successfully in both traditional image forensics (Lyu 2013) and deepfake detection: discrete Fourier transform (DFT), discrete cosine transform (DCT), and the reduced spectrum, which can be viewed as a ID repre- sentation of the DFT. While DFT and DCT visualize frequency artifacts, the reduced spectrum can be used to identify spectrum discrepancies. The formal definitions of all transforms are provided in Appendix I Furthermore, we evaluate the influence of different parameters during training and sampling of DMs and hypothesize on the origin of spectrum discrepancies.
6.1 SPECTRAL PROPERTIES OF DM- VS GAN-GENERATED IMAGES
Figure depicts the average absolute DFT spectrum of 10k images from each GAN and DM in our dataset Before applying the DFT, images are transformed grayscale and, following previous works (Marra et al. 2019: Wang et al.] 2020), high-pass filtered by subtracting a median-filtered version of the image. For all GANs we observe sigmificant artifacts, predominantly in the form of a regular grid. Taking the reduced spectra in Figure Ba into account we confirm the previously reported elevated high frequencies (see Section 2). For ProjectedGAN and Diff- Diff-ProjecteGGAN, this characteristic is also visible. However, the spectral density does not exceed that of real images. Diff Stff-StyleGA22 is an exception as shows slight decrease towards the end of the spectrum.
In contrast, the DFT spectra of images generated by DMs (see Figure 2b). except LDM, are signif- icantly more similar to the real spectrum with almost no visible artifacts. LDM, however, exhibits a thin grid across its spectrum, as well as the increased amount of high frequencies which is char- acteristic for GANs (see Figure [b).. As mentioned in Scction 4 the architecture of LDM differs from the remaining DMs as the actual image is generated using adversarially trained autoen- coder, which could explain the discrepancies. This supports previous findings which suggest that the discriminator is responsible for spectrum deviations.
Although the reduced spectra of the remaining DMs do not exhibit the GAN-like rise towards the end of the spectrum, several DMs deviate stronger from the spectrum of real images than GANs. Especially for ADM, IDDPM, and DDPM, we observe an underestimation of the spectral density which becomes stronger towards higher frequencies (relative to the overall spectral density level). In Section 6.3 we discuss potential causes.
We also report the DCT spectra of both GANs and DMs in Appendix C.2 which produce similar findings.

---

Under review as a conference paper at ICLR 2023
Real
ProGAN
StyleGAN
ProjectedGiA
Din--SyieGAN2
DII-ProRRIIUANN
(a) GANs
Real
DDPM
IDDPM
ADM
PNDM
LDM
(b) DMs
Figure 2: Mean of DFT spectrum from real and generated images. To increase visibility, the
color bar is limited to [10 100 '], with values lying outside this interval being clipped.
109
10
10
101
201
Spectral Deasty
Spectral Density
10
109
20
10
Real PRIGAN
SykGAN
StleGANE
10
Real
ID0PM
PNOM
10
10
fi
fl
(a) GANs
(b) DMs
Figure Mean reduced spectrum from real and generated images. The higher end of the spectrum where GAN CAN-characteristic discrepancies occur is magnified.
6.2 ARE DM-GENERATED IMAGES EASIER TO DETECT IN THE FREQUENCY DOMAIN?
So far showed that although DMs do not exhibit strong frequency artifacts, their spectrum de- viates from that of real images. natural question is therefore whether these discrepancies can be used detect DM-generated images more effectively. Following Frank et al. (2020), we perform simple logistic regression on each dataset with different transforms: Pixel (no transform). DFT (and taking the absolute value), and DCT. We 20k samples for training, 2k for validation, and 20k for testing, each set equally split between real and fake images. To reduce the number of features, transform all images to grayscale and take center crop with dimensions 64x64. Additionally, all features are independently standardized to have zero mean and unit variance. We apply L2 regu- larization and identify the optimal regularization weight by performing grid search over the range 100* h E (-4,- .......4)
The accuracy of GANs and DMs in our dataset is given in Table 3 We also report the results for log-scaled DFT and DCT coefficients, as this leads to significant improvements for some generators. For both GANs and DMs, using information from the frequency domain increases classification accuracy. On average, the performance gain of the best transform compared 5.72% and 10.6%, respectively. Although the gain for DMs is more than double that for GANs, the overall maximum accuracy is significantly lower (90.9% for GANs and 63.1% % for DMs average). Therefore, we do not conclude that DMs exhibit stronger discriminative features in the frequency domain compared GANs.
transform is

---

Under review as a conference paper at ICLR 2023
Table Accuracy of logistic regression on pixels and different transforms. Next each trans- form column we report the gain compared to the accuracy on pixels.
Pixel
DFT
logrL3FT)
DCT
logtDCTT)
PoGAN
64.8
74.7
+9.9
73.6
+7.8
65.4
+0.6
74,9
+10.1
PrjjcledGAN
90.0
90.8
90.3
+0.3
91.0
+1.0
95.3
+5.3
DHI-SSylaGANN
92.4
80.3
-12.0
90.5
-11.9
93.8
+14
87.6
-4.7
DiE PrejecteGAAN
87.4
93.9
+6.5
92.1
+5.7
88.1
+0.5
97.7
+10.3
DUPM
51.7
64.2
+12.6
64.2
+12.5
52.4
10.7
64.3
+12.
IDDMM
51.6
61.7
52.3
50.1
+3.7
PNDM
32.5
57.0
14.3
58.2
45.3
51.1
-1.4
61.4
+8.9
LDM
26.5
63.7
+7.1
66.1
+9.3
58.5
+1.9
73.0
+16,3
file
fibes
(a) St 1000
(b)0 100
Figure 4: Evolution of spectral density error throughout the denoising process. The error is computed relatively to the spectrum real images. We display the error for (a) all sampling steps and (b) close-up of the last 100 steps. The colorbar is clipped at and 1.
6.3 INVESTIGATING THE SOURCE OF SPECTRUM DISCREPANCIES
As explained in Section several recent works attempted to identify the source of spectrum dis- crepancies in GAN-generated images. In this section, we make an effort to launch a similar line of work for DMs. For the experiments, used code and model from ADM (Dhariwal Nichol, 2021 pre-trained on LSUN Bedroom. Spectra are generated using 512 grayscale images.
Spectrum Evolution During the Denoising Process Our first analysis targets the denoising pro- cess. We generate samples at different time steps and compare their average reduced spectrum to that of 50k real images. The results are shown in Figure We adopt the figure type from Schwarz et al. (2021) and depict the relative spectral density error Sar Stake/ S bar clipped at and 1. At t = 1000, the image is pure Gaussian noise, which causes the strong spectrum deviations. Around
with the color-
300, the error starts decrease, but interestingly it appears to not continuously improve until / = 0. With regard to high frequencies, we observe an optimum at approximately 10 followed by increasing underestimation. In Appendix
C.3 additionally provide the spectral density error between the denoising process and the diffusion process at the cor- responding step. which further visualizes this behavior. However, at
10, the images are casier to detect and also visually noisier (see Appendix Figures I6 and 17) Thus, stopping the denoising process early is not effective means to make synthetic images less detectable.
We hypothesize that the underestimation towards higher frequen- cies due training objective that is suboptimal for captur- ing high frequency content. As noted by Nichol & Dhariwal (2021), the objective proposed by Ho et al. (2020), Lximple [lke Exxxo.elle
MSP
co(x,, 2), does not lead to good likelihel values. These depend on high-frequency details synthesized at the challenging denoising tasks near t (Kingma et al. (222)) In contrast, the classical variational lower bound Lvth forces the model to focus on the denoising steps towards = 0. How- Lylb is extremely difficult to optimize (see e.g., Nichol &
10
300
100
400
800
1000
Figure 5: Mean and standard deviation of the mean-squared error at different steps.
Dhariwal (202)) )) the modified objective Layhrid = Lsimpte XL
(with À 0.00T) is proposed by (Nichol & Dhariwall 2021) and
8

---

Under review as a conference paper at ICLR 2023
100 500 200 100 30
1000
1.0
Denoising Steps
Denossing Steps
500
200
100
010
0.0
0.5
0.5
1.0
1.0
file
(a) Without DDIM
(b) With DDIM
Figure 6: Evolution of spectral density error for different numbers of denoising steps. The error is computed relatively to the spectrum of real images. The colorbar is clipped at -1 and 1. Note that the y-axis is not scaled linearly.
also employed by (Dhariwal & Nichol 2021 at different t (see 5). we observe steep increase near = O, suggesting the model struggles with accurately removing the small perturbations which correspond to high frequencies. We con- clude that the training objective used in state-of-the-arr DMs enables impressive perceptual image quality (also regarding FID), but struggles to accurately model the high-frequency content of real images which in turn can be exploited for detection. We elaborate in Appendix C.5
And indeed, when looking the mean-squared error
Effect of Number of Sampling Steps In second step, we analyze how varying the number of sampling steps affects the spectral properties of generated images. While DMs are usually trained using T 1000 steps, sampling speed can be increased by reducing the number of steps (Nichol & Dhariwal ). For ADM on LSUN Bedroom, however, images were sampled using 1000 steps
since this led to much better results (Dhariwal & Nichol, 2021). The authors also evaluate DDIM Song et al. 2022a), an alternative sampling method which allows for sampling with fewer steps.
Using the previous experimental settings, we analyze the reduced spectra of images sampled using different numbers of steps (Figure 6). With only a few steps, the spectral density is too low across the entire spectrum, resulting blurry images with little contrast (see Appendix C.4.. Performing more steps leads to lower eπrоrs, with DDIM improving faster than normal sampling. This finding coherent with previous results, as DDIM achieves better samples at fewer sampling steps (Song et al., (022a).. In Appendix (C.nnndxxx..4jcc we also analyze the detectability with different numbers of sampling steps. For both sampling methods, images become easier to detect with fewer sampling steps.
7 CONCLUSION
In this work, we make much-needed first step towards the detection of DM-generated images. We find that existing detectors, which claim synthesized by different DMs. This hints at fundamental differences between images generated by DMs and GANs, one of which we identify as different spectral propertics. While DMs produce little to no frequency artifacts, there appears to be systematic underestimation of high frequen- cies. Our hypothesis is that during training, too little weight is attached to these frequencies due to the choice of the training objective. However, this is appropriate choice because correctly reproducing high frequencies is less important to the perceived quality of generated images than matching lower frequencies. Whether spectral discrepancies can be reduced (e.g., by adding spec- tral regularization (Durall et al., (0200) without impairing image quality should be the subject of future work. Most importantly, more efforts are needed develop novel detection methods, both universal and DM-specific For tackling this task, a sufficiently large number of DM-generated im- ages is necessary, which poses a challenge due to the slow generation speed compared to GANs. We therefore urge researchers presenting novel generation methods to publish sample images alongside pre-trained models, to enable the development and evaluation of detection methods without the need for immense computational resources. This is especially relevant due to the emergence of powerful text-to-image image models based on DMs like DALL-E (Ramesh et al., 2022). Imagen (Saharia et al. 2022), and Stable Diffusion (Rombach et al. 2022). We hope that our work can spark further search in this direction and emphasizes the fact that advances in realistic media synthesis should always be accompanied by considerations regarding deepfake detection methods.
universal, fail effectively recognize images

---

Under review as a conference paper at ICLR 2023
ETHICS STATEMENT
While generative models have many beneficial applications, deepfakes pose serious threat to soci- ety by undermining trust in media and digital communication (e.g., via fake profiles in social media that are used in misinformation campaigns). This work explores the challenges of deepfake detec- tion for the novel promising class of diffusion models. We believe that the urge for reliable deepfake detection - ideally agnostic to the underlying generative model - is amplified by the ever casier use of generative models (c.g., via text-to-image synthesis). By highlighting the (current) lack of "uni- versal" detectors, we strongly encourage the community to address this shortcoming, and our results starting point for future work in this direction. While insights from such research could potentially be used to exacerbate or eventually elude detection, we strongly believe further insights this area are much needed in order to sustain identifiability of deepfakes.
REPRODUCIBILIIT STATEMENT
To ensure reprodueibility, we provide instructions to recreate the dataset in Appendix [A] and, for the additional datasets, in Appendix D All datasets considered in this work are publicly available. We train/fine-tune the detector proposed by Wann et al. (020)) using the settings provided by the authors and describe training and validation data in the text (Section (5.2). All tables and figures can be reproduced using the source code provided in the supplementary materials.
REFERENCES
G. Burton and Ian R. Moorhead, Color and spatial structure in natural scenes. Applied Optics, 26 (1):157-170, January 1987.
Lucy Chai, David Bau, Ser-Nam Lim, and Phillip Isola. What makes fake images detectable? Understanding properties that generalize. In Computer Vision ECCV 2020, Lecture Notes in Computer Science, pp. 103-120, 2020.
Keshigeyan Chandrasegaran, Ngoc-Trung Tran, and Ngai-Man Cheung, A closer look fourier spectrum discrepancies for CNN-generated images detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7200-7209, 2021.
Yuanqi Chen, Ge Li, Cece Jin, Shan Liu, and Thomas Li. SSD-GAN: Measuring the realness in the spatial and spectral domains. Proceedings of the AAAI Conference on Artificial Intelligence, 35 (2):1105-1112) May 2021.
Jooyoung Choi, Jungbeom Lee, Chachun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11472-11481, 2022.
Davide Cozzolino, Justus Thies, Andreas Rössler, Christian Riess, Matthias Nießner, and Luisa Verdoliva. ForensicTransfer Wcakly-supervised domain adaptation for forgery detection. 1812v.1812.022 10 [es], November 2019.
Davide Cozzolino. Diego Gragnaniello, Giovanni Poggi, and Luisa Verdoliva. Towards universal GAN image detection. In 202/ International Conference Visual Communications and Image Processing (VCIP), pp. 1-5, December 2021.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs image synthesis. In Advances in Neural Information Processing Systems, volume 34, pp. 8780-8794, 2021.
Ricard Durall, Margret Keuper, and Janis Keuper. Watch your up- up-convolution: CNN based gener- ative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Tarik Dzanic, Karan Shah, and Freddie Witherden. Fourier spectrum discrepancies in deep network generated images. In Advances in Neural Information Processing Systems, volume 33, pp. 3022 3032, 2020.
10

---

Under review as a conference paper at ICLR 2023
Patrick Esser, Robin Rombach, and Bjöm Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 12873-12883, 2021.
D. Field. Wavelets, vision and the statistics of natural scenes. Philosophical Transactions: Math- ematical, Physical and Engineering Sciences, 357(1760)2227-2542. 1999.
David J. Field. Relations between the statistics of natural images and the response properties of cortical cells. JOSA A, 4(12):2399-2394, December 1987.
Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recogmition. In Proceedings of the 37th International Conference on Machine Learning, volume 1 119 of Proceedings of Machine Learning Research, pp. 3247-3258, July 2020.
Krzysztof J. Geras and Charles Sutton. Scheduled denoising autoencoders. In 3rd International Conference on Learning Representations, ICIR 2015, San Diego, CA, USA, May 7-9, 2015, Con- ference Track Proceedings, 2015.
Krzysztof J. Geras and Charles Sutton. Composite denoising autoencoders. In Machine Learning and Knowledge Discovery in Databases, Lecture Notes in Computer Science, pp. 181-6886666666 2016.
Sharath Girish, Saksham Suri, Sai Saketh Rambhatla, and Abhinav Shrivastava. Towards discovery and attribution of open-world GAN generated images. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 14094-14103 2021.
D. Gragnaniello, D. Cozzolino, F. Marra, G. Poggi, and L. Verdoliva. Are GAN generated images easy detect? A critical analysis of the otate-of-the-art. 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, July 2021.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sa test. Journal of Machine Learning Research, 13(25):723-773, 2012.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEFE Conference Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge a local nash equilibrium. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Pro- cessing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6626-6637, (166-6667,2117 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistie models. In Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851, 2020.
Nils Hulzebosch, Sarah Ibrahimi, and Marcel Worring. Detecting CNN-generated facial images in real-world scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 642-643, 2020.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for im- proved quality, stability, and variation. In International Conference on Learning Representations, 2018.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. Proceedings of the IÉEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4401-4410, June 2019.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyz- ing and improving the image quality of StyleGAN. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Mahyar Khayatkhoei and Ahmed Elgammal. Spatial frequency bias in convolutional generative adversarial networks. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7):1152- 7159, June 2022.
11

---

Under review as a conference paper at ICLR 2023
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In Advances in Neural Information Processing Systems, volume 34, pp. (169--17707,20 2021.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations, February 2022.
Zhengzhe Liu, Xiaojuan Qi, and Philip H. S. Torr. Global texture enhancement for fake face detec- tion in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8060 8069, 2020.
Siwei Lyu. Natural image statistics in digital image forensics. Digital Image Forensics: There Is More Picture than Meets the Eye, pp. 239-256. 2013.
Sara Mandelli, Nicolò Bonettini, Paolo Bestagini, and Stefano Tubaro. Detecting GAN-generated images by orthogonal training of multiple CNNs. arXiv:2203. 02246 [cs], March 2022a.
Sara Mandelli, Davide Cozzolino, Edoardo D. Cannas, João P. Cardenuto, Daniel Moreira, Paolo Bestagini, Walter J. Scheirer, Anderson Rocha, Luisa Verdoliva, Stefano Tubaro, and Edward J. Delp. Forensic analysis of synthetically generated western blot images. IEEE Access, 10:59919- 59932, 2022b.
Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi. Do GANs leave artifi- cial fingerprints? In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 506-511, March 2019.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Proceedings of the 38th International Conference Machine Learning, pp. 8162-8171, July 2021.
Sophie J. Nightingale and Hany Farid. Al-synthesized faces are indistinguishable from real faces and more trustworthy. Proceedings of the National Academy of Sciences, 119(8), February 2022.
Gaurav Parmar, Richard Zhang, and Jun Yan Zhu. aliased resizing and surprising subtleties in GAN evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11410-11420, 2022.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text- conditional image generation with CLIP latents, April 2022.
Severi Rissanen, Markus Heinonen, and Arno Solin. Generative modelling with inverse heat dissi- pation, July 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115 (3):211-22) 2522, December 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kam- yar Seyed Ghasemipour, Bureu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Sali- mans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffu- sion models with deep language understanding, May 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference Learning Representations, March 2022.
Axel Sauer, Kashyap Chitta, Jens Müller, and Andreas Geiger. Projected GANs converge faster. In Advances in Neural Information Processing Systems, volume 34, pp. 17480-17492, 2021.
Katja Schwarz, Yiyi Liao, and Andreas Geiger. On the frequency bias of generative models. In
Advances in Neural Information Processing Systems, volume 34, pp. 181 18126618136, 2021.
12

---

Under review as a conference paper at ICLR 2023
Jascha Sohl-Dickster Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Con- ference Machine Learning, pp. 2256-22631 June 2015.
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- lional Conference on Learning Representations, February 2022a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, volume 32, 2019.
Yang Song and Stefano Ermon. Improved techniques for training score- score-based generative models. In Advances in Neural Information Processing Systems, volume 33, pp. 12438-12448, 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, February 2022b.
Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural net- works. In Proceedings of the 36th International Conference on Machine Learning, pp. 6105- 6114 May 2019.
D. Tolhurst, Y. Tadmor, and Tang Chao. Amplitude spectra of natural images. Ophthalmic and Physiological Optics, 12(2):229-25) 2222222, 1992.
Antonio Torralba and Aude Oliva. Statistics of natural image categories. Network: Computation in Neural Systems, 14(3):391-44 192- January 2003.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, (86):2579-266)) 2008.
Luisa Verdoliva. Media forensics and DeepFakes: An overview. IEEE Journal of Selected Topics in Signal Processing, 14(5):910-932, 2020.
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in deep network with local denoising criterion. Journal of Machine Learning Research, 111100)3371--300 2010.
Gregory K. Wallace. The JPEG still picture compression standard. Communications of the ACM, 34(4):30 44, April 1991.
Sheng- Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A. Efros. CNN- generated images surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Diffusion- GAN: Training gans with diffusion, June 2022.
Moritz Wolter, Felix Blanke, Raoul Heese, and Jochen Garcke. Wavelet-packee for deepfake image analysis and detection. Machine Learning, August 2022.
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion GANs. In International Conference Learning Representations, May 2022.
Xinsheng Xuan, Bo Peng. Wei Wang, and Jing Dong. On the generalization of GAN image forensics. In Biometric Recognition, Lecture Notes in Computer Seience, pp. 134-141, 2019.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. LSUN: Construction of a large-scale image dataset using deep learning with humans the loop, June 2016.
Xu Zhang, Svebor Karaman, and Shih-Fu Chang. Detecting and simulating artifacts in GAN fake images. In 2019 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1-6, December 2019.
13

---

Under review as a conference paper at ICLR 2023
A
DETAILS ON THE DATASET
LSUN Bedroom (Yu et al., 2016) We download and extract the Imbd database files using the official repository
The images are center-cropped to 256x256 pixels.
ProGAN (Karras et al. 2018)
We download the first 10k samples from the non conntcd collection
provided by the authors]
StyleGAN (Karras et al.
We download the first 10k samples generated with v 0.5 from the non-curated collection provided by the authors
2019)
ProjectedGAN (Sauer al. 2021 We sample 10k images using code and pre-trained models
provided by the authors using the default configuration (--trunc=1.00
Diff-StyleGAN2 (Wang et al., 2022) and Diff-ProjecteGGAN (Wang et al., 2022) We sample 10k images using code and pre- pre-trained models provided by the authors using the default configura-
tion
DDPM (Ho al. 2020), IDPM (Nichol & Dhariwal 2021), and ADM (Dhariwal & Nichol, 2021 We download the samples provided by the authors of ADM and extract the first 10k sam- ples for each generator. For ADM LSUN, we select the models trained with dropout.
PNDM (Liu et al. 2022) We sample 10k images using code and pre-trained model provided by authors We specify --method F-PNDM and --sample samplesspeed sspeedd 20 for LSUN Bedroom and 1--s--amp samp e_speed 10 for LSUN Church, these are the settings leading to the lowest FID according to Tables 5 and 6 in the original publication.
LDM (Rombach et al. 2022) We sample 10k images using code and pre-trained models provided
by the authors using settings from the corresponding table in the repository 13 For LSUN Church there is inconsistency between the repository and the paper, choose 200 DDIM steps (-c 2 (00 as reported in the paper.
httpe://iiihubb.cc////////aam
gan
httee////kauuicoo///m/ooorrraasr GAN
14

---

Under review as a conference paper at ICLR 2023
Peal
BuGAN
SukkAN
PrgessKAAN
InliSokcAANe ENGRRaaatat
DOPS
ADM
PHESA
LOW
Amalk
Figure 7: Non-curated examples for real LSUN Bedroom, GAN-generated, and DM-generated images.
15

---

Under review as a conference paper at ICLR 2023
B DETECTION
B.1 DESCRIPTIONS OF DETECTORS
Wang al. (2000) In this influential work, the authors demonstrate that a standard deep convolu- tional neural network (CNN) trained on data from single generator performs surprisingly well on unseen images. They train a ResNet-50 (He et al. 2016) on 720k images from 20 LSUN (Yu al, 2016) categories (not including Bedroom and Church), equally divided into real images and images generated by ProGAN (Karras et al. 2018). The trained binary classifier is able to distinguish real from generated images from variety of generative models and datasets. The authors further show that extensive data augmentation in the form of random flipping. blurring, and JPEG compression generally improves generalization. Moreover, the authors provide two pre-trained model configura- tions, Blur+JPEG (0.1) and Blur+JPEG (0.5), where the value in parentheses denotes the probability of blurring and JPEG compression, respectively. They achieve average precision of 92.6% and 90.8%, respectively. The work suggests that CNN-generated images contain common artifacts (or fingerprints) which make them distinguishable from real images.
Gragnaniello et al. 2021) Building upon the architecture and dataset from Wang et al. 2020),
the authors of this work experiment with different variations further improve the detection perfor- mance in real-world scenarios. The most promising variant, no-down, removes downsampling from the first layer, increasing the average accuracy from 80.71 % to 94.42%, at the cost of more trainable parameters. They also train model with the same architecture images generated by StyleGAN2 KKrras et al. 2020) instead of ProGAN, which further improves accuracy to 98.48%.
Mandelli al. (002aa) Unlike the other two methods, this work uses ensemble of five orthogo- nal CNNs to detect fake images not seen during training. All CNNs are based on the EfficientNet-B model (Tan & Le Le| 2019) but are trained on different datasets. Dataset orthogonality refers to images having different content, different processing (e.g.. JPEG compression), or being generated by dif- ferent GANs. They argue that by having different datasets, each CNN learns to detect different characteristics of real and generated images, improving the overall performance and generalization ability. For each CNN, a score is computed for ss 200 random patches extracted from the image. These scores are combined using novel patch aggregation strategy which assumes that an image fake at least one patch is classified as being fake. Finally, the output score is computed by averaging the individual scores of all five CNNs. It should be noted that, besides several GANs, the training dataset also includes samples generated by [Sng et al. (2022b). a score- bosed model.
B.2 DETECTION RESULTS FOR CLASSIFIERS TRAINED FROM SCRATCH
We repeat the experiment described in Section 5.2 but train each classifier from seratch instead of
fine-tuning it, the results are shown in Figure S Note that here "from scratch" means that we start from ResNet-50 model pre-trained on ImageNet, which is exactly how the authors trained their models (Wang et al. 2020). For most constellations of training and test data, the results are similar compared to fine-tuning, with a tendency towards worse scores for the models trained from scratch.
B.3 FEATURE SPACE ANALYSIS
The experiments in Section 5.1 demonstrate that current detection methods struggle reliably iden- tify DM-generated images while detecting GAN- GAN-generated images with near-optimal accuracy. We thus hypothesize that GAN-generated images exhibit features that are considerably distinct from those of DM-generated images.
To further investigate this hypothesis we examine the discrepancy of real and fake images in the feature spaces of the detection methods under consideration. More precisely, we employ Maximum Mean Discrepancy (MMD: Gretton et al. 2012) to estimate the distance between the distributions of real and generated images in feature space. We extract the 2048-dimensional features before the last fully-connecter layer of the detectors by Wang et al. (2020) and Gragnaniello et al. (202T). We employ Gaussian kernel with a as the median distance between instances in the combined sample as suggested in Gretton et al. (2012) and calculate the MMD between the representations of 10k generated images and 10k real images.
16

---

Under review as a conference paper at ICLR 2023
Trained on
Trained on
rait Sty WGGAN2
Dan Phojenaate Bora
Datt AACAA
Prode Profceeraan
Procda
DEVP
PINZM
PeCDAN
LXM4
ProGAN 1000
ProGAN
1000
SekGAN
StyeGAAN
ProjovaGiAN
66:
100.0
90.0
0030
84.0
$3.5
$2.0
1080
ST.9
1000
IrggeisttAN
StleGANE
StylGAN
Tested on
Uhif ProjeenGGG
91.2
1000
27.7
720
DOPM
6000
10010
1910
DOPM
150
90
1000
I00P
ADM
21.8
30.0
60.0
13.2
58.0
90.9
87.
10.0
70.
1000
1000
ATM
1000
9550
LIM
34:7
71.3
130
600
77.
59.1
964
59.9
59.3
1902
1000
LIM
1000
2015
1000
(a) AUROC
(b) PD@ 1%.
Figure 8: Detection performance for detectors trained from scratch.
Grinanielle2021 (PwriAN)
Gragnaniello2021 (SyjaCANN))
Wang2020 (MieePiGG (X)
Wang (Miae-IV (1.5)
0.5
GANs
DM
0.5
GAYe
DMS
GAN
IIM
CIANs
nMs
MMD (Xxaa)) [(X
pre-trained
0.0
0.0
0.6
0.0
0.5
0.5
0.5
0.5
0.2
0.7
0.2
0.2
Wang2020 (ite-tuned
Wong2020 (tee-raned
Wang2020 (lte- DMB
GANs
DMs
GANS
DMs
MMD[F (x.)a))) 2h ((moddd))
50
1.0
fine-tuned
PeeGAN
IOMM
StylGAN
ProjnwaGAM
0.4
0.4
SyleGGGN
PNDDM
Dill PoajeteoAAN
LDM
0.0
0.0
0.0
Wong2020
Wang2020
Wang2020 DMS)
GANs
trained from scratch
MMD (f (Xeea) ((X
1.0
1.0
1.0
0.5
0.5
0.0
0.5
0.6
0,4
04
0.7
0.2
Figure 9: MMD between feature representations of real and generated images on LSUN Bed- room for different detectors (pre-trained, fine-tuned and trained from scratch). The features f(+) correspond to the representation prior the last fully-connected layer of the respective detec- tion method. For further information on the detectors see Section 5.2 (fine-tuned) and Appendix B.2 (trained from scratch).
17

---

Under review as a conference paper at ICLR 2023
We consider three sets of detection methods, namely pre-trained, fine-tuned and trained from scratch.
Starting with pre-trained detection methods (top row in Figure we observe that the MMD be- tween representations of DM-gennn 2M-gennrattdd images and those of real images considerably lower in comparison to GAN-generated images. With the results from Table Z mind, we can conclude that the pre-trained detectors extract features that allow to reliably separate GAN-generated images from real images, while these features are less informative to discriminate DM-generated images. This findings support the experiments conducted Section 5.1
We repeat the above experiment for fine-tuned and re-trained detectors from Wang et al. (020) (middle row and bottom tune/train on all GAN-generated images, DM-generated images and both (as described in Section 5.2 and Appendix B.2). The results fine-tuned and re- re-trained detectors are qualitatively similar: When on generated images from both, DMs and GANs, the MMDs in feature space are roughly on par for both model sets. This strengthens the finding that reliable detection feasible provided the generative model class is known. When fine-tuning/training solely images from one model class we observe an imbalance: Detectors fine-tuned or trained on images from DMs are able to achieve relatively higher MMDs for GAN-generated images than vice versa. This imbalance is already evident in the experiments in Section 5.2
in Figure P9.. We consider the version Blur+JPEG (0.5) and fine-
training/fing wwtggg/inettutiin
The findings give rise the following hypothesis: GAN- and DM-generated images share some common patterns. However, detectors trained solely on GAN-generated images focus mostly on GAN-specific (arguably more prominent) patterns, while detectors trained DM-generated images focus on common patterns. The hypothesis is corroborated by the t-SNE visualizations (van der Maaten & Hinton 2008) provided in Figure 10 The detectors trained on DM-generated images evidently learn reatures/representationn which are common to both GANs and DMs (right column), while detectors which saw EAN--geneaate images during training map GAN- and DM- DM-generated images to distinct representations (center columns). Moreover, detectors trained solely on GAN- generated images lead to representations of real and DM-generated images which are clearly separated, which fits the reduced performance.
W0ng (ir-1P9G
Wang2020 (teesteed
20ng diresteed GANG
Wang 2020 (iee-teed
150
100
110
100
100
01
100
100
100
Whng2020 moined on Al
20ng orined GANo)
Wang 2020 ennred in DMoo
DOPM
100
-50
50
100
100
-10
100
100
-50
50
100
100
LDS
PreGAN
StyleGNN
PojesteGiAN
-50
DrS-InggesstAAN
100
Figure 10: t-SNE visualization of representations of real and generated images on LSUN Bed- room for different detectors (pre-trained, fine-tuned and trained from seratch) in two dimen- sions. The representatioos/featuus f(-) correspond to the representation prior to the last fully- connected layer of the respective detection method and are of dimension 2048. For further informa- tion on the detectors see Section 3.2 (fine-tuned) and Appendix (trained from scratch).
"We used the scikit-leam implementation of t-SNE with the default settings, details are given at https: 7/scikite www.iikittetenn.nnemmmm oo
mani Fold.1.... 8.S.KK html
18

---

Under review as a conference paper at ICLR 2023
Table Effect of image perturbations on detection performance. We use the pre-trained detector
by Gragnaniello et al. (02T) trained on ProGAN images and compute metrics from 10k samples.
AUROC/ Pd0e5s PS
Cleas
Crop
JPEG
Noise
ProGAN
100.0/ 100.0/ 100.0
1000/100.0/ 100.0
100.0/ 100.0/
100.0
100.0/
106.0/ 100.0
99.21
95.8
82.5
StylGAN
100.0/
100.0/
ProjecteOAAN
100.0/ 99.97 99.3
99.21 96.37 87.6
99.91 99.71 98.0
100.0: 100.0/ 99.6
96.6
79.9/
52.8
DHT-SSylacTAN
100.0/ 100.0/ 100.0
1000/ 100.0/ 100.0
100.0/ 100.0/ 100.0
100.0/ 100.0 1000
96.7/
79.5/
56.3
Did ProjectedGAN
99.9/ 99.9/ 99.2
99.07
91.9/ $4.1
99.9/ 99.71
97.0
100./ 100.0/ 99.4
96.1/
77.0/
51.0
DDPM
96.57 79.4/ 39.1
78.51 23.77
8.1
95.51 73.21
35.2
960: 76.1/
37.3
85.1/
34.0/
11.2
IDDMM
94.31 64.8
25.7
75331 18.57
5.5
62.07
24.6
93.51 60.77
23.4
30.91
25.0/
7.0
77.8/
78.3/ 214/
5.5
76.8
18.7/
44
8./
1.7
PNDM LDM
91.6/ 52.07
16.6
86.71
38.87
14.3
91.17 51.9/
19.7
92.5 56.6/
20.6
81.2/
274/
9.3
96.71
79.97
42.1
87.11
42.87
17.1
96.87
81.8/ 48.9
96.31
77.3/
40.0
82.2/
28.6/
8.8
Table 5: Effect of image perturbations on detection performance. We use the detector by Wang ct al. (2020) fine-tuned on all DM-generated images and compute metrics from 10k samples.
AUROC/ / Pdre/ / 118
Clean
Blur
Crop
JPEG
Noise
DDPM
100.0/100.0/ 100.0
100.0/ 100.0/ 99.9
100.0/ 100.0/ 99.9
100.0/ 100.0/
99.7
69.8
22.7/
11.3
IDPP
100.0/ 100.0/ 100.0
10007 100.0/ 100.0
100.0/ 100.0/ 100.0
100.0/ 100.0/
99.7
70.2/
22.5/
11.1
ADM PNDM 1DM
100.0/ /
100.0/ 100.0/
25.2/
100.0/ 100.0/ 100.0
1000/ 99.9/ 99.8
100.0/ 100.0/ 100.0
100.0: 100.0/
99.7
74.87
29.2/
14.6
100.0/ 100.0/ 100.0
1000/ 100.0/ 100.0
100.0/ 100.0/ 100.0
100.0/
100.0/
99
71.91
25.7/
12.9
B.4 EFFECT OF IMAGE PERTURBATIONS
In most real-world scenarios, like uploading to social media, images processed, which is why several previous works consider common image perturbations when evaluating detectors (Wang jet al. 2020 Hulzebosch et al. 2020 Liu al. 2020, Frank al., 2020). We follow the pro- tocol of Frank et al.j (2020) and apply blurring using a Gaussian filter (kernel size sampled from {3, 5, 7,9]. cropping with subsequent upsampling (crop factor sampled from (5, 20)), JPEG com- pression (quality factor sampled from (10, 75)), and Gaussian noising (variance sampled from (5, 20)). Unlike Frank et al. 2020), we apply each perturbation with probability of 100% to study its effect on the detection performance.
Table 4 shows the results for the dest-performing detector by Gragnamiello al. (2021) trained on ProGAN images. Note that this detection method employs training augmentation using blurring and JPEG compression. We copy the results on "clean" images from Table 2 as a reference. For GAN-generated images, blurring, cropping, and compression have only very small effect on the detection performance, with the latter even improving the results. We attribute this to the fact that the detector uses data augmentation during training. While cropping and compression cause an average decrease in AUROC of 0.34% and 0.36%, respectively, blurring leads to a significant performance degradation of 12.66%. The addition of noise deteriorates both GAN- and DM-generated images, with AUROC decreasing by 3.4% and 12.54% on average, respectively. Overall, we observe that perturbations have stronger effect DM-generated images compared to GAN-generated images. An explanation could be that the pre-trained detector performs significantly worse on DM MM-generated images, even without any perturbations. We repeat the experiment using the detector by Wang et al. 2020) fine-tuned on images from all DMs (see Section 5.2p. As the results in Table 5 show, the effect of blurring, cropping, and JPEG compression almost negligible. Adding noise, however, causes significant performance drop. This could to be related to the fact that the image generation process of DMs involves noise. However, another potential explanation is that the detector is shown blurred and compressed images during training but not those with added noise.
B.5 FAKENESS RATING
We evaluate whether DM-generated images exhibit some visual cues used by the detector to distin-
guish them from real images. Inspired by Wann et al. (2020), we rank all images by the model's
predictions (higher value means "more fake") and show examples from different percentiles. We
consider two detectors, the best-performing pre-trained detector from Gragnaniello et al. (2021) trained on ProGAN (Table [T] and the detector from Wang et al.] (2020) fine-tuned on all DM- generated images (Table 12 Details the fine-tuning process given in Section 5.2 Using the
19

---

Under review as a conference paper at ICLR 2023
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(a) DDPM
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
K
(b) IDDPM
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(c) ADM
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
Bent
(d) PNDM
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(c) LDM
Figure 11: "Fakeness" rating based of LSUN Bedroom on predictions from detector by Grag- naniello et al. (2021) trained on ProGAN images. Images are ranked by the model's output, i.e., images in the 100th percentile are considered most fake.
detector from Gragnaniello et al. (021), , we make the observation that for most DMs, images which the model assigns high "fakeness" score contain many pixels which are purely white black. On the other hand, images considered less fake appear to be more colorful. We were able to reproduce
this behavior for other datasets in Figure [3] However, this finding does not hold for the ranking
provided by the fine-tuned detector (see Figure 12). which should provide more accurate results as
its detection performance is significantly higher. Overall, we agree with Wang et al. (2020) that there is no strong correlation between the model's 's predictions and visual quality.
20

---

Under review as a conference paper at ICLR 2023
0th Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(a) DDPM
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(b) IDDPM
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
1
(c) ADM 50th Percentile
0th Percentile
25th Percentile
75th Percentile
100th Percentile
(d) PNDM
0th Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(c) LDM
Figure 12: "Fakeness" rating of LSUN Bedroom based on predictions from detector by Wang et al. (2020) fine-tuned on images from all DMs. Images are ranked by the model's 's output, i.e.. images in the 100th percentile are considered most fake.
21

---

Under review as a conference paper at ICLR 2023
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(a) LDM (FFHQ)
0th Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(b) ADM' (FFHQ)
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
(c) ADM (ImageNet)
Oth Percentile
25th Percentile
50th Percentile
75th Percentile
100th Percentile
MI
(d) PNDM (L.SUN Church)
Figure 13: "Fakeness" rating of DMM-enerated images based on predictions from detector by Gragnaniello et al. (2021) trained on ProGAN images. Images are ranked by the model's 's output, l.e., images in the 100th percentile are considered most fake.
22

---

Under review as a conference paper at ICLR 2023
C FREQUENCY ANALYSIS
C.I FREQUENCY TRANSFORMS
Discrete Fourier Transform (DFT) The DFT maps a discrete signal to the frequency domain by expressing it as sum of periodic basis functions. Given grayscale image 1 with height H and width W, the two-dimensional DFT (with normalization term omitted) defined as
H-1W-1
IDF [,, =
Σ 2=0
Σ
Ma, y) exp
2mr # exp
2 17
(4)
y=u
with k -....,H- shifted to the center of the spectrum. Therefore, coefficients towards the edges of the spectrum correspond to higher frequencies.
H -111 and | = 0, O..... W - 1. For visualization, the zero- vero-frequency component
Discrete Cosine Transform (DCT) The DCT is closely related to the DFT, however it uses real- valued cosine functions basis functions. used in the JPEG compression standard due to its high degree of energy compaction (Wallace 1991). which ensures that large portion of a signal's energy can be represented using only few DCT coefficients. The type-Il DCT, which the term DCT usually refers to, is given as
W-1
1
[ Σ Σ
Ma. cos
x. +
ki
y
ky
(5)
H
W
2
again omitting the normalization factor. In the resulting spectrum, the low frequencies are located in the upper left corner, with frequencies increasing along both spatial dimensions.
Reduced Spectrum While the DFT provides a useful visual representation of an image's spec- trum, less suitable for comparing different spectra quantitatively. Therefore, previous works use the reduced spectrun a one-dimensional representation of the Fourier spectrum (Durall et al.,
2020 Dzanic al.] 20200; Schwarz al. 2021; Chandrasegaran al., (021). obtained by azimuthally averaging over the spectrum in normalized polar coordinates E [0, 0 E (0,2T) according to
{ Jo
2x
k2 12
(r)
0)d0 with
and alann((k;
(6)
2x
[ + W2)
with [],, |D [k: 2 being the squared magnitudes of the Fourier coefficients. The maximum
frequency is given by the Nyquist frequency fayq Vk² + 12 H/V2 for square image with H W.
C.2 DCT SPECTRA
Figure 14 depicts the DCT spectra of both GANs and DMs. Similar to DFT, images generated by DMs exhibit fewer artifacts, with the exception of LDM.
C.3 SPECTRUM EVOLUTION DURING THE DENOISING PROCESS
Analogous to Figure 4 we show the spectrum error evolution during the denoising process in Fig- ure 15 Here, however, the error is computed relative to the spectrum of noised images at the corresponding step during the diffusion process. Similar to the denoising process, the spectra of the diffusion process averaged over 512 samples. While the relative error is close to zero for long time during the denoising process, the model fails to accurately reproduce higher frequencies towards f 0. More precisely, much high-frequency components removed, which explains why the sweet spot does not seem to be at f but around = 10 (see Figure
1he definition of the reduced spectrum slightly differs hetween different existing works, decide to follow that of Schwarz al. (20TT)
23

---

Under review as a conference paper at ICLR 2023
One could think that, by stopping the denoising process early, DM DM-generated images might be harder to detect. To test this hypothesis, we perform logistic regression at every 0 I t S 100 to distinguish real from increasingly denoised generated images. The model is trained using 512 real and 512 generated samples, from which 20% are used for testing. We select the optimal regularization weight by performing 5-fold cross-validation we compare the performance on pixels and different transforms. The results in Figure [6 do not indicate that around 10 fake images are less detectable. In Figure [7 it becomes apparent that at this the images are noticeably noisier, which probably explains the increased accuracy.
{10* k (-4, 33, ..., 4) }. Similar to Section 6.2
Real
ProGAN
StyleGAN
ProjectedGAN
Dift' StyleGAN2
Dif ProjectedGAN
10
10
(a) GANs
Real
DDPM
IDDPM
ADM
PNDM
LDM
10
(b) DMs
Figure 14: Mean of DCT spectrum from real and generated images. To increase visibility, the color bar is limited to [10 100). with values lying outside this interval being clipped.
1000
05
0.5
000
0.0
400
N0
-0.5
20
-05
1/
05
03
1.0
1.0
1lbex
(a) 0 S 1000
(b) Ob) 100
Figure 15: Evolution of spectral density error throughout the denoising process. The error is computed relatively to the corresponding step in the diffusion process. The colorbar is clipped at -1 and 1. If the model was able to perfectly denoise, the difference would be 0 at all
100
Picel
FFT
Accuracy
Figure 16: Accuracy of logistic regression during the denoising process. Note that only the last 100 steps of the denoising process depicted.
24

---

Under review as a conference paper at ICLR 2023
C.4 EFFECT OF NUMBER OF SAMPLING STEPS
We repeat the logistic regression experiment from the previous section for samples generated with different numbers of timesteps, the results are shown in Figure [8 We also provide example images in Figure 19 As expected. images become harder to detect with more sampling steps and therefore quickly, which is likely because
higher image quality. For DDIM, the accuracy decreases was found to generate images of higher quality with fewer sampling steps (Song et al. 2022a).
Figure 17: Example images generated by ADM different When zoomed in, the high- frequency noise towards higher ł becomes apparent.
Denoisin Steps
830
log
16
0.9
1.0
06
Accuracy
Accuracy
(a) Without DDIM
(b) With DDIM
Figure 18: Accuracy of logistic regression for different numbers of sampling steps. Note that the y-axis is not scaled linearly.
25

---

Under review as a conference paper at ICLR 2023
2.kgo
sage
loskpe
skpo
30 lage
sipp
500 sage
1000 sigs
(a) Without DDIM
2gkgo
skge
loskpa
Aago
50
100sage
200
500
1000
(b) With DDIM
Figure 19: Example images generated by ADM with different numbers of sampling steps.
C.5 POTENTIAL SOURCE OF THE SPECTRUM DISCREPANCIES
Our analysis in the frequency domain (see Figures 3 and 23 suggests that current state-of-the-art
DMs do not match the high-frequency content well. To further analyze these findings, we build on insights from the denoising autoencoder (DAE) literature. Roughly speaking, the task DAEs face is conceptually similar to the task of the noise predictor eœ in DMs single time step: denoising disturbed input at fixed noise level. Note that while we believe that DMs and DAFs can be concep- tually related, the concepts are distinct several ways: DMs use parameter sharing to perform noise prediction at multiple noise levels to up the generation DAEs make use of latent space to learn suitable representations for reconstruction, classically at fixed noise level. Nevertheless, account.
iterative process. On the other hand,
convinced that it may be useful take these insights into
A handy observation from DAEs relates the level of corruption to the learned feature scales: Denois- ing an image at small noise levels requires to accurately model fine granular details the image, while coarse/lagee-saale features still be recovered high noise levels (see e.g., Vincent et al.
26

---

Under review as a conference paper at ICLR 2023
2010) Geras & Sutton (2015: 2016)). Transferring this insight to DMs, we observe that the training objective guides the reconstruction performance across the different noise levels.
Recall that the objective of many prominent DMs can be stated as weighted of mean squared error terms
((
Σ
cx(x,, t)
1=0
usually with T = 1000. The theoretically derived variational lower bound Liib (in the case of E(1) = I) corresponds to the weighting scheme
B
w((t) 20 2, a4)
(8)
with a2 and ã₂ derived from the noise schedule 3e (see Section 3). However, Lvbb tums out to be extremely difficult to optimize in practice (see e.g., Dhariwal & Nichol (202I)), arguably due to large influence of the challenging denoising tasks near / = 0. To circumvent this issue, Laimple was and turned out to be stable to optimize
proposed by Ho et al. 2020) which corresponds to w(l) and already leads to remarkable perceptual quality. In order to achieve both, perceptual image quality and high likelihood values, values,[Nichol NNichl Dhariwal (202I) proposed Lhy Liippee + ALvib with À 0.001 which increases the influence of low noise levels. The relative importance of reconstruction at different noise levels for the above discussed objectives is depicted in Figure 20
10
Pairpple
10
1 -
Lib
10
Latal Hayteid
10
Listeid
10
(a) <t 1000.
(b)S(S 50.
Figure 20: Relative importance of the reconstruction tasks for prominent DMs. We show the relative influence (t) w(t)/ Ei= w()) for (a) the whole denoising-diffusion process and (b) a close-up on the lowest noise levels. For Iol we depict both, lower and upper bound for the denoising variance of (given as of Be and of 3 as defined in Ho et al. (022)). the case of Lhybrid, used by IDDPM and ADM, we use oi for visualization purposes instead of diagonal covariance matrix and plot wt) for the range of admissible variances o/² All plots assume the linear noise schedule 3, by Ho et al. (2020).
To put the above discussion in nutshell, hypothesize that the ability of a DM to match the real frequency spectrum is governed by the reconstruction performance at the corresponding noise levels. Importantly, successful error prediction low noise scales, i.e., near / = 0, requires to capture the content of an image. We deduce that the relative down wwwn-weighting of the influence of low noise levels when using Limple or Lhybrid (compared to the theoretically derived Lvbb which would account for high likelihoods) results in the observed mismatch of high frequencies.
The mean reduced spectra of various DMs (Figure support this hypothesis: Both Nichol & Dhari- wall 202I) and Dhariwal & Nichol 2021) train their model with Luybrid (which incorporates relatively higher weight on low noise levels), and are able reduce the gap to the real spectrum when compared to the baseline trained with Lsimple (Ho et al.) 2020). Clearly, we believe that not only the weighting scheme w((() accounts for the resulting spectral properties, but more importantly the model's is capabilities to successfully predict the low level noise in the first place. Nevertheless, the weighting scheme acts as a proxy that encourages the model to focus on specific noise levels.
We conclude that the objectives of DMs are well designed to guide the model to high perceptual quality (or benchmark metrics such as FID), while falling short on providing sufficient information to accurately model the high frequency content of the target images, which would be better captured by a likelihood-batt objective like Lylh-
27

---

Under review as a conference paper at ICLR 2023
D RESULTS ON ADDITIONAL DATASETS
To demonstrate the generalization of our findings, we perform classification and frequency analysis additional data from ADM, PNDM, and LDM. Note that ADM-G-U refers to the two-stage up- sampling stack which images are generated at resolution of 64x64 and subsequently up-sampled 256 pixels using a second model (Dhariwal & Nichol 202T). The generated images are obtained according to the instructions given in Section
A Due to the relevance of facial images in the context of deepfakes, we also include two DMs not yet considered. P2 and ADM (Choi al. 2022). trained on FFHQ (Karras et al. 2019). ADM is smaller version of ADM with 93M instead of more than 500M parameters P2 is similar to ADM' but features a modified weighting scheme which improves performance by assigning higher weights to diffusion steps where perceptually rich contents are learned (Choi et al. 2022). We download checkpoints for both models from the official repository and sample images according to the authors" instructions.
Real images from LSUN (Yu et al. 2016). ImageNet (Russakovsky et al. 2013). and FFHQ (Karras et al. 2019) are downloaded from their official sources. Images from LSUN Cat/Horse, FFHQ, and ImageNet are resized and cropped to 256 226 pixels by applying the same pre-prceessing that was used when preparing the training data for the model they are compared against. All analyses performed using 10k real and 10k generated images.
For results other GANs, refer to the original publications of the detectors (Wang al. 2020 Gragnaniello et al. 2021 Mandelli et al. 2022a).
D.1 PERFORMANCE OF UNIVERSAL DETECTORS
Similar Table 2 we report AUROC, Pd@5%, and Pd@ 1% for the additional datasets in Ta- b Overall, the results do not invalidate the claim that universal GAN-detectors are incapable of effectively detecting images generated by DMs. Both PNDM and LDM trained on LSUN Church, however, are detected significantly better by all classifiers Mandelli et al. (2022a). This might be caused by the relatively high FID (compared to the state of the art) both models achieve on this dataset (8.69 for PNDM, 4.02 for LDM).
Table 6: Detection performance of pre-trained universal detectors on additional datasets. For Wang et al. (2020) and Gragnaniello al. (202I) we consider two different variants, respectively. The best score (determined by the highest Pd 1%) for each generator is highlighted in bold.
Wang et al.
2020
Cimgnamiclio et al
2021
Mandelli et
2022A
AUROC/ 5%6 1%
Blur-IPEG (0.5)
Blar-PPEG (0.))
PreGAN
StyleCIA22
5.51
ADM (1.UNN Horse)
55.51
6.7/
1.5
53.4/
607
56.17
7.5%
50.27
4.8
44.21
2.4/
0.5
ADM (maggNNt)
69.17
13.9/
4.1
71.77
15.57
4,5
72.17
13.2/
3,5
83.9
38.5/
16.6
60.1/
6.8
0.9
ADM G- (ImspeNNet)
6721
11.7/
3.7
62.31
6.2
1.2
66.81
6.0/
78.9/
26.7/
10.2
60.07
766
1.0
(LSUN Chunch)
76.1
LDM (LSUN Church)
8633
42.1/
19.8
82.2/
33.97
14.2
98.87
93.77
75.5
99.5/
98.2/
90.2
58.97
53/
1.3
LDM (FFTQQ)
69.17
14.2/
3.6
7L.0/
14.77
3.5
911/
54.2/
25.4
67.2/
10.27
2.1
63.0/
5.67
0.6
ADMI (I-HQQ)
2621
41.9/
17.8
17.2
2,0
(FFHQQ)
79.51
26.4/
83.2/
30.1
89.2
40.57
11.5
91.1/
5197
18.9
D.2 FREQUENCY ANALYSIS
We analyze the DFT (Figure 21p. DCT (Figure 22). and reduced spectra (Figure 23) using a similar process in Section 6.1 Regarding frequency artifacts, the results are consistent with that from LSUN Bedroom, LDM exhibits grid-like artifacts while ADM and PNDM do not. The spectra of ADM on LSUN Cat and LSUN Horse do contain irregular, vertical structures, which we did not observe for any other model and dataset. However, these are substantially different and not as pronounced as GAN artifacts. The DFT and DCT of (real and generated) FFHQ images clearly deviate from the remaining spectra, which we attribute to the homogeneity of the dataset. Note that trained differently processed versions of the real images, which is why
LDM and ADM /P2 we include the spectra of both variants. While we observe the known artifacts for LDM, ADM' and P2 do not contain such patterns.
httpss: thu.i http:///wwwccciee////eeeeeeiiina
28

---

Under review as a conference paper at ICLR 2023
The reduced spectra have largely the same characteristics for LSUN Bedroom, with the exception of ImageNet. Here we observe an overestimation towards higher frequencies, which is the opposite of what see for ADM other datasets. A possible explanation could be that the authors sampled images from LSUN using 1000 and from ImageNet using only 250 steps. We suppose that the number of spectral discrepancies is highly training dependent.
Real
ADM
Real
ADM
LSUN Cat
(b) LSUN Horse
Real
ADM
ADM-G-U
Real
PNDM
LDM
10-1
1033
10
10
(c) ImageNet
(d) LSUN Church
Real*
LDM
Real**
ADM
10
(c) FFHQ (*: pre-processing according to Rombach et al. 2022 **; pre-processing according to Choi ct al. 2022
Figure 21: Mean DFT spectrum from real and generated images from additional datasets.
To increase visibility, the color bar is limited to [10 6,06,, 1.00), with values lying outside this interval being clipped.
29

---

Under review as a conference paper at ICLR 2023
Real
ADM
100
Real
ADM
10
10
10-66
(a) LSUN Cat
(b) LSUN Horse
ADM
ADM-G-U
16f1
Real
PNDM
LDM
100
10
10
10-6
(c) ImageNet
(d) LSUN Church
Real*
LDM
Real**
ADM
P2
10
(e) FFHQ (*: pre-processing according to Rombach 2022
2022 *3; pre-processing according to Choi al.
Figure 22: Mean of DCT spectrum from real and generated images from additional datasets.
To increase visibility, the color bar is limited to [10-b, 100), with values lying outside this interval being clipped.
30

---

Under review as a conference paper at ICLR 2023
10
Spectal Density
Spectral Density
09
-0
80
10
f/
(a) LSUN Cat
(b) LSUN Horse
10
10
102
102
10
10
Spectral Denskty
Spectral Density
10
10
Doak 16
Real
AUM
ADMA-U
PNDM ---
LIM
10
f/fo
(c) ImageNet
(d) LSUN Church
Spectral Density
10-
f/feN
(c) FFHQ (*: pre-processing according Rombach
2022
pre- processing according Choi
2022
Figure 23: Mean reduced spectrum from real and generated images from additional datasets.
The higher end of the spectrum where GAN-characteristter discrepancies occur is magnified.
31


Reviews:
[{"review_id": "Rw44q8XQcV", "reviewer": "Reviewer_iMjn", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors focus on studying the detection of images generated by diffusion models. Specifically, this paper evaluates the performance of current methods and then analyzes the difference between GAN-generated and DM-generated images.", "summary_of_the_review": "This paper focuses on an ignored problem. However, the novelty is limited and the experiments are inadequate. ", "strength_and_weaknesses": "Strengths:\n\n1.\tThis paper focus on an ignored problem, the detection of DM-generated images.\n\n2.\tA benchmarking of the DM deepfake is conducted.\n\n3.\tThis manuscript finds that the DM-generated images are different from GAN-generated images.\n\n\nWeaknesses:\n\n1.\tThere exist various generated models. However, this paper just analyzes the DM-generated and GAN-based models.\n\n2.\tThis paper finds that existing detectors fail to effectively recognize DM-generated images and gives some suggestions. Unfortunately, they do not propose an effective method for deepfake detection.\n\n3.\tOnly three detection methods are benchmarked in this paper. It is recommended to study more methods.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9QQZR_C4uGj", "reviewer": "Reviewer_3UTf", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents an in-depth review of detection of diffusion-model generated images. This paper evaluates SOTA detectors on a range of DMs and then analyzes the differences of many aspects between DM-generated images and GAN generated images in addition to image fidelity. ", "summary_of_the_review": "I think the topic of this paper is meaningful and useful to a decent group of people. Conclusions are clear. And the experimental setup is well-rounded. In general, it is a good paper and I vote for acceptance.", "strength_and_weaknesses": "The experimental setup of this paper is reasonable and adequate. The writing is clear and smooth. The results and conclusions are intuitive. I like how the authors present their results by asking questions and leading readers to answers.\n\nI do not observe obvious drawbacks of this paper. It might make sense to include some results that can be easily classified as fake or misclassified from different approaches. This can intuitively show the features of those generated images that are confidently classified as fake.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3ZmQqFqmkbJ", "reviewer": "Reviewer_gmjk", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors conduct an early study of the detection problem of diffusion model deepfakes. They prove that existing of-the-shelf deepfake detectors trained on GAN-generated images generalize badly to DM-generated deepfakes, while they can be effectively adapted to DM domain through fine-tuning. The authors further analyze the differences between the fake images generated by GAN and DM in the frequency domain, showing that DM-generated images exhibit unique characteristics.", "summary_of_the_review": "This paper gives the first investigation into the deepfake detection problem on diffusion model-generated images. The authors present several meaningful insights into the unique properties of DM-generated deepfakes through well-designed experiments and detailed analysis. However, more in-depth discussions could be done to make the conclusions richer and more complete.", "strength_and_weaknesses": "Strengths:\n\n1.  High-fidelity fake images generated by recent diffusion models have become a new threat in deepfake area. This paper presents a pioneer, systematic examination of this scenario with well-designed experiments.\n2.  The authors provide several interesting observations from two different views: 1) the feasibility of detecting DM deepfakes using existing methods and 2) the characteristics of DM-generated images in frequency domain. These observations prove that DM deepfakes are more challenging to detect and have quite unique characteristics compared to GAN deepfakes, which brings useful insights to both deepfake detection and DM-based generation community.\n3. The paper is overall well-written. All experiment details are presented clearly in the manuscript and supplementary materials with elaborate analysis.\n\nWeaknesses:\n\nMost of the main experiments are relatively simple and trivial. More in-depth analysis could be done to obtain richer and more complete conclusions. Specifically, I have the following suggestions:\n\n1. In section 5.2, the authors make an interesting observation that detectors trained on DM-generated images can generalize better to GAN deepfakes than vice versa. They thus hypothesize that this is because detecting DM deepfakes is a more challenging task. Although the hypothesis seems to be supported by frequency domain analysis, a more intuitive interpretation might be achieved by further analysis on the feature space, e.g., a comparison between distributions of DM/GAN-generated features extracted by DM-trained detectors and GAN-trained detectors.\n\n2. In section 6.2, the authors compare the effectiveness of detecting DM and GAN deepfakes in frequency domain using a simple logistic regression approach. However, some existing deepfake detection works absorb frequency info in a more 'soft' way, e.g. [1]. These frequency-aware deepfake methods' performance on DM deepfakes could also be examined. Will they perform better or worse on DM deepfakes than GAN deepfakes? Will the conclusion made in section 5.1 and 5.2 keep the same?\n\n3. Most existing deepfake datasets (FF++, DFDC, DF1.0, etc.) mainly deal with forgery on human faces, which is an important and meaningful deepfake scenario. However in this paper, most conclusions are made on the LSUN bedroom dataset. The only face-relevant result (FFHQ) is presented in supplementary materials, but from Figure 18(e) we can observe a totally different pattern that contradicts the authors ' conclusion on high-frequency. Unique patterns of FFHQ can also be seen in Figure 16-17. I therefore suggest that the human face deepfake generated by DM should be discussed separately, and its uniqueness should be explained.\n\n4. Since the training and inference of diffusion models are highly related to noise, the effect of perturbations widely used in deepfake datasets (Gaussian noise, blurring, compression, etc.) on DM deepfake detection might also be interesting to look at.\n\n\nReference\n\n[1] Luo, Yuchen, et al. \"Generalizing face forgery detection with high-frequency features.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "26KvynOpjhC", "reviewer": "Reviewer_rYBt", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provides extensive experimentation on how state of the art deepfake detectors behave on diffusion model generated images.\nThey also propose an analysis in the frequency domain of diffusion model generated images compared to GAN and real images.\n", "summary_of_the_review": "While the experiments and their interpretation are very interesting in trying state of the art detectors on DM fakes, this works lacks maturity in providing a novel approach to detect DM fakes.", "strength_and_weaknesses": "Strength:\n- extensive experimentation on said state of the art deepfake detectors\n- extensive experimentation on the influence of DM hyper parameters (sampling steps)\n- discussion on the optimization target of these deepfake detectors\n\nWeaknesses:\n- need more maturation on a innovative methods to detect DM deepfakes", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "0o11uac8K_", "reviewer": "Reviewer_Gcjc", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper evaluates the performance of deepfake detectors on images generated by five diffusion models (DM) and five GANs. It also analyzes the spectral representation of DM-generated images.", "summary_of_the_review": "The paper is easy to read and provides important information for a better understanding DM, and although the information is important, the paper does not provide any novelty.", "strength_and_weaknesses": "(+) The paper is an easy-to-read paper evaluating how well images generated by DM can be detected The paper evaluates the performance of existing deepfake detectors on images generated by DM. It reports performance results of finetuned existing deepfake detectors to DMs. \n(-) The paper does not provide any novelty but instead provides more information regarding DM and compares detection results with GAN-generated images. The paper analyzes the spectral content of DM-generated images comparing GAN-generated images.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject