[RE] CNN-generated images are surprisingly easy to spot... for now
Anonymous Author(s)
AfﬁliationAddress email
Reproducibility Summary
This work evaluates the reproducibility of the paper “CNN-generated images are surprisingly easy to spot... for now”by Wang et al.
published at CVPR 2020.
The paper addresses the challenge of detecting CNN-generated imagery,which has reached the potential to even fool humans.
The authors propose two methods which help an image classiﬁerto generalize from being trained on one speciﬁc CNN to detecting imagery produced by unseen architectures, trainingmethods, or data sets.
Scope of Reproducibility
The paper proposes two methods to help a classiﬁer generalize: (i) utilizing different kinds of data augmentations and(ii) using a diverse data set.
This report focuses on assessing if these techniques indeed help the generalization process. Furthermore, we perform additional experiments to study the limitations of the proposed techniques.
Methodology
We decided to implement the methods from scratch to highlight possible hurdles for practitioners who want to adopt theresults.
For our experiments, we utilized the training and test data provided by the authors, as well as, creating our owndata set to analyze how the proposed techniques perform on different data sets.
Note that overall we estimate the entireexperiments to take around 590 GPU hours.
Results
In general, we were able to replicate the overall results reported in the paper. However, we also found signiﬁcantdifferences in our experiments.
After further investigations, we discovered that we implemented two data augmentationsslightly different. Additionally, we identiﬁed three test data sets, which exhibit ﬂuctuating results across multipleruns with the same setup.
However, the overall trends of our experiments matched the original publication, i.e., datadiversity and augmentations help generalization.
We also performed several additional experiments on other data sets,highlighting limitations and clarifying the methods.
What was Easy
The paper is very detailed and we could faithfully replicate the experiments (bar the aforementioned exception).
Weonly utilized the authors’ code sparingly, but the repository is very well documented and provides pre-trained models.
What was Difﬁcult
The of one augmentations caused confusion on part the paper of the proposed technique method actually improved on the original results.
Communication with Original Authors
Most of our question could be resolved by comparing our implementation against the authors’ code.
We contacted theﬁrst author regarding our different results, he was very responsive and answered all of our questions.
Introduction
High-quality implementations of state-of-the-art image synthesis methods are available to the general public [Karraset al., 2020].
This technology is astonishing and concerning at the same time.
One the one hand, such algorithms can beused for practical use cases like image-to-image translation [Isola et al., 2017, Liu et al., 2017, Zhu et al., 2017] orimage super-resolution [Lai et al., 2017, Ledig et al., 2017, Lim et al., 2017].
On the other hand, they can generateimages which even trick human observers [Simonite, 2019].
Recently, fake news and tampered media have becomepractical problems which even have the potential to inﬂuence democratic processes [Thompson and Lapowsky, 2017,
Hao, 2019].
Recognizing these problems, Wang et al.
[2020] propose a “universal” detector for CNN-generated images.
Whilecreating a detector for know architectures has been done before [Zhang et al., 2019, Wang et al., 2019a], the authorsof this paper ask whether it is conceivable to train a classiﬁer which generalizes to new, unseen architectures.
Theauthors demonstrated that with a diverse data set and with carefully chosen data augmentation methods, a standardimage classiﬁer can achieve this goal.
More speciﬁcally, they train a ResNet
[He et al., 2016] classiﬁer on a data setconsisting of images generated by 20 different instances of ProGAN [Karras et al., 2018].
Additionally, they augmentthe training process by utilizing aggressive image ﬁltering techniques (i.e., Gaussian Blur and JPEG compression).
Combining both techniques allows them to create an image classiﬁer which even generalized to StyleGAN2 [Karraset al., 2020], which was published concurrently.
Scope of reproducibility: We opted to reimplement the methods from scratch, aiming at highlighting difﬁculties forpractitioners who want to adapt these methods.
While the authors released a very well documented code repository,we initially abstained from inspecting the source code to not prime ourselves.
In the later stages of our analysis, weused their repository to uncover some small implementation differences, which lead to our implementation actuallyimproving on the original results.
Using our implementation, we aim at verifying the two main claims of the paper:
The proposed data augmentations (usually) improve generalization. More diverse data sets help the classiﬁer generalize to unseen architectures.
We recreate parts of the original experiments which directly investigate these two claims (see Section 4.2, Section 4.3,
Table 2, and the Figures 2 and 3 in the original paper [Wang et al., 2020]).
Departing from their work, we perform severaladditional experiments: First, we further investigate the ﬁrst claim, training multiple additional data set combinations toclarify differences between our results and the original paper.
Second, we generate a new training set from a different
CNN generator (StyleGAN2 [Karras et al., 2020]) and evaluate if the results transfer.
Finally, we investigate if themethod is exclusive to the speciﬁc image classiﬁer used in the paper.
To this end, we train two other classiﬁers andcompared them to the original results: A different image classiﬁer architecture [Simonyan and Zisserman, 2015] and aclassiﬁer which leverages spectral analysis [Frank et al., 2020].
Methodology
In this section, we provide an overview of our methodology.
We start by presenting the different data sets used in ourexperiments.
Then, we describe the overall experimental setup, including the models we studied, data augmentationsperformed, details on the training procedure, our evaluation criteria, and computational requirements.
All of ourreproducibility efforts are based on the arXiv version of the paper (v2 – submitted 04.
April 2020) and the corresponding
GitHub repository 1 (commit f692c13 – 26. October 2020).
We also make our implementation, our pre-trained modelsand our data set publicly available 2 .
2.1 Data Sets
For training our networks, we used two different data sets: The ﬁrst one is provided by the original publication.
Theauthors used 20 ProGAN [Karras et al., 2018] models, each trained on a different LSUN category [Yu et al., 2015],to sample 18K generated images per model.
They also collected 18K training images for each of the correspondingclasses, resulting in a total data set of (18 .
+ 18 . 000) [-∗ 
-]· 20 = 720 . 000 images.
Additionally, we created a new data set to study the proposed techniques on a different base generator.
We downloaded
Using these models, we generated a training set of train (18K fake and 18K real) images for the LSUN cat and horse data sets, following the pre-/post-processing stepsoutlined in the original publication.
Since these categories are also contained in the data set by Wang et al.
, this allowsfor a one-to-one comparison.
We exclusively used the test data set provided by the authors.
The data set consists of images from 11 different synthesismodels:
• GANs: The data set contains samples from six different Generative Adversarial Networks (GANs): Pro-
GAN [Karras et al., 2018], StyleGAN [Karras et al., 2019], BigGAN Brock et al.
[2019], GauGAN [Park et al.,
2019], CycleGAN [Zhu et al., 2017] and StarGAN [Choi et al., 2018].
• Perceptual loss: The authors also include samples from generative models which are directly trained to
optimize a perceptual loss: Cascaded Reﬁnement Networks (CRN)
[Chen and Koltun, 2017] and ImplicitMaximum Likelihood Estimation (IMLE)
[Li et al., 2019].
• Image manipulation models:
It includes samples from Seeing In The Dark (SITD)
[Chen et al., 2018], whichapproximates long-exposure photography, and the Second-Order Attention Network (SAN)
[Dai et al.
, 2019],which generates images at a higher resolution.
• Deep fakes: Finally, the authors include samples from the FaceForensics++ [Rössler et al., 2019] data set.
2.2 Experiment Setup
In the following, we provide an overview of the different experiments performed in the paper.
We start by discussingthe different models used, present the data augmentations proposed by Wang et al., and, proceed by presenting thetraining schedule used in all experiments.
Finally, we conclude by discussing how the evaluation and with an overviewof the computational requirements needed for reproducing this report.
Models used: Following the original paper, we use a ResNet50 [He et al., 2016] model as the image classiﬁer for themajority of our experiments.
The classiﬁer is pre-trained on ImageNet [Russakovsky et al., 2015].
We use randomcrops (to 224 × 224 pixels) and horizontal ﬂipping during training of all of our classiﬁers.
Additionally, we scale theimage to the range [0 , 1] , remove the mean, and scale them to unit variance.
For the experiments presented in Section 3.3.2, we also train a VGG-11 [Simonyan and Zisserman, 2015] model and aclassiﬁer which utilizes spectral information [Frank et al., 2020].
The VGG model is also pre-trained on ImageNetand uses the same preprocessing as our ResNet models.
The spectral classiﬁer also uses random crops and horizontalﬂipping as outlined above.
Afterwards, we follow Frank et al.
and use Discrete Cosine Transfrom-II [Ahmed et al.
(DCT-II) to transform the images from the spatial to the spectral domain.
Afterwards, we use MinMax-scaling toscale the images to the range [0 , 1] and train a ResNet50 model from scratch on the transformed data.
Data augmentations: Wang et al.
proposed several data augmentations which help their classiﬁer generalize tounseen architectures:
• Gaussian blur (Blur): Before cropping, with 50% probability, the images are blurred with σ ∼ Uniform { 0 , 3 } • JPEG-compressiong (JPEG): with 50% probability images are JPEG-ed by two popular image libraries,OpenCV and PIL, with quality ∼ Uniform { 30 , . .
• Blur + JPEG (Blur & JPEG (0.5)): images are possibly blurred and JPEG-ed, each with 50% probability.
• Blur + JPEG (Blur & JPEG (0.1)): similar to the previous, but the probability is dropped to 10%.
All augmentations are applied before cropping and ﬂipping (to the entire training set).
Additionally, we also evaluate noaugmentations ( No Aug. ) as a baseline.
Note that the JPEG standard only speciﬁes quality setting in the range [30 , 95] .
However, the authors sample in the range [30 , 100] .
We follow their sampling, but clip all values to the allowed range.
Training details: At the start of the training, we randomly sample 10% of the training set for validation.
We train allmodel using Adam [Kingma and Ba, 2015] with β 1 = 0 .
9 , β 2 = 0 .
999 , a batch size of 64, and an initial learning rateof 10 − 4 .
We drop the learning rate by 10 × when the validation accuracy stagnates for 5 epochs.
When the learning ratereaches 10 − 7 , we terminate training and select the best classiﬁer based on the validation accuracy.
We abstained from validating additional combinations of (Adam-) hyperparameters, learning rate, and batch size.
The selection of data augmentation and data set variety is already computational challenging and can be viewed as ahyperparameter in its own right.
However, we note that this might be an interesting direction for future experiments. [Table]
We use the same settings for training the VGG network.
When training the DCT-ResNet we change the initial learningrate to 10 − 3 since we train the underlying model from scratch.
Evaluation criteria:
We follow the authors and use Average Precision (AP) as our evaluation metric.
It is a rankingbased metric which is not sensitive to the “base rate” of the fraction of fake images and thus commonly used throughoutthe literature [Zhou et al., 2018, Huh et al., 2018, Wang et al., 2019b].
Additionally, we compute the mean Average
Precision (mAP) over all data sets, to compute an overall tendency.
During testing, all images are center croppedwithout resizing to match the training dimension (without augmentations).
Computational requirements: All our experiments are run on two desktop machines.
Each machine is running
Ubuntu 18.04, with 64GB RAM, a AMD Ryzen 7 3700X 8-Core Processor, and two GeForce RTX 2080Ti.
Training asingle ResNet50 model on one GPU on the entire data set (20 classes) takes around 42 hours.
Most of the experimentsare conducted on smaller data set (2–8 classes), which signiﬁcantly reduces the training time to roughly 6 hours.
Overall,we estimate the entire experiments to take around [-
-]590 GPU hours, not including initial testing runs.
Results
Overall we were able to reproduce the trends of the original results and verify the two claims of the publication.
In Section 3.1, we examine the ﬁrst claim, studying in detail the different data augmentations proposed by the authors.
In
Section 3.2, we examine the second claim, evaluating if more diverse data help generalization.
Finally, in Section 3.3,we depart from the original experiments and investigate further to gain additional insights into the proposed methods. 3.1 Claim: Data augmentations improve generalization
First, we investigate the claim that data augmentations improve generalization and reproduce the second half of Table 2in the original publication.
Speciﬁcally, we train ﬁve classiﬁer on the entire training set, with each classiﬁer trained onone data augmentation introduced in Section 2.2.
The results are presented in Table 1.
Results: We can successfully recreate the general trend that data augmentations help generalization. However, forspeciﬁc cases our results differ heavily from the reported results by Wang et al.. This is especially noticeable for the bluraugmentation, where our results differ in multiple data sets (StylGAN, BigGAN, CycleGAN, CRN, SITD and SAN). The results for the SAN data set even differ by 30 .
We also seem to slightly outperform the results by Wang et al..
We veriﬁed our implementation against the original published code and noticed two substantial differences: First, weimplemented Gaussian ﬁltering with the build-in methods provided by PyTorch [Paszke et al., 2019], while Wanget al.
build their own method using SciPy [Virtanen et al., 2020].
Second, we apply blurring in conjunction with JPEGcompression with a probability of 50%.
In contrast, Wang et al.
ﬁrst apply blurring with a probability of 50% and thenapply JPEG with a probability of 50%. Note that the paper describes this correctly.
[Table]
To investigate if the results are indeed connected to our implementation, we perform an additional experiment. Weimplemented a corrected version of the Blur + JPEG augmentation and rerun the experiment.
The results are mixed.
While we achieve a closer result for the DeepFake data set, the changes lead to even higher differences in the SAN and SITD data sets. We repeat the experiment two additional times to investigate if this might be related tochance. We obtained widely varying results on the SITD ( 84 . 8 / 85 .
9 / 90 . 7 ), the SAN ( 72 . 6 / 74 . 1 / 78 . 3 ), and the DeepFake ( 72 . 4 / 73 . 0 / 76 . 1 ) data sets. A full table can found in the supplementary material. The authors already noted the SAN and DeepFake data set as exceptions.
Our model recovers better from using acombination of blurring and JPEG compression for these sets. Thus, we hypothesize that our implementation may applymore aggressive preprocessing (we always apply blurring and JPEG compression), but on fewer training examples (ourimplementation is all or nothing; Wang et al. can apply blurring, JPEG compression, or both). The probabilistic nature ofthese data augmentations can also explain the ﬂuctuation in the SITD, SAN, and DeepFake results.
Obtaining statisticalguarantees for our results would be ideal. However, we would need to perform multiple runs of ours’ and Wang et al.
’simplementation. With a single run already taking 42 GPU hours, we abstain from further investigation.
We also do notinvestigate the hand-crafted blurring implementaion. We assume that future practitioners will preferably use the build-in PyTorch method we utilized (unavailable at the time of original publication). We hypothesize that our experimentsbetter reﬂect future results.
Second, we investigate how diversity of the data set affects the results by reproducing the rest of Table 2.
Similar to theoriginal paper, we train ﬁve different classiﬁers on subsets of the original training data ( { 2 , 4 , 8 , 16 , 20 } -classes) anduse blurring and JPEG compression with a probability of 50% during training.
The results are displayed in Table 2. Results: Overall, we can again recreate the general trend that data diversity improves generalization. The resultsmostly match, but we can again observe widely different results for the SITD, SAN and DeepFake data sets.
Weattribute these differences to the earlier observed sensitivity to the random character of these experiments.
The original publication observed that data diversity helped with generalization up to a speciﬁc point.
The authorshypothesized that at this point (16 to 20 classes)
the training set may be diverse enough for practical generalization.
Weagree that data set diversity helps with generalization.
We cannot fully agree that there exists a point of diminishingreturns.
Utilizing more classes allows our classiﬁers to achieve signiﬁcantly better results on the DeepFake ( 76 . 8 → 93 .
AP) and SITD ( 84 . 5 → 92 . 9 AP) data sets.
While the diminishing returns might be related to chance, we additionally investigate if they are connected to theaggressive data augmentations used in the original experiments.
We perform a control experiment where we again trainmultiple classiﬁers on subsets of the training data with all augmentations disabled.
Note the original publication onlyassessed subsets with augmentations enabled.
The results are presented in Figure 1, a full table can be found in the supplementary material.
Contrary to our hypothesis,the data augmentation help the model generalize in the presence of more diverse data.
However, they also lead to [Figure]
i [Figure] n
[Figure]
v
[Figure]
r ag e P r ec i s i o n
[Figure]
[Figure]
P r ec i s i
[Figure]
r ag e P r ec
i s i o n
[Figure]
varying results on the SITD, SAN, and
DeepFake data set.
We assume the same probabilistic nature observed in Section 3.1 is related to the ﬂuctuations. For the other data sets, we assume that the augmentations act as a regularizer,preventing the model from overﬁtting to the training set.
We study this phenomena further in Section 3.3.1. Overall weconclude that more diverse data improves generalization.
3.3 Results beyond the original paper
We perform further experiments to get more insights into the original paper.
First, we investigate if the results aredependent on the generator used to create the training set by sampling a second data sets from StyleGAN2 [Karras et al.,2020].
Second, we investigate if the results are dependent on the classiﬁer used by comparing the ResNet50 against adifferent image classiﬁer (VGG-11) and a classiﬁer which leverages spectral analysis [Frank et al., 2020].
3.3.1 Research Question: Do the results depend on the generator which creates the data set?
We want to investigate if the generalization capabilities depend on the generator which creates the training set.
Asdescribed in Section 2.1, we utilized pre-trained StyleGAN2 models to create a novel data set. [Table]
We then trained ﬁve ResNet50 classiﬁer using the augmentation proposed by Wang et al..
For comparison, we alsotrained ﬁve classiﬁer on LSUN cat and horse data generated by ProGAN.
In contrast to our previous experiments, wealso include a StyleGAN2 test set which was provided by Wang et al.
after the original publication.
The results aredisplayed in Figure 2 and the corresponding Table can be found in the supplementary material.
1) Data augmentations help ProGAN generalise to StyleGAN2: When utilizing the proposed data set augmentation, the classiﬁers trained on ProGAN generalize to detect images from StyleGAN2 ( 87 .
→ { 86 . 3 , 87 .
1 , 89 . 2 , 97 . 1 }AP).
However, the same does not apply for StyleGAN2.
Data generated by StyleGAN2 already achieves quite a highperformance on ProGAN, thus adding data augmentations does only slightly improve or even hurts performance.2) Data augmentations are speciﬁc to the data set: Blurring seems to generally hurt the ProGAN
classiﬁer (CycleGAN, GauGAN, CRN, IMLE, SAN, and overall mAP), while it allows the StyleGAN2 classiﬁer to generalize to the
ProGAN, the StarGAN, the SITD, and, the DeepFake data set. It also achieves the best overall performance.
Thus, dataaugmentations seem to be speciﬁc to the training data set used and should be evaluated carefully.3)
ProGAN serves as a better prior for generalization: If we compare the individual performance and the overallmAP, the classiﬁers trained on ProGAN seem to better generalize to other architectures.
We hypothesize that this is dueto the fact that StyleGAN2 is a very recent proposal (released in 2020), while all other generators were proposed earlier(2016-2019).
ProGAN is itself an older generator (2018), thus the images generated by it might have more in commonwith older architectures.
We conclude that the results indeed depend on the generator used for creating the data set.
The data augmentations donot transfer and our results imply that going forward, practitioners might have to continuously update their training datasets to generalize to new architectures.
Note that we only performed experiments with two classes, adding more classesmight improve the performance of StyleGAN2 as a training set.
3.3.2 Research Question: Do data augmentations transfer to other classiﬁer?
Finally, we want to investigate the question if the results transfer to other classiﬁers.
We train two other classiﬁers tocompare our results: a VGG-11 model and a classiﬁer which utilizes spectral information (DCT-ResNet).
The resultsare depicted in Table 3, additionally, we provide a plot of the results in the supplementary material.
When we reportrelative numbers, these are in reference to the results without augmentations (No Aug.).
When using (No. Aug/Blur) augmentations the VGG model is incapable of generalizing to the GauGAN ( 60 . 7 / 56 .
6 AP),
CRN ( 56 . 8 / 51 . 5 AP) and IMLE ( 56 . 8 / 51 . 5 AP) data sets. This can be circumvented by utilizing the JPEG augmentation,which also leads to the overall best result ( 93 . 1 mAP).
Yet, the augmentation does cause a drop in AP for the StarGAN,
SITD, SAN and DeepFake data sets.
This performance drop can also not be offset by using both augmentations inconjunction.
The performance of the DCT based classiﬁer is worse overall ( 76 . 1 / 78 .
5 / 82 . 4 mAP). Blur seem to help generalization, allowing for better results on CycleGAN, StarGAN, CRN and IMLE data sets. JPEG augmentation allowsthe classiﬁer to drastically improve its results on SITD ( +52 . 8 AP) and helps generalization to CRN ( +14 . 1 AP), IMLE ( +20 . 1 AP), SAN ( +9 . 2 AP) and DeepFake ( +14 . 9 AP).
However, it severely hurts performance for Star-
GAN ( − 39 . 2 AP) and GauGAN ( − 52 . 1 AP), and, to a lesser extend, StyleGAN, BigGAN and CycleGAN. Combiningboth augmentations leads to a good compromise, achieving the best overall performance ( 82 . 4 mAP) and reducing thenegative effects for StarGAN ( − 8 .
7 AP) and GauGAN ( − 21 .
1 AP). Overall, we conclude that the proposed data augmentations transfer to other classiﬁers.
However, there exists no silverbullet which helps generalization in general.
Discussion
Since we focused our efforts on reproducing the paper from scratch, we can only partially comment on the source code.
However, we later used it to compare our implementation against.
In the following, we want to give an overview of ourobservations.
4.1 What was easy?
The overall description in the paper is very detailed.
The authors discuss in detail how they collected the data for thetest set, provide a good description of their training procedure, and summarize their results clearly with the aid of tablesand ﬁgures.
Additionally, the authors include a long appendix with further details.
We did ﬁnd one description in thepaper confusing, which we discuss in more detail in Section 4.2.
As discussed above, we only had a limited exposure to using the authors’ code.
The parts we looked at were of goodcode quality, however, sometimes could use more comments.
The README ﬁle of the repository is very detailed, itincludes clear instruction on how to set up the code, download the corresponding data, train models from scratch, andevaluate the results.
Additionally, the authors include links to download pre-trained models.
We used these models tovalidate our results and found them quite intuitive to use.
4.2 What was difﬁcult?
We came across very few hurdles when reproducing the code.
However, we did end up slightly miss-implementingone of the data augmentations.
In the paper, the authors describe their introduced data augmentations in Section 4.1.
The description for the Blur+JPEG augmentation states: “the image is possibly blurred and JPEG-ed, each with 50%probability”.
However, the shorthand explicitly uses a plus sign and in the remainder of the paper only the shorthand isused.
Thus, based on the frequent use of the plus sign, we assumed the augmentation always applies the blurring and
JPEG compression together.
This resulted in us implementing a slightly different version of the data augmentation,where we apply both in conjunction with 50% probability.
We want to stress that the paper states this correctly, wesimply missed the last part of the description.
4.3 Communication with original authors
Due to the quality of the paper and the availability of the code repository, we could resolve every question.
We didmessage the authors regarding our different results for the Blur and JPEG augmentation.
They agreed that it is difﬁcultto conclude an exact reason why and how augmentations help generalization.
For further research direction, theysuggested further research into which features the CNN learns.
Conclusion
In summary, we believe the paper to be reproducible.
We successfully recreated the original experiments from scratch.
While our results are different, the overall trend remains the same.
However, our additional experiments revealed severallimitations to these claims.
More diverse data sets help the classiﬁer generalize to unseen architectures, but: the choice of generator is crucialand data set-speciﬁc augmentations have to be used.
When using no augmentations, more data diversity hurts theperformance of the classiﬁer trained on ProGAN.
When we instead use StyleGAN2 as the generator for the dataset, we achieve signiﬁcantly lower performance for all tested conﬁgurations.
This suggest the possibility that futurepractitioners might have to constantly update their training sets with new architectures.
Also, the data augmentations seem to be speciﬁc for the training data set used.
When using ProGAN generated dataas the training set, the augmentations transfer across different classiﬁers.
However, only blurring helps StyleGAN2generalize, while the other proposed augmentations hurt performance.
Thus, we advice practitioners to carefullyinvestigate which augmentations help their current situation.