Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a wavelet-based approach for deepfake image detection. This approach is the first using wavelet for the task and could be of great contribution to the community mixing spatial and frequency information. Experimental results are presented for:

Binary classification (FFHQ vs. Stylegan 1/2)
CelebA+LSUN vs. GANs
Unknown generator test
Comparison with SOTA are carried out only vs. none of the best paper on the field
main_review: The paper is well structured, mathematical details are well presented and experimental results demonstrate the effectiveness of the technique. However I have some concerns:

Section 4.2 and 4.3 presents results obtained on different image-sets. Please compare with more advanced SOTA techniques (as reported in related work)

Given the demonstration that first layers of CNNs encode something extremely similar to wavelets (Mallat, Stéphane. "Understanding deep convolutional networks." Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 374.2065 (2016): 20150203.) please provide explanation of the contribution of employing wavelet + CNN and not directly a CNN without the wavelet extraction phase.

Moreover the method does not take into account any kind of manipulation or transformation on images. How does compression affects the technique? Resizing? Rotation? Etc.

Many cited papers obtain better results in detection and classification. The authors compare for parameters. There are works that employs analytical solutions with extremely few parameters. They should be taken into account and improvements w.r.t. them should be emphasized
summary_of_the_review: The paper is ok... but lacks of a contribution for community in my opinion and does not advance state of the art in any terms (computations speed, parameters, overall accuracy results, explaibaility).
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper leverage wavelet-based representation in deepfake detection. By applying wavelet transformation, several features are obtained. In this work, the authors shows the wavelet features at different stages are more robust than previous widely used frequency or image based representation. The representation can be applied like image itself to perform deepfake detection, for example CNN and PRNU. This paper leverage synthetic images from different GAN models to conduct their experiments, including cross-domain detection from an unknown domain.
main_review: This paper proposes a very simple solution for deepfake detection based on wavelet representation.
Strength:
+ The wavelet has not been exploited in deepfake detection, which can provide multiscale representation from low-frequency representation to high-frequency representation.
+ From the experiments shown in this paper, the wavelet representation looks promising in deepfake detection, that many features at different stages can successful perform deepfake detection, and they achieve better results than image or DCT based representation.=
+ Overall, the paper is clearly written and easy to follow.

Weakness:
- The solution is trivial. It seems more study and designs can be exploited. Since wavelet provides multiscale representation, how can you leverage those features together for deepfake detection. However, this paper only simply learns a CNN to achieve classification. The idea of applying wavelet looks good and promising, but need a better solution for more favorable detection.
- The experiments are not solid. Even though authors test their methods on a set of GAN models, they don't test it on the synthetic images with attacks, such as cropping, JPEG compression, low-pass filtering etc. Besides, detecting GAN generated images is interesting, however, there are many existing deepfake detection benchmarks. The authors need to show results on the public benchmark and compare your method with more state-of-the-art models.
- References are not enough. The authors focus on discussing the image and frequency based representation. However, there are many other works in this area recently. Such as CVPR21, ICCV21, IJCAI21.
  * C. Wang et al., Representative Forgery Mining for Fake Face Detection, CVPR, 2021.
  * H. Zhao et al., Multi-Attentional Deepfake Detection, CVPR, 2021.
  * T.Zhao et al., Learning Self-Consistency for Deepfake Detection, ICCV, 2021.
  * Y. He et al., Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis, IJCAI, 2021.

As a result, I think this work requires a lot of extensions for publication, and rate this submission as marginally below the acceptance threshold.



summary_of_the_review: My main concerns are that the experiments are solid enough and the method is trivial. The authors only apply wavelet to extract multiscale features and test them separately.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a framework to perform source identification using wavelet-packet-based analysis of GAN-generated images. They implement boundary-wavelet filters which make use of the share identical network architectures across different wavelet lengths to perform GAN source classification.
main_review: Strengths:
1. The paper uses a different representation of images i.e wavelet-packets to solve source identification problem which has not been used before. 
2. The analysis on using different levels  for Daubechies and symlets wavelets provides better understanding of which level is useful for detecting GAN generated images.

Weaknesses:

1. The main idea of using wavelet coefficients to perform deepfake detection is not a novel approach. Wavelet representation is localized in space and frequency domain. However, both domains have already been studied in the past [1], [2]. Therefore, using a representation combining both of them is not a very exciting idea. Looking at the source identification results in Tab. 2 highlights the fact that the combined representation is performing very similar to previous approaches with worse performance for LSUN dataset. This makes me question the contribution of the proposed approach for the community. Using a new representation although gives an idea to look at a new direction, but the performance doesn't contribute positively for the same. 

2. The authors provide a thorough analysis of their approach in Tab. 2 which talks about the source identification task. However, the problem of source identification is quite simple and previous approaches already have made significant advancement in the performance. The authors try to go one step further by removing one GAN from the training set and evaluate their approach. However, they don't provide any baseline results to compare with for proposed testing protocol. This gives no conclusion whether the proposed approach works better than pervious works or not. 


3. The paper proposes to use wavelet coefficients at a certain level to perform a transformation of input image using quadruple filters as shown in Eq. 2. The filters defined in Eq. 1 are used to perform the above transformation. These filters denote the low-low, low-high, high-low and high-high coefficients. However, the intuition for using these four types of coefficients is still not clear. Each type of coefficient can have different performance for deepfake detection. It is not clear which of the four is responsible for the reported result in Tab. 2 and 4.

4.  In page 4, there is a discussion for the wavelet transform being invertible. This is done using analysis filters H and inverse filters F. The paragraph further talks about perfect reconstruction and anti-aliasing conditions to select these filters. However, a brief introduction of the above mentioned conditions and how these conditions are used to select filters for the proposed framework should be mentioned clearly. 

5. The authors tackle the problem of source identification. But it would be interesting to see how the proposed method would perform when trained and tested on deepfake datasets like FaceForensics++, DFDC, UADFV, etc. Real vs fake binary classification performance would provide much more clarity on whether the proposed approach actually works for deepfake detection or not as depicted by the main title of the paper. 

6. In section 4.1, the authors use a linear regression model to separate the 3rd level Haar wavelet packets for classification. However, in section 4.2, the authors use a CNN-classifier to perform classification. it is not clear why different models were used for different datasets. 

7. The authors use Haar and Daubechies wavelets as well as symlets wavelets in their experiments. However, it would be beneficial to include a brief introduction for both type of wavelets in section 3 to better understand the intuition behind selecting these two for this work.


8. The setting of experiment on FFHQ images dataset mentions that the images are used at a resolution of 128 x 128. However, FFHQ has 1024 x 1024 images. Same is the output for StyleGAN and StyleGAN2. So my concern is that the patterns observed in the wavelet visualization in Figure 1 might be formed due to downsampling of these images. Therefore, the difference in the visualization might come from downsampling rather than GAN itself. 

References: 
1. Ning Yu, Larry S Davis, and Mario Fritz. Attributing fake images to gans: Learning and analyzing gan fingerprints. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7556–7566, 2019
2. Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 3247–3258. PMLR, 2020. 
summary_of_the_review: The paper lacks enough novelty in the technical approach. The authors tackle a very simple problem of source identification. The authors try to perform evaluation on unseen GANs but lack of baselines to compare it with makes the analysis redundant. The experimental evaluation is not thorough across different datasets for deepfake detection which seems to be the main motivation of the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents an approach to detect GAN generated Deepfake images using wavelet features. The paper also showed linear separation of GANs using the proposed approach. Experiments presented on many datasets showed that the approach is promising and presents a method that integrates both spatial and frequency domain information. 
main_review: Strengths:

Usage of wavelets to detect GAN generated images is a definite strength of this paper. Both pixel domain and frequency domain methods have been applied in the past to detect GAN generated images. Usage of wavelets integrates these. 

The source separation experiments are also strengths in this paper.

Weakness:

One main drawback of the paper is that no robustness analysis is done on the proposed method. While the authors mention that they will do this as future work, this analysis is an important part in media forensics and it is essential to know how a detection method would perform when post processing operations are performed on the image. It would have been good if the authors presented the results on at least some post processing operations like JPEG compression and noise addition. 

It will also be good to consider more GAN datasets such as CycleGAN, StarGAN, BIGGAN and others, such that the GANs are all diverse. 

Many past papers have shown patterns in the frequency domain for GAN generated images that are flagged due to upsampling. It will be good to compare those patterns and the wavelet patterns side by side. Patterns can be found in:

Ref (a). Zhang, Xu, Svebor Karaman, and Shih-Fu Chang. "Detecting and simulating artifacts in gan fake images." In 2019 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1-6. IEEE, 2019.

Ref (b). Wang, Sheng-Yu, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A. Efros. "CNN-generated images are surprisingly easy to spot... for now." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8695-8704. 2020.


summary_of_the_review: While the paper provides an interesting method to detect GAN generated images using wavelets, thus combining pixel and frequency domain, more experiments are needed to provide a good assessment of the method.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose to learn a model for the detection of synthetic images based on the wavelet-packet representation of natural and generative adversarial neural network (GAN)-generated images. In experimental evaluation, the proposed method was compared with FFHQ, CelebA, and LSUN source identification problems.
main_review: #Strengths:
1) The study pursued should be pursued with importance given the rise in deepfake images in the internet.
2) The methods section is well written.

#Weaknesses:
The paper can be improved based on the following points:
1) In section 1, it would be better to include a motivational case study to emphasize the importance of the research being pursued for a broader group of readers to be motivated.
2) Please briefly mention why previous works only relied on identifying fingerprints in either the spatial or frequency domain? What were the main challenges associated and why/how is this research work is able to overcome that? This wasn’t mentioned properly in section 1 or 2.
3) In section 3.3, for Fast Wavelet Transform (FWT) please briefly mention the use of FWT and its associated justification. Please do the same for 3.4 i.e. Boundary Wavelet as well. 
4) Though in section 3 the different wavelet methods are mentioned well, for the equations the symbols used are not intuitive and explanatory. Please specify them properly.
5) In the experimental section, the comparative study seemed to be weak. Why were the methods chosen? Could you please provide a variety of comparative study to prove efficacy?
summary_of_the_review: The paper is well written in some places, however, does has the opportunity to be improved, especially it was lacking justification in some places and could be helpful to include some more comparative study to prove the efficacy. Please refer to the points mentioned under weaknesses section in the main review to get details.
review_title: null

Meta-review: This paper received 5 quality reviewers, where 3 of them rated 5 and 2 rated 3. While this work has merits, many concerns are raised by various reviewers. The AC agrees with the reviewers that this paper is not ready for publication at its current form.
Decision: Reject