Title: FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge
Venue: NeurIPS 2024
Accepted: yes
Keywords: DeepFake Detection

Security and Privacy

Multimedia Forensics
Abstract: Generating synthetic fake faces, known as pseudo-fake faces, is an effective way to improve the generalization of DeepFake detection. Existing methods typically generate these faces by blending real or fake faces in spatial domain. While these methods have shown promise, they overlook the simulation of frequency distribution in pseudo-fake faces, limiting the learning of generic forgery traces in-depth. To address this, this paper introduces {\em FreqBlender}, a new method that can generate pseudo-fake faces by blending frequency knowledge. Concretely, we investigate the major frequency components and propose a Frequency Parsing Network to adaptively partition frequency components related to forgery traces. Then we blend this frequency knowledge from fake faces into real faces to generate pseudo-fake faces. Since there is no ground truth for frequency components, we describe a dedicated training strategy by leveraging the inner correlations among different frequency knowledge to instruct the learning process. Experimental results demonstrate the effectiveness of our method in enhancing DeepFake detection, making it a potential plug-and-play strategy for other methods.

FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge

Abstract

Generating synthetic fake faces, known as pseudo-fake faces, is an effective way to improve the generalization of DeepFake detection. Existing methods typically generate these faces by blending real or fake faces in spatial domain. While these methods have shown promise, they overlook the simulation of frequency distribution in pseudo-fake faces, limiting the learning of generic forgery traces in-depth. To address this, this paper introduces FreqBlender , a new method that can generate pseudo-fake faces by blending frequency knowledge. Concretely, we investigate the major frequency components and propose a Frequency Parsing Network to adaptively partition frequency components related to forgery traces. Then we blend this frequency knowledge from fake faces into real faces to generate pseudo-fake faces. Since there is no ground truth for frequency components, we describe a dedicated training strategy by leveraging the inner correlations among different frequency knowledge to instruct the learning process. Experimental results demonstrate the effectiveness of our method in enhancing DeepFake detection, making it a potential plug-and-play strategy for other methods.

1 Introduction

DeepFake refers to face forgery techniques that can manipulate facial attributes, such as identity, expression, and lip movement [1]. The recent advancement of deep generative models [2, 3] has

greatly sped up the evolution of DeepFake techniques, enabling the creation of highly realistic and visually imperceptible manipulations. However, the misuse of these techniques can pose serious security concerns [4], making DeepFake detection more pressing than ever before.

There have been many methods proposed for detecting DeepFakes, showing their effectiveness on public datasets [5, 6, 7, 8, 9]. However, with the continuous growth of AI techniques, new types of forgeries constantly emerge, posing a challenge for current detectors to accurately expose unknown forgeries. To address this challenge, recent efforts [10, 11, 12, 13] have focused on improving the generalizability of detection, i.e. , the ability to detect unknown forgeries based on known examples. One effective approach to address this problem is to enhance the training data by generating synthetic fake faces, known as pseudo-fakes [14, 15, 16]. The intuition behind this approach is that the DeepFake generation process introduces artifacts in the step of blending faces, and these methods generate pseudo-fake faces by simulating various blending artifacts. By training on these pseudofake faces, the models can be driven to learn corresponding artifacts. However, existing methods concentrate on simulating the spatial aspects of face blending (see Fig. 1 (right)). While they can make the pseudo-fake faces resemble the distribution of wild fake faces in the spatial domain, they do not explore the distribution in the frequency domain. Thus, current pseudo-fake faces lack frequency-based forgery clues, limiting the models to learn generic forgery features.

In this paper, we shift our attention from the spatial domain to the frequency domain and propose a new method called FreqBlender to generate pseudo-fake faces by blending frequency knowledge (see Fig. 1 (left)). To achieve this, we analyze the composition of the frequency domain and accurately identify the range of forgery clues falling into. Then we replace this range of real faces with the corresponding range of fake faces to generate pseudo-fake faces. However, identifying the frequency range of forgery clues is challenging due to two main reasons: 1) this range varies across different fake faces due to its high dependence on face content, and 2) forgery clues may not be concentrated on a single frequency range but could be an aggregation of various portions across multiple ranges. Thus, general low-pass, high-pass, or band-pass filters are incapable of precisely pinpointing the distribution.

To address this challenge, we propose a Frequency Parsing Network (FPNet) that can adaptively partition the frequency domain based on the input faces. Specifically, we hypothesize that the faces are composed of three frequency knowledge, which represents semantic information, structural information, and noise information , respectively, and the forgery traces are likely hidden in structural information. This hypothesis is validated in our preliminary analysis (refer to Sec. 3 for details). Based on this footstone, we design the network consisting of a shared encoder and three decoders to extract corresponding frequency knowledge. The encoder transforms the input data into a latent frequency representation, while the decoders estimate the probability map of the corresponding frequency knowledge.

Training this network is non-trivial since no ground truth of frequency distribution is provided. Therefore, we propose a novel training strategy that leverages the inner correlations among different frequency knowledge. To be specific, we describe dedicated-crafted objectives that are performed on various blending combinations of the output from each decoder and emphasize the properties of each frequency knowledge. The experimental results demonstrate that the network successfully parses the desired frequency knowledge within the proposed training strategy.

Once the network is trained, we can parse the frequency component corresponding to the structural information of a fake face, and blend it with a real face to generate a pseudo-fake face. It is important to note that our method is not in conflict with existing spatial blending methods, but rather complements them by addressing the defect in the frequency domain. Our method is validated on multiple recent DeepFake datasets ( e.g. , FF++ [5], CDF [6], DFDC [8], DFDCP [7], FFIW [9]) and compared with many state-of-the-art methods, demonstrating the efficacy of our method in improving detection performance.

The contributions of this paper are summarized in three-fold: 1) To the best of our knowledge, we are the first to generate pseudo-fake faces by blending frequency knowledge. Our method pushes pseudofake faces closer to the distribution of wild fake faces, enhancing the learning of generic forgery features in DeepFake detection. 2) We propose a Frequency Parsing Network that can adaptively partition the frequency components corresponding to semantic information, structural information, and noise information, respectively. Since no ground truth is provided, we design dedicated objectives

to train this network. 3) Extensive experimental results on several DeepFake datasets demonstrate the efficacy of our method and its potential as a plug-and-play strategy for existing methods.

2 Related Works

The rapid progress of AI generative models has spawned the development of DeepFake detection methods. These methods mainly rely on deep neural networks to identify the inconsistency between real and fake faces using various features, including biological signals [17], spatial artifacts [18, 14, 15, 16], frequency abnormality [13, 19, 20], auto-learned clues from dedicated-designed models [21, 22]. These methods have shown promising results on public datasets. However, some of their performance significantly deteriorates when confronted with unknown DeepFake faces due to the large distribution discrepancy resulting from limited training datasets. To tackle this issue, many methods have been proposed to improve their generalizability by learning the generic DeepFake traces, e.g. , [10, 14, 15, 16, 12, 23, 24]. One effective approach is to create synthetic fake faces during training, known as pseudo-fake faces, e.g. , [10, 14, 15, 16]. FWA [10] is a pioneering method that conducts self-blending to simulate fake faces. Several extended variants (Face X-ray [14], PCL [15], SBI [16], BiG-Arts [25]) have been proposed to blend faces using curated strategies, further improving detection performance. By increasing the diversity of training faces, the gap in the distribution of wild fake faces can be reduced, allowing the models to learn the invariant DeepFake traces across different distributions. To generate the pseudo-fake faces, existing methods usually design spatial blending operations to combine different faces. This involves extracting the face region from a source image and blending it into a target image. However, these methods overlook the distribution of wild fake faces in the frequency domain. While the synthetic faces may resemble the spatial-based distribution, the lack of consideration for frequency perspective hinders the models from learning the fundamental generic DeepFake traces.

3 Preliminary Analysis

We perform a statistical analysis of the frequency distribution of real and fake faces and present preliminary results for the main frequency components corresponding to semantic information, structural formation, and noise information, respectively.

Inspiration and Verification. The investigation in previous works [26, 27] has indicated that the forgery traces mainly exist in high-frequency areas. However, the precise range of these areas has not been described, driving us to re-investigate the frequency distribution of forgery traces.

Specifically, we conduct verification experiments using FaceForensics++ (FF++) [5] datasets. We extract the frames from all videos and randomly select 3 , 000 real images and 3 , 000 fake images for each manipulation method ( e.g. , DF, F2F, FS, and NT). Then we crop out the face region in these selected images using a face detector [28] and apply DCT [29] to generate frequency maps. For analysis, we sum up all frequency maps of real and fake images and adopt the visualization process of azimuthal average described in previous work [30, 27]. This process involves logarithmic transformation and the calculation of azimuthally-averaged flux in circular annuli apertures. By placing the center of the circular annuli aperture at the top-left corner of the frequency map, we can obtain a one-dimensional array representing the spectrum diagram. The visual results of their distribution are shown in Fig. 2 (top). It can be observed that this figure is consistent with the results in [26]. However, when we directly plot their distribution differences without logarithmic operation, the results do not match the previous figure. It can be seen that the disparities in high-frequency regions are not as substantial as expected, while the differences in the lower range become more noticeable (see Fig. 2 (bottom)). This is because the logarithmic operation mitigates the degree of differences in lower frequency ranges, causing the illusion that only the high-frequency range exhibits differences between real and fake faces. Therefore, we conjecture that the forgery traces may not only be concentrated in a very high-frequency range but could possibly spread to the low-frequency range.

Hypothesis and Validation. As shown in Fig. 2, the most significant difference can be observed in the range of very low frequency. Given the significant dissimilarity in appearance between real and fake faces, we hypothesize that the semantic information is mainly represented in this low-frequency band. Moreover, we hypothesize that the mid-to-high frequency components capture the structural information, making them more susceptible to containing forgery traces. Furthermore, we hypothesize

Deepfakes

Face2Face

FaceSwap

NeuralTextures

1.0

1.0

1.0

1.0

real

real

real

real

fake

fake

fake

fake

0.5

0.5

0.5

0.5

↑

↑

个

0.0七

0.0

0.0

0.0t

0

200

400

0

200

400

0

200

400

0

200

400

18

difference

18

difference

14

difference

18

difference

9

9

7

9

0

0

200

400

0

0

200

400

0

0

200

400

0

0

200

400

Deepfakes

Face2Face

FaceSwap

NeuralTextures

0

0

0

100

100

100

100

200

200

200

200

300

300

300

300

400

400

400

400

0

100

200

300

400

0

100

200

300

400

0

100

200

300

400

0

100

200

300

400

that the highest frequency components likely correspond to the noise introduced by various video preprocessing operations, such as compression, decompression, and encoding.

To validate our hypothesis, we directly visualize the difference between real and fake faces on their frequency maps in Fig. 3. By observing these results, we empirically split the frequency map into three non-overlap bands. The split operations follow the general band-pass filters. Denote the position in the frequency map as ( x, y ) , where (0 , 0) , (1 , 1) denotes the top-left corner and bottom-right corner. Specifically, we identify the region where x + y ≤ 1 / 16 as containing semantic information, the region where 1 / 16 < x + y ≤ 1 / 2 as containing structural information, and the region where x + y > 1 / 2 as containing noise information. The corresponding results are visualized in Fig. 4, validating that these three ranges provide empirical evi-

Frequency

Component

Real

Fake

Real

Fake

dence that aligns with our frequency distribution hypothesis.

4 FreqBlender

We describe a new method to create pseudo-fakes by blending specific frequency knowledge. The motivation is that existing methods only focus on spatial domain blending, which overlook the disparity between real and fake faces in the frequency domain. By considering the frequency distribution, the pseudo-fakes can closely resemble the fake faces. To achieve this, we propose

GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

(ℰ(GLYPH(cmap:d835)(GLYPH(cmap:d835))))

⊗

⊗

⊗

ℒGLYPH(cmap:d835)GLYPH(cmap:d835)

ℒGLYPH(cmap:d835)GLYPH(cmap:d835)

ℒGLYPH(cmap:d835)GLYPH(cmap:d835)

ℒGLYPH(cmap:d835)GLYPH(cmap:d835)

Encoder

Decoder

Decoder

Decoder

Input Image: x

GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

(ℰ(GLYPH(cmap:d835)(GLYPH(cmap:d835))))

GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

(ℰ(GLYPH(cmap:d835)(GLYPH(cmap:d835))))

GLYPH(cmap:d835)

GLYPH(cmap:d835)

-1

(GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

(GLYPH(cmap:d835)))

GLYPH(cmap:d835)

-1

(GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

(GLYPH(cmap:d835)))

GLYPH(cmap:d835)

-1

(GLYPH(cmap:d835)

GLYPH(cmap:d835)GLYPH(cmap:d835)GLYPH(cmap:d835)

(GLYPH(cmap:d835)))

Corollary1

Corollary2

Corollary3

Corollary4

dct

+

Encoder

Decoder

×

3

Input

Output

Cony

Cony

Shufne

a Frequency Parsing Network (FPNet) to partition the frequency domain into three components, corresponding to semantic information, structural information, and noise information, respectively. We then blend the structural information of fake faces with the real faces to generate the pseudo-fakes. The details of the Frequency Parsing Network are elaborated in Sec. 4.1, and the objective and training process for this network is described in Sec. 4.2. Then we introduce the deployment of our method with existing methods in Sec. 4.3.

4.1 Frequency Parsing Network

Overview. The Frequency Parsing Network (FPNet) is composed of one shared encoder and three independent decoders. The encoder transforms the input faces into frequency-critical features and the decoders aim to decompose the feature from the encoder and extract the respective frequency components.

Denote the encoder as E and three decoders as D sem , D str , D noi respectively. Given an input face image x ∈ X ∈ { 0 , 255 } h × w × 3 , we first convert this face to the frequency map as ϕ ( x ) ∈ R h × w × 3 , where ϕ denotes the operations of Discrete Cosine Transform (DCT). Then we send this frequency map into model and generate three distribution maps as D sem ( E ( ϕ ( x ))) ∈ [0 , 1] h × w , D str ( E ( ϕ ( x ))) ∈ [0 , 1] h × w , and D noi ( E ( ϕ ( x ))) ∈ [0 , 1] h × w respectively. Each distribution map indicates the probability of the corresponding frequency component distributed in the frequency map. Given these distribution maps, we can select the corresponding frequency components conveniently. For example, the frequency component corresponding to semantic information can be selected by ϕ sem ( x ) = ϕ ( x ) D sem ( E ( ϕ ( x ))) and the same is for other two frequency components, i.e. , ϕ str ( x ) = ϕ ( x ) D str ( E ( ϕ ( x ))) and ϕ noi ( x ) = ϕ ( x ) D noi ( E ( ϕ ( x ))) . The overview of FPNet is shown in Fig. 5 (left).

Network Architecture. The encoder simply consists of four convolution layers with a kernel size of 3 × 3 , a stride of 2 , and a padding of 1 . Each decoder also consists of four layers, and each layer is a combination of a convolutional layer and PixelShuffle operation [31] (see Fig. 5 (right)).

4.2 Objective Design for FPNet

The most challenging and crucial aspect of our method is to train the network for frequency parsing, as there is no ground truth available for the different frequency components. Note that the only available resources for supervising the training are the preliminary analysis results in Section 3. Nevertheless, these results are not precise and can not be adaptive to different inputs, which are insufficient for model training. Therefore, we meticulously craft a couple of auxiliary objectives to instruct the learning of networks, allowing for the self-refinement of the network.

These objectives are designed based on the following proposition.

Proposition 1. Each frequency component exhibits the following properties:

Semantic information can reflect the facial identity.

Structural information serves as the carrier of forgery traces.

Noise information has minimal impact on visual quality.

The preliminary analysis findings are generally applicable.

Corollary 1. For a given face x , the transformed face based on its semantic information will retain the same facial identity as x , i.e. , ∀ x ∈ X ∈ { 0 , 255 } h × w × 3 , F ( ϕ -1 ( ϕ sem ( x ))) = F ( x ) , where F denotes a face recognition model and ϕ -1 denotes the Inverse Discrete Cosine Transform (IDCT).

Facial Fidelity Loss. We introduce a facial fidelity loss L ff to penalize the discrepancy in identity between the input face image and the spatial content represented by semantic information. To measure the identity discrepancy, we employ the MobileNet [32] as our face recognition model and train it using ArcFace [33, 34]. We select MobileNet for its balance between computational efficiency and recognition accuracy. Let F be the MobleNet and F f ( x ) be the facial features extracted from the face image x . The facial fidelity loss can be defined as

Note that the input face x can be either real or fake, as the identity information is present in both cases.

Corollary 2. For a given real face x r , it can be detected as fake if and only if it is inserted the structural information from a fake face x f , i.e. , D ( x r ) = 0 iff x r ← x r ⊕ ϕ str ( x f ) , where D denotes a Deepfake detector with labels of fake and real in { 0 , 1 } , ⊕ indicates the inserting operation.

Authenticity-determinative Loss. This loss is designed to emphasize the determinative role of structural information. To evaluate the authenticity of faces, we develop a DeepFake detector D , which is implemented using a ResNet-34 [35] trained on real and fake faces. Then we construct two sets of faces by blending frequency components.

The first set contains three types of faces transformed from frequency components corresponding to 1) the semantic information of the real face, 2) the semantic information of the fake face, and 3) the semantic information of the real face blended with the structural information of the real face. We denote this set as C r = { ϕ -1 ( ϕ sem ( x r )) , ϕ -1 ( ϕ sem ( x f )) , ϕ -1 ( ϕ sem ( x r ) + ϕ str ( x r )) } . Since there is no structural information from fake faces in this set, all the faces should be detected as real.

Similarly, the second set contains two types of faces: 1) blending the semantic information of the fake face with the structural information of the fake face, and 2) blending the semantic information of the real face with the structural information of the fake face. We denote this set as C f = { ϕ -1 ( ϕ sem ( x f ) + ϕ str ( x f )) , ϕ -1 ( ϕ sem ( x r ) + ϕ str ( x f )) } . Since all blended faces in this set contain the structural information of fake faces, they should be detected as fake. Thus the authenticitydeterminative loss L ad can be written as

where CE denotes the cross-entropy loss.

Corollary 3. The face should exhibit no visible change if the frequency component of noise information is removed, i.e. , ∀ x ∈ X ∈ { 0 , 255 } h × w × 3 , x ≈ x ⊖ ϕ noi ( x f ) , where ⊖ indicates the removing operation.

Quality-agnostic Loss. As noise information does not contain decisive details for the overall depiction of the image, the face image is expected to be similar to the face image transformed using the frequency components of semantic and structural information. This similarity can be quantified using the quality-agnostic Loss L qa , defined as

where the face x can be either real or fake.

Corollary 4. Each frequency component is bound by the preliminary results, i.e. , there should be no significant deviation between the predicted frequency component and the approximate frequency distribution in preliminary analysis.

Prior and Integrity Loss. According to our analysis in the Preliminary Analysis section, we have an initial understanding of the approximate frequency distribution. Denote the initial frequency maps for semantic, structural, and noise information as m sem , m str , m noi , respectively. These maps are utilized to accelerate the convergence of the model towards the desired direction. Moreover, we add a constraint on the integrity of their distributions, ensuring that their combination covers all elements of the frequency map. This loss L pi can be expressed as

where 1 denotes a mask where all the elements in it is 1 .

Overall Objectives. The overall objectives are the summation of all these loss terms, as

where λ 1 , λ 2 , λ 3 , λ 4 are the weights for different loss terms.

4.3 Deployment of FreqBlender

Given a fake face x r and a real face x f , we can generate a pseudo-fake face by

Note that in our method, it is not necessary to perform the blending using wild fake faces. Instead, we can tactfully substitute wild fake faces with the pseudo-fake faces generated by existing spatial face blending methods. It allows us to overcome the limitations in the frequency distribution of existing pseudo-fake faces.

5 Experiments

5.1 Experimental Setups

DataSets. Our method is evaluated using several standard datasets, including FaceForensics++ [5] (FF++), Celeb-DF (CDF) [6], DeepFake Detection Challenge (DFDC) [8], DeepFake Detection Challenge Preview (DFDCP) [7], and FFIW-10k (FFIW) [9] datasets. Specifically, the FF++ dataset consists of 1000 pristine videos and 4000 manipulated videos corresponding to four different manipulation methods, that are Deepfakes (DF), Face2Face (F2F), FaceSwap (FS), and NeuralTextures (NT). CDF dataset comprises 590 pristine videos and 5639 high-quality fake videos created from DeepFake alterations of celebrity videos available on YouTube. DFDC is a large-scale deepfake dataset, that consists of 100 , 000 video clips, and DFDCP is a preview version of DFDC, which is also widely used in evaluation. The FFIW dataset contains 8250 pristine videos and 8250 DeepFake videos with multi-face scenarios. We follow the original training and testing split provided by the datasets for experiments.

Implementation Details. Our method is implemented using PyTorch 2.0.1 [36] with a Nvidia 3090ti. In the training stage of FPNet, the image size is set to 400 × 400 . The batch size is set to 8 and the Adam optimizer is utilized with an initial learning rate of 1 e -4 . The training epoch is set to 200 . The hyperparameters in the objective function in Eq. (5) are set as follows: λ 1 = 1 / 12 , λ 2 = 1 , λ 3 = 1 e -3 , λ 4 = 1 / 4 . For DeepFake detection, we employ the vanilla EfficientNet-b4 [37] as our model following [16]. In the training phase, we create pseudo-fake faces on-the-fly. We first generate synthetic faces using the spatial-blending method [16] and then blend them with real faces using our method with a probability of α = 0 . 2 . Other training and testing settings are the same as [16]. More analysis of parameters are provided in Supplementary.

5.2 Results

To showcase the effectiveness of our method, we train our method solely on the FF++ dataset and test it on the other different datasets. We employ the Area Under the Receiver Operating Characteristic Curve (AUC) as the evaluation metric following previous work [16]. Our method is compared with five video-based detection methods, including Two-branch [38], DAM [9], LipForensics [1],

FTCN [39] and SST [24]. Moreover, we involve thirteen frame-level state-of-the-art methods for comparison, which are DSP-FWA [10], Face X-ray [14], F3-Net [29], LRL [40], FRDM [41], PCL [15], DCL [42], SBI [16], TALL-Swin [22], UCF [12], SST [24], F-G [43], LSDA [23],respectively.

Cross-dataset Evaluation. We evaluate the cross-dataset performance of our method compared to other counterparts in Table 1. The best performance is highlighted in blue and the second-best is marked by red. It should be noted that our method operates on pseudo-fake faces generated by SBI, thus we do not need fake faces. In comparison to video-level methods, our method achieves the best performance, which outperforms all the methods by a large margin.

When compared to frame-level methods, our method still outperforms the others. For example, our method improves upon the performance of the most relevant counterpart SBI by 1 . 41% , 2 . 17% , 1 . 41% , 1 . 31% on CDF, DFDC, DFDCP, and FFIW respectively. This improvement can be attributed to the incorporation of frequency knowledge in pseudo-fake faces, enhancing the generalization of detection models. Note that the performance of the compared methods (except SBI* and F3-Net) is extracted from their original papers. SBI* denotes the performance obtained using the officially released codes, and F3-Net is reproduced using the codes implemented by others 3 . The results closely align with the reported scores, which verifies the correctness of our configuration of their codes. In subsequent experiments, we employ their release codes for comparison.

Cross-manipulation Evaluation. Since SBI is the most recent and effective method, we compare our method with it for demonstration. Specifically, we compare our method with two variants of the SBI method. The first is trained using the raw set of real videos in the FF++ dataset, while the second is trained using the c23 set. According to the standard protocols, all methods are tested on c23 videos. The results are shown in Table 2. It can be seen that our method outperforms SBI-raw by 3 . 62% and SBI-c23 by 3 . 89% , demonstrating the efficacy of our method on cross-manipulation scenarios.

5.3 Analysis

Effect of Each Objective Term. This part studies the effect of each objective term on CDF, DFDC, and DFDCP datasets. The results are shown in Table 3. Note that Baseline denotes only using prior and integrity loss L pi and 'w/o' denotes without. It can be seen that without one certain objective

term, the performance drops on all datasets, which demonstrates that different objective terms have distinct impacts, and their collective contributes most to our method.

Complementary to Spatial-blending Methods. To validate the complementary of our method, we replace the SBI method with other spatial-blending methods and study if the performance is improved. Specifically, we reproduce the pseudo-fake face generation operations in DSP-FWA, I2G, and Face X-ray, and combine them with our method. Note that I2G and Face X-ray have not released their codes, we re-implement them rigorously following their original settings. The results on FF++, CDF, DFDCP, FFIW datasets are presented in Table 4. It can be seen that by combining with DSP-FWA, our method improves 0 . 12% averagely. A similar trend is also observed in the I2G, which improves 0 . 84% on average. For Face X-ray, we improve the performance by 3 . 08% on average. It is noteworthy that I2G and Face X-ray have not released their codes yet. We rigorously follow the instructions as in their papers and run the codes widely used by others 4 5 .

Different Network Architectures. This part validates the effectiveness of our method on different networks, including ResNet-50 [35], EfficientNet-b1 [37], VGG16 [44], Xception [45], ViT [46], F3-Net [29], and GFFD[41]. We compare our method with SBI on these networks, which are tested on CDF, FF++, DFDCP, and FFIW datasets. The results are shown in Table 5. It can be observed that our method improves the performance by 1 . 78% , 0 . 41% , 1 . 51% , 0 . 2% , 2 . 75% , and 4 . 68% averagely on ResNet-50, EfficientNet-b1, Xception, Vit networks, F3-Net, and GFFD networks respectively. It is noteworthy that our method slightly reduces the performance of VGG16 by 0 . 01% . It is possibly because the capacity of VGG16 is limited than other networks, and learning spatial pseudo-fake faces almost fills up this capacity, leaving no room for the learning of frequency knowledge.

DF

SBI

Ours

F2F

FS

NT

SBI

Ours

SBI

Ours

SBI

Ours

Saliency Visualization. We employ Grad-CAM [47] to visualize the attention of our method compared to SBI on four manipulations in the FF++ dataset. Compared to SBI, our method concentrates more on the structural information, such as the manipulation boundaries. For example, our method highlights the face outline in DF, F2F, and FS, while focusing on the mouth contour in NT.

Effect of Using Wild Fake or Spatial Pseudo-fake Faces. As described in Sec. 4.3, our method is performed using real and spatial-blending pseudo-fake (SP-fake) faces. The rationale is that SP-fake faces are greatly diversified, containing more spatial forgery traces. Applying our method to these faces can consider both frequency and spatial traces effectively. To verify this, we directly perform our method on real and wild fake faces. The results in Table 6 indicate a notable performance drop when only wild fakes are used.

Limitations. Our method is designed to address the drawbacks of existing spatial-blending methods. Hence, it inherits the assumption that the faces are forged by face-swapping techniques. Further research is needed to validate our performance on other types of forgery operations, such as wholeface synthesis and attribute editing.

6 Conclusion

This paper describes a new method called FreqBlender that can generate pseudo-fake faces by blending frequency knowledge. To achieve this, we propose a Frequency Parsing Network that adaptively extracts the frequency component corresponding to structural information. Then we can blend this information from fake faces into real faces to create pseudo-fake faces. The extensive Experiments demonstrate the effectiveness of our method and can serve as a complementary module for existing spatial-blending methods.

Acknowledgement. This work is supported in part by the National Natural Science Foundation of China (No.62402464), Shandong Natural Science Foundation (No.ZR2024QF035), and China Postdoctoral Science Foundation (No.2021TQ0314; No.2021M703036). Jiaran Zhou is supported by the National Natural Science Foundation of China (No.62102380) and Shandong Natural Science Foundation (No.ZR2021QF095, No.ZR2024MF083). Baoyuan Wu is supported by Guangdong Basic and Applied Basic Research Foundation (No.2024B1515020095), National Natural Science Foundation of China (No. 62076213), and Shenzhen Science and Technology Program (No. RCYX20210609103057050). Bin Li is supported in part by NSFC (Grant U23B2022, U22B2047).

References

Alexandros Haliassos, Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. Lips don't lie: A generalisable and robust approach to face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5039-5049, 2021.

Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, and Ziwei Liu. Forgerynet: A versatile benchmark for comprehensive forgery analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 4360-4369, 2021.

Kunlin Liu, Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Wenbo Zhou, and Weiming Zhang. Deepfacelab: Integrated, flexible and extensible face-swapping framework. Pattern Recognition(PR) , 141:109628, 2023.

Supasorn Suwajanakorn, Steven M Seitz, and Ira Kemelmacher-Shlizerman. Synthesizing obama: learning lip sync from audio. ACM Transactions on Graphics (ToG) , 36(4):1-13, 2017.

Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV) , pages 1-11, 2019.

Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 3207-3216, 2020.

Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) preview dataset. arXiv preprint arXiv:1910.08854 , 2019.

Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397 , 2020.

Tianfei Zhou, Wenguan Wang, Zhiyuan Liang, and Jianbing Shen. Face forensics in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 5778-5788, 2021.

Yuezun Li and Siwei Lyu. Exposing deepfake videos by detecting face warping artifacts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , June 2019.

Jiaming Li, Hongtao Xie, Jiahong Li, Zhongyuan Wang, and Yongdong Zhang. Frequencyaware discriminative feature learning supervised by single-center loss for face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 6458-6467, 2021.

Zhiyuan Yan, Yong Zhang, Yanbo Fan, and Baoyuan Wu. Ucf: Uncovering common features for generalizable deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 22412-22423, 2023.

Yuan Wang, Kun Yu, Chen Chen, Xiyuan Hu, and Silong Peng. Dynamic graph learning with content-guided spatial-frequency relation reasoning for deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 7278-7287, 2023.

Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 5001-5010, 2020.

Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, and Wei Xia. Learning selfconsistency for deepfake detection. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV) , pages 15023-15033, 2021.

Kaede Shiohara and Toshihiko Yamasaki. Detecting deepfakes with self-blended images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 18720-18729, 2022.

Yipin Zhou and Ser-Nam Lim. Joint audio-visual deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 14800-14809, 2021.

Weiming Bai, Yufan Liu, Zhipeng Zhang, Bing Li, and Weiming Hu. Aunet: Learning relations between action units for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 24709-24719, 2023.

Changtao Miao, Zichang Tan, Qi Chu, Nenghai Yu, and Guodong Guo. Hierarchical frequencyassisted interactive networks for face manipulation detection. IEEE Transactions on Information Forensics and Security (TIFS) , 17:3008-3021, 2022.

Qiqi Gu, Shen Chen, Taiping Yao, Yang Chen, Shouhong Ding, and Ran Yi. Exploiting finegrained face forgery clues via progressive enhancement learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 735-743, 2022.

Junyi Cao, Chao Ma, Taiping Yao, Shen Chen, Shouhong Ding, and Xiaokang Yang. Endto-end reconstruction-classification learning for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4113-4122, 2022.

Yuting Xu, Jian Liang, Gengyun Jia, Ziming Yang, Yanhao Zhang, and Ran He. Tall: Thumbnail layout for deepfake video detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 22658-22668, 2023.

Zhiyuan Yan, Yuhao Luo, Siwei Lyu, Qingshan Liu, and Baoyuan Wu. Transcending forgery specificity with latent space augmentation for generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8984-8994, 2024.

Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, and Jongwon Choi. Exploiting style latent flows for generalizing deepfake video detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1133-1143, 2024.

Han Chen, Yuezun Li, Dongdong Lin, Bin Li, and Junqiang Wu. Watching the big artifacts: Exposing deepfake videos via bi-granularity artifacts. Pattern Recognition (PR) , 135:109179, 2023.

Shuai Jia, Chao Ma, Taiping Yao, Bangjie Yin, Shouhong Ding, and Xiaokang Yang. Exploring frequency adversarial attacks for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4103-4112, 2022.

Ricard Durall, Margret Keuper, and Janis Keuper. Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 78907899, 2020.

Davis King. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research (JMLR) , Dec 2009.

Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by mining frequency-aware clues. In European conference on computer vision (ECCV) , pages 86-103. Springer, 2020.

Ricard Durall, Margret Keuper, Franz-Josef Pfreundt, and Janis Keuper. Unmasking deepfakes with simple features. arXiv preprint arXiv:1911.00686 , 2019.

Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , pages 1874-1883, 2016.

Andrew G Howard. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 , 2017.

Jiankang Deng, Jia Guo, Jing Yang, Niannan Xue, Irene Cotsia, and Stefanos P Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , page 1-1, Jan 2021.

Jiahao Liang, Huafeng Shi, and Weihong Deng. Exploring disentangled content information for face forgery detection. In European Conference on Computer Vision (ECCV) , pages 128-145. Springer, 2022.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , Jun 2016.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.

Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning (ICML) , volume 97 of Proceedings of Machine Learning Research , pages 6105-6114. PMLR, 09-15 Jun 2019.

Iacopo Masi, Aditya Killekar, Royston Marian Mascarenhas, Shenoy Pratik Gurudatt, and Wael AbdAlmageed. Two-branch recurrent network for isolating deepfakes in videos. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VII 16 , pages 667-684. Springer, 2020.

Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. Exploring temporal coherence for more general video face forgery detection. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV) , pages 15044-15054, 2021.

Shen Chen, Taiping Yao, Yang Chen, Shouhong Ding, Jilin Li, and Rongrong Ji. Local relation learning for face forgery detection. In Proceedings of the AAAI conference on artificial intelligence , volume 35, pages 1081-1088, 2021.

Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. Generalizing face forgery detection with high-frequency features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 16317-16326, 2021.

Ke Sun, Taiping Yao, Shen Chen, Shouhong Ding, Jilin Li, and Rongrong Ji. Dual contrastive learning for general face forgery detection. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 2316-2324, 2022.

Li Lin, Xinan He, Yan Ju, Xin Wang, Feng Ding, and Shu Hu. Preserving fairness generalization in deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 16815-16825, 2024.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , pages 1251-1258, 2017.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR , 2021.

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision (IJCV) , page 336-359, Feb 2020.

Ziqi Huang, Kelvin C.K. Chan, Yuming Jiang, and Ziwei Liu. Collaborative diffusion for multi-modal face generation and editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023.

Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 4401-4410, 2019.

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) , pages 8110-8119, 2020.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Machine Learning (ICML) , 2015.

Jiakai Wang. Adversarial examples in physical world. In IJCAI , pages 4925-4926, 2021.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR) , 2018.

Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE symposium on security and privacy (SP) , pages 39-57. IEEE, 2017.

A Appendix / supplemental material

Various Loss Weights λ 1 , λ 2 , λ 3 , λ 4 . These weights are empirically selected based on experimental results. As shown in Table 7, we evaluate the performance across various set of λ 1 , λ 2 , λ 3 , λ 4 . The results exhibit that our method is not particularly sensitive to the settings of loss weights, with performance variations within approximately ∼ 0 . 5% . For our main experiments, we select the set of λ 1 = 1 / 12 , λ 2 = 1 , λ 3 = 0 . 001 , λ 4 = 1 / 4 as it yields the best performance.

More Details of Complementary to Spatial-blending Methods. Table 8 shows the detailed results of every manipulation in the FF++ dataset. It can be seen that our method improves the performance of all manipulation methods, averaging 1 . 32% for DSP-FWA, 4 . 28% for I2G, and 1 . 77% for Face X-ray. This improvement further demonstrates the effectiveness of our method.

Probability α in FreqBlender. As shown in Table 9, we evaluate the effect of using various probability α . Note that α denotes the probability of generating a synthetic face whether using spatialblending or using FreqBlender after spatial-blending. α = 1 denotes only using FreqBlender, while α = 0 means only using spatial-blending. It can be observed that solely using our method can not perform well, as no color space knowledge is involved, hindering the overall effectiveness of pseudofake faces. In contrast, solely using spatial-blending operations can reach a favorable performance. However, when inserting pseudo-fake faces generated by FreqBlender, the generalization performance is further enhanced, as our method can compensate for the loss of frequency knowledge. The optimal effect on CDF evaluation is achieved when α is set to 0.2. This experiment demonstrates the complementary effect of our method to existing spatial-blending methods. More generalization experiments on more datasets have been conducted in the main paper, demonstrating the effectiveness of our method.

Visual Demonstration of FreqBlender. The goal of our method is to create pseudo-fake faces that resemble the frequency distribution of wild fake faces. To verify this, we conduct a visual experiment on the FF++ dataset. Specifically, we randomly select 3 , 000 images from each manipulation method (Deepfakes, FaceSwap, Face2Face, NeuralTextures) and calculate the average frequency map for each manipulation method respectively. Then we create the same number of pseudo-fake faces using

Deepfakes

FaceSwap

Face2Face

NeuralTextures

GLYPH(cmap:d835) = 1

GLYPH(cmap:d835) = 0.5

GLYPH(cmap:d835) = 0.2

GLYPH(cmap:d835) = 0

our method and calculate their average frequency map. Finally, we visualize the frequency difference between our method and wild fake faces. The results are illustrated in Fig. 7. It is important to note that α represents the probability of applying FreqBlender. Hence, α = 1 means all pseudo-fake faces are created using our method, while α = 0 denotes our method is degraded to SBI. From this figure, we can see that the difference is minimal when α = 1 . As α decreases, the difference becomes larger as the pseudo-fake faces are more likely created by SBI. This demonstrates that our method effectively simulates the frequency distribution of wild fake faces.

Performance on FF++ Low-quality (LQ). To investigate the performance of our method on lowquality videos, we conduct experiments on the FF++ Low-quality (LQ) set. The results in Table 10 show that while all methods experience substantial performance drops on the LQ set, our method consistently achieves the highest performance, demonstrating better generalization capability on low-quality videos compared to other methods.

Tentative Validation on Various Synthesized Faces. In addition to validation on the standard datasets, we investigate a new scenario: Diffusion-based face-swap deepfake detection. In this scenario, we employ a recent diffusion model (Collaborative Diffusion [48]) to synthesize faces, which are then blended into original videos. We create 200 fake faces and evaluate our method in Table 11. It can be seen that our method is effective in detecting such forged faces.

Moreover, although our method is designed for face-swapping techniques, we also test it on face images generated by StyleGAN [49] and StyleGAN2 [50]. The results, also shown in Table 11, reveal a notable performance drop across all methods. However, our method still outperforms the others.

Performance against Evasion Attacks. We employ a widely recognized library TorchAttacks with four well-known attack methods: FGSM [51], BIM [52], PGD [53] and CW [54]. The attack experiment is conducted on the CDF dataset with six models and the attack configuration is set by default. The results are shown in Table 12. It can be seen without any defense strategies, all models can be easily attacked in the white-box attacking mode, which exactly aligns with our expectations and the discoveries in the papers of FGSM, BIM, etc.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: After the preliminary analysis section of the paper combined with data analysis, along with our experimental results, we have completed our work comprehensively.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We introduced relevant limitations in the fifth part of our paper.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We present our assumptions in the Preliminary analysis section of this paper, and our experimental results confirm the validity of our assumptions.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: You can perfectly reproduce our results according to our paper, and we will also open-source our code in the future.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: The code will be thoroughly organized and released after the paper is accepted. Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: In the Experimental Setups section of our paper, we have provided detailed descriptions of our experimental details.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: Our paper's experiments do not include error bars.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: In the Experimental Setups section of our paper, we have provided detailed descriptions of our compute resources.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have reviewed the code of ethics and are confident that we have adhered to this guideline.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Although our paper does not discuss the related impacts, our work enhances the capabilities of deepfake detection, which we believe has positive societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our model is designed to enhance the generalization capabilities of deepfake detection, and there is no risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have properly attributed the work of others in our paper and have followed licensing and usage terms.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Our paper does not introduce new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not have this research.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not have this research.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe generalization of DeepFake detection can be addressed by enhancing training data with synthetic fake faces, known as pseudo-fakes. Traditional methods generate these faces by spatial blending operations. However, the limitations of these methods are that they ignore to simulate the frequency distribution, where additional important forgery clues might be found.\n\nThis paper introduces an interesting method called FreqBlender, which attempts to blend proper frequency knowledge to enhance the effectiveness of pseudo-fake faces. This method involves identifying forgery clues in the frequency domain and blending the corresponding frequency components from fake faces into real faces. This process is challenging due to the variability and spread of forgery clues across different frequency ranges.\n\nTo achieve frequency decomposition, the authors propose a Frequency Parsing Network (FPNet) that can adaptively partition the frequency domain. The network, consisting of an encoder and three decoders, extracts semantic, structural, and noise information from the faces. Training FPNet is difficult due to the lack of ground truth for frequency distribution, so the authors devise a novel training strategy leveraging inner correlations among different frequency components.\n\nOnce trained, FPNet can extract the structural information of a fake face's frequency component and blend it with a real face to generate a pseudo-fake face. This method complements existing spatial blending methods and improves detection performance on multiple DeepFake datasets, demonstrating its effectiveness.\n\nSTRENGTHS:\n1. This papeer enhancing the generalization of DeepFake detection is a critical and current issue in AI safety, as real-world DeepFakes are likely generated by unknown models. This method addresses this challenge by taking an innovative approach, creating pseudo-fake faces through blending frequency knowledge rather than the conventional spatial knowledge used in existing methods.\n\n2. This work identifies the limitations of existing methods, noting that spatial blending only mimics the visual similarity of wild fake faces but overlooks their frequency characteristics. By analyzing the frequency distribution, the authors describe a Frequency Parsing Network (FPNet) to parse the frequency range carrying forgery traces. Interestingly, this method doesn't require wild fake faces for blending; instead, it uses existing spatial-based pseudo-fake faces as surrogates.\n\n6. The method's efficacy is validated across multiple recent DeepFake datasets, demonstrating its effectiveness with various backbones and complementing existing spatial-blending methods. This showcases its robustness and practical applicability in enhancing detection performance.\n\nWEAKNESSES:\nThis paper proposes an interesting and intuitive method that seems reasonable to me. Here are some recommendations for future improvements:\n\n1. Investigate more fine-grained frequency ranges to uncover the detailed composition of artifacts. This could potentially enhance the model's performance by providing a more nuanced understanding of forgery clues.\n\n2. Develop algorithms to enhance the explainability of the model’s decisions. This will help users understand how and why a particular face is identified as a DeepFake, increasing trust and usability in practical applications.\n\n3. Extend the method to handle other types of forgery operations beyond face-swapping, such as whole-face synthesis and attribute editing. This will broaden the method's applicability and ensure it remains effective across a wider range of DeepFake techniques.\n\n4. Utilizing LLMs may help you parse the frequency more precisely, as LLMs can provide the prior knowledge which may compensate for the lack of ground truth when training FPNet.\n\nQUESTIONS:\n1.\tAs described in Section 3, the statistics of the frequency distribution are calculated using an azimuthal average, which includes a logarithmic transformation and the calculation of azimuthally-averaged flux in circular annular apertures. Could you provide more details on why placing the center of the circular annular aperture at the top-left corner of the frequency map results in a one-dimensional spectrum diagram?\n2.\tThe FPNet uses three independent decoders to analyze the frequency domain. The output of the decoders is not clearly explained, as it is visualized in grayscale as the frequency map in Fig. 5. I understand that the output is a soft mask ranging from [0,1] for the frequency map. Therefore, I suggest using a color version to replace the DCT map in Fig. 5.\n3.\tIn the discussion on the loss of authenticity determination, the authors aim to create two sets of faces, with and without forgery traces. Does the set C_r represent the face sets that do not contain structural information of fake faces, while C_f corresponds to the face sets that have structural information of fake faces?\n4.\tIn Prior and Integrity Loss, the final term is designed to ensure that the combination of three frequency components covers the entire frequency domain. This term sums these frequency components and calculates the distance from a mask of 1. However, this term might not guarantee that the sum of these frequency components equals 1, but it can reduce deviations. What is the rationale behind this design??\n\nLIMITATIONS:\nThis paper lists its limitations in main body. Since the proposed method is designed to address the limitations of existing spatial-blending techniques. it operates under the assumption that faces are forged using face-swapping methods. It unlikely tackle other types of forgeries, such as whole-face synthesis.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies generalizable deepfake detection. The proposed method is motivated by a new data augmentation method that blends real and fake faces in the frequency domain. The paper claims the forgery can be found in three different frequency bands and proposes an unsupervised learning method, Frequency Parsing Network. Empirical results indicate that the proposed method's performance achieves results comparable to state-of-the-art.\n\nSTRENGTHS:\n1. unsupervised way of learning the frequency component, which can favor its usage. \n2. I like the analysis from Fig 2 and 3, showing the difference between real and fake residing in the high-frequency band.\n3. I enjoy the problem formulation as well as the learning object in section 4.2.\n\nWEAKNESSES:\n1. the proposed method does not achieve state-of-the-art detection performance in table 1.\n2. table 2 only has a few methods on the FF++ dataset. It should not be the case that SBI is only a baseline to compare. \n3. the proposed approach might be limited in standard face-swap fake face, or gan-generated face. How about diffusion model generated face?\n4. in terms of approach. the proposed method works when capturing semantic, structural and noise information. What if it fails to capture any one of those?\n\nQUESTIONS:\n1. line 31 says the motivation of simulating various blending artifacts. However, these artifacts are quite \"old\" compared to that from current diffusion models. That says, how would your method perform on the face generated by stable diffusion? such as instantid, photomaker?\n2. did you compare with DIRE and HiFi-Net?\nR1: DIRE for Diffusion-Generated Image Detection, ICCV2023\nR2: Hierarchical fine-grained image forgery detection and localization, CVPR2023\n\nLIMITATIONS:\nI am impressed by the learning objective formulation, but I am inclined to reject it because it does not achieve reasonable sota performance and does not show generalization to diffusion-generated images.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes FreqBlender, a new method to generate pseudo-fake faces that effectively simulate the frequency distribution of wild fake faces. Unlike common blending techniques done in the spatial domain, their method blends frequency knowledge. An analysis is conducted, showing that three frequency components are present in faces, namely semantic information, structural information, and noise information. They demonstrated that structural information contains forgery traces.\n\nTo this end, the first stage of their method employs FPNet, a novel architecture built to disentangle the input fake face into the three different frequency components. As no ground truth exists for this task, carefully crafted objectives provide the necessary supervision.\n\nIn the second stage, the trained network is used to parse the frequency components, and the structural component is extracted from a given fake face. It is then blended into a real face to obtain the pseudo-fake.\n\nThe method outperforms the state-of-the-art across different relevant datasets.\n\nSTRENGTHS:\nOriginality:\nThis method is the first to mine frequency structural information and propose a way of generating pseudo-fake faces that mimic the frequency distribution of fake faces.\n\nQuality:\nThe method is well evaluated on different datasets, and the results are consistent.\n\nClarity:\nThe paper is well written, and the method is well explained. Figures, along with a preliminary analysis of the frequency components present in faces, are provided, which help to understand the method.\n\nSignificance:\nThis method provides a new pseudo-fake mechanism that complements spatial blending, which is the reference in the literature. The authors have shown that their method improves the results of different frame and spatial blending-based methods such as DSP-FWA, I2G, Face-Xray, and SBI. The authors claim that the code will be released in the future. To ease the reproduction and adoption of this technique, we encourage them to also release the pretrained weights of FPNet.\n\nWEAKNESSES:\nReferences Missing:\nSome references are missing, for example, methods that are frame-based and only use real faces during training [1, 2], as well as other recent detectors [3, 4].\n\nAdditional Overhead:\nDuring the training of the detector, the inference of FPNnet (1 encoder and 3 decoders) is required for generating a pseudo-fake. This adds an overhead during training. The authors should include an efficiency analysis.\n\n[1] Li et al., Pixel bleach network for detecting face forgery under\ncompression, in IEEE Transactions on Multimedia 2023\n[2] Larue et al., SeeABLE: Soft Discrepancies and Bounded Contrastive Learning\nfor Exposing Deepfakes, in ICCV 2023\n[3] Dong et al., Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization, in CVPR 2023\n[4] Guo et al., AltFreezing for More General Video Face Forgery Detection, in CVPR 2023\n\nQUESTIONS:\nClarification Needed on FPNet Training:\nWhile the authors explain that for deepfake detection, pseudo-fakes generated using SBI are used as a substitute for fake faces from FF++, the authors should include the results when real fake faces from FF++ are used in Table 1.\n\nUse of PixelShuffle:\nWhy is a PixelShuffle used in the decoder of FPNet?\n\nFPNet Direct Application:\nCan the FPNet be used directly for deepfake detection? What happens when a real face is input into the FPNet?\n\nAblation Study and Frequency Knowledge:\nThe authors successfully conduct an ablation study on the backbone (Table 5) and show that results improve consistently when compared to SBI. Why does this method introduce frequency knowledge when all the tested backbones are spatial (e.g., EfficientNet-b4)? It would be interesting to compare the proposed method with a built-in frequency-based backbone (e.g., Face Forgery Network (F3-Net) [3], AFFGM [1], or the multi-scale high-frequency feature extractor from [2]).\n\nResults on Highly Compressed Data:\nWhat are the results of the method on highly compressed data, i.e., FF++ LQ?\n\n[1] Li et al., Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection, in CVPR 2021\n[2] Luo et al., Generalizing Face Forgery Detection with High-frequency Features, in  CVPR 2021\n[3] Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues, in ECCV 2020\n\nLIMITATIONS:\nAcknowledgement of Limitations:\nThe authors acknowledge the usual limitations of pseudo-fake generation methods, i.e., the hypothesis that the test face is crafted using face-swapping techniques may not hold when the test face is generated using other techniques.\n\nDiscussion on Societal Impact:\nThe race between attackers and defenders is a well-known issue in the field of deepfake detection. The authors should discuss the potential negative societal impact of their work and how it could be exploited by attackers to generate more realistic deepfakes by simply training their generators to fool the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper have introduced an effective way to improve the generalization of DeepFake detection via generating pseudo-fake faces by blending frequency knowledge.  The proposed approach achieves state-of-the-art (SOTA) results on various deepfake detecion datasets.\n\nSTRENGTHS:\n1) This paper attempts to combine frequency domain information and spatial domain information to deal with deepfake detection, which is very interesting. Spatial domain blending is very common, but it is still very rare to use it in the frequency domain, which is quite innovative.\n\n2) The writing of this paper is easy to understand and the logic is clear.\n\n3) The experiments in this paper are sufficient and effectively support the author's theoretical basis.\n\nWEAKNESSES:\n1) The idea of ​​this paper is similar to some already published papers, such as [1], [2] and [3]. I hope this paper can cite and further analyze them:\n\n[1] Tan, C., Zhao, Y., Wei, S., Gu, G., Liu, P., & Wei, Y. (2024, March). Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 5, pp. 5052-5060).\n\n[2] Yu, B., Li, W., Li, X., Lu, J., & Zhou, J. (2021). Frequency-aware spatiotemporal transformers for video inpainting detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 8188-8197).\n\n[3] Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., & Holz, T. (2020, November). Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning (pp. 3247-3258). PMLR.\n\n2) Moreover, Ref.30 is an important and widely cited work that introduces the frequency domain into deepfake detection. I don’t quite understand why the author does not compare and analyze it with this work.\n\n3) The source information, page numbers, publishers, etc. of many references are incomplete, and some even are incorrect. For example, [4] should come from CVPR2024 instead of arXiv.\n\n[4] Choi, J., Kim, T., Jeong, Y., Baek, S., & Choi, J. (2024). Exploiting Style Latent Flows for Generalizing Deepfake Video Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1133-1143).\n\n4) The faces in Figure 6 should have been resized. It would be better if the author could show the original resolution image to help readers better understand the experimental results.\n\n5)  Why is the fluctuation of  λ3 so much larger than that of other parameters? I hope the author can provide the corresponding theoretical basis and further detailed analysis.\n\n6) If the author can answer and revise the relevant questions in the final version, I will consider increasing the final score in the next round.\n\nQUESTIONS:\nSome images in the paper will be a little blurry after zooming in, so it is best to convert all images to pdf or eps format.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}