FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge

Authors: hanzhe  li
jiaran  zhou
yuezun  li
baoyuan  wu
bin  li
junyu  dong
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 04:40:21

Abstract: Generating synthetic fake faces, known as pseudo-fake faces, is an effective way to improve the generalization of DeepFake detection. Existing methods typically generate these faces by blending real or fake faces in spatial domain. While these methods have shown promise, they overlook the simulation of frequency distribution in pseudo-fake faces, limiting the learning of generic forgery traces in-depth. To address this, this paper introduces {\em FreqBlender}, a new method that can generate pseudo-fake faces by blending frequency knowledge. Concretely, we investigate the major frequency components and propose a Frequency Parsing Network to adaptively partition frequency components related to forgery traces. Then we blend this frequency knowledge from fake faces into real faces to generate pseudo-fake faces. Since there is no ground truth for frequency components, we describe a dedicated training strategy by leveraging the inner correlations among different frequency knowledge to instruct the learning process. Experimental results demonstrate the effectiveness of our method in enhancing DeepFake detection, making it a potential plug-and-play strategy for other methods.

Main paper: 
 introduction: 
 DeepFake refers to face forgery techniques that can manipulate facial attributes, such as identity, expression, and lip movement [1] . The recent advancement of deep generative models [2, 3] has greatly sped up the evolution of DeepFake techniques, enabling the creation of highly realistic and visually imperceptible manipulations. However, the misuse of these techniques can pose serious security concerns [4] , making DeepFake detection more pressing than ever before.There have been many methods proposed for detecting DeepFakes, showing their effectiveness on public datasets [5, 6, 7, 8, 9] . However, with the continuous growth of AI techniques, new types of forgeries constantly emerge, posing a challenge for current detectors to accurately expose unknown forgeries. To address this challenge, recent efforts [10, 11, 12, 13] have focused on improving the generalizability of detection, i.e., the ability to detect unknown forgeries based on known examples. One effective approach to address this problem is to enhance the training data by generating synthetic fake faces, known as pseudo-fakes [14, 15, 16] . The intuition behind this approach is that the DeepFake generation process introduces artifacts in the step of blending faces, and these methods generate pseudo-fake faces by simulating various blending artifacts. By training on these pseudofake faces, the models can be driven to learn corresponding artifacts. However, existing methods concentrate on simulating the spatial aspects of face blending (see Fig. 1 (right)). While they can make the pseudo-fake faces resemble the distribution of wild fake faces in the spatial domain, they do not explore the distribution in the frequency domain. Thus, current pseudo-fake faces lack frequency-based forgery clues, limiting the models to learn generic forgery features.In this paper, we shift our attention from the spatial domain to the frequency domain and propose a new method called FreqBlender to generate pseudo-fake faces by blending frequency knowledge (see Fig. 1 (left)). To achieve this, we analyze the composition of the frequency domain and accurately identify the range of forgery clues falling into. Then we replace this range of real faces with the corresponding range of fake faces to generate pseudo-fake faces. However, identifying the frequency range of forgery clues is challenging due to two main reasons: 1) this range varies across different fake faces due to its high dependence on face content, and 2) forgery clues may not be concentrated on a single frequency range but could be an aggregation of various portions across multiple ranges. Thus, general low-pass, high-pass, or band-pass filters are incapable of precisely pinpointing the distribution.To address this challenge, we propose a Frequency Parsing Network (FPNet) that can adaptively partition the frequency domain based on the input faces. Specifically, we hypothesize that the faces are composed of three frequency knowledge, which represents semantic information, structural information, and noise information, respectively, and the forgery traces are likely hidden in structural information. This hypothesis is validated in our preliminary analysis (refer to Sec. 3 for details). Based on this footstone, we design the network consisting of a shared encoder and three decoders to extract corresponding frequency knowledge. The encoder transforms the input data into a latent frequency representation, while the decoders estimate the probability map of the corresponding frequency knowledge.Training this network is non-trivial since no ground truth of frequency distribution is provided. Therefore, we propose a novel training strategy that leverages the inner correlations among different frequency knowledge. To be specific, we describe dedicated-crafted objectives that are performed on various blending combinations of the output from each decoder and emphasize the properties of each frequency knowledge. The experimental results demonstrate that the network successfully parses the desired frequency knowledge within the proposed training strategy.Once the network is trained, we can parse the frequency component corresponding to the structural information of a fake face, and blend it with a real face to generate a pseudo-fake face. It is important to note that our method is not in conflict with existing spatial blending methods, but rather complements them by addressing the defect in the frequency domain. Our method is validated on multiple recent DeepFake datasets (e.g., FF++ [5] , CDF [6] , DFDC [8] , DFDCP [7] , FFIW [9] ) and compared with many state-of-the-art methods, demonstrating the efficacy of our method in improving detection performance.The contributions of this paper are summarized in three-fold: 1) To the best of our knowledge, we are the first to generate pseudo-fake faces by blending frequency knowledge. Our method pushes pseudofake faces closer to the distribution of wild fake faces, enhancing the learning of generic forgery features in DeepFake detection.2) We propose a Frequency Parsing Network that can adaptively partition the frequency components corresponding to semantic information, structural information, and noise information, respectively. Since no ground truth is provided, we design dedicated objectives to train this network. 3) Extensive experimental results on several DeepFake datasets demonstrate the efficacy of our method and its potential as a plug-and-play strategy for existing methods. 
related works: 
 The rapid progress of AI generative models has spawned the development of DeepFake detection methods. These methods mainly rely on deep neural networks to identify the inconsistency between real and fake faces using various features, including biological signals [17] , spatial artifacts [18, 14, 15, 16] , frequency abnormality [13, 19, 20] , auto-learned clues from dedicated-designed models [21, 22] . These methods have shown promising results on public datasets. However, some of their performance significantly deteriorates when confronted with unknown DeepFake faces due to the large distribution discrepancy resulting from limited training datasets. To tackle this issue, many methods have been proposed to improve their generalizability by learning the generic DeepFake traces, e.g., [10, 14, 15, 16, 12, 23, 24] . One effective approach is to create synthetic fake faces during training, known as pseudo-fake faces, e.g., [10, 14, 15, 16] . FWA [10] is a pioneering method that conducts self-blending to simulate fake faces. Several extended variants (Face X-ray [14] , PCL [15] , SBI [16] , BiG-Arts [25] ) have been proposed to blend faces using curated strategies, further improving detection performance. By increasing the diversity of training faces, the gap in the distribution of wild fake faces can be reduced, allowing the models to learn the invariant DeepFake traces across different distributions. To generate the pseudo-fake faces, existing methods usually design spatial blending operations to combine different faces. This involves extracting the face region from a source image and blending it into a target image. However, these methods overlook the distribution of wild fake faces in the frequency domain. While the synthetic faces may resemble the spatial-based distribution, the lack of consideration for frequency perspective hinders the models from learning the fundamental generic DeepFake traces. 
preliminary analysis: 
 We perform a statistical analysis of the frequency distribution of real and fake faces and present preliminary results for the main frequency components corresponding to semantic information, structural formation, and noise information, respectively.Inspiration and Verification. The investigation in previous works [26, 27] has indicated that the forgery traces mainly exist in high-frequency areas. However, the precise range of these areas has not been described, driving us to re-investigate the frequency distribution of forgery traces. Specifically, we conduct verification experiments using FaceForensics++ (FF++) [5] datasets. We extract the frames from all videos and randomly select 3, 000 real images and 3, 000 fake images for each manipulation method (e.g., DF, F2F, FS, and NT). Then we crop out the face region in these selected images using a face detector [28] and apply DCT [29] to generate frequency maps. For analysis, we sum up all frequency maps of real and fake images and adopt the visualization process of azimuthal average described in previous work [30, 27] . This process involves logarithmic transformation and the calculation of azimuthally-averaged flux in circular annuli apertures. By placing the center of the circular annuli aperture at the top-left corner of the frequency map, we can obtain a one-dimensional array representing the spectrum diagram. The visual results of their distribution are shown in Fig. 2 (top). It can be observed that this figure is consistent with the results in [26] . However, when we directly plot their distribution differences without logarithmic operation, the results do not match the previous figure. It can be seen that the disparities in high-frequency regions are not as substantial as expected, while the differences in the lower range become more noticeable (see Fig. 2 (bottom)). This is because the logarithmic operation mitigates the degree of differences in lower frequency ranges, causing the illusion that only the high-frequency range exhibits differences between real and fake faces. Therefore, we conjecture that the forgery traces may not only be concentrated in a very high-frequency range but could possibly spread to the low-frequency range.As shown in Fig. 2 , the most significant difference can be observed in the range of very low frequency. Given the significant dissimilarity in appearance between real and fake faces, we hypothesize that the semantic information is mainly represented in this low-frequency band. Moreover, we hypothesize that the mid-to-high frequency components capture the structural information, making them more susceptible to containing forgery traces. Furthermore, we hypothesize that the highest frequency components likely correspond to the noise introduced by various video preprocessing operations, such as compression, decompression, and encoding.Frequency Component Real Fake Real Fake To validate our hypothesis, we directly visualize the difference between real and fake faces on their frequency maps in Fig. 3 . By observing these results, we empirically split the frequency map into three non-overlap bands. The split operations follow the general band-pass filters. Denote the position in the frequency map as (x, y), where (0, 0), (1, 1) denotes the top-left corner and bottom-right corner. Specifically, we identify the region where x + y ≤ 1/16 as containing semantic information, the region where 1/16 < x + y ≤ 1/2 as containing structural information, and the region where x + y > 1/2 as containing noise information. The corresponding results are visualized in Fig. 4 , validating that these three ranges provide empirical evidence that aligns with our frequency distribution hypothesis. 
freqblender: 
 We describe a new method to create pseudo-fakes by blending specific frequency knowledge. The motivation is that existing methods only focus on spatial domain blending, which overlook the disparity between real and fake faces in the frequency domain. By considering the frequency distribution, the pseudo-fakes can closely resemble the fake faces. To achieve this, we propose a Frequency Parsing Network (FPNet) to partition the frequency domain into three components, corresponding to semantic information, structural information, and noise information, respectively.We then blend the structural information of fake faces with the real faces to generate the pseudo-fakes.The details of the Frequency Parsing Network are elaborated in Sec. 4.1, and the objective and training process for this network is described in Sec. 4.2. Then we introduce the deployment of our method with existing methods in Sec. 4.3.Overview. The Frequency Parsing Network (FPNet) is composed of one shared encoder and three independent decoders. The encoder transforms the input faces into frequency-critical features and the decoders aim to decompose the feature from the encoder and extract the respective frequency components.Denote the encoder as E and three decoders as D sem , D str , D noi respectively. Given an input face image x ∈ X ∈ {0, 255} h×w×3 , we first convert this face to the frequency map as ϕ(x) ∈ R h×w×3 , where ϕ denotes the operations of Discrete Cosine Transform (DCT). Then we send this frequency map into model and generate three distribution maps as ] h×w respectively. Each distribution map indicates the probability of the corresponding frequency component distributed in the frequency map. Given these distribution maps, we can select the corresponding frequency components conveniently. For example, the frequency component corresponding to semantic information can be selected by ϕ sem (x) = ϕ(x)D sem (E(ϕ(x))) and the same is for other two frequency components, i.e., ϕ str (x) = ϕ(x)D str (E(ϕ(x))) and ϕ noi (x) = ϕ(x)D noi (E(ϕ(x))). The overview of FPNet is shown in Fig. 5 (left).D sem (E(ϕ(x))) ∈ [0, 1] h×w ,D str (E(ϕ(x))) ∈ [0, 1] h×w , and D noi (E(ϕ(x))) ∈ [0, 1Network Architecture. The encoder simply consists of four convolution layers with a kernel size of 3 × 3, a stride of 2, and a padding of 1. Each decoder also consists of four layers, and each layer is a combination of a convolutional layer and PixelShuffle operation [31] (see Fig. 5 (right)).The most challenging and crucial aspect of our method is to train the network for frequency parsing, as there is no ground truth available for the different frequency components. Note that the only available resources for supervising the training are the preliminary analysis results in Section 3.Nevertheless, these results are not precise and can not be adaptive to different inputs, which are insufficient for model training. Therefore, we meticulously craft a couple of auxiliary objectives to instruct the learning of networks, allowing for the self-refinement of the network.These objectives are designed based on the following proposition.Proposition 1. Each frequency component exhibits the following properties:1. Semantic information can reflect the facial identity.2. Structural information serves as the carrier of forgery traces.3. Noise information has minimal impact on visual quality.4. The preliminary analysis findings are generally applicable. Corollary 1. For a given face x, the transformed face based on its semantic information will retain the same facial identity as x, i.e., ∀ x ∈ X ∈ {0, 255} h×w×3 , F(ϕ -1 (ϕ sem (x))) = F(x), where F denotes a face recognition model and ϕ -1 denotes the Inverse Discrete Cosine Transform (IDCT).Facial Fidelity Loss. We introduce a facial fidelity loss L ff to penalize the discrepancy in identity between the input face image and the spatial content represented by semantic information. To measure the identity discrepancy, we employ the MobileNet [32] as our face recognition model and train it using ArcFace [33, 34] . We select MobileNet for its balance between computational efficiency and recognition accuracy. Let F be the MobleNet and F f (x) be the facial features extracted from the face image x. The facial fidelity loss can be defined asEQUATIONNote that the input face x can be either real or fake, as the identity information is present in both cases.Corollary 2. For a given real face x r , it can be detected as fake if and only if it is inserted the structural information from a fake face x f , i.e., D(x r ) = 0 iff x r ← x r ⊕ ϕ str (x f ), where D denotes a Deepfake detector with labels of fake and real in {0, 1}, ⊕ indicates the inserting operation.Authenticity-determinative Loss. This loss is designed to emphasize the determinative role of structural information. To evaluate the authenticity of faces, we develop a DeepFake detector D, which is implemented using a ResNet-34 [35] trained on real and fake faces. Then we construct two sets of faces by blending frequency components.The first set contains three types of faces transformed from frequency components corresponding to 1) the semantic information of the real face, 2) the semantic information of the fake face, and 3) the semantic information of the real face blended with the structural information of the real face. We denote this set as C r = {ϕ -1 (ϕ sem (x r )), ϕ -1 (ϕ sem (x f )), ϕ -1 (ϕ sem (x r ) + ϕ str (x r ))}. Since there is no structural information from fake faces in this set, all the faces should be detected as real.Similarly, the second set contains two types of faces: 1) blending the semantic information of the fake face with the structural information of the fake face, and 2) blending the semantic information of the real face with the structural information of the fake face. We denote this set asC f = {ϕ -1 (ϕ sem (x f ) + ϕ str (x f )), ϕ -1 (ϕ sem (x r ) + ϕ str (x f ))}.Since all blended faces in this set contain the structural information of fake faces, they should be detected as fake. Thus the authenticitydeterminative loss L ad can be written asEQUATIONwhere CE denotes the cross-entropy loss. Corollary 3. The face should exhibit no visible change if the frequency component of noise information is removed, i.e., ∀ x ∈ X ∈ {0, 255} h×w×3 , x ≈ x ⊖ ϕ noi (x f ), where ⊖ indicates the removing operation.Quality-agnostic Loss. As noise information does not contain decisive details for the overall depiction of the image, the face image is expected to be similar to the face image transformed using the frequency components of semantic and structural information. This similarity can be quantified using the quality-agnostic Loss L qa , defined asEQUATIONwhere the face x can be either real or fake. Corollary 4. Each frequency component is bound by the preliminary results, i.e., there should be no significant deviation between the predicted frequency component and the approximate frequency distribution in preliminary analysis.Prior and Integrity Loss. According to our analysis in the Preliminary Analysis section, we have an initial understanding of the approximate frequency distribution. Denote the initial frequency maps for semantic, structural, and noise information as m sem , m str , m noi , respectively. These maps are utilized to accelerate the convergence of the model towards the desired direction. Moreover, we add a constraint on the integrity of their distributions, ensuring that their combination covers all elements of the frequency map. This loss L pi can be expressed asEQUATIONwhere 1 denotes a mask where all the elements in it is 1.Overall Objectives. The overall objectives are the summation of all these loss terms, asEQUATIONwhere λ 1 , λ 2 , λ 3 , λ 4 are the weights for different loss terms.Given a fake face x r and a real face x f , we can generate a pseudo-fake face byEQUATION)Note that in our method, it is not necessary to perform the blending using wild fake faces. Instead, we can tactfully substitute wild fake faces with the pseudo-fake faces generated by existing spatial face blending methods. It allows us to overcome the limitations in the frequency distribution of existing pseudo-fake faces. 
experimental setups: 
 DataSets. Our method is evaluated using several standard datasets, including FaceForensics++ [5] (FF++), Celeb-DF (CDF) [6] , DeepFake Detection Challenge (DFDC) [8] , DeepFake Detection Challenge Preview (DFDCP) [7] , and FFIW-10k (FFIW) [9] datasets. Specifically, the FF++ dataset consists of 1000 pristine videos and 4000 manipulated videos corresponding to four different manipulation methods, that are Deepfakes (DF), Face2Face (F2F), FaceSwap (FS), and NeuralTextures (NT). CDF dataset comprises 590 pristine videos and 5639 high-quality fake videos created from DeepFake alterations of celebrity videos available on YouTube. DFDC is a large-scale deepfake dataset, that consists of 100, 000 video clips, and DFDCP is a preview version of DFDC, which is also widely used in evaluation. The FFIW dataset contains 8250 pristine videos and 8250 DeepFake videos with multi-face scenarios. We follow the original training and testing split provided by the datasets for experiments.Implementation Details. Our method is implemented using PyTorch 2.0.1 [36] with a Nvidia 3090ti. In the training stage of FPNet, the image size is set to 400 × 400. The batch size is set to 8 and the Adam optimizer is utilized with an initial learning rate of 1e -4 . The training epoch is set to 200. The hyperparameters in the objective function in Eq. ( 5) are set as follows:λ 1 = 1/12, λ 2 = 1, λ 3 = 1e -3 , λ 4 = 1/4.For DeepFake detection, we employ the vanilla EfficientNet-b4 [37] as our model following [16] . In the training phase, we create pseudo-fake faces on-the-fly. We first generate synthetic faces using the spatial-blending method [16] and then blend them with real faces using our method with a probability of α = 0.2. Other training and testing settings are the same as [16] . More analysis of parameters are provided in Supplementary.To showcase the effectiveness of our method, we train our method solely on the FF++ dataset and test it on the other different datasets. We employ the Area Under the Receiver Operating Characteristic Curve (AUC) as the evaluation metric following previous work [16] . Our method is compared with five video-based detection methods, including Two-branch [38] , DAM [9] , LipForensics [1] , FTCN [39] and SST [24] . Moreover, we involve thirteen frame-level state-of-the-art methods for comparison, which are DSP-FWA [10] , Face X-ray [14] , F3-Net [29] , LRL [40] , FRDM [41] , PCL [15] , DCL [42] , SBI [16] , TALL-Swin [22] , UCF [12] , SST [24] , F-G [43] , LSDA [23] ,respectively.Cross-dataset Evaluation. We evaluate the cross-dataset performance of our method compared to other counterparts in Table 1 . The best performance is highlighted in blue and the second-best is marked by red. It should be noted that our method operates on pseudo-fake faces generated by SBI, thus we do not need fake faces. In comparison to video-level methods, our method achieves the best performance, which outperforms all the methods by a large margin.When compared to frame-level methods, our method still outperforms the others. For example, our method improves upon the performance of the most relevant counterpart SBI by 1.41%, 2.17%, 1.41%, 1.31% on CDF, DFDC, DFDCP, and FFIW respectively. This improvement can be attributed to the incorporation of frequency knowledge in pseudo-fake faces, enhancing the generalization of detection models. Note that the performance of the compared methods (except SBI* and F3-Net) is extracted from their original papers. SBI* denotes the performance obtained using the officially released codes, and F3-Net is reproduced using the codes implemented by others 3 . The results closely align with the reported scores, which verifies the correctness of our configuration of their codes. In subsequent experiments, we employ their release codes for comparison.Cross-manipulation Evaluation. Since SBI is the most recent and effective method, we compare our method with it for demonstration. Specifically, we compare our method with two variants of the SBI method. The first is trained using the raw set of real videos in the FF++ dataset, while the second is trained using the c23 set. According to the standard protocols, all methods are tested on c23 videos. The results are shown in Table 2 . It can be seen that our method outperforms SBI-raw by 3.62% and SBI-c23 by 3.89%, demonstrating the efficacy of our method on cross-manipulation scenarios.Effect of Each Objective Term. This part studies the effect of each objective term on CDF, DFDC, and DFDCP datasets. The results are shown in Table 3 . Note that Baseline denotes only using prior and integrity loss L pi and "w/o" denotes without. It can be seen that without one certain objective term, the performance drops on all datasets, which demonstrates that different objective terms have distinct impacts, and their collective contributes most to our method.Complementary to Spatial-blending Methods. To validate the complementary of our method, we replace the SBI method with other spatial-blending methods and study if the performance is improved. Specifically, we reproduce the pseudo-fake face generation operations in DSP-FWA, I2G, and Face X-ray, and combine them with our method. Note that I2G and Face X-ray have not released their codes, we re-implement them rigorously following their original settings. The results on FF++, CDF, DFDCP, FFIW datasets are presented in Table 4 . It can be seen that by combining with DSP-FWA, our method improves 0.12% averagely. A similar trend is also observed in the I2G, which improves 0.84% on average. For Face X-ray, we improve the performance by 3.08% on average. It is noteworthy that I2G and Face X-ray have not released their codes yet. We rigorously follow the instructions as in their papers and run the codes widely used by others 4 5 .Different Network Architectures. This part validates the effectiveness of our method on different networks, including ResNet-50 [35] , EfficientNet-b1 [37] , VGG16 [44] , Xception [45] , ViT [46] , F3-Net [29] , and GFFD [41] . We compare our method with SBI on these networks, which are tested on CDF, FF++, DFDCP, and FFIW datasets. The results are shown in Table 5 . It can be observed that our method improves the performance by 1.78%, 0.41%, 1.51%, 0.2%, 2.75%, and 4.68% averagely on ResNet-50, EfficientNet-b1, Xception, Vit networks, F3-Net, and GFFD networks respectively. It is noteworthy that our method slightly reduces the performance of VGG16 by 0.01%. It is possibly because the capacity of VGG16 is limited than other networks, and learning spatial pseudo-fake faces almost fills up this capacity, leaving no room for the learning of frequency knowledge. Saliency Visualization. We employ Grad-CAM [47] to visualize the attention of our method compared to SBI on four manipulations in the FF++ dataset. Compared to SBI, our method concentrates more on the structural information, such as the manipulation boundaries. For example, our method highlights the face outline in DF, F2F, and FS, while focusing on the mouth contour in NT.Effect of Using Wild Fake or Spatial Pseudo-fake Faces. As described in Sec. 4.3, our method is performed using real and spatial-blending pseudo-fake (SP-fake) faces. The rationale is that SP-fake faces are greatly diversified, containing more spatial forgery traces. Applying our method to these faces can consider both frequency and spatial traces effectively. To verify this, we directly perform our method on real and wild fake faces. The results in Table 6 indicate a notable performance drop when only wild fakes are used.Limitations. Our method is designed to address the drawbacks of existing spatial-blending methods. Hence, it inherits the assumption that the faces are forged by face-swapping techniques. Further research is needed to validate our performance on other types of forgery operations, such as wholeface synthesis and attribute editing. 
conclusion: 
 This paper describes a new method called FreqBlender that can generate pseudo-fake faces by blending frequency knowledge. To achieve this, we propose a Frequency Parsing Network that adaptively extracts the frequency component corresponding to structural information. Then we can blend this information from fake faces into real faces to create pseudo-fake faces. The extensive Experiments demonstrate the effectiveness of our method and can serve as a complementary module for existing spatial-blending methods.A Appendix / supplemental material Various Loss Weights λ 1 , λ 2 , λ 3 , λ 4 . These weights are empirically selected based on experimental results. As shown in Table 7 , we evaluate the performance across various set of λ 1 , λ 2 , λ 3 , λ 4 . The results exhibit that our method is not particularly sensitive to the settings of loss weights, with performance variations within approximately ∼ 0.5%. For our main experiments, we select the set of λ 1 = 1/12, λ 2 = 1, λ 3 = 0.001, λ 4 = 1/4 as it yields the best performance. More Details of Complementary to Spatial-blending Methods. Table 8 shows the detailed results of every manipulation in the FF++ dataset. It can be seen that our method improves the performance of all manipulation methods, averaging 1.32% for DSP-FWA, 4.28% for I2G, and 1.77% for Face X-ray. This improvement further demonstrates the effectiveness of our method. Probability α in FreqBlender. As shown in Table 9 , we evaluate the effect of using various probability α. Note that α denotes the probability of generating a synthetic face whether using spatialblending or using FreqBlender after spatial-blending. α = 1 denotes only using FreqBlender, while α = 0 means only using spatial-blending. It can be observed that solely using our method can not perform well, as no color space knowledge is involved, hindering the overall effectiveness of pseudofake faces. In contrast, solely using spatial-blending operations can reach a favorable performance. However, when inserting pseudo-fake faces generated by FreqBlender, the generalization performance is further enhanced, as our method can compensate for the loss of frequency knowledge. The optimal effect on CDF evaluation is achieved when α is set to 0.2. This experiment demonstrates the complementary effect of our method to existing spatial-blending methods. More generalization experiments on more datasets have been conducted in the main paper, demonstrating the effectiveness of our method. Visual Demonstration of FreqBlender. The goal of our method is to create pseudo-fake faces that resemble the frequency distribution of wild fake faces. To verify this, we conduct a visual experiment on the FF++ dataset. Specifically, we randomly select 3, 000 images from each manipulation method (Deepfakes, FaceSwap, Face2Face, NeuralTextures) and calculate the average frequency map for each manipulation method respectively. Then we create the same number of pseudo-fake faces using Note that α = 0 denotes that our method is degraded to SBI [16] .our method and calculate their average frequency map. Finally, we visualize the frequency difference between our method and wild fake faces. The results are illustrated in Fig. 7 . It is important to note that α represents the probability of applying FreqBlender. Hence, α = 1 means all pseudo-fake faces are created using our method, while α = 0 denotes our method is degraded to SBI. From this figure, we can see that the difference is minimal when α = 1. As α decreases, the difference becomes larger as the pseudo-fake faces are more likely created by SBI. This demonstrates that our method effectively simulates the frequency distribution of wild fake faces.Performance on FF++ Low-quality (LQ). To investigate the performance of our method on lowquality videos, we conduct experiments on the FF++ Low-quality (LQ) set. The results in Table 10 show that while all methods experience substantial performance drops on the LQ set, our method consistently achieves the highest performance, demonstrating better generalization capability on low-quality videos compared to other methods.Tentative Validation on Various Synthesized Faces. In addition to validation on the standard datasets, we investigate a new scenario: Diffusion-based face-swap deepfake detection. In this scenario, we employ a recent diffusion model (Collaborative Diffusion [48] ) to synthesize faces, which are then blended into original videos. We create 200 fake faces and evaluate our method in Table 11 . It can be seen that our method is effective in detecting such forged faces. Method FF++(LQ) I2G [15] 52.20 Face X-ray [14] 65.41 SBI [16] 76.11 FreqBlender 77.56 Moreover, although our method is designed for face-swapping techniques, we also test it on face images generated by StyleGAN [49] and StyleGAN2 [50] . The results, also shown in Table 11 , reveal a notable performance drop across all methods. However, our method still outperforms the others.Performance against Evasion Attacks. We employ a widely recognized library TorchAttacks with four well-known attack methods: FGSM [51] , BIM [52] , PGD [53] and CW [54] . The attack experiment is conducted on the CDF dataset with six models and the attack configuration is set by default. The results are shown in Table 12 . It can be seen without any defense strategies, all models can be easily attacked in the white-box attacking mode, which exactly aligns with our expectations and the discoveries in the papers of FGSM, BIM, etc. Justification: The code will be thoroughly organized and released after the paper is accepted.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: In the Experimental Setups section of our paper, we have provided detailed descriptions of our experimental details.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA] Justification: Our paper's experiments do not include error bars.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: Although our paper does not discuss the related impacts, our work enhances the capabilities of deepfake detection, which we believe has positive societal impacts. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our model is designed to enhance the generalization capabilities of deepfake detection, and there is no risk of misuse.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We have properly attributed the work of others in our paper and have followed licensing and usage terms.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: Our paper does not introduce new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: Our paper does not have this research.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: Our paper does not have this research.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: F3-Net:https://github.com/Leminhbinh0209/F3NetI2G: https://github.com/jtchen0528/PCL-I2GFace X-ray: https://github.com/AlgoHunt/Face-Xray 


Decision: This paper received mixed reviews, ranging from "Borderline reject" to "Weak Accept". Reviewers acknowledged the innovative DeepFake detection method proposed in the paper (Z1eA, Xqko, RzXt). With Xqko stating that "This method is the first to mine frequency structural information and propose a way of generating pseudo-fake faces that mimic the frequency distribution of fake faces" (Xqko), and noted the analysis it provided (Xqko, GPf8) and quality of writing (RzXt, Xqko).

The reviews raised a number of concerns, including limited novelty compared to published work that was not all referenced (RzXt, Xqko). The reviewers were concerned with the experiments, noting it did not always achieve state-of-the-art detection (GPf8) and the limited evaluation against recent DeepFake detection methods and datasets, particularly those using diffusion models (GPf8, Z1eA). 

The authors provided detailed responses, though the recommendations ultimately remained divergent. I see the remaining concerns as insufficient reasons for a rejection, preferring to err on the side of acceptance in such cases.

Reviews: review: paper_summary: paper_summary: This paper have introduced an effective way to improve the generalization of DeepFake detection via generating pseudo-fake faces by blending frequency knowledge.  The proposed approach achieves state-of-the-art (SOTA) results on various deepfake detecion datasets.
strength_weakness: strength_weakness: 1) The idea of ​​this paper is similar to some already published papers, such as [1], [2] and [3]. I hope this paper can cite and further analyze them:

[1] Tan, C., Zhao, Y., Wei, S., Gu, G., Liu, P., & Wei, Y. (2024, March). Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 5, pp. 5052-5060).

[2] Yu, B., Li, W., Li, X., Lu, J., & Zhou, J. (2021). Frequency-aware spatiotemporal transformers for video inpainting detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 8188-8197).

[3] Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., & Holz, T. (2020, November). Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning (pp. 3247-3258). PMLR.

2) Moreover, Ref.30 is an important and widely cited work that introduces the frequency domain into deepfake detection. I don’t quite understand why the author does not compare and analyze it with this work.

3) The source information, page numbers, publishers, etc. of many references are incomplete, and some even are incorrect. For example, [4] should come from CVPR2024 instead of arXiv.

[4] Choi, J., Kim, T., Jeong, Y., Baek, S., & Choi, J. (2024). Exploiting Style Latent Flows for Generalizing Deepfake Video Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1133-1143).

4) The faces in Figure 6 should have been resized. It would be better if the author could show the original resolution image to help readers better understand the experimental results.

5)  Why is the fluctuation of  λ3 so much larger than that of other parameters? I hope the author can provide the corresponding theoretical basis and further detailed analysis.

6) If the author can answer and revise the relevant questions in the final version, I will consider increasing the final score in the next round.
questions: questions: Some images in the paper will be a little blurry after zooming in, so it is best to convert all images to pdf or eps format.
limitations: limitations: Yes.
paper_summary: paper_summary: This paper proposes FreqBlender, a new method to generate pseudo-fake faces that effectively simulate the frequency distribution of wild fake faces. Unlike common blending techniques done in the spatial domain, their method blends frequency knowledge. An analysis is conducted, showing that three frequency components are present in faces, namely semantic information, structural information, and noise information. They demonstrated that structural information contains forgery traces.

To this end, the first stage of their method employs FPNet, a novel architecture built to disentangle the input fake face into the three different frequency components. As no ground truth exists for this task, carefully crafted objectives provide the necessary supervision.

In the second stage, the trained network is used to parse the frequency components, and the structural component is extracted from a given fake face. It is then blended into a real face to obtain the pseudo-fake.

The method outperforms the state-of-the-art across different relevant datasets.
strength_weakness: strength_weakness: References Missing:
Some references are missing, for example, methods that are frame-based and only use real faces during training [1, 2], as well as other recent detectors [3, 4].

Additional Overhead:
During the training of the detector, the inference of FPNnet (1 encoder and 3 decoders) is required for generating a pseudo-fake. This adds an overhead during training. The authors should include an efficiency analysis.

[1] Li et al., Pixel bleach network for detecting face forgery under
compression, in IEEE Transactions on Multimedia 2023
[2] Larue et al., SeeABLE: Soft Discrepancies and Bounded Contrastive Learning
for Exposing Deepfakes, in ICCV 2023
[3] Dong et al., Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization, in CVPR 2023
[4] Guo et al., AltFreezing for More General Video Face Forgery Detection, in CVPR 2023
questions: questions: Clarification Needed on FPNet Training:
While the authors explain that for deepfake detection, pseudo-fakes generated using SBI are used as a substitute for fake faces from FF++, the authors should include the results when real fake faces from FF++ are used in Table 1.

Use of PixelShuffle:
Why is a PixelShuffle used in the decoder of FPNet?

FPNet Direct Application:
Can the FPNet be used directly for deepfake detection? What happens when a real face is input into the FPNet?

Ablation Study and Frequency Knowledge:
The authors successfully conduct an ablation study on the backbone (Table 5) and show that results improve consistently when compared to SBI. Why does this method introduce frequency knowledge when all the tested backbones are spatial (e.g., EfficientNet-b4)? It would be interesting to compare the proposed method with a built-in frequency-based backbone (e.g., Face Forgery Network (F3-Net) [3], AFFGM [1], or the multi-scale high-frequency feature extractor from [2]).

Results on Highly Compressed Data:
What are the results of the method on highly compressed data, i.e., FF++ LQ?

[1] Li et al., Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection, in CVPR 2021
[2] Luo et al., Generalizing Face Forgery Detection with High-frequency Features, in  CVPR 2021
[3] Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues, in ECCV 2020
limitations: limitations: Acknowledgement of Limitations:
The authors acknowledge the usual limitations of pseudo-fake generation methods, i.e., the hypothesis that the test face is crafted using face-swapping techniques may not hold when the test face is generated using other techniques.

Discussion on Societal Impact:
The race between attackers and defenders is a well-known issue in the field of deepfake detection. The authors should discuss the potential negative societal impact of their work and how it could be exploited by attackers to generate more realistic deepfakes by simply training their generators to fool the proposed method.
paper_summary: paper_summary: This work studies generalizable deepfake detection. The proposed method is motivated by a new data augmentation method that blends real and fake faces in the frequency domain. The paper claims the forgery can be found in three different frequency bands and proposes an unsupervised learning method, Frequency Parsing Network. Empirical results indicate that the proposed method's performance achieves results comparable to state-of-the-art.
strength_weakness: strength_weakness: 1. the proposed method does not achieve state-of-the-art detection performance in table 1.
2. table 2 only has a few methods on the FF++ dataset. It should not be the case that SBI is only a baseline to compare. 
3. the proposed approach might be limited in standard face-swap fake face, or gan-generated face. How about diffusion model generated face?
4. in terms of approach. the proposed method works when capturing semantic, structural and noise information. What if it fails to capture any one of those?
questions: questions: 1. line 31 says the motivation of simulating various blending artifacts. However, these artifacts are quite "old" compared to that from current diffusion models. That says, how would your method perform on the face generated by stable diffusion? such as instantid, photomaker?
2. did you compare with DIRE and HiFi-Net?
R1: DIRE for Diffusion-Generated Image Detection, ICCV2023
R2: Hierarchical fine-grained image forgery detection and localization, CVPR2023
limitations: limitations: I am impressed by the learning objective formulation, but I am inclined to reject it because it does not achieve reasonable sota performance and does not show generalization to diffusion-generated images.
paper_summary: paper_summary: The generalization of DeepFake detection can be addressed by enhancing training data with synthetic fake faces, known as pseudo-fakes. Traditional methods generate these faces by spatial blending operations. However, the limitations of these methods are that they ignore to simulate the frequency distribution, where additional important forgery clues might be found.

This paper introduces an interesting method called FreqBlender, which attempts to blend proper frequency knowledge to enhance the effectiveness of pseudo-fake faces. This method involves identifying forgery clues in the frequency domain and blending the corresponding frequency components from fake faces into real faces. This process is challenging due to the variability and spread of forgery clues across different frequency ranges.

To achieve frequency decomposition, the authors propose a Frequency Parsing Network (FPNet) that can adaptively partition the frequency domain. The network, consisting of an encoder and three decoders, extracts semantic, structural, and noise information from the faces. Training FPNet is difficult due to the lack of ground truth for frequency distribution, so the authors devise a novel training strategy leveraging inner correlations among different frequency components.

Once trained, FPNet can extract the structural information of a fake face's frequency component and blend it with a real face to generate a pseudo-fake face. This method complements existing spatial blending methods and improves detection performance on multiple DeepFake datasets, demonstrating its effectiveness.
strength_weakness: strength_weakness: This paper proposes an interesting and intuitive method that seems reasonable to me. Here are some recommendations for future improvements:

1. Investigate more fine-grained frequency ranges to uncover the detailed composition of artifacts. This could potentially enhance the model's performance by providing a more nuanced understanding of forgery clues.

2. Develop algorithms to enhance the explainability of the model’s decisions. This will help users understand how and why a particular face is identified as a DeepFake, increasing trust and usability in practical applications.

3. Extend the method to handle other types of forgery operations beyond face-swapping, such as whole-face synthesis and attribute editing. This will broaden the method's applicability and ensure it remains effective across a wider range of DeepFake techniques.

4. Utilizing LLMs may help you parse the frequency more precisely, as LLMs can provide the prior knowledge which may compensate for the lack of ground truth when training FPNet.
questions: questions: 1.	As described in Section 3, the statistics of the frequency distribution are calculated using an azimuthal average, which includes a logarithmic transformation and the calculation of azimuthally-averaged flux in circular annular apertures. Could you provide more details on why placing the center of the circular annular aperture at the top-left corner of the frequency map results in a one-dimensional spectrum diagram?
2.	The FPNet uses three independent decoders to analyze the frequency domain. The output of the decoders is not clearly explained, as it is visualized in grayscale as the frequency map in Fig. 5. I understand that the output is a soft mask ranging from [0,1] for the frequency map. Therefore, I suggest using a color version to replace the DCT map in Fig. 5.
3.	In the discussion on the loss of authenticity determination, the authors aim to create two sets of faces, with and without forgery traces. Does the set C_r represent the face sets that do not contain structural information of fake faces, while C_f corresponds to the face sets that have structural information of fake faces?
4.	In Prior and Integrity Loss, the final term is designed to ensure that the combination of three frequency components covers the entire frequency domain. This term sums these frequency components and calculates the distance from a mask of 1. However, this term might not guarantee that the sum of these frequency components equals 1, but it can reduce deviations. What is the rationale behind this design??
limitations: limitations: This paper lists its limitations in main body. Since the proposed method is designed to address the limitations of existing spatial-blending techniques. it operates under the assumption that faces are forged using face-swapping methods. It unlikely tackle other types of forgeries, such as whole-face synthesis.

Comments: title: Review Comments
comment: We sincerely appreciate your valuable time and additional comments you’ve provided.

**Q1. FF++ is the core of deep fake detection, and cross-manipulation evaluation is common in the community [R1,R2,R3,R4,R5]. However, table 2 from the main table merely reports 2 methods, and NONE of these references are discussed in the submission. This is insufficient to conclude that the proposed method is effective enough.**

**R1**. We would like to clarify that **the goal of our method is to create effective pseudo-fake faces solely using real faces, as in (Face X-ray [14], PCL [15], SBI [16])**. But the difference (novelty) is that we introduce FreqBlender to incorporates frequency information into these pseudo-fake faces.


Typically, the efforts of this direction are trained on **real faces in FF++ without using fake faces.** This allows them to be fairly validated across all four tracks in FF++, demonstrating effectiveness in cross-manipulation scenarios. Therefore, we follow the protocol of these methods and compare our approach with theirs. **Since SBI shows the second-best performance, we limit our comparison to it in Table 2. However, additional studies involving more methods, including  [14,15,16]), are presented in Table 7 (Supplementary)** 

**Following your suggestion, we thoroughly review these papers and found that R2 and R4 do not conduct cross-manipulation evaluations, while R3 employs a less challenging scenario (training on three tracks and testing on one). Thus, R2, R3, and R4 are not suitable for direct comparison.** 

R1 and R5 are trained on the DF track of FF++. Comparing with these two methods are relative fair. The results, shown in **Table A**, highlight the notable superiority of our approach. We will include this comparison in the revision.

**Table A: Cross-manipulation comparison.**
|             | DF    | F2F  | FS    | NT    | Avg   |
|:-----------:|:-----:|:----:|:-----:|:-----:|:-----:|
| R1 (trained on DF)       | 99.65 |70.66 |74.29  |67.34  |77.99 |
| R5 (trained on DF)         | 99.22 |60.18 |68.19  |61.17  |72.19 |
|FreqBlender       | 99.18 |96.76 |97.68  |90.88  |96.13 |


**Q2: Table 1 in the rebuttal is invalid, and I am not convinced by this experiment in two aspects: 1) no method was used for the comparison, only showing your performance is not enough. 2) why can't authors evaluate more commonly used diffusion-based methods when easily accessible tools are available, such as stable 1.5, stable 2.1, instantiated, and Dalle2? Being selective on the face generation method is not fair. For example, [R6] reports the generalization performance on StarGAN, DDPM, DDIM, and SD**

**R2**: In the first round of rebuttal, we follow the suggestion to show the generalization of our method to diffusion models. The performance is 94.74, which we believe can demonstrate the efficacy. **As suggested, we evaluate more methods (I2G, Face X-ray, SBI) as in Table B**, which also demonstrate the efficacy of our method.

To create diffusion-based face-swap deepfakes, **Collaborative Diffusion (CVPR 2023) is more user-friendly and efficient than Stable 1.5, Stable 2.1, Instantiated, and DALL-E 2**, as it allows for more effective editing of facial attributes compared to the suggested models.


**Please note that evaluating on StarGAN, DDPM, DDIM, and SD is not the primary focus or contribution of R6, which is why its performance on these models is not satisfactory (around 73% on average)**. Following the suggestion, we have made an effort to validate our method on StyleGAN and StyleGAN2 (which provide ready-made face sets) and present the results in **Table C**. **Our method outperforms others but achieves performance comparable to R6**.


**Table B: Performance of Diffusion-based face-swap deepfake detection.**
|             | Diffusion-based  |
|:-----------:|:----:      |
| I2G |     63.51     |
| Face X-ray |     89.81     |
| SBI |    91.70     |
| FreqBlender |     94.74     |


**Table C: Results in Gan-generated images.**
|             | StyleGan  |   StyleGan2   |
|:-----------:|:---------:|:-------------:|
| I2G         |  47.89    |    43.86      |
| Face X-ray  |  59.11    |    66.54      |
| SBI         |  63.99    |    72.88      |
| FreqBlender |  64.39    |    76.70      |

**Q3: The proposed method is frequency-based, then what is the performance when the forgery trace largely occurs in the RGB domain whereas less on the frequency domain. For example, these cartoon faces with large eyes generated from SD-based method? will the performance decline?**

**R3**: We would like to emphasize that our method does not rely solely on frequency information. **As highlighted in L241, our approach integrates frequency knowledge into the existing spatial-blending pseudo-fake faces, allowing it to address both spatial and frequency aspects effectively.**

We hope this explanation clarifies your concerns and encourages a re-evaluation of our work’s contribution.
Thanks for the clarification from the authors. I am sorry that I mistakenly thought the method did not achieve the sota detection performance. 

However, I still keep my scores for the proposed method's limitation on FF++ and lack of convincing experiments to show the generalization ability to the diffusion face-swap. 

1. FF++ is the core of deep fake detection, and cross-manipulation evaluation is common in the community [R1,R2,R3,R4,R5]. However, table 2 from the main table merely reports 2 methods, and NONE of these references are discussed in the submission. This is insufficient to conclude that the proposed method is effective enough.

[R1] End-to-End Reconstruction-Classification Learning for Face Forgery Detection, CVPR2022

[R2] Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues, ECCV2020

[R3] UCF: Uncovering Common Features for Generalizable Deepfake Detection, ICCV2023

[R4] Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain, CVPR 2021

[R5] Exploring Disentangled Content Information for Face Forgery Detection, ECCV 2022

2. Table 1 in the rebuttal is invalid, and I am not convinced by this experiment in **two** aspects: 

- no method was used for the comparison, only showing your performance is not enough. 

- why can't authors evaluate more commonly used diffusion-based methods when easily accessible tools are available,  such as stable 1.5, stable 2.1, instantiated, and Dalle2? Being selective on the face generation method is not fair. For example, [R6] reports the generalization performance on StarGAN, DDPM, DDIM, and SD.

[R6] Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection, CVPR 2024

3. The proposed method is frequency-based, then what is the performance when the forgery trace largely occurs in the RGB domain whereas less on the frequency domain. For example, these cartoon faces with large eyes generated from SD-based method? will the performance decline?
Thanks for your comments! I am very glad to hear that our response addresses your concerns and resolves your doubts. We appreciate that the novelty of our idea is highly recognized and its positive significance to the community is well acknowledged.
Dear Reviewer GPf8,

We sincerely appreciate the time and thoughtful comments you’ve provided. With the remaining time being limited, we are eager to receive your feedback, especially on the issues we've addressed in our rebuttal. 

In our response, we have carefully reviewed your comments and provided the following summaries:
1. Clarified the concern regarding SOTA and its application to diffusion-based models.
2. Analyzed the suggested DIRE and HIFI-Net.
3. Addressed the questions about using SBI in Table 2 and the shortcomings of FPNet.

We hope the new experiments and analysis have demonstrated the merits of our work. We deeply appreciate your time and effort!

Best regards,  
Authors
Dear Reviewer Xqko,

Thank you once again for your insightful comments! We look forward to receiving your feedback. We hope the new experiments and additional explanations have demonstrated the merits of this paper. If you have any further questions, please do not hesitate to reach out.

Best regards, Authors
Dear Reviewer RzXt,

We sincerely appreciate your time and thoughtful comments. We eagerly look forward to your feedback, especially on the issues we've addressed in our rebuttal. Our main goal is to ensure that our response aligns closely with your suggestions. Your input is invaluable to improving our work.

Best regards, Authors
Thank you for the reply. The author responded to my concern directly, resolving my doubts. I believe this article has a positive significance for the community and the idea is quite novel, so I keep my score.