FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge
Authors: Hanzhe LI
Jiaran Zhou
Yuezun Li
Baoyuan Wu
Bin Li
Junyu Dong
Venue: NeurIPS 2024 poster (2024)
Keywords: DeepFake Detection
Security and Privacy
Multimedia Forensics
Primary area: safety_in_machine_learning

Abstract: Generating synthetic fake faces, known as pseudo-fake faces, is an effective way to improve the generalization of DeepFake detection. Existing methods typically generate these faces by blending real or fake faces in spatial domain. While these methods have shown promise, they overlook the simulation of frequency distribution in pseudo-fake faces, limiting the learning of generic forgery traces in-depth. To address this, this paper introduces {\em FreqBlender}, a new method that can generate pseudo-fake faces by blending frequency knowledge. Concretely, we investigate the major frequency components and propose a Frequency Parsing Network to adaptively partition frequency components related to forgery traces. Then we blend this frequency knowledge from fake faces into real faces to generate pseudo-fake faces. Since there is no ground truth for frequency components, we describe a dedicated training strategy by leveraging the inner correlations among different frequency knowledge to instruct the learning process. Experimental results demonstrate the effectiveness of our method in enhancing DeepFake detection, making it a potential plug-and-play strategy for other methods.

Freqerender: Enhancing DeepFake Detection by Blending Frequency Knowledge
Hanzhe Li!." Jiaran Zhou Yuezun Li Baoyuan Wu Bin Li
Junyu Dong'
School of Computer Science and Technology. Ocean University of China School of Data Science, The Chinese University of Hong Kong, Shenzhen, China Guangdong Provincial Key Laboratory of Intelligent Information Processing. Shenzhen University, Shenzhen, China
Distribution
Filee
Froqueccy
Spatial
Rod
Take
Fake
Fake
Figure I: Overview of our method. In contrast to the existing pertial-lenndigg methods (right part), our method explores face blending in frequency domain (left part). By leveraging the frequency knowledge, method generate pseudo-fake faces that closely resemble the distribution of wild fake faces. Our method complement and work in conjunction with existing spatial-blending methods.
Abstract
Generating synthetic fake faces, known as pseudo-fake faces, is an effective way improve the generalization of DeepFake detection. Existing methods typically generate these faces by blending real or fake faces spatial domain. While these methods have shown promise, they overlook the simulation of frequency distribution in pseudo-fake faces, limiting the learning of generic forgery traces in-depth. To address this, this paper introduces FreqBlender, can generate pseudo-fake faces by blending frequency knowledge. Concretely, we investigate the major frequency components and propose Frequency Parsing Network to adaptively partition frequency components related to forgery traces. Then we blend this frequency knowledge from fake faces into real faces to generate pseudo-fale fake faces. Since there is ground truth for frequency components, we describe dedicated training strategy by leveraging the inner correlations among different frequency knowledge to instruct the learning process. Experimental results demonstrate the effectiveness of our method in enhancing DeepFake detection, making it potential plug-and-play strategy for other methods.
method that
Introduction
DeepFake refers to face forgery techniques that can manipulate facial attributes, such as identity, expression, and lip movement [1]. The recent advancement of deep generative models [2, 3] has
"Equal contribution
CCorresponds to Yuezun Li (liyuezunoouc.en) edw.ooucceedddd.e.o..eee.eeeeee. edu. cn)
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

greatly sped up the evolution of DeepFake techniques, enabling the creation of highly realistic and visually imperceptible manipulations. However, the misuse of these techniques can pose serious security concerns [4], making DeepFake detection more pressing than ever before.
There have been many methods proposed for detecting DeepFakes, showing their effectiveness on public datasets [5, 6, 7, 8, 9]. However, with the continuous growth of AI techniques, new types of forgeries constantly emerge, posing a challenge for current detectors to accurately expose unknown forgeries. To address this challenge, recent efforts [10, 11, 12, 13] have focused on improving the generalizability of detection, i.e., the ability to detect unknown forgeries based on known examples. One effective approach to address this problem is to enhance the training data by generating synthetic fake faces. known as pseudo-fake fakes [14, 15, 16]. The intuition behind this approach is that the DeepFake generation process introduces artifacts in the step of blending faces, and these methods generate pseudo-fake faces by simulating various blending artifacts. By training on these pseudo- fake faces, the models can be driven to learn corresponding artifacts. However, existing methods concentrate on simulating the spatial aspects of face blending (see Fig. (right)). While they can make the pscudo-fake faces resemble the distribution of wild fake faces in the spatial domain, they do not explore the distribution in the frequency domain. Thus, current pseudo-fake faces lack frequency-based forgery clues, limiting the models to learn generic forgery features.
In this paper, we shift our attention from the spatial domain to the frequency domain and propose a new method called FreqBlender to generate pseudo-fake faces by blending frequency knowledge (see Fig. (left)). To achieve this, we analyze the composition of the frequency domain and accurately identify the range of forgery clues falling into. Then we replace this range of real faces with the corresponding range of fake faces to generate pseudo-fake faces. However, identifying the frequency range of forgery clues is challenging due to two main reasons: 1) this range varies across different fake faces due to its high dependence on face content, and 2) forgery clues may not be concentrated on single frequency range but could be aggregation of various portions across multiple ranges. Thus, general low-pass, high-pass, or band-pass filters are incapable of precisely pinpointing the distribution.
To address this challenge, we propose Frequency Parsing Network (FPNet) that can adaptively partition the frequency domain based on the input faces. Specifically. we hypothesize that the faces are composed of three frequency knowledge, which represents semantic information, structural information, and noise information, respectively, and the forgery traces are likely hidden in structural information. This hypothesis is validated in our preliminary analysis (refer to Sec. 3 for details). Based on this footstone, we design the network consisting of shared encoder and three decoders to extract corresponding frequency knowledge. The encoder transforms the input data into a latent frequency representation, while the decoders estimate the probability map of the corresponding frequency knowledge.
Training this network is non-trivial since no ground truth of frequency distribution is provided. Therefore, we propose a novel training strategy that leverages the inner correlations among different frequency knowledge. To be specific, we describe dedicated-crafted objectives that are performed on various blending combinations of the output from each decoder and emphasize the properties of each frequency knowledge. The experimental results demonstrate that the network successfully parses the desired frequency knowledge within the proposed training strategy.
Once the network is trained, we can parse the frequency component corresponding to the structural information of a fake face, and blend it with real face generate a pseudo-fake face. It is important to note that our method is not in conflict with existing spatial blending methods, but rather complements them by addressing the defect in the frequency domain. Our method is validated on multiple recent DeepFake datasets (e.g., FF++ [5], CDF [6]. DFDC [8], DFDCP [7]. FFIW [9]) and compared with many of-e-of-thee-rr methods, demonstrating the efficacy of our method in improving detection performance.
The contributions of this paper are summarized in three-fold: 1) To the best of our knowledge, we the first to generate pseudo-fake faces by blending frequency knowledge. Our method pushes pseudo- fake faces closer to the distribution of wild fake faces, enhancing the learning of generic forgery features in DeepFake detection.2) We propose a Frequency Parsing Network that can adaptively partition the frequency components corresponding to semantic information, structural information. and noise information, respectively. Since no ground truth is provided, we design dedicated objectives
2

---

to train this network. 3) Extensive experimental results on several DeepFake datasets demonstrate the efficacy of our method and its potential as a plug-and-ple strategy for existing methods.
2 Related Works
The rapid progress of AI generative models has spawned the development of DeepFake detection methods. These methods mainly rely on deep neural networks to identify the inconsistency between real and fake faces using various features, including biological signals [17], spatial artifacts [18, 14, models [21, These methods have shown promising results on public datasets. However, some of their performance significantly deteriorates when confronted with unknown DeepFake faces due to the large distribution discrepancy resulting from limited training datasets. To tackle this issue, many methods have been proposed to improve their generalizability by learning the generic DeepFake traces, e-g-, [10, 14, 15, 16, 12, 23, 24]. One effective approach is to create synthetic fake faces during training, known as pseudo-fake faces, egg. [10, 14, 15, 16]. FWA [10] is a pionerring method that conducts self-blending to simulate fake faces. Several extended variants (Face X-ray [14], PCL [15], SBI [16]. BiG-Arts [25]) have been proposed to blend faces using curated strategies, further improving detection performance. By increasing the diversity of training faces, the gap in the distribution of wild fake faces can be reduced, allowing the models to learn the invariant DeepFake traces across different distributions. To generate the pseudo-fake faces, existing methods usually design spatial blending operations to combine different faces. This involves extracting the face region from a source image and blending into a target image. However, these methods overlook the distribution of wild fake faces in the frequency domain. While the synthetic faces may resemble the spatial- based distribution, the lack of consideration for frequency perspective hinders the models from learning the fundamental generic DeepFake traces.
15, 16], frequency abnormality [13, 19, 20], auto-learned clues from dedicated-
Preliminary Analysis
We perform a statistical analysis of the frequency distribution of real and fake faces and present preliminary results for the main frequency components corresponding to semantic information, structural formation, and noise information, respectively.
Inspiration and Verification. The investigation in previous works [26, 27] has indicated that the forgery traces mainly exist in high-frequency areas. However, the precise range of these areas has not been described, driving us to re- re-investigate the frequency distribution of forgery traces.
Specifically, we conduct verification experiments using FaceForensics++ (FF++) [5] datasets. We extract the frames from all videos and randomly select 3, 000 real images and 3. 000 fake images for each manipulation method (e.g., DF, F2F, FS, and NT). Then we crop out the face region in these selected images using face detector [28] and apply DCT [29] to generate frequency maps. For analysis, we sum up all frequency maps of real and fake images and adopt the visualization process of azimuthal average described in previous work [30, 27J. This process involves logarithmic transformation and the calculation of azimuthally- placing the center of the circular annuli aperture at the top-left corner of the frequency map, we can obtain a one- med-meeniional array representing the spectrum diagram. The visual results of their distribution are shown in Fig. 2 (top). It can be observed that this figure is consistent with the results [26]. However, when we directly plot their distribution differences without logarithmic operation, the results do not match the previous figure. It can be seen that the disparities in high- figh-frequency regions are not as substantial as expected, while the differences in the lower range become more noticeable (see Fig. 2 (bottom)). This is because the logarithmic operation mitigates the degrec of differences in lower frequency ranges, causing the illusion that only the high- frequency range exhibits differences between real and fake faces. Therefore, we conjecture that the forgery traces may not only be concentrated a very high-frequecyy range but could possibly spread to the low cowffrequeccy range.
flux in circular annuli apertures. By
Hypothesis and Validation. As shown in Fig. 2, the most significant difference can be observed in the range of very low frequency. Given the significant dissimilarity in appearance between real and fake faces, we hypothesize that the semantic information is mainly represented in this low-frequency band. Moreover, we hypothesize that the mid-to-high frequency components capture the structural information, making them more susceptible to containing forgery traces. Furthermore, we hypothesize
3

---

Deeplakes
Fare2Face
Face Swap
Neuraleextures
1.0
real take
real
fake
fake
Take
0.5
0.0
0.0
0.0
200
400
0
200
400
200
400
200
400
18
difference
18
difference
14
difference
18
difference
9-
0
200 400
200 400
200
400
200
400
Figure 2: Statistics of frequency distribution. The top part shows the frequency distribution of real and fake faces using algorithms in [30, 27J. The bottom part shows the frequency difference between real and fake. The values on the vertical axis are logarithmic with 2.
Deepfakes
Face2Face
FaceSwap
Neural Textures
100
100
100
100
300
300
.000
400
100
200
300
400
Figure 3: Visualization of the frequency difference between real and fake faces. The lighter color indicates the larger difference.
that the highest frequency components likely correspond to the noise introduced by various video preprocessing operations, such as compression, decompression, and encoding.
To validate our hypothesis, we directly visualize the difference between real and fake faces on their frequency maps in Fig. 3. By observing these results, we empirically split the frequency map into three non-overlap bands. The split operations follow the general band-pass filters. Denote the position in the frequency map (v. y), where (0, 0), (1, 1) denotes the top-left comer and bottom-right comner Specifically, identify the region where & + S 1/16 as con- taining semantic information, the region where 1/16 < r + S. 1/2 as containing structural in- formation, and the region where + 1/2 containing noise information. The correspond- ing results are visualized in Fig. 4, validating that these three ranges provide empirical evi- dence that aligns with our frequency distribution hypothesis.
Compenent
Frequeasy
Real
Figure Image visualization corresponding to different frequency components.
4 FrenBr
describe a new method create pseudo-fakes by blending specific frequency knowledge. The motivation is that existing methods only focus spatial domain blending, which overlook the disparity between real and fake faces the frequency domain. By considering the frequency distribution, the pseudo-fake fakes can closely resemble the fake faces. To achieve this, we propose

---

Decode:
Curotayy
Pum
Ayy
Comllar??
Faai
Origut
de
Eacodet
Comalary
lepat Irage
(8(00)))
(+)+(+))
Aig
Dui
Comolayy
(E0(E())O)
(ex(
Enseder
Daseder x3
Figure 5: Overview of the proposed Frequency Parsing Network (FPNet). Given imput face image, method can partition into three frequency components, correspouding to the semantic information, structural information, and noise information respectively. Since there is no ground truth, we propose four corollaries to supervise the training. The architecture of the encoder and decoders is shown in the right part.
a Frequency Parsing Network (FPNet) to partition the frequency domain into three components, corresponding to semantic information, structural information, and noise information, respectively. We then blend the structural information of fake faces with the real faces to generate the pseudo-fakes. The details of the Frequency Parsing Network elaborated in Sec. 4.1, and the objective and training process for this network is described in Sec. 4.2. Then we introduce the deployment of our method with existing methods in Sec. 4.3.
4.1 Frequency Parsing Network
Overview. The Frequency Parsing Network (FPNet) is composed of one shared encoder and three independent decoders. The encoder transforms the input faces into frequency-cring -critical features and the decoders aim to decompose the feature from the encoder and extract the respective frequency components.
Denote the encoder & and three decoders Deems Dstr. Dnoi respectively. Given in- put face image x E X E 20,255) hxxx3, we first convert this face to the frequency map as (x)) E R where á denotes the operations of Discrete Cosine Transform (DCT). Then we send this frequency map into model and generate three distribution maps Deem (((()))) [0, 1/ (E((x(() E [0. 1| and Duusi (E(()))) [0, 1|^ respectively. Each distri- bution map indicates the probability of the corresponding frequency component distributed in the frequency map. Given these distribution maps, we can select the corresponding frequency components conveniently. For example, the frequency component corresponding to semantic information can be selected by @sem (x) = ((x)Dxem (E((())) and the same is for other two frequency components, sur ( ((((x))) and Pnoi (x) ((x)D (E
ie., Dste (x) ) = (xx)
The overview of FPNet
shown in Fig. (left).
Network Architecture. The encoder simply consists of four convolution layers with a kernel size of 3 x 3, stride of 2, and a padding of 1. Each decoder also consists of four layers, and each layer is combination of convolutional layer and PixelShuffle operation [31] (see Fig. (right)).
4.2 Objective Design for FPNet
The most challenging and crucial aspect of our method train the network for frequency parsing, as there is no ground truth available for the different frequency components. Note that the only available resources for supervising the training are the preliminary analysis results in Section 3. Nevertheless, these results are not precise and can not be adaptive to different inputs, which are insufficient for model training. Therefore, we meticulously craft a couple auxiliary objectives to instruct the learning of networks, allowing for the self-refinement of the network.
These objectives designed based the following proposition.
Proposition 1. Each frequency component exhibits the following properties:
1. Semantic information can reflect the facial identity.
5

---

2. Structural information serves as the carrier forgery traces.
3. Noise information has minimal impact on visual quality.
4. The preliminary analysis findings are generally applicable.
Corollary 1. For given five the transformed face based on its semantic information will retain the same facial identity as X, i.c., V x E E {0, 255] 255)xxxx3, F [Qeem (x))) F(x), where FF denotes a face recognition model and Φ 1 denotes the Inverse Discrete Cosine Transform (IDCT).
Facial Fidelity Loss. We introduce facial fidelity loss Cy to penalize the discrepancy in identity between the input face image and the spatial content represented by semantic information. To measure the identity discrepancy, we employ the MobileNet [32] as our face recognition model and train using ArcFace [33, 34]. We select MobileNet for its balance between computational efficiency and recognition accuracy. Let F be the MobleNet and Fy (x) be the facial features extracted from the face image The facial fidelity loss be defined as
(x)
( (Øsem (x)) Fy (x)
(I)
Note that the input face can be either real or fake, as the identity information is present in both cases.
Corollary 2. For a given real face Xv, can be detected as fake if and only if it is inserted the structural information from fake face x,, i.e., D(x,) = 0 iff X, Lh &x (x)() where 'D denotes a Deepfake detector with labels of fake and real in {0, 1}. @ indicates the inserting operation.
Authenticit--deeemmnatte Loss. This loss is designed to emphasize the determinative role of structural information. To evaluate the authenticity of faces, we develop a DeepFake detector D, which is implemented using ResNet-34 [35] trained on real and fake faces. Then construct two sets of faces by blending frequency components.
The first set contains three types of faces transformed from frequency components corresponding to 1) the semantic information of the real face, 2) the semantic information the fake face, and 3) the semantic information of the real face blended with the structural information of the real face. We denote this set as C, = {Φ (Øsem (x,))
(deem (x))) )), o (@eem (x,) + (x, ))}- Since there no structural information from fake faces in this set, all the faces should be detected real.
Similarly, the second set contains two types of faces: 1) blending the semantic information of the fake face with the structural information of the fake face, and 2) blending the semantic information of the real face with the structural information of the fake face. We denote this set as Cj {o (déee (x (xy)++m + Oatr (x))), o ( (0sem + Pstr (x)))) Since all blended faces this contain the structural information of fake faces, they should be detected as fake. Thus the authenticity- determinative loss Lad can be written as
(x,, x()
Σ CE(x, xEC,
Σ xEC,
CE((, 0)
Cr
|c
where CE denotes the cross-entropy loss.
Corollary 3. The face should exhibit no visible change if the frequency component of noise infor- mation is removed, i.e., x E X {0,255) (0522555xxxx hxwx3, removing operation.
@ ( where è indicates the
Quaoity- Loss. As noise information does contain decisive details for the overall depiction the image, the face image is expected to be similar to the face image transformed using the frequency components of semantic and structural information. This similarity can be quantified using the quality-agnostic Loss Lyav defined as
Lqu (x) ||x 10sem Ostr (x))()))
where the face x can be either real or fake.
Corollary 4. Each frequency component is bound by the preliminary results, i.e., there should be no significant deviation between the predicted frequency component and the approximate frequency distribution in preliminary analysis.

---

Prior and Integrity Loss. According to our analysis in the Preliminary Analysis section, we have an initial understanding of the approximate frequency distribution. Denote the initial frequency maps for semantic, structural, and noise information as Msmm, Mettr, Mmnii respectively. These maps are utilized to accelerate the convergence of the model towards the desired direction. Moreover, we add constraint on the integrity of their distributions, ensuring that their combination covers all elements of the frequency map. This loss Cpi can be expressed as
pp = =I] (E()(x))) - Meem 1 + 1 Dau (E(((()(()) - mate 12 + || - mooi
II (E()(x))) + ser ((r(((((())))) + Dnoi (E((((())))) - l1::
(4)
where 1 denotes a mask where all the elements in it is 1.
Overall Objectives. The overall objectives are the summation of all these loss terms,
= Cr + X2Cad + Azhaa + Aa
(5)
where \1, X2, A3, A are the weights for different loss terms.
4.3 Deployment of FregBlender
Given a fake face x, and a real face X,, we can generate a pseudo- face by
= - x ( + o(x;) Dtr (E(((x))) + 6(x, (no ).
(E(((x)))
(6)
1 Note that in our method, it is not necessary to perform the blending using wild fake faces.
Instead, we can tactfully substitute wild fake faces with the pseudo-fake faces generated by existing spatial face blending methods. It allows us to overcome the limitations in the frequency distribution of existing pseudo-fake faces.
Experiments
5.1 Experimental Setups
Our method is evaluated using several standard datasets, including FaceForensics++ [5] (FF++), Celeb-DF (CDF) [6], DeepFake Detection Challenge (DFDC) [8]. DeepFake Detection Challenge Preview (DFDCP) [7], and FFIW-10K (FFTW) [9] datasets. Specifically, the FF++ dataset consists of 1000 pristine videos and 4000 manipulated videos corresponding to four different manipu- lation methods, that are Deepfakes (DF), Face2Face (F2F), FaceSwap (FS), and NeuralTextures (NT). CDF dataset comprises 590 pristine videos and 5639 high-quality fake videos created from DeepFake alterations of celebrity videos available on YouTube. DFDC is a large-scale deepfake dataset, that consists of 100, 000 video clips, and DFDCP is preview version of DFDC, which is also widely used in evaluation. The FFIW dataset contains 8250 pristine videos and 8250 DeepFake videos with multi-face scenarios. We follow the original training and testing split provided by the datasets for experiments.
DataSets.
Implementation Details. Our method is implemented using PyTorch 2.0.1 [36] with Nvidia 3090ti. In the training stage of FPNet, the image size is set 400 x 400. The batch size is set 8 and the Adam optimizer is utilized with an initial learning rate of 1e-4, The training epoch set to 200. The hyperparameters
the objective function in Eq. (5) are set as follows: A 1/12, A2 = 1, = 1e-3, A = 1/4. For DeepFake detection, we employ the vanilla EfficientNet-b4 [37] as our model following [16]. In the training phase, we create pseudo-fake faces on-the-fly. We first generate synthetic faces using the spatial-blending method [16] and then blend them with real faces using our method with a probability a 0.2. Other training and testing settings are the same [16]. More analysis of parameters provided in Supplementary.
5.2 Results
To showcase the effectiveness of our method, we train our method solely on the FF++ dataset and test it on the other different datasets. We employ the Area Under the Receiver Operating Characteristic Curve (AUC) as the evaluation metric following previous work [16]. Our method compared with five video-based detection methods, including Two-branch [38], DAM [9], LipForensics [1],
7

---

Table 1: The cross-dataset evaluation of different methods. Blue indicates hest and red indicates second hest.
Training Set
Test Set AUC (%)
Method
Input Type
Real
Fake
CDF
DFDC
DFDCP
FFIW
Two-branch (FCCC'220) [38]
Video
76.65
DAM (CVPR'21) [9]
Video
75.3
72.8
LipForensics (CVPR'21) [1]
Video
82.4
73.50
FTCN (ICCV'21) [39]
Video
86.9
71.00
74.0
74.47
SST (CVPR'24) [24]
Video
89.0
DSP-FWA (CVPRW' 19 [10])
Frame
69.30
Face X-ray (CVPR'20) [14]
Frame
71.15
Face X-ray (CVPR'20) [14]
Frame
F3-Net (ECCV'20) [29]
Frame
72.93
61.16
81.96
61.58
LRL (AAAI'21) [40]
Frume
78.26
76.53
FRDM (CVPR'2I) [41]
Frame
79.4
79.7
PCI+12G (ICCV*21) [15]
Frame
90.03
67.52
74.37
DCL (AAAI'22) [42]
Frame
82,30
76.71
71.14
SBI" (CVPR'22) [16]
Frame
92.94
72.08
85.51
85.99
SBI (CVPR'22) [16]
Frame
93.18
72.42
86.15
84.83
TALL-Swin (ICCV'23) [22]
Frame
90.79
76.78
UCF (ICCV'23) [12]
82.4
BiG-Arts (PR'23) [25]
Frame
77.04
80.48
F-G (CVPR'24) 22) [43]
Frame
74.42
61.47
LSDA (CVPR 24) [23]
73.6
FreqBlender (Ours)
Frame
94.59
74.59
87.56
86.14
FTCN [39] and SST [24]. Moreover, we involve thirteen frame-level soatee-ff-theart methods for comparison, which are DSP-FWA [10], Face X-ray [14], F3-Net [29], LRL [40]. FRDM [41], PCL [15], DCL [42], SBI [16], TALL-Swin [22], UCF [12], SST [24], F-G [43], LSDA (e3),respectivell,
Cross-dataset Evaluation. We evaluate the cross-dataset performance our method compared to other counterparts in Table 1. The best performance is highlighted in blue and the second-best is marked by red. It should be noted that our method operates on pseudo-fake faces generated by SBI, thus do not need fake faces. In comparison video-level methods, our method achieves the best performance, which outperforms all the methods by a large margin.
When compared to frame-level methods, method still outperforms the others. For example, method improves upon the performance of the most relevant counterpart SBI by 1.41%, 2.17%. 1.41%. 1.31% on CDF, DFDC, DFDCP, and FFIW respectively. This improvement to the incorporation of frequency knowledge in pseudo-fake faces, enhancing the generalization of detection models. Note that the performance of the compared methods (except SBI* and F3-Net) extracted from their original papers. SBI* denotes the performance obtained using the officially released codes, and F3-Net is reproduced using the codes implemented by others The results closely align with the reported scores, which verifies the correctness of our configuration of their codes. In subsequent experiments, employ their release codes for comparison.
be attributed
Cross- mosss-manpplation Evaluation. Since SBI is the most recent and effective method, we compare method with it for demonstration. Specifically, SBI method. The first is trained using the raw set real videos in the FF++ dataset, while the second According to the standard protocols, all methods tested c23 videos.
compare method with two variants of the
trained using the c23 The results are shown in Table 2. can he seen that our method outperforms SBI-raw by 3.62% and SBI-c23 by 3.89% demonstrating the efficacy of method cross-manipulatioon scenarios.
5.3 Analysis
Effect of Each Objective Term. This part studies the effect of each objective term on CDF, DFDC, and DFDCP datasets. The results are shown in Table 3. Note that Baseline denotes only using prior and integrity loss Cpi and "w/o" denotes without. It can seen that without one certain objective

---

Table 2: The cross-manipulation evaluation of different methods.
FF++
Method
Avg
DF
F2F
FS
NT
SBI-raw [16]
98.35
91.07
96.92
83.69
92.51
SBI-c23 [16]
98.60
92.60
95.44
82.30
92.24
FregBlender (Ours)
99.18
96.76
97.68
90.88
96.13
Table 3:
Effect of each objective term.
Table 4: Effect of our method complementary to spatial-
Setting
CDF
DFDC
DFDCP
Avg
blending methods.
Bascline
91.69
72.69
86.67
83.68
Method
FF++
CDF
DFDCP
FFIW
Avg
w/o La
94.01
74.30
86.86
85.06
DSP-FWA [TO]
48.14
62.91
60.74
40.65
53.11
Lud
93.31
72.97
86.32
[10]
65.47
41.81
I2G [15]]
59.56
53.55
48.02
46.75
51.97
Cea
94.28
74.42
87.25
85,32
12G [15] Ours
63.84
48.89
49.53
48.96
52.81
Lpi
93.78
74.03
85,99
84.60
X-ray [14]
63.65
All
94.59
74.59
87.56
85.58
Face X-ray [14] Ours
84.03
76.05
63.90
67.24
72.81
Table 5: The effect of method different networks.
Method
CDF
FF++
DFDCP
FFIW
Avg
ResNct-50 [35] + SBI
84.82
95.39
73.51
81.67
83.85
ResNet-50 [35] Ours
85.44
94.61
76.16
86.32
85.63
EfficientNet-bl [37] SBI
90.25
94.66
87.54
82.55
88.75
EfficientNer-bl [37] Ours
90.53
94.65
87.70
83.76
89.16
VGGI6 [44] SBI
78.22
93.05
74.13
87.26
83.16
VGGI6 [44] Ours
78.38
93.10
73.47
87.63
83.15
Xception [45 + SBI
87.00
91.40
75.68
70.24
81.08
Xception [45] + Ours
90.52
93.32
76.07
70.43
82,59
ViT [46] SBI
85.85
96.09
87.71
86.05
88.92
VIT [46] + Ours
86.34
96.10
87.17
86.88
89.12
F3-Net [29] SBI
84.94
93.42
79.29
73.42
82.77
F3-Net [29] Ours
88.10
95.16
84.32
74,49
85.52
GFFD [41] SBI
81.34
91.81
77.19
65.53
78.97
GFFD [41] Ours
86.71
92.18
78.25
77.45
83.65
term, the performance drops on all datasets, which demonstrates that different objective terms have distinct impacts, and their collective contributes most to our method.
Complementary to Spatial-blending Methods. To validate the complementary of our method, replace the SBI method with other sratial-blending methods and study if the performance is improved. Specifically, we reproduce the pseudo-fake face generation operations in DSP-FWA, 12G, and Face X-ray, and combine them with our method. Note that I2G and Face X-ray have released their codes, we re-implement them rigorously following their original settings. The results on FF++, CDF, DFDCP, FFIW datasets are presented in Table It can seen that by combining with DSP-FWA, our method improves 0.12% averagely. A similar trend is also observed in the 12G, which improves 0.84% on average. For Face X-ray, we improve the performance by 3.08% on average. It is noteworthy that I2G and Face X-ray have not released their codes yet. We rigorously follow the instructions in their papers and run the codes widely used by others*
Different Network Architectures. This part validates the effectiveness our method on different networks, including ResNet-50 [35], EfficientNet-bl [37], V [44], Xception [45], ViT [46], F3-Net [29], and GFFD[41]. We compare our method with SBI on these networks, which are tested on CDF, FF++ DFDCP, and FFIW datasets. The results are shown in Table 5. It can be observed that our method improves the performance by 1.78%, 0.41%, 1.51%, 0.2%. 2.75%, and 4. 4.68% averagely on R5sNet-50. EfficientNet-bl. Xception, Vit networks, F3-Net, and GFFD networks respectively. is noteworthy that our method slightly reduces the performance of VGG16 by 0.01%. It is possibly because the capacity of VGG16 is limited than other networks, and learning spatial pseudo-fake faces almost fills up this capacity, leaving no room for the learning of frequency knowledge.
4 122G httpa://githin wwwpttttu..coccceeeeeeeee com/jjtchen0228/O1---2
Face X-ray: https:/ //t:::/gttttt githhb..... cat.uu..om/Alooooi www./llgollnnt/aaaeeeeeoooooo Xray

---

DF
SBI
Ows
F2F
SBI
Owrs
FS
SBI
Ours
NT
SBI
Ows
Figure Grad-CAM visualization SBI and our method on four manipulation types FF++ dataset. Compared SBI, our method focuses more on the manipulated structural boundaries.
Table 6: Effect of using wild fake or SP-fake faces.
Sa-fake
CDF
DFDC
DFDCP
FFIW
Avg
75.79
66.30
66.30
67.70
72.95
94.59
74.59
74.59
86.14
85.72
Saliency Visualization. We employ Grad-CAM [47] to visualize the attention of our method com- pared to SBI on four manipulations in the FF++ dataset. Compared to SBI, our method concentrates more on the structural information, such as the manipulation boundaries. For example, our method highlights the face outline in DF, F2F, and FS, while focusing on the mouth contour in NT.
Effeet of Using Wild Fake or Spatial Pseudo-fake Faces. As described in Sec. 4.3, our method is performed using real and spatial-blending pseudo-fake (SP-fake) faces. The rationale is that SP-fake faces are greatly diversified, containing more spatial forgery traces. Applying our method to these faces can consider both frequency and spatial traces effectively. To verify this, directly perform our method on real and wild fake faces. The results in Table 6 indicate a notable performance drop when only wild fakes used.
Our method is designed to address the drawbacks of existing spatial-blending methods. Hence, it inherits the assumption that the faces are forged by face-swapping techniques. Further research is needed to validate our performance on other types of forgery operations, such as whole- face synthesis and attribute editing.
Limitations.
6 Conclusion
This paper describes a new method called FreqBlender that can generate pseudo-fake faces by blending frequency knowledge. To achieve this, we propose Frequency Parsing Network that adaptively extracts the frequency component corresponding to structural information. Then we can blend this information from fake faces into real faces to create pseudo-fake faces. The extensive Experiments demonstrate the effectiveness of our method and can serve as a complementary module for existing spatial-blending methods.
Acknowledgement. This work is supported in part by the National Natural Science Foundation of China (No.624022 62464440,,,) Shandong Natural Science Foundation (No. NN..2R2024QF0055, and China Postdoctoral Science Foundation (No.202 22202TTQQS TQQ0314; No. N0.2021MM030036 Jiaran Zhou is supported by the National Natural Science Foundation of China (No.6210238) and Shandong Natural Sci- ence Foundation (No. NN..2R2021QU00 dong Basic and Applied Basic Research Foundation 1 51500995)) National Natural Science Foundation of China (No. 62076213), and Shenzhen Science and Technology Program (No. RCYX20210609 (1XX002200001100070500 Bin Li is supported in part by NSFC (Grant U23B2022, U22B2047).
No. N0.2RR224MF088. Baoyuan Wu is supported by Guang-
References
[1] Alexandros Haliassos, Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. Lips don't lie: A generalisable and robust approach to face forgery detection. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 5039-5049, 2021.
10

---

[2] Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, and Ziwei Liu. Forgerynet: A versatile benchmark for comprehensive forgery analysis. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition (CVPR), pages 4360 4369, 2021.
conference
[3] Kunlin Liu, Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Wenbo Zhou, and Weiming Zhang. Deepfacelab: Integrated, flexible and extensible face-swapping framework. Pattern Recogni- Pi), 141:109628, 2023.
[4] Supasorn Suwajanakorn, Steven M Seitz, and Ira Kenelmacher-Shiizzmaa., Synthesizing obama: learning lip sync from audio. ACM Transactions on Graphics (ToG), 36(4):1-13, 2017.
[5] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics+:: Learning detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pages 1-11, 2019.
[6] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A large-scale challenging dataset for deepfake forensics. Proceedings of the IEEE/CVF conference on computer vision and patlern recognition (CVPR), pages 3207-3216, 2020.
[7] Brian Dolhansky, Russ Howes, Ben Pilaum, Nicole Baram, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) preview dataset. arXiv preprint arXiv:1910.0885 2019.
[8] Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397, 2020.
[9] Tianfei Zhou, Wenguan Wang, Zhiyuan Liang, and Jianbing Shen. Face forensics in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 5778-5788, 2021.
[10] Yuezun Li and Siwei Lyu. Exposing deepfake videos by detecting face warping artifacts. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019.
[11] Jiaming Li, Hongtao Xie, Jiahong Li, Zhongyuan Wang, and Yongdong Zhang. Frequency- aware discriminative feature learning supervised by single-center loss for face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 6458 4458-6467, 2021.
[12] Zhiyuan Yan, Yong Zhang. Yanbo Fan, and Baoyuan Wu. Ucf: Uncovering common features for generalizable deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12412-22423, 2023.
[13] Yuan Wang, Kun Yu, Chen Chen, Xiyuan Hu, and Silong Peng. Dynamic graph learning with content-guided spatial-frequent relation reasoning for deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7278 7178-7227.220 2023.
[14] Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 5001-5010, 2020.
[15) Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, and Wei Xia. Learning self- consistency for deepfake detection. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pages 15023-15033, 2021.
[16] Kaede Shiohara and Toshihiko Yamasaki. Detecting deepfakes with self-blended images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18720-18729, 2022.
[17] Yipin Zhou and Ser-Nam Lim. Joint audio-visual deepfake detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 14800-14809, 2021.

---

[18] Weiming Bai, Yufan Liu, Zhipeng Zhang, Bing Li, and Weiming Hu. Aunet: Learning relations between action units for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24709-24711 2023.
[19] Changtao Miso, Zichang Tan, Qi Chu, Nenghai Yu, and Guodong Guo. Hierarchical frequency- assisted interactive networks for face manipulation detection. IEEE Transactions on Information Forensics and Security (TIFS), 17:3008-300 -0081,,,,2200 2022.
[20] Qiqi Gu, Shen Chen, Taiping Yao. Yang Chen, Shouhong Ding, and Ran Yi. Exploiting fine- grained face forgery clues via progressive enhancement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 735 00--44,,222 2022.
[21] Junyi Cao, Chao Ma, Taiping Yao, Shen Chen, Shouhong Ding, and Xiaokang Yang. End- to-end reconstrucionncaasiffctiion learning for face forgery detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 4113-4122, 2022.
[22] Yuting Xu, Jian Liang, Gengyun Jía, Ziming Yang. Yanhao Zhang, and Ran He. Tall: Thumbnail layout for deepfake video detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22658-22668 2023.
[23] Zhiyuan Yan, Yuhao Luo, Siwei Lyu, Qingshan Liu, and Baoyuan Wu. Transcending forgery specificity with latent space augmentation for generalizable deepfake detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9984, 2024.
[24] Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, and Jongwon Choi. Ex- ploiting style latent flows for generalizing deepfake video detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1133-1143, 2024.
[25] Han Chen, Yuezun Li, Dongdong Lin, Bin Li, and Junqiang Wu. Watching the big artifacts: Exposing deepfake videos via bi-granularity artifacts. Pattern Recognition (PR), 135: 109179, 2023.
(26) Shuai Jia, Chao Ma. Taiping Yao, Bangjie Yin, Shouhong Ding, and Xiaokang Yang. Exploring frequency adversarial attacks for face forgery detection. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 4103-4112, 2022.
[27] Ricard Durall, Margret Keuper, and Janis Keuper. Watch your up-convolution: Con based generative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 7890- 7899, 2020.
[28) Davis King. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research (JMLR), Dec 2009.
[29] Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by mining frequency- clues. In European conference on computer vision (ECCV), pages 86-103. Springer, 2020.
[30] Ricard Durall, Margret Keuper, Franz-Josef Pfreundt, and Janis Keuper. Unmasking deepfakes 2019.
with simple features. arXiv preprint
[31] Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel comvolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 1874-1883, 2016.
[32] Andrew G Howard. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint
2017.
12

---

[33] Jiankang Deng, Jia Guo, Jing Yang. Niannan Xue, Irene Cotsia, and Stefanos P Zafeiriou. Areface: Additive angular margin loss for deep face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), page 1-1, Jan 2021.
[34] Jiahao Liang, Huafeng Shi, and Weihong Deng. Exploring disentangled content information for face forgery detection. In European Conference on Computer Vision (ECCV), pages 225. Springer, 2022.
[35] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2016.
[36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[37] Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning (ICML), volume 97 of Proceedings of Machine Learning Research, pages 6105-6114. PMLR, 09-15 Jun 2019.
[38] Iacopo Masi, Aditya Killekar, Royston Marian Mascarenhas, Shenoy Pratik Gurudatt, and Wael AbdAlmageed. Two-branch recurrent network for isolating deepfakes in videos. In Computer EssionEECV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VII 16, pages 667-684, Springer, 2020.
[39] Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. Exploring temporal coherence for more general video face forgery detection. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pages 15044-15054, 2021.
[40] Shen Chen, Taiping Yao, Yang Chen, Shouhong Ding, Jilin Li, and Rongrong Ji. Local relation learning for face forgery detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 108 1081-1088, 2021.
[41] Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. Generalizing face forgery detection with high-frequency features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 16317-16326, 2021.
[42] Ke Sun, Taiping Yao, Shen Chen, Shouhong Ding, Jilin Li, and Rongrong Ji. Dual contrastive learning for general face forgery detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 231 2316-2324, 2022.
[43] Li Lin, Xinan He, Yan Ju, Xin Wang, Feng Ding, and Shu Hu. Preserving fairness generalization in deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16815 16825, 2024.
[44] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 2014.
[45] François Chollet. Xception: Deep learning with depthwise separable convolutions. Pro- ceedings of the IEEE conference computer vision and pattern recognition (CVPR), pages 1251-1258, 2017.
[46] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
[47] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision (IJCV), page 336-359, Feb 2020.
13

---

[48] Ziqi Huang, Kelvin C.K. Chan, Yuming Jiang, and Ziwei Liu. Collaborative diffusion for multi-modal face generation and editing. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[49] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattem recognition (CVPR), pages 4401-4410, 2019.
[50] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition (CVPR), pages 8110-8119, 2020.
[51] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. In International Conference on Machine Learning (ICML), 2015.
[52] Jiakai Wang. Adversarial examples in physical world. In IJCAI, pages 4925 4926, 2021.
[53] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations (ICLR), 2018.
[54] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE symposium on security and privacy (SP), pages 39-57. IEEE, 2017.
14

---

A Appendix supplemental material
Various Loss Weights A1, A2. A3, As. These weights are empirically selected based on experimental results. As shown in Table 7, we evaluate the performance across various set of X1 X2, λ3. X4 The results exhibit that our method is not particularly sensitive to the settings loss weights, with performance variations within approximately 0.5%. For main experiments, select the set of A1 = 1/12, X2
1, A3 = 0.001, 2 1/4 4 as yields the best performance.
Table 7: Effect of 3ur method different loss proportions.
A
A
A
A4
CDF
FF++
FFIW
DFDCP
Avg
1
93.16
96.30
87.82
85,32
90.65
0.1
1
0.1
0.5
93.59
95.60
84.78
86.60
90.14
0.1
1
0.01
0.5
94.27
96.11
85.54
87.81
90.93
0.01
0.0001
0.1
93.60
96.00
85.85
87.38
90.71
0.001
1/4
94.59
96.13
86.14
87.56
91.11
More Details of Complementary Spatial-blendngg Methods. Table 8 shows the detailed results of every manipulation the FF++ dataset. It can be seen that our method improves the performance of all manipulation methods, averaging 1.32% for DSP-FWA, 4.28% for I2G, and 1.77% for Face X-ray. This improvement further demonstrates the effectiveness of our method.
Table &: Effect of our method complementary to
Methods.
FF++
Method
DF
F2F
FS
NT
Avg
DSP-FWA [10]
55.48
43.79
49.26
44.05
48.14
DSP-FWA [10] Ours
56.20
45.93
53.96
41.74
49.46
12G [15]
47.83
82.13
60.82
47.47
59.56
12G [15] Ours
56.90
81.26
61.66
55.52
63.84
X-ray [14]
89,38
85.02
83.45
71.17
82.26
Face X-ray [14] Ours
93.21
85.41
82.70
74.79
84.03
Probability in FreqBlender. As shown in Table 9, evaluate the effect of using various probability a. Note that a denotes the probability of generating a synthetic face whether using spatial- blending using FregBlender after spatial-blending. a denotes only using FreqBlender, while a = means only using spatial-blending color space knowledge is involved, hindering the overall effectiveness of pseudo-
It can be observed that solely using our method can not
perform well, fake faces. In contrast, solely using spatial-blending operations can reach a favorable performance. However, when inserting pseudo-fake faces generated by BreqBlender, the generalization performance is further enhanced, as our method can compensate for the loss of frequency knowledge. The optimal effect on CDF evaluation is achieved when is set to 0.2. This experiment demonstrates the complementary effect of our method to existing spatial- experiments on more datasets have been conducted in the main paper, demonstrating the effectiveness of our method.
methods. More generalization
Table 9: Effect of our method on different data proportions.
FF++
CDF
DF
F2F
FS
NT
Avg
63.18
59.85
65.79
54.61
60.86
56.34
0.8
98.58
94.80
97.69
89.83
95.23
90.27
0.5
99.03
96,99
97.81
91.68
96.38
91.83
0.3
99.12
97.15
97.93
90.91
96.28
93.76
0.2
99.18
96.76
97.68
90.88
96.13
94.59
0.1
99.11
97.11
97.93
90.91
96.26
93.76
99.17
97.63
97.77
91.35
96.50
93.58
Visual Demonstration of FregBlender. The goal of our method is to create pseudo-fake faces that resemble the frequency distribution of wild fake faces. To verify this, we conduct visual experiment on the FF++ dataset. Specifically, we randomly select 3, 000 images from each manipulation method (Deepfakes, FaceSwap. Face2Face, NeuralTexxture) and calculate the average frequency map for each manipulation method respectively. Then we create the same number of pseudo-fake faces using
15

---

0.5
0.2
Deepfakes
-1
FaceSwap
Face2Face
0
NeuralTextures
Figure 7: Heatmap visualization of the frequency difference between method and the wild fake faces (Deepfakes, FaceSwap, Face2Face, NeuralTextures) Note that denotes that method is degraded to SBI [16].
method and calculate their average frequency map. Finally, we visualize the frequency difference between our method and wild fake faces. The results are illustrated in Fig. 7. is important to note means all pseudo-fake faces
that represents the probability of applying BregBlender. Hence, are created using our method, while a denotes our method is degraded to SBI. From this figure, see that the difference is minimal when larger as the pseudo-fake faces are more likely created by SBI. This demonstrates that our method effectively simulates the frequency distribution of wild fake faces.
1. As decreases, the difference becomes
Performance on FF++ Low-quality (LQ). To investigate the performance of our method on low- quality vidcos. conduct experiments on the FF++ Low-quality (LQ) set. The results in Table 10 show that while all methods experience substantial performance drops on the LQ set, our method consistently achieves the highest performance, demonstrating better generalization capability low-quality videos compared to other methods.
Tentative Validation on Various Synthesized Faces. In addition to validation the standard datasets, we investigate a new scenario: Diffusion-baasd face-swap deepfake detection. In this scenario, we employ recent diffusion model (Collaborative Diffusion [48]) to synthesize faces, which are then blended into original videos. create 200 fake faces and evaluate our method in Table 11 I.. It can be seen that method is effective in detecting such forged faces.
16

---

Table 10: Performance of
Table Performance Diffusion-based face-swap and Gan-Generated
method on FT+ (LQ).
images.
Method
FF++LQ)
Method
Diflusion based 48
StyleGAN [49]
StyleGAN2 [50]
12G [15]
52.20
12G [15]
63.51
47.89
43.86
Face X-ray [14]
65.41
Face X-ray [14]
89.81
59.11
66.54
SBI [16]
76.11
SBI [16]
91.70
63.99
72.88
FreqBlender
77.56
FreqBlender
94.74
64.39
76.70
Moreover, although our method designed for face-swapping techniques, we also test it on face images generated by StyleGAN [49] and StyleGAN2 [50]. The results, also shown in Table 11, reveal a notable performance drop across all methods. However, our method still outperforms the others.
Performance against Evasion Attacks. We employ widely recognized library TorchAttacks with four well-known attack methods: FGSM [51], BIM [52], PGD [53] and CW [54]. The attack experiment is conducted on the CDF dataset with six models and the attack configuration is set by
default. The results are shown in Table 12. can be seen without any defense strategies, all models
can be easily attacked in the white-box attacking mode, which exactly aligns with our expectations and the discoveries in the papers of FGSM, BIM, etc.
Table 12: AUC (%) performance against evasion attacks.
Attacked
Method
Original
FGSM
BIM
PGD)
CW
FWA [10]
62.91
12G [15]
$3.35
3.26
0
Prac-Xray [14]
67.99
54.66
0
0
SBI-c23 [16]
92.94
27.19
0
SBI-raw [16]
93.18
11.48
Ours
94.59
22.31
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: After the preliminary analysis section of the paper combined with data analysis, along with our experimental results, we have completed our work comprchensively.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We introduced relevant limitations in the fifth part of our paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

We present our assumptions in the Preliminary analysis section of this paper, and our experimental results confirm the validity of our assumptions.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: You can perfectly reproduce our results according to our paper, and we will also open-source our code in the future.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
The code will be thoroughly organized and released after the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
//niss httppsccccpbccccccceececccccclcc /ww.dee/oooeeuummiieeee /Cooeeeooiissseeeeee wwssssnollicyo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: In the Experimental Setups section of our paper, we have provided detailed descriptions of our experimental details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[NA]
Justification: Our paper's experiments do not include error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
20

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In the Experimental Setups section of our paper, we have provided detailed descriptions of our compute resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https htt///eeurppp.pp/ wwteriis...ppppuu cc/public//Bccecccccccccceeeecee Belliii///tiiittteeee ceuuiieeuielnnn ines?
Answer: [Yes]
Justification: We have reviewed the code of ethics and are confident that we have adhered to this guideline.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Although our paper does discuss the related impacts, our work enhances the capabilities of deepfake detection, which we believe has positive societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our model is designed to enhance the generalization capabilities of deepfake detection, and there is no risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly attributed the work of others in paper and have followed licensing and usage terms.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not introduce new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not have this research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not have this research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "nq9QxI2JN5", "reviewer": "Reviewer_RzXt", "summary": "This paper have introduced an effective way to improve the generalization of DeepFake detection via generating pseudo-fake faces by blending frequency knowledge.  The proposed approach achieves state-of-the-art (SOTA) results on various deepfake detecion datasets.", "questions": "Some images in the paper will be a little blurry after zooming in, so it is best to convert all images to pdf or eps format.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) This paper attempts to combine frequency domain information and spatial domain information to deal with deepfake detection, which is very interesting. Spatial domain blending is very common, but it is still very rare to use it in the frequency domain, which is quite innovative.\n\n2) The writing of this paper is easy to understand and the logic is clear.\n\n3) The experiments in this paper are sufficient and effectively support the author's theoretical basis.", "weaknesses": "1) The idea of ​​this paper is similar to some already published papers, such as [1], [2] and [3]. I hope this paper can cite and further analyze them:\n\n[1] Tan, C., Zhao, Y., Wei, S., Gu, G., Liu, P., & Wei, Y. (2024, March). Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 5, pp. 5052-5060).\n\n[2] Yu, B., Li, W., Li, X., Lu, J., & Zhou, J. (2021). Frequency-aware spatiotemporal transformers for video inpainting detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 8188-8197).\n\n[3] Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., & Holz, T. (2020, November). Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning (pp. 3247-3258). PMLR.\n\n2) Moreover, Ref.30 is an important and widely cited work that introduces the frequency domain into deepfake detection. I don’t quite understand why the author does not compare and analyze it with this work.\n\n3) The source information, page numbers, publishers, etc. of many references are incomplete, and some even are incorrect. For example, [4] should come from CVPR2024 instead of arXiv.\n\n[4] Choi, J., Kim, T., Jeong, Y., Baek, S., & Choi, J. (2024). Exploiting Style Latent Flows for Generalizing Deepfake Video Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1133-1143).\n\n4) The faces in Figure 6 should have been resized. It would be better if the author could show the original resolution image to help readers better understand the experimental results.\n\n5)  Why is the fluctuation of  λ3 so much larger than that of other parameters? I hope the author can provide the corresponding theoretical basis and further detailed analysis.\n\n6) If the author can answer and revise the relevant questions in the final version, I will consider increasing the final score in the next round.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the positive comments on the novelty, writting and experiment configuration, and for the constructive suggestions. \n\n**Q1: The idea of this paper is similar to some already published papers, such as [1], [2] and [3]. I hope this paper can cite and further analyze them.**\n\n**R1:** Thanks for highlighting these references. We analyze them in the following:\n* **Analysis of [1][3]**: References [1] and [3] focus on detecting deepfakes (e.g., GAN-generated faces) by learning frequency information. Specifically, [1] introduces a dedicated architecture to extract frequency information hidden in deepfake faces, while [3] directly analyzes the frequency spectrum of these faces. **Our method differs significantly from these methods:** \n    * **Motivation and methodology are different**: [1][3] focus on designing frequency-sensitive architectures or strategies to capture frequency information from input samples. In contrast, our method creates pseudo-fake faces for training deepfake detectors. This is achieved by a novel frequency-blending strategy to make pseduo-fake faces resemble the real-world fake faces.\n    * **Scope is different**: It should be noted that [1][3] mainly focus on detecting whole face image synthesis, whereas our method lies in the scope of face-swap deepfake detection. \n* **Analysis of [2]**: Reference [2], on the other hand, explores the use of frequency knowledge for **general image forensic tasks (e.g., object inpainting), rather than deepfake face detection**. Thus, this method could not be directly adapted to our task. \n\nIn summary, while these references also employ frequency information, the motivation, methodology and scope are different. We will include these papers and their analyses in the revision.\n\n[1] Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning. AAAI 2024.\n[2] Frequency-aware spatiotemporal transformers for video inpainting detection. ICCV 2021.\n[3] Leveraging frequency analysis for deep fake image recognition. ICML 2020.\n\n \n**Q2: [30] is an important and widely cited work that introduces the frequency domain into deepfake detection. Why the author does not compare and analyze it with this work.**\n\n**R2:** Thank you for the question. Reference [30] is an earlier work (ECCV 2020) that investigated the use of the frequency domain. We cited this work in our main text in order to provide context for using DCT (L108). Since this approach is somewhat outdated, we did not include it in Table 1 of the main text. As suggested, we retrain the model using the released code and evaluate it rigorously according to their instructions. The results are shown in **Table A**. It can be seen that [30] can only perform decently on these recent datasets. We will include this analysis in the revised version.\n\n**Table A: Performance of [30] and our method.**\n|   Method    | CDF | DFDC | DFDCP | FFIW  |\n|:-----------:|:-----:|:----:|:-----:|:-----:|\n| [30]        | 72.93 | 61.16| 81.96 | 61.58 |\n| FreqBlender | 94.59 | 74.59| 87.56 | 86.14 |\n\n**Q3: Why is the fluctuation of λ3 so much larger than that of other parameters? I hope the author can provide the corresponding theoretical basis and further detailed analysis.**\n\n**R3:** Thanks for the constructive suggestion. $\\lambda_3$ is the coefficient of Quality-agnostic Loss (L222), which helps regulate the intensity of noise information. Since Prior and Integrity Loss also play a role in controlling noise intensity (implicitly), $\\lambda_3$ should be relatively subtle compared to other coefficients. To furthter illustrate this trend, we conduct an additonal experiment with $\\lambda_3 = 0.1$. Note that $\\lambda_3 = 1,0.01,0.001$ has been studied in Table 6 of Supplementary. The results are shown in **Table B**, indicating that our method is slightly improved with a lower $\\lambda_3$. We will provide a more detailed analysis in the revision. \n\n**Table B: Effect of our method on different loss proportions.**\n| $\\lambda_1$ | $\\lambda_2$ | $\\lambda_3$ | $\\lambda_4$ | CDF | FF++ | DFDCP | FFIW  | Avg |\n|:-----------:|:----:|:----:|:-----:|:----:|:-----:|:-----:|:-----:|:-----:|\n|0.1 | 1 | 0.1  | 0.5       | 93.59 | 95.60 | 86.60 | 84.78 | 90.14 |\n|0.1 | 1 | 0.01 | 0.5       | 94.27 | 96.11 | 87.81 | 85.54 | 90.93 |\n\n**Q4: Many references need to be revised.**\n\n**R4:** We will carefully check the references and revise them accordingly.\n\n**Q5: It would be better if the author could show the original resolution image (Figure 6) to help readers better understand the experimental results.**\n\n**R5:** We will revise Figure 6 accordingly.\n\n\n**Q6: Some images in the paper are a little blurry after zooming in. It is best to convert all images to pdf or eps format.**\n\n**R6**: We will update these figures accordingly to enhance clarity.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MjvR82iQwt", "reviewer": "Reviewer_Xqko", "summary": "This paper proposes FreqBlender, a new method to generate pseudo-fake faces that effectively simulate the frequency distribution of wild fake faces. Unlike common blending techniques done in the spatial domain, their method blends frequency knowledge. An analysis is conducted, showing that three frequency components are present in faces, namely semantic information, structural information, and noise information. They demonstrated that structural information contains forgery traces.\n\nTo this end, the first stage of their method employs FPNet, a novel architecture built to disentangle the input fake face into the three different frequency components. As no ground truth exists for this task, carefully crafted objectives provide the necessary supervision.\n\nIn the second stage, the trained network is used to parse the frequency components, and the structural component is extracted from a given fake face. It is then blended into a real face to obtain the pseudo-fake.\n\nThe method outperforms the state-of-the-art across different relevant datasets.", "questions": "Clarification Needed on FPNet Training:\nWhile the authors explain that for deepfake detection, pseudo-fakes generated using SBI are used as a substitute for fake faces from FF++, the authors should include the results when real fake faces from FF++ are used in Table 1.\n\nUse of PixelShuffle:\nWhy is a PixelShuffle used in the decoder of FPNet?\n\nFPNet Direct Application:\nCan the FPNet be used directly for deepfake detection? What happens when a real face is input into the FPNet?\n\nAblation Study and Frequency Knowledge:\nThe authors successfully conduct an ablation study on the backbone (Table 5) and show that results improve consistently when compared to SBI. Why does this method introduce frequency knowledge when all the tested backbones are spatial (e.g., EfficientNet-b4)? It would be interesting to compare the proposed method with a built-in frequency-based backbone (e.g., Face Forgery Network (F3-Net) [3], AFFGM [1], or the multi-scale high-frequency feature extractor from [2]).\n\nResults on Highly Compressed Data:\nWhat are the results of the method on highly compressed data, i.e., FF++ LQ?\n\n[1] Li et al., Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection, in CVPR 2021\n[2] Luo et al., Generalizing Face Forgery Detection with High-frequency Features, in  CVPR 2021\n[3] Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues, in ECCV 2020", "limitations": "Acknowledgement of Limitations:\nThe authors acknowledge the usual limitations of pseudo-fake generation methods, i.e., the hypothesis that the test face is crafted using face-swapping techniques may not hold when the test face is generated using other techniques.\n\nDiscussion on Societal Impact:\nThe race between attackers and defenders is a well-known issue in the field of deepfake detection. The authors should discuss the potential negative societal impact of their work and how it could be exploited by attackers to generate more realistic deepfakes by simply training their generators to fool the proposed method.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality:\nThis method is the first to mine frequency structural information and propose a way of generating pseudo-fake faces that mimic the frequency distribution of fake faces.\n\nQuality:\nThe method is well evaluated on different datasets, and the results are consistent.\n\nClarity:\nThe paper is well written, and the method is well explained. Figures, along with a preliminary analysis of the frequency components present in faces, are provided, which help to understand the method.\n\nSignificance:\nThis method provides a new pseudo-fake mechanism that complements spatial blending, which is the reference in the literature. The authors have shown that their method improves the results of different frame and spatial blending-based methods such as DSP-FWA, I2G, Face-Xray, and SBI. The authors claim that the code will be released in the future. To ease the reproduction and adoption of this technique, we encourage them to also release the pretrained weights of FPNet.", "weaknesses": "References Missing:\nSome references are missing, for example, methods that are frame-based and only use real faces during training [1, 2], as well as other recent detectors [3, 4].\n\nAdditional Overhead:\nDuring the training of the detector, the inference of FPNnet (1 encoder and 3 decoders) is required for generating a pseudo-fake. This adds an overhead during training. The authors should include an efficiency analysis.\n\n[1] Li et al., Pixel bleach network for detecting face forgery under\ncompression, in IEEE Transactions on Multimedia 2023\n[2] Larue et al., SeeABLE: Soft Discrepancies and Bounded Contrastive Learning\nfor Exposing Deepfakes, in ICCV 2023\n[3] Dong et al., Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization, in CVPR 2023\n[4] Guo et al., AltFreezing for More General Video Face Forgery Detection, in CVPR 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the positive feedback regarding the originality, quality, clarity, and significance of our work, and are grateful to the constructive comments.\n\n**Q1: Some references are missing, e.g., [1, 2, 3, 4].**\n\n**R1:** Thanks for the suggestion. We will include these related references in the revision.\n\n**Q2: Additional Overhead: FPNet (1 encoder and 3 decoders) adds an overhead during training. The authors should include an efficiency analysis.**\n\n**R2:** Thanks for the constructive suggestion. Note that the encoder only contains four convolutional layers and each decoder  contains four layers made up of a convolutional layer and a PixelShuffle operation (L173-L175). Thus, FPNet is lightweight and has small overhead. **Table A shows the efficiency analysis regarding FLOPs, Params and Run-time of FPNet**. Using an Nvidia 3080ti GPU, the run-time is 248 FPS (0.004 seconds per image), and creating a pseudo-fake face costs 0.074 seconds on average. Thus our method only introduces minimal overhead in training detectors. \n\n**Table A: Efficiency analysis of FPNet.**\n| Architecture|  Params (M)  | FLOPs(G)| Run-time (FPS) |\n|:-----------:|:--------:|:-----:|:-----:|\n| FPNet (Encoder * 1,Decoder * 3)     |   20.14  | 29.10 |  248  |\n\n**Q3: While the authors explain that pseudo-fakes generated using SBI are used as a substitute for fake faces from FF++, the authors should include the results when real fake faces from FF++ are used in Table 1.**\n\n**R3:** Thanks for the constructive suggestion. We conduct an extra experiment accordingly, as shown in **Table B**. It can be seen that by using \"real\" fake faces to create pesudo-fakes with our method, the performance is notably degraded across all datasets. This is because compared to SBI, only using the fake faces from FF++ lack diversity, primarily reflecting the frequency distribution of FF++ instead of real-world fake faces. We will include these results into revised version.\n\n**Table B: Performance of using fake faces from FF++.**\n|     Method  | Type |   CDF | DFDC | DFDCP | FFIW  |\n|:-----------:|:----:|:-----:|:----:|:-----:|:-----:|\n| FreqBlender |\"real\" fake |   75.79    |  66.30   |   82.01   |  67.70 |\n| FreqBlender |SBI | 94.59 | 74.59| 87.56 | 86.14 |\n\n**Q4: Why is a PixelShuffle used in the decoder of FPNet?**\n\n**R4:** PixelShuffle is an effective upsampling operation that is widely used in generative models. This operation rearranges the channels and reshapes the features by a specified factor. Since the decoder is designed to generate frequency component masks, we employ PixelShuffle operations for upsampling. We will include these descriptions in the revision for better clarity.\n\n\n**Q5: Can the FPNet be used directly for deepfake detection? What happens when a real face is input into the FPNet?**\n\n**R5:** Thanks for the thoughtful questions. The responses are as follows:\n1) **FPNet is not intended for direct deepfake detection**. This is because that FPNet is designed to analyze the frequency components, which serves as a preprocessing step to create effective pseudo-fake faces for training deepfake detectors.  \n2) **FPNet can decompose the frequency domain into three components for both real and fake faces.** But for real faces, the frequency component corresponding to structural information does not contain forgery traces. \n\n**Q6: ITable 5) Why does this method introduce frequency knowledge when all the tested backbones are spatial (e.g., EfficientNet)?** \n\n**R6:** Thanks for the thoughtful question. Our method creates effective pseudo-fake faces by blending frequency knowledge. These pseudo-fake faces contain essential frequency knowledge and are used for training deepfake detectors. Thus, the frequency knowledge can be introduced, even though the backbones are spatial. \n\n**Q7: (Table 5) It would be interesting to compare the proposed method with a built-in frequency-based backbone (e.g., F3-Net [3], AFFGM [1], or the multi-scale high-frequency feature extractor from [2]).**\n\n**R7:** Thanks for the constructive suggestion. As suggested, we attempt to explore the effect of the backbones in frequency-based methods. Since AFFGM [1] has not released its codes, we are unable to test our method on this backbone. Nevertheless, we retrain F3-Net [3] and GFFD [2] using their released codes. The results, **shown in Table C**, indicate that the performance of these frequency-based backbones also improved, demonstrating the effectiveness of our method on frequency-based backbones. We will include these results and analysis in the revision.\n\n**Table C: Performance of our method with F3-Net [3] and GFFD [2].**\n|  Method          | CDF   | FF++ | DFDCP | FFIW  | Avg   |\n|:-----------:     |:-----:|:----:|:-----:|:-----:|:-----:|\n| F3-Net [3] + SBI | 84.94 | 93.42| 79.29 | 73.42 | 82.77 |\n| F3-Net [3] + Ours| 88.10 | 95.16| 84.32 | 74.49 | 85.52 |\n| GFFD [2] + SBI   | 81.34 | 91.81| 77.19 | 65.53 | 78.97 |\n| GFFD [2] + Ours  | 86.71 | 92.18| 78.25 | 77.45 | 83.65 |\n\n\n**Q8: Results on Highly Compressed Data: What are the results on highly compressed data, i.e., FF++ LQ?**\n\n**R8:** Thanks for the insightful question. **Table D** shows the results of our method when testing on FF++ LQ set. It can be observed that the performance of all methods significantly declines on the LQ set, which aligns with our expectations since compression operations can obscure forgery traces, making detection more challenging. However, our method still achieves the best performance compared to others, demonstrating better generalization ability on low-quality videos.\n\n**Table D: Performance of our method on FF++ LQ.**\n|    Method   | FF++ LQ  |\n|:-----------:|:-----:   |\n| I2G         |  52.20   |\n| Face x-ray  |  65.41   |\n| SBI         |  76.11   |\n| FreqBlender |  77.56   |\n\n\n**Q9: We encourage them to also release the pretrained weights of FPNet**\n\n**R9:** Thanks for the suggestion. We will release the weights along with the codes after acceptance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qzNBaw2FTw", "reviewer": "Reviewer_GPf8", "summary": "This work studies generalizable deepfake detection. The proposed method is motivated by a new data augmentation method that blends real and fake faces in the frequency domain. The paper claims the forgery can be found in three different frequency bands and proposes an unsupervised learning method, Frequency Parsing Network. Empirical results indicate that the proposed method's performance achieves results comparable to state-of-the-art.", "questions": "1. line 31 says the motivation of simulating various blending artifacts. However, these artifacts are quite \"old\" compared to that from current diffusion models. That says, how would your method perform on the face generated by stable diffusion? such as instantid, photomaker?\n2. did you compare with DIRE and HiFi-Net?\nR1: DIRE for Diffusion-Generated Image Detection, ICCV2023\nR2: Hierarchical fine-grained image forgery detection and localization, CVPR2023", "limitations": "I am impressed by the learning objective formulation, but I am inclined to reject it because it does not achieve reasonable sota performance and does not show generalization to diffusion-generated images.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. unsupervised way of learning the frequency component, which can favor its usage. \n2. I like the analysis from Fig 2 and 3, showing the difference between real and fake residing in the high-frequency band.\n3. I enjoy the problem formulation as well as the learning object in section 4.2.", "weaknesses": "1. the proposed method does not achieve state-of-the-art detection performance in table 1.\n2. table 2 only has a few methods on the FF++ dataset. It should not be the case that SBI is only a baseline to compare. \n3. the proposed approach might be limited in standard face-swap fake face, or gan-generated face. How about diffusion model generated face?\n4. in terms of approach. the proposed method works when capturing semantic, structural and noise information. What if it fails to capture any one of those?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable time and comments.\n\n**Q1: The proposed method does not achieve state-of-the-art detection performance in table 1.**\n\n**R1:** We would like to highlight that **our method achieves the highest number of top-1 rankings compared to all others (best performance on 3 out of 4 datasets)**, and ranks 3rd on the DFDC dataset among the 20 methods compared. Given the wide variety of detection strategies and deepfake datasets, we believe this achievement can demonstrate the comprehensive superiority of our method. \n\nMoreover, we respectfully believe that both the AC and all reviewers could agree with that, the value and contribution of a work should not solely be judged by empirical results, but also by the innovative insights it offers for future research. \n\n**Q2: Table 2 only has a few methods on the FF++ dataset. It should not be the case that SBI is only a baseline to compare.**\n\n**R2:** The rationale for using SBI lies in two aspects:\n* **Second-best performance**: SBI consistently performs better than other methods (**coming in second only to our method, achieving the second-best performance on 2 out of 4 datasets**). Therefore, we believe that comparing our method with SBI is representative and can reflect the efficacy of our method in cross-manipulation evaluation. \n* **High relevance**: Our method adopts SBI as the substitute for fake faces in creating frequency-blending pseudo-fake faces. Thus, comparing our results with those of SBI further highlights the effectiveness of our method. \n\nWe will include these explanations in the revision for better clarity.\n\n\n**Q3: The proposed approach might be limited in standard face-swap fake face, or gan-generated face. How about diffusion model generated face? This method does not show generalization to diffusion-generated images.**\n\n**R3:** Thanks for the thoughful question. We provide our responses in three aspects:\n* **Our method is in scope of face-swap deepfake detection**: As described in Introduction (L29-L31) and stated in the limitation section (L322-L325), our method targets for the face-swap deepfake detection, **a significant topic in recent years, as evidenced by works such as Face x-ray [14] (CVPR2020), PCL [15] (ICCV2021), SBI [16] (CVPR2022), UCF [12] (ICCV2023), BiG-Arts [25] (PR2023), F-G [43] (CVPR2024), and LSDA [23] (CVPR2024)**. This scope limitation has been noted by other reviewers as well. \n* **It is important to note that detecting diffusion-generated faces and face-swap deepfake are typically two different tasks**: Detecting Diffusion-generated images typically falls under the category of whole image synthesis, as seen in works such as DIRE (ICCV2023, the suggested work by reviewer), AVG (ICASSP2023). In these works, they do not validate themselves on face-swap deepfake datasets studied in our paper. Therefore, **while detecting diffusion-generated images is also an important task, it falls outside the scope of our method.** \n* **In response to the suggestion, we investigate whether our method can facilitate the detection of Diffusion-based generated images.** We conduct an additional scenario: **Diffusion-based face-swap deepfake detection**. In this scenario, a recent diffusion model (Collaborative Diffusion (CVPR2023)) is used to synthesize a face, which is then blended into original videos. We use the dataset provided by this work to create 200 fake faces, duo to the time contraints. **Table A** presents the performance of our method in this context, demonstrating its effectiveness in detecting such forged faces. We will include this experiment into the revision.\n\n**Table A: Performance of Diffusion-based face-swap deepfake detection.**\n|             | Diffusion-based  |\n|:-----------:|:----:      |\n| FreqBlender |     94.74     |\n\n\n**Q4: The proposed method works when capturing semantic, structural and noise information. What if it fails to capture any one of those?**\n\n**R4:** Thanks for the thoughtful question. The proposed method is capable of decomposing any input face into three frequency components. However, in certain exceptional cases (e.g., whole face synthsis or attribute editing), the captured structural information may not fully cover the forgery traces, affecting the effectiveness of pseudo-fake faces and subsequently limiting the efficacy of deepfake detectors. \n\n**Q5: \"Line 31 says the motivation of simulating various blending artifacts. However, these artifacts are quite \"old\" compared to that from current diffusion models. How would your method perform on the face generated by stable diffusion?**\n\n**R5:** See response in R3.\n\n**Q6: Did you compare with DIRE and HiFi-Net?**\n\n**R6:** Thanks for the value question. \n* **DIRE is intended for detecting fully synthesized faces, rather than specifically identifying face-swap deepfakes.** To compare with ours, we adapt this method under our scenario. **Table B** illustrates the performance of both DIRE and our method, demonstrating that our method significantly outperforms DIRE.\n* **HiFi-Net, on the other hand, is built for general image manipulation localization (i.g., segmentation) and could not be used for face forgery detection.** Thus, HiFi-Net is not included in our comparisons. \n\n**Table B: Comparison with DIRE and HiFi-Net.**\n|             | CDF   | DFDC | DFDCP | FFIW  |\n|:-----------:|:-----:|:----:|:-----:|:-----:|\n| DIRE        | 46.71 | 51.97| 45.24 | 50.82 |\n| FreqBlender | 94.59 | 74.59| 87.56 | 86.14 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q8bYNPk2jN", "reviewer": "Reviewer_Z1eA", "summary": "The generalization of DeepFake detection can be addressed by enhancing training data with synthetic fake faces, known as pseudo-fakes. Traditional methods generate these faces by spatial blending operations. However, the limitations of these methods are that they ignore to simulate the frequency distribution, where additional important forgery clues might be found.\n\nThis paper introduces an interesting method called FreqBlender, which attempts to blend proper frequency knowledge to enhance the effectiveness of pseudo-fake faces. This method involves identifying forgery clues in the frequency domain and blending the corresponding frequency components from fake faces into real faces. This process is challenging due to the variability and spread of forgery clues across different frequency ranges.\n\nTo achieve frequency decomposition, the authors propose a Frequency Parsing Network (FPNet) that can adaptively partition the frequency domain. The network, consisting of an encoder and three decoders, extracts semantic, structural, and noise information from the faces. Training FPNet is difficult due to the lack of ground truth for frequency distribution, so the authors devise a novel training strategy leveraging inner correlations among different frequency components.\n\nOnce trained, FPNet can extract the structural information of a fake face's frequency component and blend it with a real face to generate a pseudo-fake face. This method complements existing spatial blending methods and improves detection performance on multiple DeepFake datasets, demonstrating its effectiveness.", "questions": "1.\tAs described in Section 3, the statistics of the frequency distribution are calculated using an azimuthal average, which includes a logarithmic transformation and the calculation of azimuthally-averaged flux in circular annular apertures. Could you provide more details on why placing the center of the circular annular aperture at the top-left corner of the frequency map results in a one-dimensional spectrum diagram?\n2.\tThe FPNet uses three independent decoders to analyze the frequency domain. The output of the decoders is not clearly explained, as it is visualized in grayscale as the frequency map in Fig. 5. I understand that the output is a soft mask ranging from [0,1] for the frequency map. Therefore, I suggest using a color version to replace the DCT map in Fig. 5.\n3.\tIn the discussion on the loss of authenticity determination, the authors aim to create two sets of faces, with and without forgery traces. Does the set C_r represent the face sets that do not contain structural information of fake faces, while C_f corresponds to the face sets that have structural information of fake faces?\n4.\tIn Prior and Integrity Loss, the final term is designed to ensure that the combination of three frequency components covers the entire frequency domain. This term sums these frequency components and calculates the distance from a mask of 1. However, this term might not guarantee that the sum of these frequency components equals 1, but it can reduce deviations. What is the rationale behind this design??", "limitations": "This paper lists its limitations in main body. Since the proposed method is designed to address the limitations of existing spatial-blending techniques. it operates under the assumption that faces are forged using face-swapping methods. It unlikely tackle other types of forgeries, such as whole-face synthesis.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This papeer enhancing the generalization of DeepFake detection is a critical and current issue in AI safety, as real-world DeepFakes are likely generated by unknown models. This method addresses this challenge by taking an innovative approach, creating pseudo-fake faces through blending frequency knowledge rather than the conventional spatial knowledge used in existing methods.\n\n2. This work identifies the limitations of existing methods, noting that spatial blending only mimics the visual similarity of wild fake faces but overlooks their frequency characteristics. By analyzing the frequency distribution, the authors describe a Frequency Parsing Network (FPNet) to parse the frequency range carrying forgery traces. Interestingly, this method doesn't require wild fake faces for blending; instead, it uses existing spatial-based pseudo-fake faces as surrogates.\n\n6. The method's efficacy is validated across multiple recent DeepFake datasets, demonstrating its effectiveness with various backbones and complementing existing spatial-blending methods. This showcases its robustness and practical applicability in enhancing detection performance.", "weaknesses": "This paper proposes an interesting and intuitive method that seems reasonable to me. Here are some recommendations for future improvements:\n\n1. Investigate more fine-grained frequency ranges to uncover the detailed composition of artifacts. This could potentially enhance the model's performance by providing a more nuanced understanding of forgery clues.\n\n2. Develop algorithms to enhance the explainability of the model’s decisions. This will help users understand how and why a particular face is identified as a DeepFake, increasing trust and usability in practical applications.\n\n3. Extend the method to handle other types of forgery operations beyond face-swapping, such as whole-face synthesis and attribute editing. This will broaden the method's applicability and ensure it remains effective across a wider range of DeepFake techniques.\n\n4. Utilizing LLMs may help you parse the frequency more precisely, as LLMs can provide the prior knowledge which may compensate for the lack of ground truth when training FPNet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive review on the significance of our topic, novelty and experimental results.\n\n\n**Q1: Could you provide more details on why placing the center of the circular annular aperture at the top-left corner of the frequency map results in a one-dimensional spectrum diagram?**\n\n**R1:** Thanks for the question. We follow the process in [26,27], where we use the top-left corner of frequency map as the center for several circular annuli apertures. The radius of these apertures is varied from 0 up to the side length of the frequency map, with a fixed interval. Then we average the signals inside the interval area between adjacent circular annuli apertures. In this way, we convert a two-dimensional frequency map into a one-dimensional frequency spectrum. We will revise correpsonding descriptions to improve clarity.\n\n**Q2: I understand that the output is a soft mask ranging from [0,1] for the frequency map. Therefore, I suggest using a color version to replace the DCT map in Fig. 5.**\n\n**R2:** Thanks for your suggestion. We will replace it with a color version.\n\n**Q3: Does the set C_r represent the face sets that do not contain structural information of fake faces, while C_f corresponds to the face sets that have structural information of fake faces?**\n\n**R3:** Yes, your understanding is correct.\n\n**Q4: The Prior and Integrity Loss sums these frequency components and calculates the distance from a mask of 1. What is the rationale behind this design?**\n\n**R4:** Thanks for the insightful question. We expect these frequency components will span the entire frequency domain (all elements in frequency map) while minimizing overlap. Therefore, we calculate the distance of the sum of these frequency components using mask 1. We will enhance the clarity of the related descriptions in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers for their valuable time and professional comments. We are encouraged by the positive feedback and suggestions on the following aspects:
* **Soundness, Presentation, and Contribution**: 
    * **All reviewers** rate these sections as **Good** in their reviews.
* **Novelty of our method**: 
    * **Reviewer RzXt** acknowledges that our method is quite interesting and innovative in terms of frequency blending. 
    * **Reviewer Xqko** recognizes the originality and significance of our method, particularly in the new pseudo-fake mechanism. 
    * **Reviewer Z1eA** highlights that the method addresses an existing challenge by taking an innovative approach of frequency blending. 
* **Extensive and sufficient experiments**: 
    * **Reviewer RzXt** notes that the experiments are sufficient and effectively support the theoretical basis. 
    * **Reviewer Xqko** comments that our method is well evaluated, with consistent results. 
    * **Reviewer Z1eA** emphasizes that our method is valdate across various DeepFake datasets with different backbones. 
* **Well-written and clear logic**:
    * **Reviewer RzXt**, **Reviewer Xqko** and **Reviewer Z1eA** comment that the paper is well-written, the logic is clear and the method is well explained.
    * **Reviewer Xqko**, **Reviewer GPf8** like the preliminary analysis in figures of frequency components in faces. 

We appreciate that **Reviewer GPf8** has a positive view of our  unsupervised way of learning frequency comopents, the analysis of frequency distribtuion and the problem formation along with learning objects. Meanwhile, **Reviewer GPf8** also raises two concerns: **"...but I am inclined to reject it because it does not achieve reasonable sota performance and does not show generalization to diffusion-generated images."**

We would like to highlight the responses to these concerns:
* **Concern of SOTA**: We would like to highlight that **our method achieves the highest number of top-1 rankings compared to all others (best performance on 3 out of 4 datasets)**, and ranks 3rd on the DFDC dataset among the 20 methods compared. Given the wide variety of detection strategies and deepfake datasets, we believe this achievement can demonstrate the comprehensive superiority of our method. **Moreover, we respectfully believe that both the AC and all reviewers could agree with that, the value and contribution of a work should not solely be judged by empirical results, but also by the innovative insights it offers for future research.** 
* **Concern of generalization to Diffusion-generated images**: We would like to thank the reviewer for this thoughful comment. 
    * **Our method is in scope of face-swap deepfake detection**: As described in Introduction (L29-L31) and stated in the limitation section (L322-L325), our method targets for the face-swap deepfake detection, **a significant topic in recent years, as evidenced by works such as Face x-ray [14] (CVPR2020), PCL [15] (ICCV2021), SBI [16] (CVPR2022), UCF [12] (ICCV2023), BiG-Arts [25] (PR2023), F-G [43] (CVPR2024), and LSDA [23] (CVPR2024)**. This scope limitation has been noted by other reviewers as well. 
    * **It is important to note that detecting diffusion-generated faces and face-swap deepfake are typically two different tasks**: Detecting Diffusion-generated images typically falls under the category of whole image synthesis, as seen in works such as DIRE (ICCV2023, the suggested work by reviewer), AVG (ICASSP2023). In these works, they do not validate themselves on face-swap deepfake datasets studied in our paper. Therefore, **while detecting diffusion-generated images is also an important task, it falls outside the scope of our method.** 
    * **In response to the suggestion, we investigate whether our method can facilitate the detection of Diffusion-based generated images.** We conduct an additional scenario: **Diffusion-based face-swap deepfake detection**, where the diffusion model is adapted to a face-swap scenario. In this scenario, a diffusion model (Collaborative Diffusion (CVPR2023)) is used to synthesize faces, which are then blended into original videos. The results are shown in **R3** of our response to **Reviewer GPf8**, demonstrating the efficacy of our method in detecting Diffusion-based face-swap deepfakes. We will include this experiment in the revision.

We hope above responses could address the concerns of **Reviewer GPf8**, and facilitate AC to make a more comprehensive assessment of the value and contribution of our work.

Meta-review: This paper received mixed reviews, ranging from "Borderline reject" to "Weak Accept". Reviewers acknowledged the innovative DeepFake detection method proposed in the paper (Z1eA, Xqko, RzXt). With Xqko stating that "This method is the first to mine frequency structural information and propose a way of generating pseudo-fake faces that mimic the frequency distribution of fake faces" (Xqko), and noted the analysis it provided (Xqko, GPf8) and quality of writing (RzXt, Xqko).

The reviews raised a number of concerns, including limited novelty compared to published work that was not all referenced (RzXt, Xqko). The reviewers were concerned with the experiments, noting it did not always achieve state-of-the-art detection (GPf8) and the limited evaluation against recent DeepFake detection methods and datasets, particularly those using diffusion models (GPf8, Z1eA). 

The authors provided detailed responses, though the recommendations ultimately remained divergent. I see the remaining concerns as insufficient reasons for a rejection, preferring to err on the side of acceptance in such cases.
Decision: Accept (poster)