From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes
Authors: Long Ma
Zhiyuan Yan
Jin Xu
Yize Chen
Qinglang Guo
Zhen Bi
Yong Liao
Hui Lin
Venue: NeurIPS 2025 poster ()
Keywords: Universal / Generalizable Deepfake Detection
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Detecting deepfakes has been an increasingly important topic, especially given the rapid development of AI generation techniques.
In this paper, we ask: How can we build a universal detection framework that is effective for most facial deepfakes?
One significant challenge is the wide variety of deepfake generators available, resulting in varying forgery artifacts (e.g., lighting inconsistency, color mismatch, etc).
But should we ``teach" the detector to learn all these artifacts separately? It is impossible and impractical to elaborate on them all.
So the core idea is to pinpoint the more common and general artifacts across different deepfakes.
Accordingly, we categorize deepfake artifacts into two distinct yet complementary types: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). 
FIA arise from the challenge of generating all intricate details, inevitably causing inconsistencies between the complex facial features and relatively uniform surrounding areas.
USA, on the other hand, are the inevitable traces left by the generator's decoder during the up-sampling process.
This categorization stems from the observation that all existing deepfakes typically exhibit one or both of these artifacts.
To achieve this, we propose a new data-level pseudo-fake creation framework that constructs fake samples with only the FIA and USA, without introducing extra less-general artifacts.
Specifically, we employ a super-resolution to simulate the USA, while  utilise  image-level self-blending on diverse facial regions to create the FIA.
We surprisingly found that, with this intuitive design, a standard image classifier trained only with our pseudo-fake data can non-trivially generalize well to previously unseen deepfakes.

From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes
Long Ma'
Zhiyuan Yan Jin Xu* Yize Chen²
Qinglang Guo Zhen Bi
Yong Liao Hui Lin?
School of Cyber Science and Technology, University of Science and Technology of China Schhoo of Electronic and Computer Engineering, Peking University
SShooo of Data Science, The Chinese University of Hong Kong
"Scoool of Information Science and Technology. University of Science and Technology of China Huuzoo University Banbu AI Foundation
3 China Academy of Electronics and Information Technology (longm@mail, yliao@)ustc. edu.
Abstract
Detecting deepfakes has been an increasingly important topic, especially given the rapid development of Al generation techniques. this paper, we ask: How can we build universal detection framework that is effective for most facial deepfakes? One significant challenge is the wide diversity of existing deepfake generators, which produced varied types of forgery artifacts (e.g., lighting inconsistency, color mismatch, etc). But should we "teach" the detector to learn all these artifacts sepa- rately? It is impossible and impractical to elaborate on them all. So the core idea is to pinpoint the more common and general artifacts across different deepfakes. Through systematic analysis of shared technical frameworks in existing deepfake algorithms, we categorize deepfake artifacts into two distinct yet complementary types: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). FIA arise from the challenge generating all intricate details, inevitably causing incon- sistencies between the complex facial features and relatively uniform surrounding areas. USA, on the other hand, are the inevitable traces left by the generator's 's decoder during the up-sampling process, with all existing deepfakes exhibiting either or both artifacts. Subsequently. we propose novel image-level pscudo-fake creation framework that constructs fake samples with only the FIA and USA, with- out introducing extra less-general artifacts. Specifically, we reconstruct the target face to simulate the USA, while utilize image-level blend on diverse facial regions to create the FIA. We surprisingly found that, with this intuitive design, standard image classifier trained only with our pseudo-fake data can non-trivially generalize well to unseen deepfakes.
Introduction
In recent years, A growing number of facial manipulation techniques have advanced due to the rapid development of generative models [35] 36 58 75). which facilitate the production of highly realistic and virtually undetectable alterations. As a result, the risks of personal privacy being violated and the spread of misinformation have increased significantly. Therefore, developing universal deepfake detector that can be used to detect the existing diverse fake methods has become an urgent priority.
*Corresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a). Face Inconsistency Artifacts (FIA)
Face-level Inconsistencies
Region-level Inconsistencies
Real
Pake
Real
DOPM
Byebrow
Moe: Deuils
Los Detaila
More Desils
Los Details
Mouth
Nove
Frechead
Al-genratee
facc replon compared surrounding
(b). Up-Sampling Artifacts (USA)
Real
Fake
Super-Resolution
RQ
LM
976
148
IKS
160
66
Hin6-2: Al-generated often ap-sanpting artffctss the decoders during gener process.
Figure I: Illustration of the identified general forgery artifacts different face deepfakes: (a) Face Inconsistency Artifacts and (b) Up- Sampliniigg Artifacts. We show that the existing face deepfakes typically exhibit both or one of these two artifacts.
this paper, we pose the question: How can we build a universal detection framework that is effective for most facial deepfakes? A significant challenge lies in the wide range of deepfake generators, which lead to different types of forgery artifacts. Specifically, a large number of existing works dedicated creating detection algorithms that carefully designed identify counterfeit artifacts within specific handcrafted designs, including eye blinking [42]. pupil morphology [25), and corneal specularity [32]. However, these methods mainly effective for identifying particular forgery techniques. But should we train the detector to learn each of these artifacts separately? is impossible and impractical to cover them all. So, where should we start from?
The core idea is to encourage the detection model to learn the more generalizable artifacts. As indicated in previous works [8] 63 70,, different fakes might share some common forgery patterns, and the unlimited deepfake artifacts could be generalized and summarized by the limited generalizable artifacts. Motivated by this, conduct in-depth preliminary exploration of the common patterns of existing deepfake artifacts, through which we identify two generalizable deepfake artifacts and categorize them into two distinct yet complementary categories: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA), as depicted in Figure I FIA represents an inherent limitation that deepfakes struggle overcome, originating from the generator's incapacity to precisely reproduce facial attributes and nuances, as well as the inescapable disparities between regions induced by amalgamation processes. Unlike FIA, Up-Sampling Artifacts (USA) are not readily apparent the unaided eye and originate from the generator's inherent up-samnling process that the generator's de- coder cannot adequately substitute. Crucially, since decoding and up- up-sampling constitute fundamental operations in facial manipulation generators, such artifacts become inevitable byproduct across all methods this domain. Numerous prior investigations have substantiated this observation [22 65
To date, all existing deepfakes characteristically display one or both of these artifact categories.
To enable the model simultaneously capture FIA and USA, we propose sophisticated data aug- mentation method: FIA-USA, which leverages super-resolution models [71 and autoencoders [57] introduce Up-Sampling Artifacts (USA) by reconstructing the face, and employs strategy of generating multiple masks to blend the reconstructed face with the original, thereby introducing Face Inconsistency Artifacts (FIA) by creating discrepancies at both the facial and regional levels. Augmented with the aforementioned dual artifacts, our method effectively conditions the model to generalize across unseen deepfakes. Furthermore, maximize the efficacy FIA-USA, we intro- duce two complementary technique: Automatic Forgery-awan Feature Selection (AFFS), which streamlines feature dimensionality and augments feature discernment through adaptive selection. Region-aware Contrastive Regularization (RCR), by juxtaposing the features of the manipulated

---

region against those of the authentic region, RCR enhances the model's focus on the upsampling traces within the manipulated areas and the inconsistencies between different regions.
Extensive experiments conducted on seven widely used Deepfake detection datasets (encompassing over 58 distinct forgery methods, spanning facial manipulation categories including identity swapping, expression reenactment, generative faac synthesis, and attribute editing) validate the superior efficacy of our method. Detailed ablation experiments and robustness experiments have demonstrated the effectiveness of each component in our method and the robustness of our method to disturbances.
Table I: Comparison of our framework and state-of-th--ar((SOTA) methods using pseudo- deepfake synthesis. Our method differs from previous methods in terms of the level at which alterations are applied (local vs global), the introduced artifacts, and the level at which facial inconsistencies are applied.
Alteration artifact
Introduced Artifacts
Methods
Face Inconsistency face-level
global
local
Up-Sampling
region-level
Face Xray 40
PCL
89
OST
SLADD 3
SBI [3]
SeeABLE
17
RBI 64
Plug-and-Pie
3
Ours
2
Related Work
Classic Deepfake Detection. Classic deepfake detection can be categorized into multiple aspects: On
the one hand, typical deepfake image detection methods encompass frequency-domaie analysis [56
49
identity information 17 69]. contrastive learning (23 87). network structure improvement [8
17
watermark [85] 722 robustness [38] [5), interpretability [16] 622 and so forth. On the other
hand, deepfake video detection methods include innovative video representation formats (78), self-
supervised learning [2] 27 (2]. anomaly detection [19 (2)). biological signal [76] 8) multi-modal In addition to the above two aspects, the the remainder of the work involves forgery multi task learning [1 86). adversarial attack 44, and so on.
based 5
localization 11 26
Deepfake Detectors Based on Data Synthesis. A notable approach involves synthesizing pseudo- fake faces during training to enhance generalization capability. Early works like Face X-ray [40] pioneered blending-based augmentation by fusing facial regions from different identities simulate forgery boundaries. Subsequent methods improved upon this paradigm through different artifact simulation strategies: SLADD 3 focused local adversarial perturbations, SBI
(63 proposed self- blending to amplify facial inconsistencies. SecABLE (37] which proposes fine-grained
blending, and Plug-and-Play Play (83] employs facial feature masks rather than full-face manipulation for enhanced face synthesis precision. However, as highlighted in Table [ existing methods predom- inantly focus on isolated artifact types (Mainly focused on FIA) and single-scale modifications (global or local), fundamentally limiting their ability to capture the artifacts interplay inherent in real deepfakes. The paradigm of this data augmentation method can be represented as:
IF M + 4 + (1 M) M)@
2 La
where, I, represents the target face, I, represents the source face, and M represents the mask. Our proposed FIA-USA method injects new vitality into this paradigm by generating multiple types of masks, introducing USA to Jt, as well random combination of artifacts.
3 Method
In this section, we begin by outlining two distinct and prevalent types of forgery artifacts found in deepfakes: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). We then detail the construction of data augmentation techniques (FIA-USA) designed to enable the model to learn to
3

---

FIA-USA
Traing Phase
AE
SR
Bise
Bland
APPS
Psoudo fake
Randomly select recorstructed images
Backboon
FPN
Jacc
Roal
Real
Base image
imsaee
Frosion copres buil
Extract Landmark
Reduce
distamoo
Combining facial features
Patch-kerel Contrastive Regulartation
Generale zzeek
Incataso distance
A Automatic Fowgery-aware Feature Selection (AFTS)
distamon
Image el Contrastive Regularization
lays
Nga
Dueing TFN
tubuZ Eechhundds
Region-aware Contrastive
mastie mnvech boar
Regularization (RCR)
Inference Phase
lon simmiluttly
F"(1 Calziltte the difiemnna forwech dusrn and serfors sealing
features
fretems Jowa Gisensionn
Bockbooe
Real
Kassy Calcalato amilanty and meant indes dissenions siculaity building
a. Initialization
h. Used in Training
lesting image
The overall framework of proposed method consists of: 1) FIA-USA enhances the capability of standard classifiers to effectively detect and generalize to unknown deepfake techniques by generating pseudo-fake data containing Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). 2) Region-aware Contrastive Regularization (RCR) enables the model to focus simultaneously forged boundaries and spatial artifacts through the contrastive regularization of features from forged and real regions. 3) Automatic Forgery-aware Feature Selection (AFFS) achieves efficient feature selection by assessing the similarity between the feature maps of each channel in each layer and the mask.
Figure 2:
detect these artifacts concurrently. Subsequently, we present Automatic Forgery-aware Feature Selection (AFFS) and Region-aware Contrastive Regularization (RCR).
3.1 FIA-USA.
Face Inconsistency Artifacts (FIA), As depicted in Figure D Face Inconsistency Artifacts (FIA) frequently occur in deepfakes due to the inherent difficulty in rendering intricate details, which often results in discrepancies between intricate facial features and the more uniform surrounding regions. In essence, until deepfake technology advances to the point where it can flawlessly replicate real faces, these artifacts are likely persist. We explore the origins of Face Inconsistency Artifacts by categorizing deepfake techniques into two domains: Full Face Synthesis and Face Swapping. In deepfake-based full face synthesis, Face Inconsistency Artifacts inherently persist regardless of generator's capabilities, while their severity directly correlates with the model's performance. For face swapping, these artifacts arise from two primary sources: 1). the efficacy the generator. 2). the blending boundary created during the face swappin''s 's blending procedure. Furthermore, categorize face swapping into two main types based on the forged area: a. macro-editing realize face swapping through 4 or 81 facial keypoints. and b. micro-editing, which enables granular control over individual facial features. In section B of the appendix, we provide a detailed description of the classification of forgery techniques and their differences.
Up-Sampling Artifacts (USA). These artifacts consistently present synthetic facial regions originate from upsampling operations within the generator's 's decoder architecture, where interpolation processes inevitably introduce characteristic distortion patterns, with the extent USA (Up-Sampling Artifacts) introduced varying significantly across different generator implementations. To investigate

---

more universal up-sampling artifacts, we categorize forgery techniques into two types: One category generates images of forged regions that are clear and free of blur as the original image. And another type result in blurred tampered areas, requiring additional facial super-resolution models (SR) to process the blurred areas.
FIA-USA. While FIA and USA represent two fundamental artifact categories in deepfakes, con- ventional data augmentation methods predominantly focus on simulating isolated forgery traces, thereby hindering detection models from effectively learning cross-domain forgery patterns. To bridge this critical gap, we propose FIA-USA- a dual-artifact collaborative enhancement framework engineered for universal detection. Our framework systematically emulates the forgery process through a three-pronged methodology:
1. Mylti-Type Mask Synthesis(MTTMS) The Multi-Type Mask Synthesis module aims to compre- hensively model Face Inconsistency Artifacts (FIA) through hierarchical mask generation strategy, covering both macro-editing boundaries and micro-editing traces. This process involves two comple- mentary approaches:
a. Macro-Editing Mask. We generate facial contours through two configurations: (1) a high-precision mode using 81 facial keypoints compute detailed convex hull boundaries, and (2) a rectangular approximation mode derived from 4 keypoints. The initial mask M is formulated as:
M = M(KK), n E {4,81)
where H-(-) denotes convex hull computation and K,, represents the keypoints set. As same [63], the initial mask M subsequently undergoes erosion or dilation by applying two Gaussian filters with distinct kernel sizes, where erosion occurs when the kernel of the first filter is larger than that of the second, while dilation manifests when the first kernel is comparatively smaller.
b. Micro-Editing Masks. Given set of 81 facial keypoinnss Kg1 (k1,A2.. ... kgg), J. first extract two subsets: eyes-specific keypoints K Keyes C K and mouth-specific keypoints Kmouth C K. Using convex hull computation H-)), we derive two binary masks: the eye mask M,- = H H(Keyes) and the mouth mask Mm. H (.moothh. These masks are then logically combined through union operations
to generate three distinct facial regions: (1) M Me. (2) M Mm. and (3) M Me U Mn (combined mouth and eyes). Since other facial regions such as eyebrows and nose are rarely targeted in localized facial manipulations (e.g., attribute editing, expression synthesis), these features were intentionally excluded from our mask generation framework to prioritize regions most susceptible to forgery (eyes and mouth).
2. Multi-Modal Reconstruction (MMM) We employ two complementary reconstruction paradigms reconstruct the target face([+) in Formula thereby systematically simulating the Up-Sampling Artifacts (USA) inherent in diverse deepfake generators: a. Autoencoder (AE) Reconstruction: Reconstruct 7, through AE (57] to simulate the excessive dependency relationship between adjacent pixels (65] caused by upsampling in GAN / Diffusion models. b. SSper-
(SR) Recon- struction: Applies SR models (71) upsample Lt, thereby replicating the characteristic artifacts inherent in deepfake post-processing pipelines. Specifically, given an input face image I, we generate reconstructed versions LAE- = (E( and Isn SR()..
3. Random Artifact Combination(RAC)): To ensure comprehensive coverage of artifact interactions, we blend original images(source face) with reconstructed variants(target face) (IAR, Isn, I) using mask M, based on Equation [ More specifically, given a facial image/.. first MTMS generates mul- tiple masks M with equal probability, then MMR generates a reconstructed images (LAre, ISR), and finally RAC blend the / with randomly selected reconstruction variants 1, from (LAE, IsR, 1) using randomly selected mask M.. Before blending, 7, and 7, will undergo data cnnnenntttionnchangenn color tone, saturation, contrast, blur, etc) to enhance the inconsistency of statistical information. As shown in Table [ compared to existing methods, FIA-USA has achieved collaborative artifact enhancement across local global, facial/ / regional levels. Please refer to Algorithm|] | for the complete algorithmic process.
3.2
Loss Function
3.2.1
Automatic Forgery-aware Feature Selection
While FIA-USA generates discriminative training samples through dual-artifact augmentation, the inherent feature redundancy in Feature Pyramid Networks (FPN) [45] may drive models to focus on
5

---

Algorithm Pseudocode for FIA-USA
Input: Base image of size (H, W, 3), facial landmarks L of size (81,2)
Output: Image I with FIA and USA
1: def T(I):
Source-Target Augmentation
2:
ColorTransform())
3: I FrequencyTransform/ Transform())
4: return
5: def Recon(I): :
Reconstruct the source image
6:
if Uniform mmnn = O, max = 1) E [0,0.3]
7:
I f AE
8:
else Uniform min = 0, max = 1) E (0.3, 0.5)
9:
It SR(T)
10:
else
11:
Tt I return I
13: I, l +- Rec
if Uniform(mmp
max = 1) < 0.5:
15:
1o < I
else
17:
T,, 4 < I.. T(,)
18: M + CombineFaccillFeetuesss ConvexHull(L)
19: IPF + I.G M 0 -M-
non-critical forgery patterns, thereby compromising detection generalization. This limitation stems from FPN's naive aggregation of all channel-wise features without adaptively emphasizing FIA/USA- correlated representations. To address this limitation, we propose Automatic Forgery-aware Feature Selection (AFFS), statistically-driver feature compression paradigm that quantifies channel-wise sensitivity to forgery regions, thereby dynamically constructing lightweight yet discriminative feature pyramids.
Given a pretrained backbone network é, let f: = (I) E R C. denote the feature map from the i-th layer, where R& represents the input image. To identify artifact-sensitive feature dimensions, we leverage pseudo-fake pairs RR, Fns M, b generated by FIA-USA A and compute the normalized response discrepancy between real and forged regions. For the k-tth channel f E R H; in layer the selection criterion is formulated as:
Lik APPS
-(((()))) Mnl12 N
where F denotes feature normalization and spatial interpolation to align f with mask M,. Channels are ranked by ascending Ci. AFFS and the top-m, channels with minimal errors are retained to form a compressed feature map Jí E IR. +m, for FPN construction. AFFS leverages a pre-trained model initialization, operating supervised feature ranking mechanism that reduces feature redundancy.
3.2.2 Region-aware Contrastive Regularization
Building upon the artifact-sensitive features selected by AFFS, design Region-aware Contrastive Regularization (RCR) to explicitly model the divergence between forged and authentic regions through multi multi-granularic granularity contrastive learning. Given the Feature Pyramid Network (FPN) P. we denote the feature maps of real and fake faces HR (R)(R), Hp = P()(F)) For H the areas inside and outside are real, while for Hr, the areas inside M are manipulated and the areas outside M are real. We define the region within M as H Hin and the region outside M as Hout Hout, RCR operates on the refined feature pyramid from AFFS, implementing aulti-granularity contrast through:

---

Patch-level Contrastive Regularization. For real faces, the features f of H and HR are consistent, thus we aim to minimize the distance between them. Conversely, for fake faces, the features f of H and H are distinct, thus we aim to maximize the distance between them. Therefore, the loss function Patch-level Contrastive Regularization (PCR) can formulated as:
Σ RREMR R /T
elfi S/T
L1 llg Σ
WT
Σ
JREHR
freHt
where f represents a pixel feature, T is temperature parameter and 8(-) is the normalized cosine similarity between two features, as:
8(f1 f2) ΣE ΣEAL fill F
(5)
SEL BEf
Image-level Contrastive Regularization. Since both the region of real face and fake face outside M are real, we aim to minimize the distance between them and maximize the distance between the fake face and the real face in the area inside M. Therefore, the loss function for Image-level Contrastive Regularization (ICR) can be formulated as:
Σ HouLL Hoo
L2 log
(6)
E fA
50R SUR-FE
Jost Hout
Overall Loss. The network is optimized using the following loss:
L XacE + daLi AsL2,
where LICE denotes the cross-entropy classification loss. LBCE, Li and La weighted by the hyperparame- ters λ1 A2 and λą, respectively.
4
Experiments
4.1
Settings
Datasets. To evaluate the effectiveness our proposed method, we conducted extensive experiments on seven
widely-adopted benchmark datasets spanning both classical facial manipulation paradigms and emerging
generative deepfake architectures.at a. Traditional Deepfake Datasets: 1. FaccForensies+4 (FF++) [59] 2.
Deepfake Detection (DFD) 12] 3. Deepfake Detection Challenge (DFDC) 15. 4. preview version of DFDC
(DFDCP) [4. and 5. CelebDF (CDF) 43). FF++ comprises 1,000 original videos and 4,000 fake videos forged by four manipulation methods, namely, Deepfakes(DF) B Face2Face(F2F) 68), FaceSwap(F'S) [8] and NeuralTexxures(NT) (67] FF++ offers three levels of compression: raw, lightly compressed (c23), and heavily compressed (c40), under storage constraints, our implementation adopts the FF++ c23 (including DFD) with training conducted following the SBI protocol (63), employing exclusively real facial samples from FF++_c2 _=t___333_ subset. Although many previous studies have utilized the same dataset both training and testing, the preprocessing and experimental configurations can differ, making fair comparisons difficult. Therefore, in addition to testing on the raw data of the aforementioned datasets, we also performed generalization assessments on the unified new benchmark for traditional deepfakes, (DeepfakeBench) [ b. Generative Deepfake Datasets. Diffusion Facial Forgery (DiFF) contains over 500,000 images synthesized using 13 different generation methods under four conditions: Text to Image (T2I), Image Image (I21). Face Swapping (FS) and
Face Editing (FE). Text to Image encompasses four generation methods: Midjourney [50). Stable Diffiusion XL
(SDXL) 55 Free-DoM T [84). and HPS [77]. Image to Image comprises Low Runk Adaptationí LoRA) 31
DreamBooth (60, SDXL Refiner (55 and Free-DoM_J. Face Swapping includes DiffFace (36) and DCFace 36
Face Editing comprises Imagic (35] Cycle Difflusion(CycleDif)) 75), and Collaborative Diffussinn( CoDiff)
DF40 80) encompasses 40 state-of-the-art deepfake generation methodologies spanning four core
categories: face swapping, facial reenactment, full-face synthesis, and advanced facial manipulation.
Significantly surpassing existing benchmarks in both scope and volume, DF40 integrates cutting-egge generative architectures from 2024 alongside widely adopted commercial software solutions.
Evaluation Metrics. We report the Frame-Level Area Under Curve (AUC) metric on the Deepfake and DIFF dataset, report the Video-Level Area Under Curve (AUC) metric on the traditional deepfake datasets, to compare our proposed method with prior works. Implementation Details. We adopt EfficientNeIB4 [66] as the backbone network architecture (We also
7

---

Table 2: Cross-dataset Evaluations: Cross-dataset evaluations conducted using the Frame- level AUC. All experiments were trained on the c23 version FF++ and tested on other datasets. # indicates the results cited from [10] and indicates reproduction results using the checkpoints provided by the authors, otherwise, the results are from DeepfakeBench 82
Detector
Backbone
Venues
CDF-v1
CDF-v2
DFD
DFDCP
Avg.
FWA [4
Xception
CVPRW"18 18
0.790
0.668
0.740
0.638
0.714
CapsuleNel
Capsule
ICASSP'19
0.791
0.747
0.684
0.657
0.720
CNN-Aug
ResNet
CVPR'20
0.742
0.703
0.646
0.617
0.677
Face X-ray
HRNet
CVPR'20
0.709
0.679
0.766
0.694
0.712
FFD 11
Xception
CVPR'20
0.784
0.744
0.802
0.743
0.768
F3Net 56
Xception
ECCV*20
0.777
0.798
0.702
0.735
0.749
SPSL 47
Xception
CVPR'2E 20
0.815
0.765
0.812
0.741
0.783
SRM 49
Xception
CVPR'21
0.793
0,755
0.812
0.741
0.775
CORE 53
Xception
CVPRW'22
0.780
0,743
0.802
0.734
0.764
Recce
2
Designed
CVPR'22
0.768
0,732
0.812
0.734
0.761
SBI 62
EfficientNet-B4 B4
CVPR'22
0,813
0.774
0.799
UCF 81
Xception
ICCV'23
0.779
0.753
0.807
0.759
0.774
DD* [
ResNet-34
AAA1'24
0.818
0,864
0.851
ProDet
6
EfficienINet-B4
NeurlPS 24
0.909
0,842
0.848
0.774
0.843
LSDA*
79
EfficientNet-B44
CVPR'24
0.867
0.830
0.880
0.815
0.848
Forensic-Adapter
10
CLJIP (ViT-B/16)
CVPR'25
0.837
0.799
Ours
EfficientNet-BA
0.901
0.867
0.821
0.818
0.852
investigate alternative network architectures and their respective outcomes), trained for 50 epochs using the SAM optimizer [21 with a batch size 12 and initial Icarning rate of 0.001. For video processing, each input uniformly sampled into 32 frames during both training and inference phases. Our data augmentation pipeline combines the proposed FIA-USA strategy with conventional techniques including RandomHoriiontalPPig RandomCutOut, and AddGaussianNoise. The loss coefficients Y₁ A2. À empirically set 1, 2.5, and 0.25 respectively (We also explored the impact of other variants the detection results), with the temperature parameter fixed
experiments were conducted on a single NVIDIA 3090.
4.2
Generalization Evaluation
4.2.1
Traditional Deepfake Datasets.
We first compare our method with previous work the frame level. To ensure the faimess of the experiment, we conducted the experiment the DeepfakeBench, and adhered to the data preprocessing and experimental settings provided by them. For previous work, we utilized the experiment data provided by DeepfakeBench. As shown Table 2 our method outperforms other methods on the majority of deepfake datasets and competes with state-of- the-art methods on the CDF-vl dataset. In addition to comparing at the frame-level, we also compared our method the video-level with state-of-the art detection algorithms, including various data augmentation methods. The comparison results, as shown in Table strongly demonstrate the effective generalization of our method.
Table Comparison with SOTA methods us- Table 4: Comparison with universal deepfake detection methods using the Frame-Level AUC on the DiFF dataset. indicates models that are designed for deepfake detection, while } signifies models intended for generated image detection.
ing the Video-Level AUC.
Model
Venues
CDF-22
DFDC
DFDCP
Face X Ray 40
CVPR'20 20
0.711
LipForensics
28
CVPR'21
FTCN
(90
ICCY"21
0.869
0.740
12G
ICCV'21
0.900
0.743
Test Suhset
HCIL
24
BCCV'22
0.790
0.692
Method
T21
121
FS
FF
ICT
CVPR'22
0.857
SBI 63
CVPR 22
0.928
0.719
0.855
Xception i [59
62.43
56.83
85,97
$8.64
AliFreezing
24
CVPR'23
0.895
N-NNet S6
66.87
67.64
81.01
60.60
SAM
CVPR 24
0.890
EfficienNNet
66
74.12
57.27
82.11
57.20
LSDA
CVPR 24
0.911
0.770
DIRB (73
44.22
64.64
84,98
57.72
CFM
TIFS'24
0.897
0.802
LAA-Ne(f
51
CVPR'24 0.840
0.741
SBI 63
80.20
80.40
85.08
68.79
Ours
0.941
0.732
0.866
Ours
86.05
84.95
89,42
72.73
4.2.2
Generative Deepfake Datasets
Our evaluation on DIFF DF40 datasets demonstrates the superior generalization capabilities our archi- tecture compared to state-or state-of-ther the-art detection methods. Comprehensive metrics are detailed in Table 4 and Ta As shown in Table our method achieves an average AUC of 87.8%. 1,,, surpassing RECCE by 9.7 % and outperforming SBI by 23,4 % and enables 10% higher AUC than LSDA's latent space augmentation

---

Table 5: Comparative Analysis of Detection Performance Between the Proposed Method and State-of-the-Art Approaches on Six Representative Face Swapping Forgery Types in DF40 [80].
Method
Venues
uniface
c4s
facedancer
fsgan
inswap
simswap
Avg.
RECCE 2
CVPR 2022
84.2
65.2
78.3
88.4
79.5
73.0
78.1
SBI 68
CVPR 2022
64.4
69.0
44.7
87.9
63.3
56.8
64.4
CORE 53
CVPRW 2022
81.7
63.4
71.7
91.1
79.4
69.3
76.1
IID 33
CVPR 2023
79.5
71.0
79.0
86.4
74.4
64.0
75.7
UCF 81
ICCV 2023
78.7
69.2
80.0
88.1
76.8
64.9
77.5
LSDA
CVPR 2024
85.4
68.4
75.9
83.2
81.0
72.7
77.8
CDFA
ECCV 2024
76.5
67.4
75.4
84.8
72.0
75.9
Progressivelett
NeurlPS 2024
84.5
71.0
73.6
86.5
78.8
77.8
78.7
Ours
91.8
87.5
83.0
86.3
87.4
91.0
87.8
(a) Ablation Study of Method Components. We re- port the Vidco-Level AUC for traditional deepfake datasets and the Frame-Level AUC for generative deepfake datasets. SR signifies process super-
(b) Ablation Study on Model Architectures.
Test Dataset
Model
Traditional
Generative
Avg.
CDFv22
DEDCP
FS
FF
resolution, and AE refers to the reconstruction achieved
Res50
82.45
79.14
68.82
58.59
72.25
through autoencoders.
Rcs101
85.51
82.35
71.26
60.44
74.89
Effbl
91.88
82.30
83.73
70.69
82.15
Test
Edtb4
94.10
86.66
89.42
72.73
85.72
Setting
Traditional
Generative
Avg.
CDF-v2
DFDCP
FS
FE
Experiment on Hyperparameter Configuration.
W/n MTMS
91.99
87.06
88.00
71.93
84.74
w/o SR
93.25
86.81
80.73
61.07
80.46
Test
w/o AE
92.53
86.37
88.67
70.31
84.47
Ar, Ao, As
Traditional
Generative
Avg.
w/o FIA-USA
93.18
89.45
80.68
62.99
81.57
CDF-v2
DFDCP
FS
FE
PCR
90.12
86.13
89.61
66.87
1,
90.86
85.01
89.91
70.19
83.99
W/o ICR
90.85
85.00
89.95
75.01
85,20
1,
91.88
86.01
89.91
72.19
85.00
w/o RCR
90.14
86.13
89.61
66.87
83.18
1.5.
91.82
86.00
89.95
71.01
84.70
94.70
86.30
70.62
0.255,
91.11
85.59
89.61
70.92
84.30
Ours
94.10
86.66
89.42
72.73
85.72
0.25
94.10
86.66
89.42
72.73
85.72
Table 6: Ablation experiment. We conducted ablation experiments on method components, model frameworks, and hyperparameter configuration. The best results are presented in bold.
average. The results notably contradict recent findings 79 about RGB-hased methods' limitations against generative deepfakes, proving that properly designed RGB-level artifacts retain critical discriminative signals. For comprehensive experimental results on forgery techniques supported by the DF40 and DIFF, please refer to the extended analysis in Appendix Section G
4.2.3 Robustness
In Figure 3 we assessed the influence of diffcrent levels of random perturbations on the detection perfor- mance. We quantified the effects of different levels of Gaussian Blur, Block Wise, Change Contrast, and JPEG Compression the detectors. Notably, gauge accuracy, employed the reduction rate of Video-Level AUC assess the robustness
detector different degrees of disturbance, which could expressed as RAUe = (AUC - AU Crow. ) AU Craw. where AU Craw refers to no perturbations. As shown the Figure our method exhibits superior robustness compared to other methods.
AJF ROT CNDY RA0
Figure 3: Robustness to Unseen Perturbations. We reported the Video-Level AUC Reduction Ratio for four specific types of perturbations at five different levels.

---

4.3
Ablation Study
The impact of method components. To systematically evaluate the contribution of each component in our frame- work, we conducted comprehensive ablation studies across seven benchmark datasets. As detailed in Table (6a our analysis focuses on three core innovations: (1) the FIA- USA augmentation mechanism, (2) Automatic Forgery- aware Feature Selection (AFFS), and (3) Region-aware Contrastive Regularization (RCR). We adopt video-level AUC for traditional deepfakes and frame-level AUC for generative deepfakes. The empirical results demonstrate progressive performance degradation when removing indi- vidual components, with the complete framework achieving optimal detection accuracy. The combination of FIA USA, AFFS, and RCR yields optimal performance. As intended in our design, AFFS and RCR are crucial for fully leverag- ing the potential of FIA-USA. Removing any component degrades performance, with the absence of FIA-USA hav- ing the most significant impact (a 4.15% decrease). It is important to clarify that "w/o FIA-USA" specifically refers to using only the data augmentation proposed by SBI. Furthermore, RCR exerts a stronger influence FIA-USA's effectiveness compared to AFFS.
Ococtral
Mask
Oroainal
Out
DD
16
NI
Tendtiooll
Genentive
Figure 4: GradCAM visualization of fake samples across various deepfake datasets.
The ablation specifically targeting FLA-USA (components within it) reveals: Various data augmentation tech- niques contribute differently. SR and AE are shown he effective for detecting generative forgeries. Conversely, MTMS appears to suppress performance on this specific forgery type. This aligns with our theoretical analysis: generative forgeries constitute global manipulations that produce fewer Face Inconsistency Artifacts (FIA) but primarily exhibit Sp-Sampling Artifacts (USA). As seen from MTMS's ' performance on traditional forgeries. increasing the diversity of FIA proves beneficial for detecting face swapping forgery.
Regarding the RCR ablation: The results align well with our chosen hyperparameters They indicate that the PCR contributes substantially more the model's performance than the ICR. This observation is consistent with the hyperparameter sensitivity analysis presented in Table 6c: increasing the loss weight for PCR while decreasing that for ICR leads to significant performance gains, Conversely, increasing the ICR weight while reducing the PCR weight results in only marginal improvements.
The impact of model framework. Our architectural analysis in Table 6b systemmtttall benchmarks detection performance across backbone networks, quantitatively comparing the effects of EfficientNet [66], ResNet (29). and different depth configurations on detection performance, demonstrates the compatibility of our method with various network frameworks, while also illustrating the critical impact of model parameter size and architectural design on detector performance. This highlights the ongoing importance of developing and specialized deep learning architectures for counterfeit detection tasks.
robust
Impact of hyperparameters. We also examined the effect of hyperparameters of the loss function on model performance, as detailed in the Table 6c| Our analysis of the weights revealed that increasing the weight of da relative to the weights of cross entropy enhances model performance, whereas decreasing the weight of As relative entropy also improves performance, which is consistent with the ablation experiment results for RCR components in Table 6a However, conversely, model performance will not be significantly improved.
5
Visualizations
Our GradCAM 99 analysis in Figure demonstratee precise localization of facial forgery artifacts across both conventional and emerging generative architectures. Notably, the visualization framework not only pinpoints manipulation traces in traditional deepfakes but also effectively in state-of-the-art synthetic media, validating our method's generalization capability.
6 Conclusion
In conclusion, this study presents a universal framework for deepfake detection by focusing on common artifacts that span various types of face forgeries. By categorizing deepfake artifacts into Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA), enhance the generalization capability of detection models. This targeted approach allows the model to focus on detecting fundamental artifact patterns, and potentially improving its performance across diverse deepfake variations. In doing so, our framework may help address some challenges in current detection methods and could provide a promising foundation for developing more adaptable deepfake detection systems in future research.
10

---

Acknowledgments and Disclosure of Funding
This work was supported in part by Key Science & Technology Project of Anhui Province Zhejiang Provincial Natural Science Foundation of China under Grant (No. LQN25F020023).
References
[1] Zhongjie Ba, Qingyu Liu, Zhenguang Liu, Shuang Wu, Feng Lin, Li Lu, and Kui Ren. Exposing the deception: Uncovering more forgery clues for deeptake detection. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 719-728, 2024.
[2] Junyi Cao, Chao Ma, Taiping Yao, Shen Chen, Shouhong Ding, and Xiaokang Yang. End-to-end reconstructonn-haafffcaatii learning for face forgery detection. In Proceedings of the IEEE/CVF Confer- Computer Vision and Puttrrn Recognition, pages 41 1113-1122, 2022.
[3] Liang Chen, Yong Zhang, Yibing Song. Lingqiao Liu, and Jue Wang. Self supervie learning of adversarial example: Towards good generalizations for deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18710-18719, 2022.
[4] Liang Chen, Yong Zhang, Yibing Song, Jue Wang, and Lingqiao Liu, Ost: Improving generalization of deepfake detection via one-shot test-time training, Advances in Neural Information Processing Systems, 35:24597-24610, 2022.
[5] Harry Cheng. Yangyang Guo. Tianyi Wang. Liqiang Nie, and Mohan Kankanhalli. Diffusion facial forgery detection. arXiv preprint arXiv:2401.15852 2024.
[6] Jikang Cheng, Zhiyuan Yan, Ying Zhang, Yuhao Luo, Zhongyuan Wang, and Chen Li. Can leave deepfake data behind in training deepfake detector? arXiv preprint arXiv:2408. 17052, 2024.
Jongwook Choi, Tachoon Kim, Yonghyun Jeong, Scungryul Back, and Jongwon Choi. Exploiting style latent flows for generalizing deepfake video detection. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 1133-1143, June 2024.
[8] Umur Aybars Ciftci, llke Demir, and Lijun Yin. Fakecatcher: Detection of synthetic portrait videos using biological signals. IEEE transactions pattern analysis and machine intelligence, 2020.
[9] Davide Cozzolino, Andreas Rössler, Justus Thics, Matthia Nießner, and Luisa Verdoliva. Id-reveal: ldentity-aware deepfake video detection. In Proceedings of the IEEF/CVF international conference on computer vision, pages 15108-1511 2021.
[10] Xinjie Cui, Yuezun Li, Ao Luo, Jiaran Zhou, and Junyu Dong. Forensics adapter: Adapting clip for generalizable face forgery detection. arXiv preprint arXiv:2411.19715. 2024.
[11] Hao Dang, Feng Liu, Joel Stehouwer. Xiaoming Liu, and Anil K Jain. the detection of digital face manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern recognition, pages 5781-5790, 2020.
[12] Deepfakedetection, contributing tow.tt--ddppaaeeeeettiooo....e html
2021.
http://aa..goolleeeo...oo/o////a Accessed 2021-11-13.
[13] DeepFakes, 2020. www www.github..com giithu...com/epptaaee///scsassss Acces// Accessed 2020-09-02.
[14] B Dolhansky. The dee pfake detection challenge (dfdc) pre view dataset. arXiv preprint ai::::
2019.
[15] Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang and Cristian Canton Ferrer. The deepfake detection challenge (dfde) dataset. arXiv preprint arXiv:2006.07397 2020.
[16] Shichao Dong, Jin Wang, Jiajun Liang, Hanqiang Fan, and Renhe Ji. Explaining deepfake detection by analysing image matching. In European conference on computer vision, pages 18-35. Springer, 2022.
[17] Xiaoyi Dong. Jianmin Bao, Dongdong Chen, Ting Zhang, Weiming Zhang, Nenghai Yu. Dong Chen, Fang Wen, and Baining Guo. Protecting celebrities from deepfake with identity consistency transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9468-9478. 2022.
[18] FaceSwap. 2021. www. www.github... httpa///aw.aaaeeeoooooa//ea/aaaa
2020-09-03.

---

[19] Jianwei Fei, Yunshu Dai, Peipeng Yu, Tianrun Shen, Zhihua Xia, and Jian Weng. Learning second order local anomaly for general face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 20270-20280, 2022.
[20] Chao Feng, Ziyang Chen, and Andrew Owens. Self-supervised video forensies by audio-visual anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10491-10503, 2023.
[21] Pierre Foret, Ariel Kleiner. Hossein Mobahi, and Behnam Neyshabur Sharpness- awware minimization for efficiently improving generalization. arXiv preprini arXiv:2010.01412, 2020.
[22] Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. International conference on machine learning, pages 3247-3258 PMLR, 2020.
[23] Sheldon Fung, Xuequan Lu, Chao Zhang, and Chang-Tsun Li. Deepfakeuel: Deepfake detection via unsupervised contrastive learning. In 2021 international joint conference neural networks (IJCNN), pages 1-8. IEEE, 2021.
[24] Zhihao Gu, Taiping Yao, Yang Chen, Shouhong Ding, and Lizhuang Ma. Hierarchical contrastive inconsistency learning for deepfake video detection. European Conference on Computer Vision, pages 693.613.60 Springer. 2022.
[25] Hui Guo, Shu Hu, Xin Wang, Ming-Ching Chang, and Siwci Lyu. Eyes tell all: Irregular pupil shapes reveal gan-generated faces. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2904-2908. IEEE, 2022.
[26] Xiao Guo, Xiaohong Liu, Zhiyuan Ren, Steven Grosz, Iacopo Masi, and Xiaoming Liu. Hierarchical fine-grained image forgery detection and localization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 3155-3165, 2023.
[27] Alexandros Haliassos, Rodrigo Mira, Stavros Petridis, and Maja Pantic. Leveraging real talking faces via self-supervision for robust forgery detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14950-14962, 2022.
[28] Alexandros Haliassos, Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. Lips don't lie: A generalisable and robust approach to face forgery detection. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 5039-5049, 2021.
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
[30] Hillobar, 2023. httpe://github.. httpa//.ccom/iilbb///eeeeee /Rpee
Accessed 2023.
[31] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arxiv:2006. 106.6655, 2021.
[32] Shu Hu, Yuezun Li, and Siwei Lyu. Exposing gan can-generated faces using inconsistent corneal specular highlights. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2500-2504. IEFF, 2021.
[33] Baojin Huang. Zhongyuan Wang, Jifan Yang. Jiaxin Ai. Qin Zou, Qian Wang, and Dengpan Ye. Implicit identity driven deepfake face swapping detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4490-499, 2023.
[34] Ziqi Huang, Kelvin CK Chan, Yuming Jiang, and Ziwei Liu. Collaborative diffusion for multi-moda face generation and editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6080-6090, 2023.
[35] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang. Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: based real image editing with diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007-6017, 2023.
[36] Kihong Kim, Yunho Kim, Seokju Cho, Junyoung Seo, Jisu Nam, Kychul Lee, Seungryong Kim, and KwangHee Lee. Diffflac:: Diffusion-bass face swapping with facial guidance. arXiv preprint arXiv:2212.13344, 2022.
12

---

[37] Nicolas Larue, Ngoc-Son Vu, Vitomir Struc. Peter Peer, and Vassilis Christophides. Seeable: Soft discrepancies and bounded contrastive learning for exposing deepfakes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21011-21021, 2023.
[38] Binh M Le and Simon Woo. Quality-agnostic deepfake detection with intra-model collaborative learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22389, 2023.
[39] Hanzhe Li, Jiaran Zhou, Bin Li, Junyu Dong, and Yuezun Li. Freqblender: Enhancing deepfake detection by blending frequency knowledge. arXiv preprint arXiv:2404. 13872, 2024.
[40] Lingzhi Li, Jianmin Bao, Ting Zhang. Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5001-5010, 2020.
[41] Y Li. Exposing deepfake videos by detecting face warping artif acts. arXiv preprint arXiv: 1.00066,, 2018.
[42] Yuezun Li, Ming- Ching Chang, and Siwei Lyu. In ictu oculi: Exposing ai created fake videos by detecting eye blinking. In 2018 IEEE International workshop on information forensics and security (WIFS), pages 1-7. Icce, 2018
[43] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 3207-3216, 2020.
[44] Jiawei Liang, Siyuan Liang, Aishan Liu, Xiaojun Jia, Junhao Kuang, and Xiaochun Cao. Poisoned forgery face: Towards backdoor attacks on face forgery detection. arXiv preprint 2rXi: 1143, 2024.
[45] Tsung- Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 21 17-2125, 2017.
[46] Yuzhen Lin, Wentang Song. Bin Li, Yuezun Li, Jiangqun Ni, Han Chen, and Qiushi Li. Fake it till you make it: Curricular dynamic forgery augmentations towards general deepfake detection. In European Conference on Computer Vision, pages 104-122. Springer, 2024.
[47] Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue. Weiming Zhang, and Nenghai Yu. Spatial-phase shallow learning: rethinking face forgery detection in frequency domain. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 772-781, 2021.
[48] Anwei Luo, Chenqi Kong, Jiwu Huang, Yongjian Hu, Xiangui Kang, and Alex C. Kot. Beyond the prior forgery knowledge: Mining critical clues for general face forgery detection. IEEE Transactions on Information Forensics and Security, 19: 168-1182, 2024.
[49] Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. Generalizing face forgery detection with high- frequeney fcatures. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16317-16326, 2021.
[50] Midjourney, 2022. htttps://ww. midjourney.....mm com, Accessed 2022.
[51] Dat Nguyen, Nesryne Mejri, Inder Pal Singh, Polina Kuleshova, Marcella Astrid, Anis Kacem, Enjie Ghorbel, and Djamila Aouada. Laa-net: Localized artifact attention network for quality-agnosii and generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 77395-17405, 2024.
[52] Huy Nguyen, Junichi Yamagishi. and Isao Echizen. Capsule-forenstee Using capsule networks to detect forged images and videos. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP). pages 2307-230 2311. IEEE, 2019.
[53] Yunsheng Ni, Depu Meng, Changqian Yu. Chengbin Quan. Dongchun Ren, and Youjian Zhao. Core: Consistent representation learning for face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12-21, 2022.
[54] Trevine Oorloff, Surya Koppisetti. Nicolò Bonettini, Divyaraj Solanki, Ben Colman, Yaser Yacoob, Ali Shahriyari, and Gaurav Bharaj. Avif: Audio-visual feature fusion for video deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27102- 27112, 2024.
13

---

[55] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:230.001952 2023.
[56] Yuyang Qian, Guojun Yin. Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by mining frequency-aware clues. European conference computer vision, pages 86-103. Springer, 2020.
[57] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[58] Felix Rosberg, Eren Erdal Aksoy, Fernando Alonso-Femaandes and Cristofer Englund. Facedancer: Pose-and occlusion-aware high fidelity face swapping. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3454 3463, 2023.
[59] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics*#:: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pages 1-11, 2019.
[60] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pages 22500- 22510, 2023.
[61] Somdev Sangwan, 2023.
wttp:thhb..co//mmmd////e////ae/e Accessed 2023.
[62] Rui Shao, Tianxing Wu, and Ziwei Liu. Detecting and recovering sequential deepfake manipulation. In European Conference on Computer Vision, pages 712-728. Springer, 2022.
[63] Kaede Shiohara and Toshihiko Yamasaki. Detecting deepfakes with selnddlended images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18720- 18729, 2022.
[64] Yuyang Sun, Huy H Nguyen, Chun-Shinn Shinnn Lu, Zhi Zhang, Lu Sun, and Isao Echizen. Generalized deepfakes detection with reconstrucedd-blent images and multi-scale feature reconstruction network. arXiv preprint 2rXiv:2312.08022 2023.
[65] Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wci. Rethinking the up- sampling operations in enn-based generative network for generalizable deepfake detection. Pn Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28130-28133 2024.
[66] Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference machine learning, pages 6105-61 61. PMLR, 2019.
[67] Justus Thies, Michael Zollhöfer, and Matthias Nießner. Deferred neural rendering: Image synthesis using neural textures. Journal of ACM Transactions on Graphics, 38(4): 12, 2019.
[68] Justus Thics, Michael Zollhofor, Marc Stamminger, Christian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2016.
[69] Patrick Tinsley, Adam Czajka, and Patrick Flynn. This face does not exist... but it might be yours! identity leakage in generative models. In Proceedings the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1320-1328, 2021.
[70] Sheng- Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8695-8777 8704, 2020.
[71] Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. Towards real-world blind face restoration with generative facial prior. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[72] Xueyu Wang, Jiajun Huang, Siqi Ma, Surya Nepal, and Chang Xu. Deepfake disrupter: The detector of deepfake is my friend. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 14920-14929, 2022.
[73] Zhendong Wang. Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion-genennat generated image detection. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 224455,2255,2222 2023.
14

---

[74] Zhendong Wang. Jianmin Bao, Wengang Zhou, Weilun Wang, and Houqjang Li. Altfreezing for more general video face forgery detection. In Proceedings of the IEEE/CVE conference on computer vision and puttern recognition, pages 41 4129-4138, 2023.
[75] Chen Henry Wu and Fernando De la Torre A latent space of stochastic diffusion models for zero-shot image editing and guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7378-7387, 2023.
[76] Jiahui Wu, Yu Zhu. Xiaoben Jiang. Yatong Liu, and Jiajun Lin. Local attention and long- tig-diitance interaction of rppg for deepfake detection. The Visual Computer, 40(2): (002):1083-1000 1094, 2024.
[77] Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Better aligning text-to-image models with human preference. arXiv preprint arXiv:2303. 14420, 1(3), 2023.
[78] Yuting Xu, Jian Liang, Gengyun Jia, Ziming Yang. Yanhao Zhang, and Ran He. Tall: Thumbnail layout for deepfake video detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22658 22668, 2023.
[79] Zhiyuan Yan, Yuhao Luo, Siwei Lyu, Qingshan Liu, and Baoyuan Wu. Transcending forgery specificity with latent space augmentation for generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8984-8994, 2024.
[80] Zhiyuan Yan, Taiping Yao, Shen Chen, Yandan Zhao, Xinghe Fu, Junwei Zhu, Donghao Luo. Li Yuan, Chengjie Wang, Shouhong Ding, et al. DI40: Toward next-generation deepfake detection. arXiv preprint arXiv:2406. 13495. 2024.
[81] Zhiyuan Yan, Yong Zhang, Yanbo Fan, and Baoyuan Wu. Ucf: Uncovering common features for generalizable deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
[82] Zhiyuan Yan, Yong Zhang, Xinhang Yuan, Siwei Lyu, and Baoyuan Wu. Deepfakebench: comprehensive benchmark of deepfake detection. arXiv preprint arXiv:2307 00426, 2023.
[83] Zhiyuan Yan, Yandan Zhao, Shen Chen, Mingyi Guo, Xinghe Fu, Taiping Yao, Shouhong Ding, and Li Yuan. Generalizing deepfake video detection with plug-and-play. Video-level blending and spatiotemporal adapter tuning. arXiv preprint ar0iv:2408. I 17065, 2024.
[84] Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training- free energy- guided conditional diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 23174 23174223184 23184, 2023.
[85] Ning Yu, Vladislav Skripniuk. Dingfan Chen, Larry Davis, and Mario Fritz. Responsible disclosure of generative models using scalable fingerprinting. arXiv preprint arXiv:2012.08726, 2020.
[86] Yike Yuan, Xinghe Fu, Gaoang Wang. Qiming Li, and Xi Li. soreey--oomm---sueeeee deepfake detection with non-negative constraint. IEEE Signal Processing Letters, 29:2512-2516. 2022.
[87] Cong Zhang, Honggang Qi, Yuezun Li, and Siwei Lyu. Contrastive multi-faceforensics:: An end-to-end bi bi-grained contrastive learning approach for multi-face forgery detection. arXiv preprint arXiv:2308. 01520, 2023.
[88] Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. Multi- attentional deepfake detection. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 2185-2194, 2021.
[89] Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, and Wei Xia. Learning self-consistency for deepfake detection. In Proceedings of the IEEE/CVF international conference computer vision, pages 15023,1503,, 2021.
[90] Yinglin Zheng. Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. Exploring temporal coherence for more general video face forgery detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15044-15054 2021.
[91] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 2921-2929, 2016.
[92] Wanyi Zhuang, Qi Chu. Zhentao Tan, Qiankun Liu, Haojie Yuan, Changtao Miao, Zixiang Luo, and Nenghai Yu. Uia-vit: Unsupervised inconsistency aware method based on vision transformer for face forgery detection. In European conference computer vision, pages 391-407 Springer, 2022.
15

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the central claims articulated in both the abstract and introduction rigorously align with the paper's technical contributions and methodological scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The ahstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can he expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, the limitations of our methodology systematically analyzed in Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper limitations, but those are not discussed in the paper.
authors are encouraged create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested a few datasets with few runs. In general, empirical results often depend implicit assumptions, which should he articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to- text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection,
outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Although our paper focuses on deepfake detection rather than theoretical analysis, all architectural propositions receive rigorous validation through quantitative experiments and visual demonstrations.
Guidelines:
The answer NA means that the paper does not include theoretical results.
16

---

All the theorems, formulas, and proofs in the paper should be numbered and cross cross-reference..
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can cither appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides sufficient information in multiple key aspects support experimental reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, contribution novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g., in the case of a large language model), relcasing model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility. which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g.. a large language model), then there should either be a way to access this model for reproducing the results a way to reproduce the model (e.g.. with open- dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed source models, it may be that access to the model is limited in some way (e.g.. registered users), hut it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: The code is currently not open source.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines for more details.
guides/Coode https///ooooouuueeiiooooooooo
While we encourage the release of code and data, we understand that this might not be possible,
so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
17

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines httpp////niaa.cppuaabbpe for more details.
Suades//odeSuumiissiooPPe
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g,, data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, the paper provides comprehensive details regarding the training and testing setup to ensure reproducibility and clarity.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error hars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some paramcter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asy asymmetric distributions, the authors should be careful not to show in tables or figures rates). If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
symmetric bars that would yield results that are of range (e.g. negative
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Yes, the paper explicitly specifies the type of computational hardware used (RTX3090). However, resource requirements are minimal, detailed information about execution time or memory consumption is not provided, as these aspects are unlikely to affect reproducibility for experiments.
Guidelines:
18

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Fthies battps:/nneurips.ccmm http:///wwri....cceebaabbe/eecae
Answer: [Yes]
Justification: We have carcfully read the NeurIPS Code of Ethics, but this paper does not involve any issues that violate ethical standards.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., if there is special considcration due laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss hoth potential positive societal impacts and negative societal impacts the work performed?
Answer: [Yes]
Justification: Yes, the impacts of our methodology are systematically analyzed in Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- ment technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimatc point out that improvement in the quality of generative models could used to generate deepfakes for disinformation. On the other hand, it not needed point out a generic algorithm optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional unintentional) misuse the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms monitor- ing misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled of the model, for example by requiring that adhere usage guidelines restrictions to access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make hest faith effort.
12. Licenses for existing assets
Question: Are creators or original owners assets (e.g., code, data, models), used the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected"?
Answer: [Yes]
Justification:
Guidelines:
The answer means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it changed) should be provided.
If this information not available online, the authors encouraged reach out the asset's creators.
13. New assets
Question: Are new assets introduced paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail as possible should included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
20

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country which research is conducted, approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- blc), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe usage of LLMs it important, original, or non non-standard component
core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the method development in this research does not involve LLMs any important, original, non-standard components.
Please refer LLM policy http::::/eaurrppe...c///otrrrere should should not be described.
for what
21

---

A
Appendix
This supplementary material provides:
Sec. We discussed the classification and basic process of forgery techniques.
Sec. We reviewed the application of data augmentation methods in forgery detection and discussed more details about FIA- IIA-
Scc.
We introduced the construction process of FPN.
Sec.
We provided a detailed introduction to AFFS was provided, along with visual evidence.
Sec.
We provide visualization example of which boundary pixels were discarded by RCR.
Sec.
We provided test results on more forgery techniques.
Sec. H
We provide experimental results combining FIA-USA with other state-of-the-art detection
methods
Sec. [
We discussed the limitations of this paper.
Sec.
We discussed the impacts of this paper.
Sec.
K
We provided more visual examples.
B
Face forgery techniques
In this section, we discuss the classification of facial forgery technology and the basic process of forgery technology, and which nodes FIA and USA will be introduced.
B.1 Classfication of facial forgery technology
According different processing procedures and technical principles, we first divide forgery techniques into two categories:
1). Full Face Synthesis: Generate the entire image/video directly through the image video generation model, without involving facial cropping and stitching in the basic process. Therefore, the artifacts involved in such forgery techniques mainly include USA and small amount of FIA, where USA is the dependency between adjacent pixels the forgery area due to the performance the generator, and FIA the inconsistency in the generator's 's ability to generate face areas with dense details and background areas with sparse details.
2). Face Swapping. Crop the original face, generate fake regions through a generator (optional), and then stitch it with the target face. Therefore, this forgery technique can be divided into two situations: It includes FIA and USA: FIA is mainly caused by cropping and stitching, well as generator performance, while USA mainly determined by generator performance. b. It only includes FIA: FIA is mainly caused by ropping and stitching. Furthermore, based on the forged area, we can divide face swapping into two parts: Macro-editing and Micro-editing. Macro-editing can be further classified into two subtypes: Editing based on 4 facial keypoints and Editing based on 81 keypoints. Micro-editing allows for precise adjustments to each facial feature. Figure 5 provides detailed overview of our classification concept.
B.2 Basic Process
In Figure 6 we depict the basic process of cach type of forgery technique. We can see from this that the main difference between full face synthesis and face swapping lics in whether they include face cropping and stitching operations, which leads to the emergence of a large number of FIA, and whether the background is generated by the generator. The main difference between macro-editing and micro-editing in face swapping is whether the forged area is the entire face can finely fabricate facial features. Further, macro editing based on 81 key points more detailed, and the edges of the forged area are more similar to edges a real face, while 4 key points result in the forged area being a square. This discovery is mainly due to a detailed observation of the widely used forgery technique ROPE 30 and roop [61], which provides two types of face cropping techniques.
B.3 At what stage will FIA and USA be introduced
In Figure 7 we demonstrate which operations lead to the emergence of FIA and USA in the basic process. The observation of phenomenon led to the refinement of our FLA-USA design.
22

---

Facial forgery technology
Not Involving face cropping and stitching
Involving face cropping and stitching
Full Face Synthesis
Face Swapping
Large area of forgery
Small foryed area
macro-editing
micro micro-editing
Crop based 81 key points
Crop based on 4 key points
Editing based 81 facial keypoints
Editing based on facial keypoints
Figure 5: We categorize forgery techniques into two main types: Full face synthesis and Face swapping. Face swapping is further divided into macro-editing (based on 81 keypoints) and micro-editing.
Details of FIA-USA
C.1 Blend-based data augmentation
In Deepfake detection, a successful approach is to manually construct negative samples for training, where data augmentation in the RGB domain is based on Blend, which can he represented as follows:
Ir M @ 1: + (1 M) @ Is,
(8)
Among them, J represents the target face, Ta represents the source face, and 7F represents the negative sample. The earliest design 400 was to use facial similarity to find the most suitable I, for I.. Later, SBI [63] proposed self-blend to improve the statistical consistency of Ir. There also works [64] that separate the background face of I,, add noise to background. and reconstruct It, using the reconstructed I, for blend. However, to our knowledge, there has been no work so far that considers the introduction of both global and local artifacts, let alone the introduction of FIA at both the full face and regional levels while introducing USA.
C.2 Other types of data augmentation
recent years, many other data augmentation methods have been proposed, such as frequency domain [3 and latent space [79. Many studies suggest 39
[99 that compared to data augmentation in other domains, data 64 exhibits weak robustness and limited effectiveness. However, we have demonstrated through extensive experiments that proposed data augmentation in the RGB domain not weaker or even better than that other domains. is worth mentioning that our proposed data augmentation belongs to the image level, and we surprisingly found that it is superior to the most advanced data augmentation methods at the video level (74).
augmentation in the RGB domain 40
63
C.3
The detailed process of generating masks in FIA-USA
Here have detailed processing of generating masks FIA-USA shown Figure 8 Firstly, we divide mask generation into two types: 1) Macro editing masks and 2) Micro editing masks. Macro editing mask: We follow the mask generation process in SBI [63], which includes the following steps: Calculate the convex hull of 81 facial keypoints to obtain the mask of facial contour b. Erosion expansion of
23

---

Full Face Synthesis
Face swapping
Real imagee /videus, conditional inputs (such as prompis, audio, sketches, elc).
Impuls:
Impults Target face and source face, conditional inputs (such prompls, audio, skelches, etc).
Tools: Image / video generation
Tools:
Facial keypoint localization model, Image generation models
(Optional).
The model dirertly generales images/iderus based condittions Inputs or modiíles images/videos based on conditional impuls.
Facial keypoint detection
Optional
Facial cropping
Outgut: Fake images /videos
The model modifles the face based on the tarpet face
Facial sitching
Output: Fake images Ivideos
(a) Basic processes of, fffiil face synthesis and face swapping.
Face swapping
Facial keypoint detection
Based on 4 key points
Based key points
Facial cropping
macro- editing
micro- editing
Facial stitching
(b) Basic processes of macro-editing (4 vs keypoints) and micro-editing
Figure 6: We illustrates the basic processes of forgery techniques, full face synthesis and face swapping differ in face cropping and stitching, while macro-editing (4 vs 8/ keypoints) and micro- editing vary in forgery scope and detail.
the mask; Besides, also considered the of calculating hull based facial keypoints, which originated the observation of two methods for extracting raw faces provided by open-source deepfake project that is truly available [61] (0).
Micro-editing masks: The main method is to simulate the forgery process of manipulating facial details randomly by randomly combining facial features.
24

---

Full Face Synthesis
Face swapping
Facial keypoint detection
The model directly generates imagen/videos besed on conditional inputs modifies images/Videos besed on conditional inputs.
Optional
Facial cropping
model modifies the source face besed the target face
Facial strtching
Both have been Introduced.
Both FLA and USA have been introduced.
Only FIA has boen intruduced.
Figure 7: The operations that led to the occurrence of FIA and USA in the basic process.
Determine Mank
editing
Pacal & or leygann deledion
Pacil losporms deseton
hall facell
fnced fistrme
Ersino cacnos hall
Ieaed
points
Figure 8: The generation of macro and micro-editing masks in FIA-USA, using convex hull and erosion / expanssinn for macro masks and random feature combination for micro masks.
Image / video generation models
Caustng blury forgory aress
The Eorged clear
Need
No need
Factal super- resolution
Farial super
Figure 9: We divide the generation models into two categories based on whether facial super- resolution models needed for post-processing.
C.4 Why use Autoencoder and Super-Resolution models to reconstruct images
As shown in Figure 9 we divide the processing flow of the generative model into two categories: One category generates images of Forged regions that are as clear and free of blur as the original image (using Diffusion models and GANs the main methods, and upsampling the latent code through AE to generate images). And another
25

---

FIA-USA
Facial forgery tochnology
Using edited masks for blending
No aesd pereertu mhode
Blend with original images
Fall Face Syuthesis
Face Swapping
genemstion
Blend with facial cngetttcelution imagis
Using Macro edited masks blonding
Using masks for Meending
zhe onsorreacte imagee
Blond with factal super-cssolttion
images
Crasing Morry forgery rnee
The Torged araa deer
macro editing
macro-editing
micro-editing
forgull une ts clear
Cassing blury forgery area
locad key
key polnts
super
Facell mesletion
Editing besed Eacaa poyptizs
basod keypainnts
Figure 10: We demonstrate through graphical representation that our proposed FIA-USA covers all possible scenarios that may lead to the occurrence of FIA and USA.
type (especially older voice driven lip shape modification techniques) will result in blurred lampered areas, requiring additional facial super-resolution models process the blurred Therefore, utoencoder (AE) to reconstruct images or facial Super-Resolution (SR) process faces, order to approximate the USA introduced by the generator in the forged area.
C.5 FIA-USA covers all possible situations that may occur in FIA and USA
As shown in the Figure 10 our proposed FIA-USA framework covers every possible situation that may occur in FIA and USA, which also the most intuitive explanation why method effective.
1x1
Layer
AFFS
Feature
Layer
AFES
1x1
Feature 4
RCR
1x1
I Layer
AFFS
Feature 3
1x1
3x3
L
AFFS
Fealure
Feature 2
Layer
1x1 represents 1 x convolution
3x3 represents 3 x convolution
2 X represents upsampling using nearest neighbor interpolation method
Image
Figure 11: We illustrated the construction process of FPN.
26

---

D
Details FPN
As shown in the Figure [T] FPN follows the framework proposed 45 using the output features of the 2nd, 3rd, 4th, and 5th layers of the backhone to construct an FPN, In our experimental setup, the number of feature map channels is 128 and 196, corresponding to EfficientNet [66]. and Resnet [29 respectively.
E
Details of AFFS
Here we provide a detailed description of the process of AFFS and a visual explanation its effectiveness.
E.1
Why is AFFS proposed
Let's consider neural network @ and FPN P. well the process of performing RCR on the FPN's maximum resolution layer. The output of is set of features (di, d2, d3, da P takes this set of features as input to construct set of feature maps with same dimensions and different resolutions, while RCR performs feature differentiation learning on the feature map with the highest resolution. One intuition is that there is significant amount of redundancy the output features of the o, and every dimension of the features d., [1, 2, 3, 4] suitable for constructing FPN for RCR. Therefore, we propose AFFS for feature dimensionality reduction each feature d, before constructing FPN. Morcover, this feature dimensionality reduction method effectively exploits the potential advantages of our proposed FIA-USA and lays the foundation for the effective execution of RCR.
Aburdn
bandayy
pinds
Figure 12: We provide visual example to illustrate which pixels will be discarded.
E.2 AFFS
First, let's consider multiple positive and negative sample pairs generated by FIA-USA, along with their corresponding masks [R.,, M..
usual method is to directly construct an FPN using a set of features Ji from a neural network 6 as input, where represents the i-th layer of the ø 2) Next, let's discuss the initialization process of AFFS. For the output of each layer of the neural network, we hope that the difference between the real image and the fake image after normalization is as consistent as possible with the mask M used represent forged areas. However, we have noticed that not every channel meets this requirement, so we would like to reserve channels that meet the requirements for cach layer. Therefore, for fi E R W,, we calculate the following loss for each channel on a set of FIA-USA generated samples [Rm, Fn, M, ) N obtain those excellent dimensions.
1 ( (JE(F)) M N
CAFES
Ff represents normalization and scaling operations, f E RW: l E [0, C.). represents the feature map of the k- th dimension of the fc. During the training process, we only select the channels reserved for ft during the AFFS initialization process. After AFFS processing, the original features Sa RR - will become f E R H.
E.3 Visual Explanation of AFFS
Here we provide set of visual examples of AFFS in Figure [3 The experiment was conducted on the 2-th layer of EfficientNedB4. Unlike the actual situation, in order enhance readability, only calculated Larrs for single epoch single pair of samples. And based on this, sort the 32 dimensions of the features in 2-th layer. Here, we present the dimensions ranked in the top 14 and bottom 4. In the actual process, we iterate for 5 epochs on all samples on the training set to initialize AFFS. In Table 7 we present the changes in feature dimensions of different backbone features before and after performing AFFS.
27

---

The similarity decreases
The top 14
Mask
Real
Fake
No. 1
No. 26 hars: 0.241
No. 18
No. 9 Larps 0.264
Lpps: 0.232
Apppy: 0.262
No.
No.
No.
No. 5
No.31
No.
No.
Lares: 0.232
Lanes: 0.272
Larpy: 0.283
Lazpy: 0.286
Lapey: 0.289
Lanes: 0.290
Larry: 0.294
The bottom
The similarity increases
Mask
Real
Fake
No.
No.
No. 4 LaFFst 0.377
No. 16 LaFes: 0.378
LAFFs:0.344 0.344
LAFFs: 0.373
Figure 13: The result of calculating LAFFS for the dimensions of the 2-th layer features of EfficientNetB4 on single sample and sorting them by similarity.
Table 7: Feature dimension parameters of AFFS.
Model
Input Dimension
Output Dimension
Res50
(256.51
(196,38,,768 768,,786,,11111
Res 101
(256,512.1024.2048)
199,,88,7688, 76,,,,6666,1166)
Effbl
(24.40,
(16.24,40,112)
Effb4
(32.56,160,448)
(24,32,56,160 160)
Table 8: We report the Video-level AUC. The combination of our data augmentation method with other SOTA detection algorithms. represents the method used in the original paper.
Method
Training set
CDFv2
DFDCP
Avg.
LAA-Net
SBI + EPPN i
0.840
0.741
0.791
LAA-Net
FIA-USA EFPN
0.875
0.782
0.829
Ours
FIA-USA FPN (AFES)
0,901
0.861
0.881
Ours
FIA-USA FPN (AFFS) +RCR
0.941
0.866
0.904
28

---

F Details of RCR
In the LAA Net [SI]. a very interesting hypothesis is the pixels on the fake boundary contain both the features of the fake and the features of the real area, in designed RCR, such pixels will be discarded. Figure provides visual example.
G Testing on more forgery techniques
In addition to several widely used forgery detection datasets mentioned in the paper, also conducted experiments on other forgery techniques based the DF40 dataset [80] where real samples were obtained from FF++ and fake samples were constructed using corresoonding forgery techniques based on FF++ settings. As shown in the Table 9a] our proposed forgery detection framework has achieved excellent detection results on the vast majority of forgery methods. Mcanwhile, Table (b] shows the comparison hetween our method and other statc-of-the-arr methods various forgery techniques in DIFF.
Table 9: Test results on more forgery techniques.
(a) We tested all forgery techniques provided on the DF40 dataset and reported Frame-Level AUC. All results with AUC greater than 80 are highlighted in bold, while those with AUC less than 80 but greater than 70 underlined.
Type AUC
mohilesmap 0.97
Midloumey 0,97
faceswap 01,99
styleclip 0.82
DIT
lia 0.91
ddim 0.97
monet 0.84
RDDM 0.70
StyleGANs
0.66
0,93
Type
pixari 0.93
whichfaccisseal 0.12
decpfacelab 0.77
Styk<GANX 0.41
heygen 0.58
lacedanner 0.83
MRAA 0.85
pirender 0.81
VOGAN 0.85
AUC
0.95
Type
StyleGAN2 0.93
facevid2vid 0,83
simswap 0.91
inswap 0.87
one sbot free 0,85
Way 0.76.
cls 0.88
Stargan22 0.48
tpsm 0,83
sd2.1 0.97
ACC
Type
blendface 0.94
hyperreenact 0,80
CollabDitf 0.95
fagan 0.86
danet 0,80
sadtallier 0.74
form 0.85
AUC
0.92
0.72
(b) Comparison with universal deepfake detection methods using the Levee-Level AUC on the DiFF dataset. The notation i indicates models that are designed for deepfake detection, while $ signifies models intended for general generated detection. All experiments were trained on the c23 version of FF++. The best result bolded.
Test Subset
Method
Cycle CoDiff'
Imagic
DiFace
DCFace
Dream
SDXL R FD
LoRA Midj
FD T SDXL
HPS
F-Neet [56
36.14
35.00
32.56
25.61
53.29
55.45
65.04
40,90
45.00
61.64 68.96
EfficientNeel
66
56.51
38.38
48.50
64.45
89.13
71.64
65.04
59.93
69.67
64.94 74.63
CNN Aug
70
50.31
46.84
75.95
43.10
80.69
58.75
60.10
43.65
47.90 61.95 60.56
SBI
63
82.32
64.70
49.84
73.64
89.92
75.66
77.73
87.67
89.18
79.56
90.55
80.37 83.30
Ours
84.76
72.61
52.92
76.86
93.63
83,43
80.54
92.68
93.48
82.57
94.85
82.15
88.40
H Combining with SoTA detection method
In this section, we aim to validate the effectiveness of our FIA-USA data augmentation by integrating it with sfatc-of-the-ar image-level detection methods. While our comprehensive evaluation currently focuses on LAA- Net (51) (CVPR 24) due to limited availability of open-souree reveals significant performance improvements. Notably, the original LAA-Net implementation was trained on raw version FF++ data, whereas our experiments utilize the c23 version. Compared with the baseline implementation, data augmentation strategy has significantly improved the performance of the original detector. To address requests detailed comparisons with L.A.-Net, our table shows two key findings: First, our method achieves a 7.5% performance advantage over LAA-Net when using identical data augmentation conditions. Second, even without employing RCR (Robust Context Refinement), our approach substantially outperforms LAA-Net's EFPN component, demonstrating the inherent strength of our core methodology.
Inage-level detection implementations, Table
I Limitations
The proposed detection framework primarily focuses on image-level data augmentation and matching detection architectures. While categorize image-level artifacts into FIA and USA, and have achieved satisfactory video-level level detection performance, we acknowledge that video-level artifacts present more complex challenges including temporal inconsistencies and audio-visual asynchrony which fall outside the current research scope but represent our intended future research direction. Second, although recent approaches increasingly incorporate large models' zero-shot capabilities for deepfake
29

---

detection, existing work has not explored the integration of data augmentation methods with such models due practical constraints including variations in training methodologies and model scales. This does not our method cannot applied to large models, currently we are still unfamiliar with large models and have not fully explored this part.
Regarding experimental validation, we mainly compared and analyzed our proposed method with previous data augmentation methods, which is consistent with the main contribution of our paper. Of course, also compared state-oftthe-at detection methods that not data augmentation based. Following established practices data augmentation research, we conducted comprehensive evaluations across four widely adopted network architectures to ensure fair comparison. However, we note that detection performance may vary across different network frameworks depending on their baseline performance and inherent compatibility with deepfake detection tasks. We cannot guarantee that all model frameworks will achieve excellent detection results after using our method.
Finally, while our method demonstrates effectiveness across substantial majority of forgery methods (yalidated on over 58 different techniques), we acknowledge relatively weaker performance in specific edge cases. Nevertheless, the framework maintains robust detection capability for most common forgery approaches, achieving satisfactory overall performance that meets research objectives.
J Broader impacts
The proposed framework offers positive societal impacts by enhancing detection of sophisticated face deepfakes, thereby mitigating disinformation campaigns, fake news, and fraudulent content that erode public trust in digital media. Additiionlly, it safeguards individual privacy by identifying forged facial manipulations identity theft, unauthorized face swapping, and other privacy violations. The method could further support content authenticity initiatives through integration into social media platforms or news verification systems prioritize legitimate visual content. However, potential negative impacts include the risk of adversaries reverse- wwerse-enginnering the detection framework refine deepfake generation methods, potentially escalating the adversarial "arms race" between forgery creators and detectors. Furthermore, false positives in detection could lead to unintended censorship, where legitimate content is erroncously flagged as fake.
enable
K
More visual examples
From Figure 14]to Figure 16 we demonstrate the fake samples that FIA-USA can generate through different reconstruction methods. including autoencoderr (AE) and facial super-resolution (SR), and multiple types of masks based on three real faces. Figure shows the negative samples randomly generated by FIA-USA during the real training process. Figures 18 to list the images in the DF40 dataset 800 that were misclassified by method.
30

---

Mask type
Mask type 2
Mask type
Real
AE
Mask
Fake
Real
SR
Mask
Fake
Roal
Real
Mask
Fake
Figure 14: More examples of FIA-USA.
31

---

Mask type 4
Mask type 5
Mask type
Real
AE
Mask
Fake
Real
SR
Mask
Fake
Real
Real
Mask
Fake
Figure 15: More examples of FIA-USA.
32

---

Mask type 7
Mask type 8
Mask type
Real
AE
Mask
Fake
Real
SR
Mask
Fake
Real
Real
Mask
Fake
Figure 16: More examples of FIA-USA.
33

---

Real
Recons
tructed
Mask
Fake
Real
Reco
nstruc
ted
Mask
Fake
Real
Reco
nstruc
ted
Mask
Fake
Figure 17: More examples of FIA-USA.
34

---

Roal
Blendface
CollabDiff
Donet
Despfacelb
DIT
ele
Figure 18: Images misclassified by our method in the DF40 dataset.
35

---

Facedancer
Faceswap
Facevid2vid
fomm
fsgan
heygen
hyperreencct
inswap
Figure 19: Images misclassified by our method in the DF40 dataset.
36

---

mcnet
MidJourney
mobileswap
MRAA
shot free
pirender
pixart
RDDM
Figure 20: Images misclassified by our method in the DF40 dataset.
37

---

Sd
i
simswap
starganv2
styleclip
StyleGAN2
N3yleGAN3
SLyleGANXL
Figure 21: Images misclassified by our method in the DF40 dataset.
38

---

uniface
VQGAN
Wav2lip
whichfaceisreal
e4s
lia
tpsm
RERECTO
Sadtalker
Figure 22: Images misclassified by our method in the DF40 dataset.
39


Reviews:
[{"review_id": "mON0CnzvUN", "reviewer": "Reviewer_cBZ3", "summary": "This paper tackles the growing problem of deepfakes by suggesting a more general way to detect them, rather than trying to identify every specific forgery technique. The authors noticed that most deepfakes, regardless of how they’re created, tend to have two main types of flaws: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). \n\nThe author generates \"pseudo-fake\" training images that deliberately include only these two types of artifacts. The idea is that by training a detector on these fundamental flaws, it becomes much better at spotting real deepfakes it hasn't seen before. They also added a couple of supporting techniques: Automatic Forgery-aware Feature Selection (AFFS) helps the detector zero in on the most relevant forgery clues, and Region-aware Contrastive Regularization (RCR) helps it distinguish between genuine and manipulated areas. Essentially, they're showing that a focused approach on universal artifacts can lead to a more robust and adaptable deepfake detection system.", "questions": "In the limitations, the authors acknowledge \"relatively weaker performance in specific edge cases\" among the 58+ forgery methods tested. Could they provide more specific examples of which forgery methods constitute these \"edge cases\"?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1) This paper seems pretty solid in terms of quality. The authors put in the work to systematically analyze existing deepfake algorithms to pinpoint common artifacts, which is a smart move. They propose a novel pseudo-fake creation framework that's designed specifically to introduce these general artifacts, avoiding less common ones, which sounds really intentional and well-thought-out.\n2) The paper definitely brings some novel ideas to the table. While data augmentation for deepfake detection isn't new, their specific approach of categorizing artifacts into FIA and USA and then explicitly designing a pseudo-fake generation framework (FIA-USA) to introduce both types is novel. They highlight that existing methods typically focus on isolated artifact types or single-scale modifications, which limits their generalization. Their Multi-Type Mask Synthesis for blending and Multi-Modal Reconstruction for USA, leveraging autoencoders and super-resolution, seem quite original in their combination and purpose.\n\nWeaknesses:\n1) While the FIA-USA framework and its components are innovative, the underlying concepts of blending and reconstruction for data augmentation have been explored before in deepfake detection.\n2) While the experiments are extensive, they mention that some results are \"cited from\" or \"our reproduction results\" for other methods.", "was_revised": true, "final_justification": "The authors provided detailed answers to my questions, and I have decided to increase the paper's score.", "rebuttal": "Dear Reviewer cBZ3:\n\nOn behalf of all authors, we sincerely appreciate the significant time and effort you dedicated to reviewing our manuscript.  Your thorough, thoughtful, and insightful feedback is invaluable to our work, and we are deeply grateful for your recognition of our **rigorous experiments and innovative methodology**. Below, we address your questions point by point:\n\n> **Q1.** While the FIA-USA framework and its components are innovative, the underlying concepts of blending and reconstruction for data augmentation have been explored before in deepfake detection.\n\n**R1.** First, regarding the innovation of our data augmentation approach: While data augmentation has long been proposed in forgery detection research and **remains a vibrant research frontier**, existing methods predominantly focus on isolated artifact types and single blend regions. \n\n**As comprehensively detailed in Table 1 of the paper**, our method provides substantially broader coverage across both artifact diversity and blend region complexity. This methodological advancement is rigorously grounded in our systematic analysis of prevalent forgery techniques (Appendix B), where we categorize forgery artifacts into Face Inconsistency Artifacts (FIA) and  Up-Sampling Artifacts (USA). \n\nThis taxonomy revitalizes data augmentation strategies and extends their applicability to previously undetectable forgery methods—particularly generative forgeries. Extensive experimental validation confirms our approach's efficacy. \n\n**Table 1: Comparison of our framework and state-of-the-art(SOTA) methods using pseudo-deepfake synthesis. Our method differs from previous methods in terms of the level at which alterations are applied (local vs global), the introduced artifacts, and the level at which facial inconsistencies are applied.**\n\n\n| | global  | local | face-level |region-level | Up-Sampling |\n| :--- | :--: | :--: | :--: | :--: | :---: |\n|**Face Xray, PCL, OST**   |$\\checkmark$  |  | $\\checkmark$  | |  |\n| **SLADD** | $\\checkmark$  |  | | $\\checkmark$  |  |\n| **SBI** | $\\checkmark$  |  | $\\checkmark$ |  | |\n| **SeeABLE** | | $\\checkmark$  |  | $\\checkmark$  |  |\n| **RBI** | $\\checkmark$ | | $\\checkmark$  |  | |\n| **Plug-and-Play** | | $\\checkmark$  | |$\\checkmark$   |  |\n| **Ours** | $\\checkmark$  |$\\checkmark$ | $\\checkmark$  | $\\checkmark$  | $\\checkmark$  |\n\n---\n\n> **Q2.** While the experiments are extensive, they mention that some results are \"cited from\" or \"our reproduction results\" for other methods.\n\n**R2.**  Please allow me to reiterate our gratitude for your suggestions regarding the experimental content. We sincerely appreciate your meticulous review of our paper. All experiments in our study primarily focus on evaluating our method's robustness against various forgery techniques, while simultaneously comparing it with state-of-the-art forgery detection methods to demonstrate its superiority. \n\nHowever, a critical issue arises: although different studies use the same datasets, their experimental settings often differ. For instance, SBI employs 8 frames for training, whereas methods like LAA-Net utilize 128 frames. Such discrepancies in experimental configurations lead to unfair comparisons. **In this work, reproducing all existing methods' experiments is not our objective. Our goal is to ensure maximally equitable experimental conditions.** Many prior studies, such as DeepfakeBench, have addressed this challenge. \n\nTo minimize workload and focus on the paper's overall presentation, we  extensively reuse results obtained under identical experimental settings. Additionally, for papers not previously reproduced with consistent settings, we conduct independent re-implementations. Consequently, many experimental results are cited from other publications. **When incorporating such data, we rigorously verify the alignment of experimental settings.**\n\n\n---\n\n\n> **Q3.** The paper acknowledges 'relatively weaker performance in specific edge cases' across 58+ forgery methods. Please explicitly state which specific forgery methods constitute these edge cases.\n> \n**R3.** Our approach demonstrates robust performance against 53 out of 58 evaluated forgery techniques while showing limitations on only 5 specific methods.  **As noted in Appendix Table 9**, we highlight successful results through **bolding and underlining**, which may inadvertently obscure visibility of underperforming cases. \n\nThese less effective scenarios primarily involve: **Which Face Is Real, StarGANv2, StyleGAN-XL,HeyGen, DiT**. Although our method shows reduced efficacy against these particular forgery techniques, its strong performance against the majority of tested methods remains encouraging. We will prioritize optimizing performance for these specific cases in future work.\n\n**Table 9:**  **We test all forgery techniques provided on the DF40 dataset and reported Frame-Level AUC.** All results with AUC greater than 80 are **highlighted** in bold, while those with AUC less than 80 but greater than 70 are **underlined**.\n\n **Type** | mobileswap    | MidJourney    | faceswap        | styleclip        | DiT             | lia               | ddim             | mcnet            | RDDM             | StyleGAN3\n:--------:|:-------------:|:-------------:|:---------------:|:----------------:|:---------------:|:-----------------:|:----------------:|:----------------:|:----------------:|:-----------------------------:\n **AUC**  | $\\textbf{0.97}$ | $\\textbf{0.97}$ |$\\textbf{0.89}$   | $\\textbf{0.82}$   | 0.66            | $\\textbf{0.91}$     | $\\textbf{0.97}$    | $\\textbf{0.84}$    | $\\underline{0.70}$ | $\\textbf{0.93}$\n **Type** | e4e           | pixart        | whichfaceisreal | deepfacelab      | StyleGANXL      | heygen            | facedancer       | MRAA             | pirender         | VQGAN\n **AUC**  | $\\textbf{0.95}$ | $\\textbf{0.93}$ | 0.42            | $\\underline{0.77}$ | 0.41            | 0.58              | $\\textbf{0.83}$    | $\\textbf{0.85}$    | $\\textbf{0.81}$    | $\\textbf{0.85}$\n **Type** | StyleGAN2     | facevid2vid   | simswap         | inswap           | one\\_shot\\_free | wav2lip           | e4s              | starganv2        | tpsm             | sd2.1\n **AUC**  | $\\textbf{0.93}$ | $\\textbf{0.83}$ | $\\textbf{0.91}$   | $\\textbf{0.87}$    | $\\textbf{0.85}$   | $\\underline{0.76 }$ | $\\textbf{0.88}$    | 0.48             | $\\textbf{0.83}$    | $\\textbf{0.97}$\n **Type** | blendface     | hyperreenact  | uniface         | CollabDiff       | fsgan           | danet             | SiT              | sadtalker        | fomm             |\n **AUC**  | $\\textbf{0.94}$ | $\\textbf{0.80}$ | $\\textbf{0.92}$   | $\\textbf{0.95}$    | $\\textbf{0.86}$   | $\\textbf{0.80}$     | $\\underline{0.72}$ | $\\underline{0.74}$ | $\\textbf{0.85}$    |"}, {"review_id": "z3Vm6hmHAl", "reviewer": "Reviewer_fmVY", "summary": "While this paper contains minor limitations, they do not detract from its overall strengths. The logical flow is exceptionally clear, experimental validation is comprehensive with impressive performance, and supplementary materials (particularly the visual analysis of misclassified samples and systematic evaluation of mainstream deepfake methods) provide substantial value. I therefore recommend a Weak Accept. Should the authors address Concern #1 and #2 raised in my review, I would be more willing to elevate my score.", "questions": "1.  It is recommended to relocate selected content from the appendix to the main body.\n2.  It is necessary to clarify the details of video-level detection.", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strenth:\nThis paper categorizes deepfake artifacts into two types: FIA (Face Inconsistency Artifacts): Arising from detail discrepancies between forged faces and backgrounds (e.g., lighting/texture discontinuities), USA (Up-Sampling Artifacts): Introduced by the up-sampling process in generator decoders (e.g., interpolation traces). By analyzing technical workflows of mainstream forgery algorithms (e.g., Full Face Synthesis and Face Swapping), it demonstrates that all existing methods inevitably produce at least one of FIA/USA artifacts. Building on this foundation, the paper proposes FIA-USA data augmentation, whose three components (Multi-Type Mask Synthesis (MTMS), Multi-Modal Reconstruction (MMR), Random Artifact Combination (RAC)), collectively cover potential artifact types in deepfakes comprehensively, addressing the limitation of prior augmentation methods that only simulate FIA locally or globally. Additionally, the complementary AFFS and RCR techniques fully leverage FIA-USA’s capabilities. Validation across 7 datasets (including traditional and generative deepfakes) encompassing 58 distinct forgery methods proves the approach’s generalizability. Though the framework involves multiple components, the authors’ thorough ablation studies confirm each element’s effectiveness, and the paper’s logical structure remains exceptionally clear.\n\nWeakness:\n1.The FIA-USA framework involves numerous components, and understanding its full operational procedure necessitates referring to the algorithmic details in the appendix. To improve accessibility, the complete algorithm should be moved to the main text. Additionally, the initialization process of AFFS relies on training dataset data—a critical implementation detail currently buried in the appendix that warrants inclusion in the main body.\n\n2.Although the proposed method still relies on the frameworks of SBI and Face X-ray, the improvements derived from the authors' artifact analysis of mainstream deepfake techniques are compelling. As clearly demonstrated in Table 1, their approach exhibits significant distinctions from prior works. However, I am perplexed: the method appears to be an image-domain enhancement—why can it be applied to video-level deepfake detection? More specifically, I wonder whether the video-level experiments use the maximum value or the average value of multi-frame results.\n\n\n3.While the method demonstrates compelling efficacy across most forgery techniques, its detection performance remains notably weak for certain generative approaches, particularly StyleGANXL, DIT, and others. Overall, the performance is still impressive.\n\n4.The framework contains too many components, making the method somewhat complex. Although the author has devoted a lot of effort to elaborating on each component, it still needs to be carefully read in conjunction with the appendix and have a certain understanding of data augmentation work in the RGB field.", "was_revised": true, "final_justification": "I maintain my original score that recommend the paper to be accepted.", "rebuttal": "Dear reviewer fmVY:\n\nOn behalf of all authors, we deeply appreciate your contributions during the review process. We are honored to benefit from your feedback and appreciate your recognition of our **article's logical framework, experimental richness, and model performance**. Below, we address your questions point by point:\n\n>**Q1.** Move the full FIA-USA algorithm and the initialization details of AFFS from the appendix to the main text for better accessibility and emphasis, as they are critical to understanding the framework.\n>\n**R1.** Regarding the manuscript's presentation, we acknowledge your observation about the inherent complexity of our methodology and the extensive experimental results. \nOur proposed universal deepfake detection framework comprises three  core components, each vital for robust detection performance: \n* The FIA-USA dual-artifact collaborative enhancement framework, which generates pseudo-fake samples containing Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA) through Multi-Type Mask Synthesis (MTMS) for hierarchical mask generation (covering macro-editing boundaries via convex hulls of 4/81 keypoints and micro-editing traces via eye/mouth feature combinations), Multi-Modal Reconstruction (MMR) using autoencoders and super-resolution models to simulate USA by reconstructing target faces, and Random Artifact Combination (RAC) to blend source and reconstructed images with randomly selected masks and artifacts; \n* Automatic Forgery-aware Feature Selection (AFFS), which reduces feature redundancy in Feature Pyramid Networks by ranking channels based on normalized response discrepancies between real and forged regions to retain artifact-sensitive dimensions; \n* Region-aware Contrastive Regularization (RCR), which enhances discrimination between manipulated and authentic areas through patch-level contrast (minimizing feature distances in real faces while maximizing them in fakes) and image-level contrast (aligning real regions across samples). \n\n**To enable the clearest possible description of each component given the inherent complexity and to adhere to the strict page limits for the main text while ensuring its overall quality**, some in-depth analyses, supplementary algorithmic details, and extensive ablation studies are necessarily included in the appendix in this submission version. **Should our paper be accepted, we will relocate the most critical of this appendix content back into the main text in the camera-ready version to provide the most comprehensive presentation possible within the final page constraints**.\n\n---\n\n>**Q2.** The proposed method's compelling improvements—derived from the authors' artifact analysis of mainstream deepfake techniques—are notable, even though it relies on the SBI and Face X-ray frameworks. Described as image-domain enhancement, a key question remains: how is this approach applicable to video-level detection?\n\n**R2.** While traditional blend-based augmentation methods like Face X-ray and SBI primarily focus on simulating isolated Face Inconsistency Artifacts (FIA) through either global or local facial manipulations—thereby limiting their ability to capture the complex artifact interplay in real deepfakes. Our FIA-USA framework pioneers a dual-artifact collaborative strategy that simultaneously addresses both FIA and Up-Sampling Artifacts (USA), which stem from inherent decoder operations in all deepfake generators. \n\nSpecifically, we overcome previous limitations through three key innovations: \n1) Multi-Type Mask Synthesis that  models both macro-editing boundaries (via 4/81-keypoint convex hulls) and micro-editing traces (through randomized eye/mouth feature combinations). \n2) Multi-Modal Reconstruction that employs autoencoder and super-resolution model to systematically simulate USA patterns across diverse generator architectures. \n3) Random Artifact Combination that blends reconstructed variants with source images to create comprehensive pseudo-fakes covering global-local and facial-regional inconsistencies. In Table 1, we have conducted a detailed comparison with previous data augmentation methods.\n\nFor video-level analysis, our methodology employs uniform sampling of 32 frames per video. We compute frame-wise forgery probabilities independently through our detection pipeline, then derive the final video-level confidence score by **calculating the mean of all sampled frame probabilities.**\n\n**Table 1: Comparison of our framework and state-of-the-art(SOTA) methods using pseudo-deepfake synthesis. Our method differs from previous methods in terms of the level at which alterations are applied (local vs global), the introduced artifacts, and the level at which facial inconsistencies are applied.**\n\n| | global  | local | face-level |region-level | Up-Sampling |\n| :--- | :--: | :--: | :--: | :--: | :---: |\n|**Face Xray, PCL, OST**   |$\\checkmark$  |  | $\\checkmark$  | |  |\n| **SLADD** | $\\checkmark$  |  | | $\\checkmark$  |  |\n| **SBI** | $\\checkmark$  |  | $\\checkmark$ |  | |\n| **SeeABLE** | | $\\checkmark$  |  | $\\checkmark$  |  |\n| **RBI** | $\\checkmark$ | | $\\checkmark$  |  | |\n| **Plug-and-Play** | | $\\checkmark$  | |$\\checkmark$   |  |\n| **Ours** | $\\checkmark$  |$\\checkmark$ | $\\checkmark$  | $\\checkmark$  | $\\checkmark$  |\n\n---\n\n>**Q3.**  While the method demonstrates compelling efficacy across most forgery techniques, its detection performance remains notably weak for certain generative approaches, particularly StyleGANXL, DIT, and others. Overall, the performance is still impressive.\n\n**R3.**  We sincerely appreciate your recognition of the overall efficacy of our method across diverse forgery techniques. As highlighted in our comprehensive evaluation (Section 4), we rigorously evaluate our deepfake detection framework across seven benchmark datasets, collectively spanning over 58 distinct forgery methods to ensure comprehensive validation of generalization capabilities. These include five traditional deepfake datasets—FaceForensics++ (FF++), Deepfake Detection (DFD), Deepfake Detection Challenge (DFDC), DFDC Preview (DFDCP), and CelebDF (CDF)—which cover classical manipulation techniques like identity swapping and expression reenactment. Additionally, two generative deepfake datasets—Diffusion Facial Forgery (DiFF) and DF40 are used to test robustness against modern threats, with DiFF incorporating 13 text/image-driven synthesis methods (e.g., Stable Diffusion, LoRA) and DF40 encompassing 40 cutting-edge techniques spanning face swapping, full-face synthesis, and fine-grained attribute editing. While our method demonstrates robust detection performance for 53 of the 58 techniques evaluated（As shown in Table 9）, we acknowledge the relatively weaker efficacy for some generative approaches such as StyleGANXL and DiT. **We plan to further improve our work in the future and develop more robust detection methods.**\n\n---\n>**Q4.** The framework contains too many components, making the method somewhat complex. Although the author has devoted a lot of effort to elaborating on each component, it still needs to be carefully read in conjunction with the appendix and have a certain understanding of data augmentation work in the RGB field.\n\n**R4.** Thank you for recognizing our paper. Our work does have numerous components, but it is not simply a pile up. Each component plays an irreplaceable role in our methodology. \n\nOur motivation for starting this paper is to notice a phenomenon: The fundamental challenge driving deepfake detection development is the inherent limitation of existing deepfake detectors in generalizing across diverse generation techniques (e.g., GANs, Diffusion models). These methods produce highly variable artifacts, causing specialized detectors to fail when encountering unseen manipulation methods. Through systematic analysis of mainstream forgery algorithms, we identity two universal flaws underlying all deepfakes: 1) Face Inconsistency Artifacts (FIA), stemming from generators’ inability to perfectly harmonize facial details with background physical properties (e.g., lighting/texture), exacerbated by blending boundaries in face-swapping techniques; and 2) Up-Sampling Artifacts (USA), mathematical distortions introduced during decoder-based image reconstruction in encoder-decoder architectures. \n\n Then，We propose FIA-USA with the aim of developing more robust forgery detection models by simulating these two more common features through data augmentation. At the same time, we notice that work such as SBI focuses more on the data augmentation process itself, which  makes us doubt  whether FIA-USA's capabilities are fully utilized? \n\nTherefore, we propose RCR (Region aware Contrastive Regularization). It amplifies three critical distinctions: 1) intra-real-sample consistency, 2) intra-fake-sample discrepancies between manipulated/authentic regions, and 3) cross-sample consistency in real regions. This forces the model to pinpoint USA distribution  and FIA boundary discontinuities, enhancing sensitivity to artifacts. \n\nComplementing this, AFFS (Automatic Forgery-aware Feature Selection) addresses feature redundancy in standard Feature Pyramid Networks (FPN). It purifies the feature space by using forgery-region masks as supervision signals to quantify each channel’s sensitivity to USA/FIA, selectively retaining artifact-correlated dimensions. \n\nIn addition, consistent with your point of view, we  recommend readers to read a series of previous works on data augmentation, including Face X-ray, SBI, RBI, etc., because these works are progressive layer by layer, and reading previous works can indeed help better understand our method."}, {"review_id": "2D4GkdLvWP", "reviewer": "Reviewer_FW6y", "summary": "The paper proposes a universal deepfake detection framework that identifies and exploits two general forgery artifacts—FIA and USA—across various types of deepfake generation. The authors introduce a novel dual-artifact augmentation technique (FIA-USA) for pseudo-deepfake sample synthesis, along with two training enhancements: AFFS and RCR. Extensive experiments demonstrate great generalization and robustness compared to SOTA methods.", "questions": "1. Why you only choose (2.5, 1, 0.25) as your hyperparameters ($\\lambda_1, \\lambda_2, \\lambda_3$)?\n2. See Weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1. The FIA-USA data augmentation strategy is simple yet remarkably effective.\n2. Extensive experiments on a wide range of datasets and rigorous ablation studies.\n3. The method has good generalizability to unseen methods and robust under perturbations.\n\n### Weaknesses\n1. The brackets in Eq. 3 are not closed, making it unclear.\n2. Although FIA-USA shows great effectiveness, it may heavily relies on the quality of pseudo-fake generation; performance might drop when faced with adversarial attacks or higher-fidelity forgeries.\n3. The paper lacks a formal theoretical justification for the strong cross-domain generalization performance of FIA and USA. Providing such analysis would strengthen the claims.\n4. There lacks a detailed analysis about the results in Tab. 6a to 6c.", "was_revised": true, "final_justification": "I appreciate the clear rebuttal made by the authors. After considering the authors' response, I decided to maintain my original score.", "rebuttal": "Dear reviewer FW6y:\n\nOn behalf of all authors, we extend our sincerest gratitude to you! We deeply appreciate the valuable time and effort you dedicated to the review process, especially during this period of significantly increased submission volume. We particularly appreciate your recognition of **the breadth of our experiments and the effectiveness of our methodology**. Below, we address your questions point by point:\n\n>**Q1.** The brackets in Eq. 3 are not closed, making it unclear.\n\n**R1.** We sincerely apologize for an unfortunate error that slipped into Equation 3 during manuscript preparation—namely, the inclusion of an extraneous left parenthesis. We regret any confusion this may have caused. **The correct expression should read\n$\\mathcal{L}\\_{AFFS}\\^{i,k}=\\frac{1}{N}\\sum\\_ {n=1}^{N}\\Vert \\mathcal{F}(f\\_{i}^{k}(R\\_n)-f\\_{i}^{k}(F\\_n))-M_n \\Vert \\_{2}\\^{2}$, where $\\mathcal{F}$ denotes feature normalization and spatial interpolation to align $f_{i}^{k}$ with mask $M_n$. We will correct this in the revised manuscript.**\n\n---\n\n>**Q2.** FIA-USA’s effectiveness likely depends heavily on the quality of its pseudo-fake generation. This creates two key vulnerabilities: 1. Performance may degrade against adversarial attacks (e.g., perturbations designed to fool the detector). 2. Performance may degrade when faced with higher-fidelity forgeries potentially emerging in the future.\n\n**R2.** We acknowledge the validity of your concerns. We  address them in two parts:\n* First, regarding adversarial attacks: Fundamentally, deepfake detection operates as a downstream classification task (e.g., image/video classification), and classification models remain highly vulnerable to adversarial attacks—including data poisoning, evasion attacks, and backdoor attacks. While many researchers are actively developing defense mechanisms against these threats, **our core contribution is orthogonal to adversarial defense. Our primary objective focuses on enhancing the model’s inherent detection performance.** That said, we find your proposed idea—developing data augmentation methods that simultaneously resist adversarial attacks and generalize to unseen counterfeits—highly valuable. This concept has been incorporated into our subsequent research plan.\n* Second, concerning the evolution of forgery methods: We fully recognize this challenge, **as the relationship between forgery and detection is inherently adversarial—an ongoing pursuit where each advance prompts countermeasures.** While our current model demonstrates strong detection capabilities, its efficacy against future generative models may diminish (as seen even with SOTA methods like SBI in detection generative methods). **Rest assured, we are committed to advancing more robust detection frameworks to address the emergent trust crisis in digital media.**\n    \n\n---\n>**Q3.** The paper lacks a formal theoretical justification for the strong cross-domain generalization performance of FIA and USA. Providing such analysis would strengthen the claims.\n\n**R3.** Thank you for your valuable feedback regarding the theoretical underpinnings of our paper. **In this work, we derive the effectiveness of our method directly from analyzing the operational processes and inherent defects of deepfake generation techniques.** Specifically, in Section 3 of the main text, we primarily discuss the **irretrievability of FIA**.  This argument stems from two fundamental constraints:\n* a. The generation bottleneck: All deepfake methods (whether full face synthesis or face swapping) require generating high-frequency facial details. However, generators are inherently limited in their ability to perfectly generate faces.\n* b. The inevitability of blend: Face swapping techniques (involving both macro-editing of 4/81 key points and micro-editing) crucially depend on blend operations(see Appendix B.1). This process forcibly concatenates heterogeneous pixels, creating local  discontinuities (like color or brightness jumps) at the boundaries, which form consistent artifact patterns across methods.\n    \nAdditionally, **USA are unavoidable**: whether using autoencoders (AE) or super-resolution models (SR), the generator’s decoder must perform upsampling operations. Appendix B provides a detailed taxonomy of deepfake methods, explaining the basic generation process and the specific types of artifacts each method is prone to introducing. Building on this analysis of generation defects, our method introduced the FIA-USA data augmentation strategy. This augmentation is specifically designed to simulate the various manifestations of FIA and USA, enhancing the model's ability to detect them. For instance:\n* AE and SR augmentation targets USA restoration.\n* Blending augmentation targets FIA restoration.\n    \nTherefore, the justification for our method’s effectiveness is straightforward and intuitive: we start from the inherent defects of the generation process itself and use targeted data augmentation to train the model to recognize these specific artifacts more accurately. **Appendix Table 10 provides empirical support for this intuition, demonstrating that our FIA-USA augmentation covers nearly the entire spectrum of observable deepfake artifacts. In addition, Section B of the appendix provides a detailed analysis of the current mainstream forgery techniques, including the classification of forgery techniques and why they are classified in this way, the basic process of each forgery, and which artifacts are introduced at which step.**\n\n\n---\n> **Q4.**  A detailed analysis about the results in Tab. 6a to 6c.\n\n**R4.** This section provides a detailed analysis of Table 6 (to be updated in the camera-ready version), focusing primarily on the ablation studies designed to evaluate our proposed method.\n\n* **Table 6a primarily investigates the effectiveness of each core component.** The combination of FIA-USA, AFFS, and RCR yields optimal performance. As intended in our design, AFFS and RCR are crucial for fully leveraging the potential of FIA-USA. Removing any component degrades performance, with the absence of FIA-USA having the most significant impact (a 4.15% decrease). It is important to clarify that \"w/o FIA-USA\" specifically refers to using only the data augmentation proposed by SBI. Furthermore, RCR exerts a stronger influence on FIA-USA's effectiveness compared to AFFS. \n\n     **The ablation specifically targeting FIA-USA (components within it) reveals:** Various data augmentation techniques contribute differently. SR and AE are shown to be effective for detecting generative forgeries. Conversely, MTMS appears to suppress performance on this specific forgery type. This aligns with our theoretical analysis: generative forgeries constitute global manipulations that produce fewer  Face Inconsistency Artifacts (FIA) but primarily exhibit  Up-Sampling Artifacts (USA). As seen from MTMS’s performance on traditional forgeries, increasing the diversity of FIA proves beneficial for detecting face swapping forgery.\n\n    **Regarding the RCR ablation:**  The results align well with our chosen hyperparameters. They indicate that the PCR contributes substantially more to the model's performance than the ICR. This observation is consistent with the hyperparameter sensitivity analysis presented in Table 6c: increasing the loss weight for PCR while decreasing that for ICR leads to significant performance gains. Conversely, increasing the ICR weight while reducing the PCR weight results in only marginal improvements.\n\n*  **Table 6b demonstrates the compatibility of our method with various network frameworks**, while also illustrating the critical impact of model parameter size and architectural design on detector performance. This highlights the ongoing importance of developing more robust and specialized deep learning architectures for counterfeit detection tasks.\n*  **Table 6c provides a detailed ablation experiment for our hyperparameter selection**, which is consistent with the ablation experiment results for RCR components in Table 6a. We find that increasing the weight of PCR loss for BCE loss significantly improves model performance, while reducing ICR for BCE loss also improves model loss. However, conversely, model performance will not be significantly improved.\n    \n\n---\n> **Q5.** Why was the current hyperparameter setting selected？\n\n**R5.**  **We conducted detailed experiments to determine the values of our hyperparameters.**  As discussed in the fourth point, our findings demonstrate that increasing the weight of PCR while decreasing the weight of ICR relative to the BCE loss enhances model performance. Conversely, applying the opposite weighting scheme reduces performance. Guided by these results, we determine the final hyperparameter settings after extensive experimentation. **Table 6.c displays  our experimental results.**\n\nWe attribute this outcome to a key observation: ICR images are subjected to different data augmentations (e.g., compression, color shifts, contrast adjustments), resulting in significantly reduced feature-level similarity relative to PCR. Consequently, ICR is less effective than PCR, justifying its lower weighting."}, {"review_id": "uS0Pwyz0HT", "reviewer": "Reviewer_uxGL", "summary": "Briefly summarize the paper and its contributions. This is not the place to critique the paper; the authors should generally agree with a well-written summary. This is also not the place to paste the abstract—please provide the summary in your own understanding after reading.\n\nThis paper introduces a universal framework for detecting facial deepfakes by focusing on identifying common and generalizable forgery artifacts. The core challenge addressed is the wide diversity of existing deepfake generators and the impracticality of teaching a detector to learn all varied artifact types separately.\n\nThe authors propose categorizing deepfake artifacts into two distinct yet complementary types:\n\nFace Inconsistency Artifacts (FIA): These arise from the difficulty of generating intricate facial details, leading to inconsistencies between complex facial features and their surrounding areas, or from blending boundaries in face swapping.\n\nUp-Sampling Artifacts (USA): These are inevitable traces left by the generator’s decoder during the up-sampling process, as decoding and up-sampling are fundamental operations in facial manipulation generators. The authors claim that all existing deepfakes are observed to exhibit either or both of these artifact types.\n\nTo enable detection models to learn these generalizable artifacts, the paper proposes an image-level pseudo-fake creation framework called FIA-USA. This framework constructs fake samples by simulating both FIA and USA without introducing less-general artifacts. Specifically, it reconstructs target faces to simulate USA using autoencoders (AE) or super-resolution (SR) models, and utilizes image-level blending with multiple mask types on diverse facial regions to create FIA. The blending process includes macro-editing masks (based on 4 or 81 facial keypoints) and micro-editing masks (randomly combining facial features like eyes and mouth).\n\nBeyond the data augmentation, the framework also includes two complementary techniques to maximize efficacy:\n\nAutomatic Forgery-aware Feature Selection (AFFS): This streamlines feature dimensionality and augments feature discernment by adaptively selecting channels in Feature Pyramid Networks (FPN) that are sensitive to forgery regions, thereby reducing feature redundancy.\n\nRegion-aware Contrastive Regularization (RCR): By juxtaposing features of manipulated regions against authentic regions, RCR enhances the model's focus on upsampling traces and inconsistencies between different areas. This includes both Patch-level Contrastive Regularization (PCR) and Image-level Contrastive Regularization (ICR).\nExtensive experiments on seven widely used deepfake detection datasets, encompassing over 58 distinct forgery methods, validate the superior efficacy and robustness of the proposed method. The results demonstrate that a standard image classifier trained with FIA-USA pseudo-fake data can generalize well to unseen deepfakes, even outperforming state-of-the-art methods, including those based on latent space augmentation. The paper concludes that this targeted approach, by focusing on fundamental artifact patterns, can help address challenges in current detection methods and provide a promising foundation for more adaptable deepfake detection systems.", "questions": "Question: The paper states that its core methodology for detecting deepfakes primarily relies on analyzing image-level artifacts, namely Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). While the evaluation includes \"satisfactory video-level detection performance\", the framework acknowledges that \"video-level artifacts present more complex challenges - including temporal inconsistencies and audio-visual asynchrony - which fall outside the current research scope\". How does the current image-level focused approach implicitly capture or respond to these video-level inconsistencies?  Is this a future area of research interest?\n\nProposed Suggestion: For the authors to strengthen the paper's contribution towards a \"universal detection framework\", it would be beneficial to elaborate on the specific temporal or inter-frame cues (if any) that their current image-based approach implicitly leverages to achieve reported video-level AUC scores. If applicable, could the authors provide a brief discussion or hypothesis on how the FIA-USA framework might be extended or complemented in future work to directly address these complex video-level artifacts (e.g., inconsistencies over time or audio-visual discrepancies), rather than solely relying on aggregating frame-level detections? This would clarify the path towards a truly universal solution for all types of deepfakes, including video-specific forgeries.\n\nQuestion: The paper explicitly acknowledges that it has \"not yet explored the integration of data augmentation methods with such [large] models due to practical constraints\". This implies a current boundary to the method's general applicability, especially as larger models increasingly dominate deepfake generation and detection.\n\nProposed Suggestion: While a full exploration of large model integration is noted as future work, could the authors expand on the primary challenges they anticipate when adapting or integrating their FIA-USA framework, AFFS, and RCR with large foundation models (e.g., Vision Transformers, very large Diffusion Models)? Are these challenges primarily computational, architectural, or related to the nature of artifacts produced by and detectable by these larger models? Providing this insight would enhance the paper's forward-looking perspective and set a clearer agenda for future research in this area.\n\nQuestion: The paper states that all experiments were conducted on \"a single NVIDIA 3090\" and indicates that resource requirements are \"minimal\". However, specific metrics like execution time for training/inference or precise memory consumption are not provided in the main text or the appendix beyond this general statement.\n\nProposed Suggestion: To ensure full experimental reproducibility and provide practical insights for researchers and practitioners considering deploying this method, could the authors provide approximate training times (e.g., per epoch or total) and inference times (e.g., frames per second) for their model on a standard dataset (e.g., FF++) using the specified hardware? Additionally, GPU memory consumption (e.g., VRAM usage) would be a useful detail. This specific data would allow others to more accurately gauge the computational feasibility and scalability of the proposed approach.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Quality\n\nStrengths:\n* Robust Experimental Validation: The paper conducts extensive experiments on seven widely used Deepfake detection datasets, encompassing over 58 distinct forgery methods, including identity swapping, expression reenactment, generative face synthesis, and attribute editing. This broad evaluation validates the superior efficacy and robustness of their method.\n\n*  Performance: The proposed method is shown to outperform other state-of-the-art (SOTA) approaches on the majority of traditional deepfake datasets and achieves competitive results on others, such as CDF-v1. Notably, it demonstrates strong generalization capabilities on generative deepfake datasets like DiFF and DF40, often surpassing previous methods by significant margins (e.g., 9.7% over RECCE and 23.4% over SBI on DF40 average AUC).\n\n*  Robustness: The method exhibits superior robustness to unseen perturbations like Gaussian Blur, Block Wise, Change Contrast, and JPEG Compression, as quantified by the reduction rate of Video-Level AUC.\n\n* Ablation Studies: Detailed ablation experiments confirm the effectiveness of each component (FIA-USA, AFFS, RCR), showing a progressive degradation in performance when individual components are removed, which strengthens the argument for their contributions.\n\n* Technical Soundness: The categorization of artifacts (FIA and USA) is grounded in a systematic analysis of shared technical frameworks in existing deepfake algorithms. The methodology for constructing pseudo-fakes (FIA-USA) is well-articulated, involving multi-type mask synthesis and multi-modal reconstruction using autoencoders and super-resolution models to simulate realistic artifacts. The integration of AFFS and RCR further refines feature learning for robust detection.\n\nWeaknesses:\n\n* Limited Evaluation of Full Video-Level Artifacts: While the paper achieves satisfactory video-level detection performance and acknowledges video-level artifacts as a future direction, the primary focus remains on image-level data augmentation and detection architectures. This leaves complex video-level challenges like temporal inconsistencies and audio-visual asynchrony largely unexplored in the current framework .\n\n* Unexplored Large Model Integration: The authors note that they have not yet explored the integration of their data augmentation methods with large models' zero-shot capabilities, citing practical constraints and unfamiliarity with large models. This could be a limitation in light of recent advancements in large generative models.\n\n* Performance in Edge Cases: Although the method is effective across a substantial majority of forgery methods (over 58), the authors acknowledge relatively weaker performance in specific edge cases. While they state it maintains robust detection for most common approaches, these edge cases could be critical.\n\nClarity\n\n\nStrengths:\n\n* Detailed Methodology: The core components, especially FIA-USA, AFFS, and RCR, are explained in detail, including their motivations and mathematical formulations. The appendix further elaborates on mask generation, reconstruction models, and the theoretical basis of AFFS.\n\n* Sufficient Experimental Details: The paper provides comprehensive details for reproducibility, including the backbone network architecture, optimizer, batch size, learning rate, training epochs, video sampling, data augmentation pipeline, loss coefficients, and hardware used (NVIDIA 3090). Data splits and datasets are clearly specified, including how FF++ was used for training.\n\nWeaknesses:\n\n* Lack of Open-Source Code: The authors explicitly state that the code is \"currently not open source\". While the paper provides extensive details, the absence of publicly available code can hinder direct reproducibility and further research by the community, despite the detailed instructions. This is noted as a \"No\" in the NeurIPS checklist .\n\n* Limited Compute Resource Details: While the GPU type is specified (NVIDIA 3090), more granular details such as memory usage and exact execution times for experiments are not provided. While authors claim this is unlikely to affect reproducibility for experiments of this scale, it could be beneficial for researchers with different computational resources.\n\nSignificance\n\nStrengths:\n*  The paper tackles the increasingly important topic of deepfake detection, driven by the rapid development of AI generation techniques and the associated risks of privacy violations and misinformation spread. Developing a universal deepfake detector is highlighted as an \"urgent priority\".\n\n* Generalizable Approach: By focusing on common and generalizable artifacts (FIA and USA) instead of specific ones, the framework moves towards a universal detection capability, enhancing the generalization of detection models across diverse and unseen deepfake variations. This is a significant step beyond methods tailored for specific forgery techniques.\n\nOriginality\n\nStrengths \n\n* Novel Pseudo-Fake Creation Framework: The paper introduces a \"novel image-level pseudo-fake creation framework\" (FIA-USA) which is a significant original contribution. Unlike prior methods that often focus on isolated artifact types or single-scale modifications, FIA-USA is a dual-artifact collaborative enhancement framework designed for universal detection, addressing both global/local and facial/regional alterations, and specifically introducing USA, which was less emphasized previously.\n\n* Complementary Techniques: The introduction of Automatic Forgery-aware Feature Selection (AFFS) and Region-aware Contrastive Regularization (RCR) as complementary techniques further contributes to the originality. AFFS is described as a \"statistically-driven feature compression paradigm\" that quantifies channel-wise sensitivity to forgery regions, and RCR explicitly models divergence between forged and authentic regions through multi-granularity contrastive learning.\n\nWeaknesses:\n* Building on Existing Blend Paradigm: While FIA-USA injects new vitality, it builds upon an existing blend-based data augmentation paradigm. The originality lies in how they extend this paradigm (multiple masks, USA introduction) rather than inventing blending from scratch. This is a minor point, as the authors clearly explain their advancements over previous blending methods.", "was_revised": true, "final_justification": "I appreciate the thorough rebuttal by the authors and after careful consideration I have decided to raise my score to accept.", "rebuttal": "Dear reviewer uxGL:\n\nOn behalf of all authors, we sincerely thank you for your thorough evaluation and generous recognition of our work. We are deeply encouraged by your positive remarks on the **rigorous experimental design, robustness of results, exceptional performance benchmarks, and meticulous ablation studies** – your insightful feedback is invaluable to our research development. Below, we provide point-by-point responses to your specific questions to further clarify and enhance the manuscript. \n\n> **Q1.**  Could the authors briefly discuss how the FIA-USA framework might be extended or enhanced to directly address complex video-level artifacts in future work, rather than relying solely on aggregating frame-level detections?\n\n**R1.** Thank you for recognizing our work. For video-level detection, our current approach is to uniformly sample 32 frames from a video and compute the average forgery probability across these frames to represent the video-level probability.\n\nPrior to this work, we try to develop a series of video data augmentation methods. These methods employ frame-level alterations (e.g., introducing artifacts using SBI) and induce temporal inconsistencies by frame rearrangement. However, this method does not significantly enhance the performance of model. Our current paper primarily explores image-level data augmentation, we find that robust detectors can be trained effectively using image-level augmentation alone. This finding suggests that developing meaningful video-level data augmentation likely requires strategies to generate more generalizable temporal artifacts. Indeed, overly simplistic temporal inconsistencies can actually degrade model performance.\n\n**A potentially promising direction is to incorporate acoustic features to create such inconsistencies, or integrate physiological signals (e.g., eye blinking patterns, facial movement patterns) into the augmentation pipeline.** However, we must honestly report that despite sustained exploration in this direction, we have yet to achieve significant breakthroughs.\n\n\n---\n\n> **Q2.**  Performance in Edge Cases: Although the method is effective across a substantial majority of forgery methods (over 58), the authors acknowledge relatively weaker performance in specific edge cases.\n\n\n**R2.**  Regarding model performance, our method currently yields only 5 unsatisfactory detection results when evaluated against over 58 distinct forgery methods, demonstrating considerable effectiveness. Nevertheless, we intend to pursue further improvements.\n \n**Table 9:**  **We test all forgery techniques provided on the DF40 dataset and reported Frame-Level AUC.** All results with AUC greater than 80 are **highlighted** in bold, while those with AUC less than 80 but greater than 70 are **underlined**.\n\n **Type** | mobileswap    | MidJourney    | faceswap        | styleclip        | DiT             | lia               | ddim             | mcnet            | RDDM             | StyleGAN3\n:--------:|:-------------:|:-------------:|:---------------:|:----------------:|:---------------:|:-----------------:|:----------------:|:----------------:|:----------------:|:-----------------------------:\n **AUC**  | $\\textbf{0.97}$ | $\\textbf{0.97}$ |$\\textbf{0.89}$   | $\\textbf{0.82}$   | 0.66            | $\\textbf{0.91}$     | $\\textbf{0.97}$    | $\\textbf{0.84}$    | $\\underline{0.70}$ | $\\textbf{0.93}$\n **Type** | e4e           | pixart        | whichfaceisreal | deepfacelab      | StyleGANXL      | heygen            | facedancer       | MRAA             | pirender         | VQGAN\n **AUC**  | $\\textbf{0.95}$ | $\\textbf{0.93}$ | 0.42            | $\\underline{0.77}$ | 0.41            | 0.58              | $\\textbf{0.83}$    | $\\textbf{0.85}$    | $\\textbf{0.81}$    | $\\textbf{0.85}$\n **Type** | StyleGAN2     | facevid2vid   | simswap         | inswap           | one\\_shot\\_free | wav2lip           | e4s              | starganv2        | tpsm             | sd2.1\n **AUC**  | $\\textbf{0.93}$ | $\\textbf{0.83}$ | $\\textbf{0.91}$   | $\\textbf{0.87}$    | $\\textbf{0.85}$   | $\\underline{0.76 }$ | $\\textbf{0.88}$    | 0.48             | $\\textbf{0.83}$    | $\\textbf{0.97}$\n **Type** | blendface     | hyperreenact  | uniface         | CollabDiff       | fsgan           | danet             | SiT              | sadtalker        | fomm             |\n **AUC**  | $\\textbf{0.94}$ | $\\textbf{0.80}$ | $\\textbf{0.92}$   | $\\textbf{0.95}$    | $\\textbf{0.86}$   | $\\textbf{0.80}$     | $\\underline{0.72}$ | $\\underline{0.74}$ | $\\textbf{0.85}$    |\n\n---\n\n> **Q3.**  What are the main challenges (computational, architectural, or artifact-related) in integrating FIA-USA/AFFS/RCR with large foundation models? This would clarify future research directions.\n\n**R3.** The primary challenge with VLM (Vision-Language Model) integration stems from our team's limited prior exposure to VLMs. We remain unfamiliar with their training paradigms, fine-tuning strategies, and underlying architectures. Although our FIA-USA framework can generate samples suitable for VLM training, we suspect simple feature concatenation may yield insufficient gains. Our future direction involves actively exploring VLMs to develop more robust detectors. However, this will require substantial research time. \n\nCritically, we believe **VLM's core superiority lies in its interactive capabilities, intrinsic interpretability, and task universality** – attributes that must be fully leveraged. Without adapting to these strengths, VLM adoption risks becoming merely a transition from one encoder architecture to another, offering marginal benefit. **Consequently, extending FIA-USA to VLMs will not treat them as mere classifiers. Instead, we aim to propose either: A more interpretable framework, or A unified multi-task framework (e.g., incorporating forgery localization).** Achieving this will likely require advanced large-model training techniques, such as reinforcement learning-based fine-tuning approaches.\n\n---\n> **Q4.** Lack of Open-Source Code.\n\n**R4.** We strongly support open source code. However, as our paper encompasses extensive experimentation, multiple methodologies, and numerous interdependent components, organizing the codebase presents significant challenges. Releasing hastily structured code would likely compromise readability and usability and may introduce technical issues, potentially consuming reviewers' valuable time unnecessarily. Please rest assured that we are currently undertaking a thorough codebase restructuring to ensure high quality and usability. **We plan to open-source the code promptly upon the paper's acceptance.**\n\n---\n\n> **Q5.**  Provide the resources needed for reproduction：1. Training time(per epoch/total).  2. Inference speed(frames per second). 3. GPU memory.\n\n**R5.** For model training and inference, we utilize the FF++ C23 version for real faces. The training set comprise 720 real videos, from which we extracted 32 frames per video, resulting in a total of 720 * 32 * 2 training images (720 * 32 real faces + 720 * 32 FIA-USA generated fake faces ). Each training epoch take approximately 48 minutes, requiring 20.2 GB of GPU memory with a batch size of 12. The validation set consist of 140 real videos, yielding 140 * 32 * 2 images. Each validation epoch take about 3 minutes. We train the model for a total of 50 epochs, performing validation once per epoch. Before training commenced, we initialize the AFFS module, initializing parameters across 4 layers—a process taking approximately 20 minutes and requiring 1.7 GB of GPU memory. Consequently, the total expected training time is 42.8 hours. The speed of inference and validation is projected to be 50 images per second，requiring 20.2 GB of GPU memory with a batch size of 12.\n\n\n\n\n**Table R5: Resource Utilization**\n| Phase          | Duration              | GPU Memory | Batch Size | Frequency          | Details                       |\n|----------------|-----------------------|------------|------------|--------------------|-------------------------------|\n| Initialization | 20 minutes           | 1.7 GB     | -          | Once               | AFFS module setup (4 layers) |\n| Training       | 48 minutes/epoch     | 20.2 GB    | 12         | 50 epochs          | Per-epoch processing         |\n| Validation     | 3 minutes/run        | 20.2 GB    | 12         | 50 epochs  | Per-epoch evaluation         |\n| **Total Training** | **42.8 hours**   | -          | -          | -                  | Initialization + 50×(Train+Val) |\n| Inference      | 50 images/second     | 20.2 GB    | 12         | -                  | -    |"}]

Author rebuttal:

Meta-review: This paper addresses the urgent challenge of generalizable deepfake detection. Rather than designing detectors for every forgery technique, the authors identify two universal artifact categories, Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA), that inevitably arise across diverse deepfake generators. Building on this insight, they propose FIA-USA, a pseudo-fake creation framework that systematically synthesizes training data containing only these artifacts, and complement it with Automatic Forgery-aware Feature Selection (AFFS) and Region-aware Contrastive Regularization (RCR).

The work is novel and well-motivated, providing a principled taxonomy of artifacts and a carefully designed augmentation pipeline. Experiments are comprehensive and rigorous, spanning seven benchmark datasets (traditional and generative, with over 58 forgery methods), and show consistent improvements in generalization and robustness. Ablation studies and visual analyses convincingly demonstrate the contributions of each component.

While the framework is complex and exhibits somewhat weaker performance on a few modern generative forgery methods (e.g., StyleGANXL, DiT), these are acknowledged by the authors and do not diminish the overall impact. Reviewers found the rebuttal clarifications satisfactory, particularly regarding novelty and video-level detection methodology.

Overall, this is a technically solid and impactful paper. It introduces a principled perspective on universal artifacts, demonstrates state-of-the-art performance, and provides extensive validation. I recommend acceptance.
Decision: Accept (poster)