Title: From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes
Venue: NeurIPS 2025
Accepted: yes
Keywords: Universal / Generalizable Deepfake Detection
Abstract: Detecting deepfakes has been an increasingly important topic, especially given the rapid development of AI generation techniques.
In this paper, we ask: How can we build a universal detection framework that is effective for most facial deepfakes?
One significant challenge is the wide variety of deepfake generators available, resulting in varying forgery artifacts (e.g., lighting inconsistency, color mismatch, etc).
But should we ``teach" the detector to learn all these artifacts separately? It is impossible and impractical to elaborate on them all.
So the core idea is to pinpoint the more common and general artifacts across different deepfakes.
Accordingly, we categorize deepfake artifacts into two distinct yet complementary types: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). 
FIA arise from the challenge of generating all intricate details, inevitably causing inconsistencies between the complex facial features and relatively uniform surrounding areas.
USA, on the other hand, are the inevitable traces left by the generator's decoder during the up-sampling process.
This categorization stems from the observation that all existing deepfakes typically exhibit one or both of these artifacts.
To achieve this, we propose a new data-level pseudo-fake creation framework that constructs fake samples with only the FIA and USA, without introducing extra less-general artifacts.
Specifically, we employ a super-resolution to simulate the USA, while  utilise  image-level self-blending on diverse facial regions to create the FIA.
We surprisingly found that, with this intuitive design, a standard image classifier trained only with our pseudo-fake data can non-trivially generalize well to previously unseen deepfakes.

From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes

Abstract

Detecting deepfakes has been an increasingly important topic, especially given the rapid development of AI generation techniques. In this paper, we ask: How can we build a universal detection framework that is effective for most facial deepfakes? One significant challenge is the wide diversity of existing deepfake generators, which produced varied types of forgery artifacts ( e.g., lighting inconsistency, color mismatch, etc ). But should we 'teach' the detector to learn all these artifacts separately? It is impossible and impractical to elaborate on them all. So the core idea is to pinpoint the more common and general artifacts across different deepfakes. Through systematic analysis of shared technical frameworks in existing deepfake algorithms, we categorize deepfake artifacts into two distinct yet complementary types: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). FIA arise from the challenge of generating all intricate details, inevitably causing inconsistencies between the complex facial features and relatively uniform surrounding areas. USA, on the other hand, are the inevitable traces left by the generator's decoder during the up-sampling process, with all existing deepfakes exhibiting either or both artifacts. Subsequently, we propose a novel image-level pseudo-fake creation framework that constructs fake samples with only the FIA and USA, without introducing extra less-general artifacts. Specifically, we reconstruct the target face to simulate the USA, while utilize image-level blend on diverse facial regions to create the FIA. We surprisingly found that, with this intuitive design, a standard image classifier trained only with our pseudo-fake data can non-trivially generalize well to unseen deepfakes.

1 Introduction

In recent years, A growing number of facial manipulation techniques have advanced due to the rapid development of generative models [35, 36, 58, 75], which facilitate the production of highly realistic and virtually undetectable alterations. As a result, the risks of personal privacy being violated and the spread of misinformation have increased significantly. Therefore, developing a universal deepfake detector that can be used to detect the existing diverse fake methods has become an urgent priority.

(a). Face Inconsistency Artifacts (FIA)

188

197

202

201

181

188

191

193

168

176

180

183

167

170

172

171

177

177

177

176

180

180

179

176

179

178

175

171

176

175

170

166

166

165

160

156

163

161

157

155

160

158

157

155

162

159

156

154

Real

Super-Resolution

Face-level Inconsistencies

Region-level Inconsistencies

(b). Up-Sampling Artifacts (USA)

DeepFake

Real              Fake

Eyes

Mouth

Teeth

Nose

Real

More Details

Forehead

Eyebrow

DDPM

Less Details

More Details

Less Details

Hint-1: AI-generated faces often exhibit inconsistencies at both the

face level

and

region level

compared to the surrounding areas.

AAA

LIIIL'AAA

VWV.11

Hint-2: AI-generated faces often exhibit up-sampling artifacts

by the decoders during the generation process.

In this paper, we pose the question: How can we build a universal detection framework that is effective for most facial deepfakes? A significant challenge lies in the wide range of deepfake generators, which lead to different types of forgery artifacts. Specifically, a large number of existing works are dedicated to creating detection algorithms that are carefully designed to identify counterfeit artifacts within specific handcrafted designs, including eye blinking [42], pupil morphology [25], and corneal specularity [32]. However, these methods are mainly effective for identifying particular forgery techniques. But should we train the detector to learn each of these artifacts separately? It is impossible and impractical to cover them all. So, where should we start from?

The core idea is to encourage the detection model to learn the more generalizable artifacts. As indicated in previous works [81, 63, 70], different fakes might share some common forgery patterns, and the unlimited deepfake artifacts could be generalized and summarized by the limited generalizable artifacts. Motivated by this, we conduct an in-depth preliminary exploration of the common patterns of existing deepfake artifacts, through which we identify two generalizable deepfake artifacts and categorize them into two distinct yet complementary categories: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA), as depicted in Figure 1. FIA represents an inherent limitation that deepfakes struggle to overcome, originating from the generator's incapacity to precisely reproduce facial attributes and nuances, as well as the inescapable disparities between regions induced by amalgamation processes. Unlike FIA, Up-Sampling Artifacts (USA) are not readily apparent to the unaided eye and originate from the generator's inherent up-sampling process that the generator's decoder cannot adequately substitute. Crucially, since decoding and up-sampling constitute fundamental operations in facial manipulation generators, such artifacts become an inevitable byproduct across all methods in this domain. Numerous prior investigations have substantiated this observation [22, 65]. To date, all existing deepfakes characteristically display one or both of these artifact categories.

To enable the model to simultaneously capture FIA and USA, we propose a sophisticated data augmentation method: FIA-USA , which leverages super-resolution models [71] and autoencoders [57] to introduce Up-Sampling Artifacts (USA) by reconstructing the face, and employs a strategy of generating multiple masks to blend the reconstructed face with the original, thereby introducing Face Inconsistency Artifacts (FIA) by creating discrepancies at both the facial and regional levels. Augmented with the aforementioned dual artifacts, our method effectively conditions the model to generalize across unseen deepfakes. Furthermore, to maximize the efficacy of FIA-USA, we introduce two complementary technique: Automatic Forgery-aware Feature Selection (AFFS) , which streamlines feature dimensionality and augments feature discernment through adaptive selection. Region-aware Contrastive Regularization (RCR) , by juxtaposing the features of the manipulated

region against those of the authentic region, RCR enhances the model's focus on the upsampling traces within the manipulated areas and the inconsistencies between different regions.

Extensive experiments conducted on seven widely used Deepfake detection datasets ( encompassing over 58 distinct forgery methods, spanning facial manipulation categories including identity swapping, expression reenactment, generative face synthesis, and attribute editing ) validate the superior efficacy of our method. Detailed ablation experiments and robustness experiments have demonstrated the effectiveness of each component in our method and the robustness of our method to disturbances.

2 Related Work

Classic Deepfake Detection. Classic deepfake detection can be categorized into multiple aspects: On the one hand, typical deepfake image detection methods encompass frequency-domain analysis [56, 49], identity information [17, 9, 69], contrastive learning [23, 87], network structure improvement [88, 17], watermark [85, 72], robustness [38, 51], interpretability [16, 62] and so forth. On the other hand, deepfake video detection methods include innovative video representation formats [78], selfsupervised learning [2, 27, 92], anomaly detection [19, 20], biological signal [76, 8], multi-modal based [54]. In addition to the above two aspects, the the remainder of the work involves forgery localization [51, 26], multi task learning [51, 86], adversarial attack [44], and so on.

Deepfake Detectors Based on Data Synthesis. A notable approach involves synthesizing pseudofake faces during training to enhance generalization capability. Early works like Face X-ray [40] pioneered blending-based augmentation by fusing facial regions from different identities to simulate forgery boundaries. Subsequent methods improved upon this paradigm through different artifact simulation strategies: SLADD [3] focused on local adversarial perturbations, SBI [63] proposed selfblending to amplify facial inconsistencies, SeeABLE [37] which proposes fine-grained region-specific blending, and Plug-and-Play [83] employs facial feature masks rather than full-face manipulation for enhanced face synthesis precision. However, as highlighted in Table 1, existing methods predominantly focus on isolated artifact types (Mainly focused on FIA) and single-scale modifications (global or local), fundamentally limiting their ability to capture the artifacts interplay inherent in real deepfakes. The paradigm of this data augmentation method can be represented as:

where, I t represents the target face, I s represents the source face, and M represents the mask. Our proposed FIA-USA method injects new vitality into this paradigm by generating multiple types of masks, introducing USA to I t , as well as random combination of artifacts.

3 Method

In this section, we begin by outlining two distinct and prevalent types of forgery artifacts found in deepfakes: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA) . We then detail the construction of data augmentation techniques ( FIA-USA ) designed to enable the model to learn to

Base image

Pseudo fake

Real image

Real

𝑳𝑩𝑪𝑬

Blend

Randomly select reconstructed images

Erosion of

convex hull

Combining

facial features

Extract

Landmark

Generate mask M

Fake

Real

AFFS

Region-aware Contrastive

Regularization (RCR)

Reduce

distance

Patch-level Contrastive Regularization

Increase

distance

Reduce

distance

Increase

distance

Image-level Contrastive Regularization

AE                    SR                     Base

Backbone

Fake

Real

Do inference with unseen testing image

Inference Phase

Traing Phase

FIA-USA

FPN

Backbone

MLP

MLP



Automatic Forgery-aware Feature Selection ( AFFS )



Backbone

features

features with

lower dimensions

feature map from

the i - th layer

𝓕(-)

•

𝓕(-

)   Calculate the difference for each channel and

perform scaling.

•

𝓛𝑨𝑭𝑭𝑺

Calculate similarity and record the index of

dimensions with high similarity for building FPN.

𝓛𝑨𝑭𝑭𝑺

high similarity

low similarity

a. Initialization



AFFS

b. Used in Training

During training, the FPN

is built using the channels

recorded in each layer

during initialization.

西

detect these artifacts concurrently. Subsequently, we present the Automatic Forgery-aware Feature Selection (AFFS) and Region-aware Contrastive Regularization (RCR) .

3.1 FIA-USA.

Face Inconsistency Artifacts (FIA). As depicted in Figure 1, Face Inconsistency Artifacts (FIA) frequently occur in deepfakes due to the inherent difficulty in rendering intricate details, which often results in discrepancies between intricate facial features and the more uniform surrounding regions. In essence, until deepfake technology advances to the point where it can flawlessly replicate real faces, these artifacts are likely to persist. We explore the origins of Face Inconsistency Artifacts by categorizing deepfake techniques into two domains: Full Face Synthesis and Face Swapping . In deepfake-based full face synthesis, Face Inconsistency Artifacts inherently persist regardless of the generator's capabilities, while their severity directly correlates with the model's performance. For face swapping, these artifacts arise from two primary sources: 1). the efficacy of the generator. 2). the blending boundary created during the face swapping's blending procedure. Furthermore, we can categorize face swapping into two main types based on the forged area: a. macro-editing realize face swapping through 4 or 81 facial keypoints. and b. micro-editing , which enables granular control over individual facial features. In section B of the appendix, we provide a detailed description of the classification of forgery techniques and their differences.

Up-Sampling Artifacts (USA). These artifacts consistently present in synthetic facial regions originate from upsampling operations within the generator's decoder architecture, where interpolation processes inevitably introduce characteristic distortion patterns, with the extent of USA (Up-Sampling Artifacts) introduced varying significantly across different generator implementations. To investigate

more universal up-sampling artifacts, we categorize forgery techniques into two types: One category generates images of forged regions that are as clear and free of blur as the original image. And another type result in blurred tampered areas, requiring additional facial super-resolution models (SR) to process the blurred areas.

FIA-USA. While FIA and USA represent two fundamental artifact categories in deepfakes, conventional data augmentation methods predominantly focus on simulating isolated forgery traces, thereby hindering detection models from effectively learning cross-domain forgery patterns. To bridge this critical gap, we propose FIA-USA-a dual-artifact collaborative enhancement framework engineered for universal detection. Our framework systematically emulates the forgery process through a three-pronged methodology:

1. Multi-Type Mask Synthesis(MTMS): The Multi-Type Mask Synthesis module aims to comprehensively model Face Inconsistency Artifacts (FIA) through a hierarchical mask generation strategy, covering both macro-editing boundaries and micro-editing traces. This process involves two complementary approaches:

a. Macro-Editing Mask. We generate facial contours through two configurations: (1) a high-precision mode using 81 facial keypoints to compute detailed convex hull boundaries, and (2) a rectangular approximation mode derived from 4 keypoints. The initial mask M is formulated as:

where H ( · ) denotes convex hull computation and K n represents the keypoints set. As same as [63], the initial mask M subsequently undergoes erosion or dilation by applying two Gaussian filters with distinct kernel sizes, where erosion occurs when the kernel of the first filter is larger than that of the second, while dilation manifests when the first kernel is comparatively smaller.

b. Micro-Editing Masks. Given a set of 81 facial keypoints K 81 = { k 1 , k 2 , . . . , k 81 } , we first extract two subsets: eyes-specific keypoints K eyes ⊂ K and mouth-specific keypoints K mouth ⊂ K. Using convex hull computation H ( · ) , we derive two binary masks: the eye mask M e = H ( K eyes ) and the mouth mask M m = H ( K mouth ) . These masks are then logically combined through union operations ∪ to generate three distinct facial regions: (1) M = M e , (2) M = M m , and (3) M = M e ∪ M m (combined mouth and eyes). Since other facial regions such as eyebrows and nose are rarely targeted in localized facial manipulations (e.g., attribute editing, expression synthesis), these features were intentionally excluded from our mask generation framework to prioritize regions most susceptible to forgery (eyes and mouth).

2. Multi-Modal Reconstruction(MMR): We employ two complementary reconstruction paradigms to reconstruct the target face( I t ) in Formula 1, thereby systematically simulating the Up-Sampling Artifacts (USA) inherent in diverse deepfake generators: a. Autoencoder (AE) Reconstruction: Reconstruct I t through AE [57] to simulate the excessive dependency relationship between adjacent pixels [65] caused by upsampling in GAN / Diffusion models. b. Super-Resolution (SR) Reconstruction: Applies SR models [71] to upsample I t , thereby replicating the characteristic artifacts inherent in deepfake post-processing pipelines. Specifically, given an input face image I , we generate reconstructed versions I AE = AE ( I ) and I SR = SR ( I ) .

3. Random Artifact Combination(RAC): To ensure comprehensive coverage of artifact interactions, we blend original images(source face) with reconstructed variants(target face) ( I AE , I SR , I ) using mask M , based on Equation 1. More specifically, given a facial image I s , first MTMS generates multiple masks M with equal probability, then MMR generates a reconstructed images ( I AE , I SR ) , and finally RAC blend the I with randomly selected reconstruction variants I t from ( I AE , I SR , I ) using a randomly selected mask M . Before blending, I s and I t will undergo data augmentation(change color tone, saturation, contrast, blur, etc) to enhance the inconsistency of statistical information. As shown in Table 1, compared to existing methods, FIA-USA has achieved collaborative artifact enhancement across local / global, facial / regional levels. Please refer to Algorithm 1 for the complete algorithmic process.

3.2 Loss Function

3.2.1 Automatic Forgery-aware Feature Selection

While FIA-USA generates discriminative training samples through dual-artifact augmentation, the inherent feature redundancy in Feature Pyramid Networks (FPN) [45] may drive models to focus on

Algorithm 1 Pseudocode for FIA-USA

non-critical forgery patterns, thereby compromising detection generalization. This limitation stems from FPN's naive aggregation of all channel-wise features without adaptively emphasizing FIA/USAcorrelated representations. To address this limitation, we propose Automatic Forgery-aware Feature Selection (AFFS), a statistically-driven feature compression paradigm that quantifies channel-wise sensitivity to forgery regions, thereby dynamically constructing lightweight yet discriminative feature pyramids.

Given a pretrained backbone network ϕ , let f i = ϕ i ( I ) ∈ R W i ∗ H i ∗ C i denote the feature map from the i -th layer, where I ∈ R H ∗ W ∗ 3 represents the input image. To identify artifact-sensitive feature dimensions, we leverage pseudo-fake pairs { R n , F n , M n } N n =1 generated by FIA-USA and compute the normalized response discrepancy between real and forged regions. For the k -th channel f k i ∈ R W i ∗ H i in layer i , the selection criterion is formulated as:

where F denotes feature normalization and spatial interpolation to align f k i with mask M n . Channels are ranked by ascending L i,k AFFS and the topm i channels with minimal errors are retained to form a compressed feature map f ′ i ∈ R W i ∗ H i ∗ m i for FPN construction. AFFS leverages a pre-trained model as initialization, operating as a supervised feature ranking mechanism that reduces feature redundancy.

3.2.2 Region-aware Contrastive Regularization

Building upon the artifact-sensitive features selected by AFFS, we design Region-aware Contrastive Regularization (RCR) to explicitly model the divergence between forged and authentic regions through multi-granularity contrastive learning. Given the Feature Pyramid Network (FPN) P , we denote the feature maps of real and fake faces as H R = P ( ϕ ( R )) , H F = P ( ϕ ( F )) . For H R , the areas inside and outside M are real, while for H F , the areas inside M are manipulated and the areas outside M are real. We define the region within M as H in R , H in F and the region outside M as H out R , H out F . RCR operates on the refined feature pyramid from AFFS, implementing multi-granularity contrast through:

Patch-level Contrastive Regularization. For real faces, the features f of H in R and H out R are consistent, thus we aim to minimize the distance between them. Conversely, for fake faces, the features f of H in F and H out F are distinct, thus we aim to maximize the distance between them. Therefore, the loss function of Patch-level Contrastive Regularization (PCR) can be formulated as:

where f i represents a pixel feature, τ is a temperature parameter and δ ( · ) is the normalized cosine similarity between two features, as:

Image-level Contrastive Regularization. Since both the region of real face and fake face outside M are real, we aim to minimize the distance between them and maximize the distance between the fake face and the real face in the area inside M . Therefore, the loss function for Image-level Contrastive Regularization (ICR) can be formulated as:

Overall Loss. The network is optimized using the following loss:

where L BCE denotes the cross-entropy classification loss. L BCE , L 1 and L 2 are weighted by the hyperparameters λ 1 , λ 2 and λ 3 , respectively.

4 Experiments

4.1 Settings

Datasets. To evaluate the effectiveness of our proposed method, we conducted extensive experiments on seven widely-adopted benchmark datasets spanning both classical facial manipulation paradigms and emerging generative deepfake architectures. a. Traditional Deepfake Datasets: 1. FaceForensics++ (FF++) [59], 2. Deepfake Detection (DFD) [12], 3. Deepfake Detection Challenge (DFDC) [15], 4. preview version of DFDC (DFDCP) [14], and 5. CelebDF (CDF) [43]. FF++ comprises 1,000 original videos and 4,000 fake videos forged by four manipulation methods, namely, Deepfakes(DF) [13], Face2Face(F2F) [68], FaceSwap(FS) [18], and NeuralTextures(NT) [67]. FF++ offers three levels of compression: raw, lightly compressed (c23), and heavily compressed (c40), under storage constraints, our implementation adopts the FF++_c23 (including DFD) with training conducted following the SBI protocol [63], employing exclusively real facial samples from FF++_c23 subset. Although many previous studies have utilized the same dataset for both training and testing, the preprocessing and experimental configurations can differ, making fair comparisons difficult. Therefore, in addition to testing on the raw data of the aforementioned datasets, we also performed generalization assessments on the unified new benchmark for traditional deepfakes, (DeepfakeBench) [82]. b. Generative Deepfake Datasets. 6. Diffusion Facial Forgery (DiFF) [5] contains over 500,000 images synthesized using 13 different generation methods under four conditions: Text to Image (T2I), Image to Image (I2I), Face Swapping (FS) and Face Editing (FE). Text to Image encompasses four generation methods: Midjourney [50], Stable Diffusion XL (SDXL) [55], Free-DoM_T [84], and HPS [77]. Image to Image comprises Low Rank Adaptation(LoRA) [31], DreamBooth [60], SDXL Refiner [55], and Free-DoM_I . Face Swapping includes DiffFace [36] and DCFace [36]. Face Editing comprises Imagic [35], Cycle Diffusion(CycleDiff) [75], and Collaborative Diffusion(CoDiff) [34]. 7. DF40 [80] encompasses 40 state-of-the-art deepfake generation methodologies spanning four core categories: face swapping, facial reenactment, full-face synthesis, and advanced facial manipulation. Significantly surpassing existing benchmarks in both scope and volume, DF40 integrates cutting-edge generative architectures from 2024 alongside widely adopted commercial software solutions.

Evaluation Metrics. We report the Frame-Level Area Under Curve (AUC) metric on the DeepfakeBench and DIFF dataset, report the Video-Level Area Under Curve (AUC) metric on the traditional deepfake datasets, to compare our proposed method with prior works.

Implementation Details. We adopt EfficientNetB4 [66] as the backbone network architecture (We also

investigate alternative network architectures and their respective outcomes), trained for 50 epochs using the SAM optimizer [21] with a batch size of 12 and initial learning rate of 0.001. For video processing, each input is uniformly sampled into 32 frames during both training and inference phases. Our data augmentation pipeline combines the proposed FIA-USA strategy with conventional techniques including RandomHorizontalFlip, RandomCutOut, and AddGaussianNoise. The loss coefficients λ 1 , λ 2 , λ 3 are empirically set to 1, 2.5, and 0.25 respectively (We also explored the impact of other variants on the detection results), with the temperature parameter τ fixed at 0.7. All experiments were conducted on a single NVIDIA 3090.

4.2 Generalization Evaluation

4.2.1 Traditional Deepfake Datasets.

We first compare our method with previous work at the frame level . To ensure the fairness of the experiment, we conducted the experiment on the DeepfakeBench, and adhered to the data preprocessing and experimental settings provided by them. For previous work, we utilized the experiment data provided by DeepfakeBench. As shown in Table 2, our method outperforms other methods on the majority of deepfake datasets and competes with state-ofthe-art methods on the CDF-v1 dataset. In addition to comparing at the frame-level, we also compared our method at the video-level with state-of-the-art detection algorithms, including various data augmentation methods. The comparison results, as shown in Table 3, strongly demonstrate the effective generalization of our method.

4.2.2 Generative Deepfake Datasets

Our evaluation on DIFF and DF40 datasets demonstrates the superior generalization capabilities of our architecture compared to state-of-the-art detection methods. Comprehensive metrics are detailed in Table 4 and Table 5. As shown in Table 5, our method achieves an average AUC of 87.8% , surpassing RECCE by 9.7% and outperforming SBI by 23.4% , and enables 10% higher AUC than LSDA's latent space augmentation on

(a) Ablation Study of Method Components. We report the Video-Level AUC for traditional deepfake datasets and the Frame-Level AUC for generative deepfake datasets. SR signifies the process of superresolution, and AE refers to the reconstruction achieved through autoencoders.

(b) Ablation Study on Model Architectures.

average. The results notably contradict recent findings [79] about RGB-based methods' limitations against generative deepfakes, proving that properly designed RGB-level artifacts retain critical discriminative signals. For comprehensive experimental results on forgery techniques supported by the DF40 and DIFF, please refer to the extended analysis in Appendix Section G.

4.2.3 Robustness

In Figure 3, we assessed the influence of different levels of random perturbations on the detection performance. We quantified the effects of different levels of Gaussian Blur, Block Wise, Change Contrast, and JPEG Compression on the detectors. Notably, to gauge accuracy, we employed the reduction rate of Video-Level AUC to assess the robustness of the detector to different degrees of disturbance, which could be expressed as R AUC = ( AUC -AUC raw ) /AUC raw , where AUC raw refers to no perturbations. As shown in the Figure 3, our method exhibits superior robustness compared to other methods.

Block Wise

Change Contrast

JPEG Compression

Average

0.0

0.0

0.0

0.0

0.0

AUC Reduction Ratio

AUC Reduction Ratio

AUC Reduction Ratio

AUC Reduction Ratio

AUC Reduction Ratio

0.1

0.1

0.1

0.1

0.2

0.2

0.2

0.2

0.3

E°0

0.3

0.4

0.4

0.4

0.4

0.4

0.5

0.5

0.5

0.5

0.5

0

1

2

Level

3

0

1

2

Level

3

4

0

1

2

Level

3

4

0

1

2

Level

3

4

0

1

2

Level

3

4

FWA

Face X-ray

SB1

PatchForensics

Ours

4.3 Ablation Study

The impact of method components. To systematically evaluate the contribution of each component in our framework, we conducted comprehensive ablation studies across seven benchmark datasets. As detailed in Table 6a, our analysis focuses on three core innovations: (1) the FIAUSA augmentation mechanism, (2) Automatic Forgeryaware Feature Selection (AFFS), and (3) Region-aware Contrastive Regularization (RCR). We adopt video-level AUC for traditional deepfakes and frame-level AUC for generative deepfakes. The empirical results demonstrate progressive performance degradation when removing individual components, with the complete framework achieving optimal detection accuracy . The combination of FIA-USA, AFFS, and RCR yields optimal performance. As intended in our design, AFFS and RCR are crucial for fully leveraging the potential of FIA-USA. Removing any component degrades performance, with the absence of FIA-USA having the most significant impact (a 4.15% decrease). It is

Original        Mask            Ours

Traditional

Original         Ours

FE

FS

Generative

DF

FS

F2F

NT

T2I

I2I

important to clarify that "w/o FIA-USA" specifically refers to using only the data augmentation proposed by SBI. Furthermore, RCR exerts a stronger influence on FIA-USA's effectiveness compared to AFFS.

The ablation specifically targeting FIA-USA (components within it) reveals: Various data augmentation techniques contribute differently. SR and AE are shown to be effective for detecting generative forgeries. Conversely, MTMS appears to suppress performance on this specific forgery type. This aligns with our theoretical analysis: generative forgeries constitute global manipulations that produce fewer Face Inconsistency Artifacts (FIA) but primarily exhibit Up-Sampling Artifacts (USA). As seen from MTMS's performance on traditional forgeries, increasing the diversity of FIA proves beneficial for detecting face swapping forgery.

Regarding the RCR ablation: The results align well with our chosen hyperparameters. They indicate that the PCR contributes substantially more to the model's performance than the ICR. This observation is consistent with the hyperparameter sensitivity analysis presented in Table 6c: increasing the loss weight for PCR while decreasing that for ICR leads to significant performance gains. Conversely, increasing the ICR weight while reducing the PCR weight results in only marginal improvements.

The impact of model framework. Our architectural analysis in Table 6b systematically benchmarks detection performance across backbone networks, quantitatively comparing the effects of EfficientNet [66], ResNet [29], and different depth configurations on detection performance, demonstrates the compatibility of our method with various network frameworks, while also illustrating the critical impact of model parameter size and architectural design on detector performance. This highlights the ongoing importance of developing more robust and specialized deep learning architectures for counterfeit detection tasks.

Impact of hyperparameters. We also examined the effect of hyperparameters of the loss function on model performance, as detailed in the Table 6c. Our analysis of the weights revealed that increasing the weight of λ 2 relative to the weights of cross entropy enhances model performance, whereas decreasing the weight of λ 3 relative to cross entropy also improves performance, which is consistent with the ablation experiment results for RCR components in Table 6a. However, conversely, model performance will not be significantly improved.

5 Visualizations

Our GradCAM [91] analysis in Figure 4 demonstrates precise localization of facial forgery artifacts across both conventional and emerging generative architectures. Notably, the visualization framework not only pinpoints manipulation traces in traditional deepfakes but also effectively in state-of-the-art synthetic media, validating our method's generalization capability.

6 Conclusion

In conclusion, this study presents a universal framework for deepfake detection by focusing on common artifacts that span various types of face forgeries. By categorizing deepfake artifacts into Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA), we enhance the generalization capability of detection models. This targeted approach allows the model to focus on detecting fundamental artifact patterns, and potentially improving its performance across diverse deepfake variations. In doing so, our framework may help address some challenges in current detection methods and could provide a promising foundation for developing more adaptable deepfake detection systems in future research.

Acknowledgments and Disclosure of Funding

This work was supported in part by Key Science & Technology Project of Anhui Province 202423l10050033, Zhejiang Provincial Natural Science Foundation of China under Grant (No. LQN25F020023).

References

Zhongjie Ba, Qingyu Liu, Zhenguang Liu, Shuang Wu, Feng Lin, Li Lu, and Kui Ren. Exposing the deception: Uncovering more forgery clues for deepfake detection. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 719-728, 2024.

Junyi Cao, Chao Ma, Taiping Yao, Shen Chen, Shouhong Ding, and Xiaokang Yang. End-to-end reconstruction-classification learning for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4113-4122, 2022.

Liang Chen, Yong Zhang, Yibing Song, Lingqiao Liu, and Jue Wang. Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 18710-18719, 2022.

Liang Chen, Yong Zhang, Yibing Song, Jue Wang, and Lingqiao Liu. Ost: Improving generalization of deepfake detection via one-shot test-time training. Advances in Neural Information Processing Systems , 35:24597-24610, 2022.

Harry Cheng, Yangyang Guo, Tianyi Wang, Liqiang Nie, and Mohan Kankanhalli. Diffusion facial forgery detection. arXiv preprint arXiv:2401.15859 , 2024.

Jikang Cheng, Zhiyuan Yan, Ying Zhang, Yuhao Luo, Zhongyuan Wang, and Chen Li. Can we leave deepfake data behind in training deepfake detector? arXiv preprint arXiv:2408.17052 , 2024.

Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, and Jongwon Choi. Exploiting style latent flows for generalizing deepfake video detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1133-1143, June 2024.

Umur Aybars Ciftci, Ilke Demir, and Lijun Yin. Fakecatcher: Detection of synthetic portrait videos using biological signals. IEEE transactions on pattern analysis and machine intelligence , 2020.

Davide Cozzolino, Andreas Rössler, Justus Thies, Matthias Nießner, and Luisa Verdoliva. Id-reveal: Identity-aware deepfake video detection. In Proceedings of the IEEE/CVF international conference on computer vision , pages 15108-15117, 2021.

Xinjie Cui, Yuezun Li, Ao Luo, Jiaran Zhou, and Junyu Dong. Forensics adapter: Adapting clip for generalizable face forgery detection. arXiv preprint arXiv:2411.19715 , 2024.

Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil K Jain. On the detection of digital face manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition , pages 5781-5790, 2020.

Deepfakedetection, 2021. https://ai.googleblog.com/2019/09/ contributing-data-to-deepfakedetection.html Accessed 2021-11-13.

DeepFakes, 2020. www.github.com/deepfakes/faceswap Accessed 2020-09-02.

B Dolhansky. The dee pfake detection challenge (dfdc) pre view dataset. arXiv preprint arXiv:1910.08854 , 2019.

Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397 , 2020.

Shichao Dong, Jin Wang, Jiajun Liang, Haoqiang Fan, and Renhe Ji. Explaining deepfake detection by analysing image matching. In European conference on computer vision , pages 18-35. Springer, 2022.

Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Ting Zhang, Weiming Zhang, Nenghai Yu, Dong Chen, Fang Wen, and Baining Guo. Protecting celebrities from deepfake with identity consistency transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9468-9478, 2022.

FaceSwap, 2021. www.github.com/MarekKowalski/FaceSwap Accessed 2020-09-03.

Jianwei Fei, Yunshu Dai, Peipeng Yu, Tianrun Shen, Zhihua Xia, and Jian Weng. Learning second order local anomaly for general face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20270-20280, 2022.

Chao Feng, Ziyang Chen, and Andrew Owens. Self-supervised video forensics by audio-visual anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10491-10503, 2023.

Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412 , 2020.

Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning , pages 3247-3258. PMLR, 2020.

Sheldon Fung, Xuequan Lu, Chao Zhang, and Chang-Tsun Li. Deepfakeucl: Deepfake detection via unsupervised contrastive learning. In 2021 international joint conference on neural networks (IJCNN) , pages 1-8. IEEE, 2021.

Zhihao Gu, Taiping Yao, Yang Chen, Shouhong Ding, and Lizhuang Ma. Hierarchical contrastive inconsistency learning for deepfake video detection. In European Conference on Computer Vision , pages 596-613. Springer, 2022.

Hui Guo, Shu Hu, Xin Wang, Ming-Ching Chang, and Siwei Lyu. Eyes tell all: Irregular pupil shapes reveal gan-generated faces. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 2904-2908. IEEE, 2022.

Xiao Guo, Xiaohong Liu, Zhiyuan Ren, Steven Grosz, Iacopo Masi, and Xiaoming Liu. Hierarchical fine-grained image forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3155-3165, 2023.

Alexandros Haliassos, Rodrigo Mira, Stavros Petridis, and Maja Pantic. Leveraging real talking faces via self-supervision for robust forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14950-14962, 2022.

Alexandros Haliassos, Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. Lips don't lie: A generalisable and robust approach to face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5039-5049, 2021.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Hillobar, 2023. https://github.com/Hillobar/Rope , Accessed 2023.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 , 2021.

Shu Hu, Yuezun Li, and Siwei Lyu. Exposing gan-generated faces using inconsistent corneal specular highlights. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 2500-2504. IEEE, 2021.

Baojin Huang, Zhongyuan Wang, Jifan Yang, Jiaxin Ai, Qin Zou, Qian Wang, and Dengpan Ye. Implicit identity driven deepfake face swapping detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4490-4499, 2023.

Ziqi Huang, Kelvin CK Chan, Yuming Jiang, and Ziwei Liu. Collaborative diffusion for multi-modal face generation and editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6080-6090, 2023.

Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6007-6017, 2023.

Kihong Kim, Yunho Kim, Seokju Cho, Junyoung Seo, Jisu Nam, Kychul Lee, Seungryong Kim, and KwangHee Lee. Diffface: Diffusion-based face swapping with facial guidance. arXiv preprint arXiv:2212.13344 , 2022.

Nicolas Larue, Ngoc-Son Vu, Vitomir Struc, Peter Peer, and Vassilis Christophides. Seeable: Soft discrepancies and bounded contrastive learning for exposing deepfakes. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 21011-21021, 2023.

Binh M Le and Simon S Woo. Quality-agnostic deepfake detection with intra-model collaborative learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22378-22389, 2023.

Hanzhe Li, Jiaran Zhou, Bin Li, Junyu Dong, and Yuezun Li. Freqblender: Enhancing deepfake detection by blending frequency knowledge. arXiv preprint arXiv:2404.13872 , 2024.

Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5001-5010, 2020.

Y Li. Exposing deepfake videos by detecting face warping artif acts. arXiv preprint arXiv:1811.00656 , 2018.

Yuezun Li, Ming-Ching Chang, and Siwei Lyu. In ictu oculi: Exposing ai created fake videos by detecting eye blinking. In 2018 IEEE International workshop on information forensics and security (WIFS) , pages 1-7. Ieee, 2018.

Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3207-3216, 2020.

Jiawei Liang, Siyuan Liang, Aishan Liu, Xiaojun Jia, Junhao Kuang, and Xiaochun Cao. Poisoned forgery face: Towards backdoor attacks on face forgery detection. arXiv preprint arXiv:2402.11473 , 2024.

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2117-2125, 2017.

Yuzhen Lin, Wentang Song, Bin Li, Yuezun Li, Jiangqun Ni, Han Chen, and Qiushi Li. Fake it till you make it: Curricular dynamic forgery augmentations towards general deepfake detection. In European Conference on Computer Vision , pages 104-122. Springer, 2024.

Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue, Weiming Zhang, and Nenghai Yu. Spatial-phase shallow learning: rethinking face forgery detection in frequency domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 772-781, 2021.

Anwei Luo, Chenqi Kong, Jiwu Huang, Yongjian Hu, Xiangui Kang, and Alex C. Kot. Beyond the prior forgery knowledge: Mining critical clues for general face forgery detection. IEEE Transactions on Information Forensics and Security , 19:1168-1182, 2024.

Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. Generalizing face forgery detection with highfrequency features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16317-16326, 2021.

Midjourney, 2022. https://www.midjourney.com , Accessed 2022.

Dat Nguyen, Nesryne Mejri, Inder Pal Singh, Polina Kuleshova, Marcella Astrid, Anis Kacem, Enjie Ghorbel, and Djamila Aouada. Laa-net: Localized artifact attention network for quality-agnostic and generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17395-17405, 2024.

Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. Capsule-forensics: Using capsule networks to detect forged images and videos. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages 2307-2311. IEEE, 2019.

Yunsheng Ni, Depu Meng, Changqian Yu, Chengbin Quan, Dongchun Ren, and Youjian Zhao. Core: Consistent representation learning for face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12-21, 2022.

Trevine Oorloff, Surya Koppisetti, Nicolò Bonettini, Divyaraj Solanki, Ben Colman, Yaser Yacoob, Ali Shahriyari, and Gaurav Bharaj. Avff: Audio-visual feature fusion for video deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2710227112, 2024.

Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 , 2023.

Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by mining frequency-aware clues. In European conference on computer vision , pages 86-103. Springer, 2020.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684-10695, 2022.

Felix Rosberg, Eren Erdal Aksoy, Fernando Alonso-Fernandez, and Cristofer Englund. Facedancer: Pose-and occlusion-aware high fidelity face swapping. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 3454-3463, 2023.

Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision , pages 1-11, 2019.

Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR , pages 2250022510, 2023.

Somdev Sangwan, 2023. https://github.com/s0md3v/roop , Accessed 2023.

Rui Shao, Tianxing Wu, and Ziwei Liu. Detecting and recovering sequential deepfake manipulation. In European Conference on Computer Vision , pages 712-728. Springer, 2022.

Kaede Shiohara and Toshihiko Yamasaki. Detecting deepfakes with self-blended images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18720-18729, 2022.

Yuyang Sun, Huy H Nguyen, Chun-Shien Lu, ZhiYong Zhang, Lu Sun, and Isao Echizen. Generalized deepfakes detection with reconstructed-blended images and multi-scale feature reconstruction network. arXiv preprint arXiv:2312.08020 , 2023.

Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethinking the upsampling operations in cnn-based generative network for generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 28130-28139, 2024.

Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning , pages 6105-6114. PMLR, 2019.

Justus Thies, Michael Zollhöfer, and Matthias Nießner. Deferred neural rendering: Image synthesis using neural textures. Journal of ACM Transactions on Graphics , 38(4):1-12, 2019.

Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2016.

Patrick Tinsley, Adam Czajka, and Patrick Flynn. This face does not exist... but it might be yours! identity leakage in generative models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 1320-1328, 2021.

Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8695-8704, 2020.

Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. Towards real-world blind face restoration with generative facial prior. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2021.

Xueyu Wang, Jiajun Huang, Siqi Ma, Surya Nepal, and Chang Xu. Deepfake disrupter: The detector of deepfake is my friend. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14920-14929, 2022.

Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion-generated image detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22445-22455, 2023.

Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, and Houqiang Li. Altfreezing for more general video face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4129-4138, 2023.

Chen Henry Wu and Fernando De la Torre. A latent space of stochastic diffusion models for zero-shot image editing and guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7378-7387, 2023.

Jiahui Wu, Yu Zhu, Xiaoben Jiang, Yatong Liu, and Jiajun Lin. Local attention and long-distance interaction of rppg for deepfake detection. The Visual Computer , 40(2):1083-1094, 2024.

Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Better aligning text-to-image models with human preference. arXiv preprint arXiv:2303.14420 , 1(3), 2023.

Yuting Xu, Jian Liang, Gengyun Jia, Ziming Yang, Yanhao Zhang, and Ran He. Tall: Thumbnail layout for deepfake video detection. In Proceedings of the IEEE/CVF international conference on computer vision , pages 22658-22668, 2023.

Zhiyuan Yan, Yuhao Luo, Siwei Lyu, Qingshan Liu, and Baoyuan Wu. Transcending forgery specificity with latent space augmentation for generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8984-8994, 2024.

Zhiyuan Yan, Taiping Yao, Shen Chen, Yandan Zhao, Xinghe Fu, Junwei Zhu, Donghao Luo, Li Yuan, Chengjie Wang, Shouhong Ding, et al. Df40: Toward next-generation deepfake detection. arXiv preprint arXiv:2406.13495 , 2024.

Zhiyuan Yan, Yong Zhang, Yanbo Fan, and Baoyuan Wu. Ucf: Uncovering common features for generalizable deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22412-22423, 2023.

Zhiyuan Yan, Yong Zhang, Xinhang Yuan, Siwei Lyu, and Baoyuan Wu. Deepfakebench: A comprehensive benchmark of deepfake detection. arXiv preprint arXiv:2307.01426 , 2023.

Zhiyuan Yan, Yandan Zhao, Shen Chen, Mingyi Guo, Xinghe Fu, Taiping Yao, Shouhong Ding, and Li Yuan. Generalizing deepfake video detection with plug-and-play: Video-level blending and spatiotemporal adapter tuning. arXiv preprint arXiv:2408.17065 , 2024.

Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-free energyguided conditional diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 23174-23184, 2023.

Ning Yu, Vladislav Skripniuk, Dingfan Chen, Larry Davis, and Mario Fritz. Responsible disclosure of generative models using scalable fingerprinting. arXiv preprint arXiv:2012.08726 , 2020.

Yike Yuan, Xinghe Fu, Gaoang Wang, Qiming Li, and Xi Li. Forgery-domain-supervised deepfake detection with non-negative constraint. IEEE Signal Processing Letters , 29:2512-2516, 2022.

Cong Zhang, Honggang Qi, Yuezun Li, and Siwei Lyu. Contrastive multi-faceforensics: An end-to-end bi-grained contrastive learning approach for multi-face forgery detection. arXiv preprint arXiv:2308.01520 , 2023.

Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. Multiattentional deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2185-2194, 2021.

Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, and Wei Xia. Learning self-consistency for deepfake detection. In Proceedings of the IEEE/CVF international conference on computer vision , pages 15023-15033, 2021.

Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. Exploring temporal coherence for more general video face forgery detection. In Proceedings of the IEEE/CVF international conference on computer vision , pages 15044-15054, 2021.

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2921-2929, 2016.

Wanyi Zhuang, Qi Chu, Zhentao Tan, Qiankun Liu, Haojie Yuan, Changtao Miao, Zixiang Luo, and Nenghai Yu. Uia-vit: Unsupervised inconsistency-aware method based on vision transformer for face forgery detection. In European conference on computer vision , pages 391-407. Springer, 2022.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Yes, the central claims articulated in both the abstract and introduction rigorously align with the paper's technical contributions and methodological scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Yes, the limitations of our methodology are systematically analyzed in Appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Although our paper focuses on deepfake detection rather than theoretical analysis, all architectural propositions receive rigorous validation through quantitative experiments and visual demonstrations.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper provides sufficient information in multiple key aspects to support experimental reproducibility.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: The code is currently not open source.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Yes, the paper provides comprehensive details regarding the training and testing setup to ensure reproducibility and clarity.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Yes, the paper explicitly specifies the type of computational hardware used (RTX3090). However, resource requirements are minimal, detailed information about execution time or memory consumption is not provided, as these aspects are unlikely to affect reproducibility for experiments. Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have carefully read the NeurIPS Code of Ethics, but this paper does not involve any issues that violate ethical standards.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Yes, the impacts of our methodology are systematically analyzed in Appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Appendix

This supplementary material provides:

Sec. B: We discussed the classification and basic process of forgery techniques.

Sec. C: We reviewed the application of data augmentation methods in forgery detection and discussed more details about FIA-USA.

Sec. D: We introduced the construction process of FPN.

Sec. E: We provided a detailed introduction to AFFS was provided, along with visual evidence.

Sec. F: We provide a visualization example of which boundary pixels were discarded by RCR.

Sec. G: We provided test results on more forgery techniques.

Sec. H: We provide experimental results combining FIA-USA with other state-of-the-art detection methods.

Sec. I: We discussed the limitations of this paper.

Sec. J: We discussed the impacts of this paper.

Sec. K: We provided more visual examples.

B Face forgery techniques

In this section, we discuss the classification of facial forgery technology and the basic process of forgery technology, and at which nodes FIA and USA will be introduced.

B.1 Classfication of facial forgery technology

According to different processing procedures and technical principles, we first divide forgery techniques into two categories:

1). Full Face Synthesis: Generate the entire image/video directly through the image / video generation model, without involving facial cropping and stitching in the basic process. Therefore, the artifacts involved in such forgery techniques mainly include USA and a small amount of FIA, where USA is the dependency between adjacent pixels in the forgery area due to the performance of the generator, and FIA is the inconsistency in the generator's ability to generate face areas with dense details and background areas with sparse details.

2). Face Swapping . Crop the original face, generate fake regions through a generator (optional), and then stitch it with the target face. Therefore, this forgery technique can be divided into two situations: a. It includes FIA and USA: FIA is mainly caused by cropping and stitching, as well as generator performance, while USA is mainly determined by generator performance. b. It only includes FIA: FIA is mainly caused by ropping and stitching. Furthermore, based on the forged area, we can divide face swapping into two parts: Macro-editing and Micro-editing . Macro-editing can be further classified into two subtypes: Editing based on 4 facial keypoints and Editing based on 81 keypoints. Micro-editing allows for precise adjustments to each facial feature. Figure 5 provides a detailed overview of our classification concept.

B.2 Basic Process

In Figure 6, we depict the basic process of each type of forgery technique. We can see from this that the main difference between full face synthesis and face swapping lies in whether they include face cropping and stitching operations, which leads to the emergence of a large number of FIA, and whether the background is generated by the generator. The main difference between macro-editing and micro-editing in face swapping is whether the forged area is the entire face or can finely fabricate facial features. Further, macro editing based on 81 key points is more detailed, and the edges of the forged area are more similar to the edges of a real face, while 4 key points result in the forged area being a square. This discovery is mainly due to a detailed observation of the widely used forgery technique ROPE [30] and roop [61], which provides two types of face cropping techniques.

B.3 At what stage will FIA and USA be introduced

In Figure 7, we demonstrate which operations lead to the emergence of FIA and USA in the basic process. The observation of a phenomenon led to the refinement of our FIA-USA design.

Facial forgery technology

Full Face Synthesis

Face Swapping

Not Involving face cropping

and stitching

Involving face cropping

and stitching

macro-editing

micro-editing

Editing based on

81 facial keypoints

Editing based on 4

facial keypoints

Large area

of forgery

Small forged

area

Crop based on

81 key points

Crop based on

4 key points

C Details of FIA-USA

C.1 Blend-based data augmentation

In Deepfake detection, a successful approach is to manually construct negative samples for training, where data augmentation in the RGB domain is based on Blend, which can be represented as follows:

Among them, I t represents the target face, I s represents the source face, and I F represents the negative sample. The earliest design [40] was to use facial similarity to find the most suitable I s for I t . Later, SBI [63] proposed self-blend to improve the statistical consistency of I F . There are also works [64] that separate the background and face of I t , add noise to the background, and reconstruct I t , using the reconstructed I t for blend. However, to our knowledge, there has been no work so far that considers the introduction of both global and local artifacts, let alone the introduction of FIA at both the full face and regional levels while introducing USA.

C.2 Other types of data augmentation

In recent years, many other data augmentation methods have been proposed, such as frequency domain [39] and latent space [79]. Many studies suggest [39, 79] that compared to data augmentation in other domains, data augmentation in the RGB domain [40, 63, 64] exhibits weak robustness and limited effectiveness. However, we have demonstrated through extensive experiments that our proposed data augmentation in the RGB domain is not weaker or even better than that in other domains. It is worth mentioning that our proposed data augmentation belongs to the image level, and we surprisingly found that it is superior to the most advanced data augmentation methods at the video level [74].

C.3 The detailed process of generating masks in FIA-USA

Here we have detailed the processing of generating masks in FIA-USA, as shown in Figure 8. Firstly, we divide mask generation into two types: 1) Macro editing masks and 2) Micro editing masks.

Macro editing mask: We follow the mask generation process in SBI [63], which includes the following steps: a. Calculate the convex hull of 81 facial keypoints to obtain the mask of facial contour b. Erosion or expansion of

Full Face Synthesis

Inputs:

Real images /videos,

conditional inputs (such as

prompts, audio, sketches, etc).

Tools:

Image / video generation

models.

The model directly generates

images/videos based on

conditional inputs or modifies

images/videos based on

conditional inputs

.

Output:

Fake images /videos

Face swapping

Inputs:

Target face and source

face, conditional inputs (such as

prompts, audio, sketches, etc).

Tools:

Facial keypoint localization

model, Image generation models

(Optional).

Facial keypoint detection

Output:

Fake images /videos

Facial cropping

Facial stitching

The model modifies the source

face based on the target face

Optional

(a) Basic processes of

full face synthesis

and

face swapping

.

Face swapping

Facial keypoint detection

Facial stitching

editing

micro-

editing

Facial cropping

Based on

4 key points

Based on

81 key points

the mask; Besides, we also considered the case of calculating convex hull based on 4 facial keypoints, which originated from the observation of two methods for extracting raw faces provided by the open-source deepfake project that is truly available [61, 30].

Micro-editing masks: The main method is to simulate the forgery process of manipulating facial details randomly by randomly combining facial features.

Full Face Synthesis

The model directly generates

images/videos based on

conditional inputs or modifies

images/videos based on

conditional inputs.

Face swapping

Facial keypoint detection

Facial cropping

Facial stitching

The model modifies the source

face based on the target face

Optional

Both FIA and USA have been

introduced.

Both FIA and USA have been

introduced.

Only FIA has been introduced.

Generate Mask

Determine the type of Mask

Macro editing

mask

Micro editing

mask

Facial 4 or 81

keypoint detection

Facial 81 keypoints

detection

Randomly select

facial features

Calculate the convex

hull of facial keypoints

Erosion of convex hull

Based on 81 key points

Based on 4 key points

Image / video

generation models

Causing blurry

forgery areas

The forged

area is clear

Facial super-

resolution

Facial super-

resolution

Need

No need

C.4 Why use Autoencoder and Super-Resolution models to reconstruct images

As shown in Figure 9, we divide the processing flow of the generative model into two categories: One category generates images of forged regions that are as clear and free of blur as the original image (using Diffusion models and GANs as the main methods, and upsampling the latent code through AE to generate images). And another

Facial forgery technology

Full Face Synthesis

Face Swapping

macro-editing

micro-editing

Editing based on

81 facial keypoints

Editing based on 4

facial keypoints

based on

81 key points

based on

4 key points

FIA-USA

Using Macro edited masks for blending

Causing blurry

forgery areas

The forged area is

clear

Blend with AE

reconstructed images

Blend with facial

super-resolution images

Facial super-

resolution

Using Macro edited

masks for blending

Using Micro edited

masks for blending

No need

generation

model

Need generation

model

Causing blurry

forgery areas

The forged

area is clear

Blend with AE

reconstructed images

Blend with facial

super-resolution images

Facial super-

resolution

Blend with

original images

type (especially older voice driven lip shape modification techniques) will result in blurred tampered areas, requiring additional facial super-resolution models to process the blurred areas. Therefore, we use utoencoder (AE) to reconstruct images or facial Super-Resolution (SR) to process faces, in order to approximate the USA introduced by the generator in the forged area.

C.5 FIA-USA covers all possible situations that may occur in FIA and USA

As shown in the Figure 10, our proposed FIA-USA framework covers every possible situation that may occur in FIA and USA, which is also the most intuitive explanation why our method is effective.

Image

Layer 1

Layer 2

Layer 3

Layer 4

Layer 5

Feature 2

Feature 3

Feature 4

Feature 5

2 x

2 x

2 x

AFFS

AFFS

AFFS

AFFS

3x3

1x1

1x1

1x1

1x1

Feature 2

RCR

•

1x1 represents 1 x 1 convolution

•

3x3 represents 3 x 3 convolution

•

2 x represents upsampling using

nearest neighbor interpolation

method

D Details of FPN

As shown in the Figure 11, our FPN follows the framework proposed by [45], using the output features of the 2nd, 3rd, 4th, and 5th layers of the backbone to construct an FPN, In our experimental setup, the number of feature map channels is 128 and 196, corresponding to EfficientNet [66], and Resnet [29], respectively.

E Details of AFFS

Here we provide a detailed description of the process of AFFS and a visual explanation of its effectiveness.

E.1 Why is AFFS proposed

Let's consider a neural network ϕ and an FPN P , as well as the process of performing RCR on the FPN's maximum resolution layer. The output of ϕ is a set of features { d 1 , d 2 , d 3 , d 4 } , P takes this set of features as input to construct a set of feature maps with same dimensions and different resolutions, while RCR performs feature differentiation learning on the feature map with the highest resolution. One intuition is that there is a significant amount of redundancy in the output features of the ϕ , and not every dimension of the features d i , i ∈ [1 , 2 , 3 , 4] is suitable for constructing FPN for RCR. Therefore, we propose AFFS for feature dimensionality reduction of each feature d i before constructing FPN. Moreover, this feature dimensionality reduction method effectively exploits the potential advantages of our proposed FIA-USA and lays the foundation for the effective execution of RCR.

Abandon

boundary

pixels

E.2 AFFS

1) First, let's consider multiple positive and negative sample pairs generated by FIA-USA, along with their corresponding masks { R n , F n , M n } N n =1 . The usual method is to directly construct an FPN using a set of features f i from a neural network ϕ as input, where i represents the i -th layer of the ϕ .

2) Next, let's discuss the initialization process of AFFS. For the output of each layer of the neural network, we hope that the difference between the real image and the fake image after normalization is as consistent as possible with the mask M used to represent forged areas. However, we have noticed that not every channel meets this requirement, so we would like to reserve channels that meet the requirements for each layer. Therefore, for f i ∈ R W i ∗ H i ∗ C i , we calculate the following loss for each channel on a set of FIA-USA generated samples { R n , F n , M n } N n =1 to obtain those excellent dimensions,

F represents normalization and scaling operations, f k i ∈ R W i ∗ H i , k ∈ [0 , C i ] , represents the feature map of the k -th dimension of the f i . During the training process, we only select the channels reserved for f i during the AFFS initialization process. After AFFS processing, the original features f i ∈ R W i ∗ H i ∗ C i will become f ′ i ∈ R W i ∗ H i ∗ m i .

E.3 Visual Explanation of AFFS

Here we provide a set of visual examples of AFFS in Figure 13. The experiment was conducted on the 2-th layer of EfficientNetB4. Unlike the actual situation, in order to enhance readability, we only calculated L AFFS for a single epoch on a single pair of samples. And based on this, sort the 32 dimensions of the features in the 2-th layer. Here, we present the dimensions ranked in the top 14 and bottom 4. In the actual process, we iterate for 5 epochs on all samples on the training set to initialize AFFS. In Table 7, we present the changes in feature dimensions of different backbone features before and after performing AFFS.

The similarity decreases

Mask

Real

Fake

No. 1

𝐿𝐴𝐹𝐹𝑆 : 0.232

No. 26

𝐿𝐴𝐹𝐹𝑆 : 0.241

No. 18

𝐿𝐴𝐹𝐹𝑆 : 0.262

No. 9

𝐿𝐴𝐹𝐹𝑆 : 0.264

No. 27

𝐿𝐴𝐹𝐹𝑆 : 0.232

No. 1

9

𝐿𝐴𝐹𝐹𝑆 : 0.272

No. 22

𝐿𝐴𝐹𝐹𝑆 : 0.283

No. 5

𝐿𝐴𝐹𝐹𝑆 : 0.286

No.3

1

𝐿𝐴𝐹𝐹𝑆 : 0.289

No. 25

𝐿𝐴𝐹𝐹𝑆 : 0.290

No. 14

𝐿𝐴𝐹𝐹𝑆 : 0.294

The bottom 4

The similarity increases

No. 0

𝐿𝐴𝐹𝐹𝑆 : 0.344

Real

Fake

Mask

No. 21

𝐿𝐴𝐹𝐹𝑆 : 0.373

No. 4

𝐿𝐴𝐹𝐹𝑆 : 0.377

No. 16

𝐿𝐴𝐹𝐹𝑆 : 0.378

00.

F Details of RCR

In the LAA Net [51], a very interesting hypothesis is the pixels on the fake boundary contain both the features of the fake area and the features of the real area, so in our designed RCR, such pixels will be discarded. Figure 12 provides a visual example.

G Testing on more forgery techniques

In addition to several widely used forgery detection datasets mentioned in the paper, we also conducted experiments on other forgery techniques based on the DF40 dataset [80], where real samples were obtained from FF++ and fake samples were constructed using corresponding forgery techniques based on FF++ settings. As shown in the Table 9a, our proposed forgery detection framework has achieved excellent detection results on the vast majority of forgery methods. Meanwhile, Table 9b shows the comparison between our method and other state-of-the-art methods on various forgery techniques in DIFF.

(a) We tested all forgery techniques provided on the DF40 dataset and reported Frame-Level AUC. All results with AUC greater than 80 are highlighted in bold, while those with AUC less than 80 but greater than 70 are underlined.

(b) Comparison with universal deepfake detection methods using the Frame-Level AUC on the DiFF dataset. The notation † indicates models that are designed for deepfake detection, while ‡ signifies models intended for general generated detection. All experiments were trained on the c23 version of FF++. The best result is bolded.

H Combining with SoTA detection method

In this section, we aim to validate the effectiveness of our FIA-USA data augmentation by integrating it with state-of-the-art image-level detection methods. While our comprehensive evaluation currently focuses on LAANet [51] (CVPR 24) due to limited availability of open-source image-level detection implementations, Table 8 reveals significant performance improvements. Notably, the original LAA-Net implementation was trained on raw version FF++ data, whereas our experiments utilize the c23 version. Compared with the baseline implementation, our data augmentation strategy has significantly improved the performance of the original detector. To address requests for detailed comparisons with LAA-Net, our table shows two key findings: First, our method achieves a 7.5% performance advantage over LAA-Net when using identical data augmentation conditions. Second, even without employing RCR (Robust Context Refinement), our approach substantially outperforms LAA-Net's EFPN component, demonstrating the inherent strength of our core methodology.

I Limitations

The proposed detection framework primarily focuses on image-level data augmentation and matching detection architectures. While we categorize image-level artifacts into FIA and USA, and have achieved satisfactory video-level detection performance, we acknowledge that video-level artifacts present more complex challenges including temporal inconsistencies and audio-visual asynchrony - which fall outside the current research scope but represent our intended future research direction.

Second, although recent approaches increasingly incorporate large models' zero-shot capabilities for deepfake

detection, existing work has not yet explored the integration of data augmentation methods with such models due to practical constraints including variations in training methodologies and model scales. This does not mean that our method cannot be applied to large models, but currently we are still unfamiliar with large models and have not fully explored this part.

Regarding experimental validation, we mainly compared and analyzed our proposed method with previous data augmentation methods, which is consistent with the main contribution of our paper . Of course, we also compared state-of-the-art detection methods that are not data augmentation based. Following established practices in data augmentation research, we conducted comprehensive evaluations across four widely adopted network architectures to ensure fair comparison. However, we note that detection performance may vary across different network frameworks depending on their baseline performance and inherent compatibility with deepfake detection tasks. We cannot guarantee that all model frameworks will achieve excellent detection results after using our method.

Finally, while our method demonstrates effectiveness across a substantial majority of forgery methods ( validated on over 58 different techniques ), we acknowledge relatively weaker performance in specific edge cases. Nevertheless, the framework maintains robust detection capability for most common forgery approaches, achieving satisfactory overall performance that meets our research objectives.

J Broader impacts

The proposed framework offers positive societal impacts by enhancing detection of sophisticated face deepfakes, thereby mitigating disinformation campaigns, fake news, and fraudulent content that erode public trust in digital media. Additionally, it safeguards individual privacy by identifying forged facial manipulations that enable identity theft, unauthorized face swapping, and other privacy violations. The method could further support content authenticity initiatives through integration into social media platforms or news verification systems to prioritize legitimate visual content. However, potential negative impacts include the risk of adversaries reverse-engineering the detection framework to refine deepfake generation methods, potentially escalating the adversarial "arms race" between forgery creators and detectors. Furthermore, false positives in detection could lead to unintended censorship, where legitimate content is erroneously flagged as fake.

K More visual examples

From Figure 14 to Figure 16, we demonstrate the fake samples that FIA-USA can generate through different reconstruction methods, including autoencoderr (AE) and facial super-resolution (SR), and multiple types of masks based on three real faces. Figure 17 shows the negative samples randomly generated by FIA-USA during the real training process. Figures 18 to 22 list the images in the DF40 dataset [80] that were misclassified by our method.

Mask type  1

Mask type  2

Mask type  3

Real

AE

Mask

Fake

Real

SR

Mask

Fake

Real

Real

Mask

Fake

Mask type  4

Mask type  5

Mask type  6

Real

AE

Mask

Fake

Real

SR

Mask

Fake

Real

Real

Mask

Fake

Mask type  7

Mask type  8

Mask type  9

Real

AE

Mask

Fake

Real

SR

Mask

Fake

Real

Real

Mask

Fake

Real

Recons tructed

Mask

Fake

Real

Reco nstruc ted

Mask

Fake

Real

Reco nstruc ted

Mask

Fake

9

9香

?香雪

Real

CollabDiff

Blendface

Danet

DDIM

Deepfacelab

DIT

e4e

86

/PF

Facedancer

Faceswap

fomm

heygen

hyperreenact

inswap

mcnet

MidJourney mobileswap

MRAA

one_shot_free pirende r

pixart

RDDM

EXO

Sd 2.1

Sit

simswap

starganv2

styleclip

StyleGAN2

StyleGAN3

StyleGANXL

uniface

VQGAN

Wav2lip

whichfaceisreal

e4s

tpsm

Sadtalker

OSITIONREJECTS

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nBriefly summarize the paper and its contributions. This is not the place to critique the paper; the authors should generally agree with a well-written summary. This is also not the place to paste the abstract—please provide the summary in your own understanding after reading.\n\nThis paper introduces a universal framework for detecting facial deepfakes by focusing on identifying common and generalizable forgery artifacts. The core challenge addressed is the wide diversity of existing deepfake generators and the impracticality of teaching a detector to learn all varied artifact types separately.\n\nThe authors propose categorizing deepfake artifacts into two distinct yet complementary types:\n\nFace Inconsistency Artifacts (FIA): These arise from the difficulty of generating intricate facial details, leading to inconsistencies between complex facial features and their surrounding areas, or from blending boundaries in face swapping.\n\nUp-Sampling Artifacts (USA): These are inevitable traces left by the generator’s decoder during the up-sampling process, as decoding and up-sampling are fundamental operations in facial manipulation generators. The authors claim that all existing deepfakes are observed to exhibit either or both of these artifact types.\n\nTo enable detection models to learn these generalizable artifacts, the paper proposes an image-level pseudo-fake creation framework called FIA-USA. This framework constructs fake samples by simulating both FIA and USA without introducing less-general artifacts. Specifically, it reconstructs target faces to simulate USA using autoencoders (AE) or super-resolution (SR) models, and utilizes image-level blending with multiple mask types on diverse facial regions to create FIA. The blending process includes macro-editing masks (based on 4 or 81 facial keypoints) and micro-editing masks (randomly combining facial features like eyes and mouth).\n\nBeyond the data augmentation, the framework also includes two complementary techniques to maximize efficacy:\n\nAutomatic Forgery-aware Feature Selection (AFFS): This streamlines feature dimensionality and augments feature discernment by adaptively selecting channels in Feature Pyramid Networks (FPN) that are sensitive to forgery regions, thereby reducing feature redundancy.\n\nRegion-aware Contrastive Regularization (RCR): By juxtaposing features of manipulated regions against authentic regions, RCR enhances the model's focus on upsampling traces and inconsistencies between different areas. This includes both Patch-level Contrastive Regularization (PCR) and Image-level Contrastive Regularization (ICR).\nExtensive experiments on seven widely used deepfake detection datasets, encompassing over 58 distinct forgery methods, validate the superior efficacy and robustness of the proposed method. The results demonstrate that a standard image classifier trained with FIA-USA pseudo-fake data can generalize well to unseen deepfakes, even outperforming state-of-the-art methods, including those based on latent space augmentation. The paper concludes that this targeted approach, by focusing on fundamental artifact patterns, can help address challenges in current detection methods and provide a promising foundation for more adaptable deepfake detection systems.\n\nSTRENGTHS AND WEAKNESSES:\nQuality\n\nStrengths:\n* Robust Experimental Validation: The paper conducts extensive experiments on seven widely used Deepfake detection datasets, encompassing over 58 distinct forgery methods, including identity swapping, expression reenactment, generative face synthesis, and attribute editing. This broad evaluation validates the superior efficacy and robustness of their method.\n\n*  Performance: The proposed method is shown to outperform other state-of-the-art (SOTA) approaches on the majority of traditional deepfake datasets and achieves competitive results on others, such as CDF-v1. Notably, it demonstrates strong generalization capabilities on generative deepfake datasets like DiFF and DF40, often surpassing previous methods by significant margins (e.g., 9.7% over RECCE and 23.4% over SBI on DF40 average AUC).\n\n*  Robustness: The method exhibits superior robustness to unseen perturbations like Gaussian Blur, Block Wise, Change Contrast, and JPEG Compression, as quantified by the reduction rate of Video-Level AUC.\n\n* Ablation Studies: Detailed ablation experiments confirm the effectiveness of each component (FIA-USA, AFFS, RCR), showing a progressive degradation in performance when individual components are removed, which strengthens the argument for their contributions.\n\n* Technical Soundness: The categorization of artifacts (FIA and USA) is grounded in a systematic analysis of shared technical frameworks in existing deepfake algorithms. The methodology for constructing pseudo-fakes (FIA-USA) is well-articulated, involving multi-type mask synthesis and multi-modal reconstruction using autoencoders and super-resolution models to simulate realistic artifacts. The integration of AFFS and RCR further refines feature learning for robust detection.\n\nWeaknesses:\n\n* Limited Evaluation of Full Video-Level Artifacts: While the paper achieves satisfactory video-level detection performance and acknowledges video-level artifacts as a future direction, the primary focus remains on image-level data augmentation and detection architectures. This leaves complex video-level challenges like temporal inconsistencies and audio-visual asynchrony largely unexplored in the current framework .\n\n* Unexplored Large Model Integration: The authors note that they have not yet explored the integration of their data augmentation methods with large models' zero-shot capabilities, citing practical constraints and unfamiliarity with large models. This could be a limitation in light of recent advancements in large generative models.\n\n* Performance in Edge Cases: Although the method is effective across a substantial majority of forgery methods (over 58), the authors acknowledge relatively weaker performance in specific edge cases. While they state it maintains robust detection for most common approaches, these edge cases could be critical.\n\nClarity\n\n\nStrengths:\n\n* Detailed Methodology: The core components, especially FIA-USA, AFFS, and RCR, are explained in detail, including their motivations and mathematical formulations. The appendix further elaborates on mask generation, reconstruction models, and the theoretical basis of AFFS.\n\n* Sufficient Experimental Details: The paper provides comprehensive details for reproducibility, including the backbone network architecture, optimizer, batch size, learning rate, training epochs, video sampling, data augmentation pipeline, loss coefficients, and hardware used (NVIDIA 3090). Data splits and datasets are clearly specified, including how FF++ was used for training.\n\nWeaknesses:\n\n* Lack of Open-Source Code: The authors explicitly state that the code is \"currently not open source\". While the paper provides extensive details, the absence of publicly available code can hinder direct reproducibility and further research by the community, despite the detailed instructions. This is noted as a \"No\" in the NeurIPS checklist .\n\n* Limited Compute Resource Details: While the GPU type is specified (NVIDIA 3090), more granular details such as memory usage and exact execution times for experiments are not provided. While authors claim this is unlikely to affect reproducibility for experiments of this scale, it could be beneficial for researchers with different computational resources.\n\nSignificance\n\nStrengths:\n*  The paper tackles the increasingly important topic of deepfake detection, driven by the rapid development of AI generation techniques and the associated risks of privacy violations and misinformation spread. Developing a universal deepfake detector is highlighted as an \"urgent priority\".\n\n* Generalizable Approach: By focusing on common and generalizable artifacts (FIA and USA) instead of specific ones, the framework moves towards a universal detection capability, enhancing the generalization of detection models across diverse and unseen deepfake variations. This is a significant step beyond methods tailored for specific forgery techniques.\n\nOriginality\n\nStrengths \n\n* Novel Pseudo-Fake Creation Framework: The paper introduces a \"novel image-level pseudo-fake creation framework\" (FIA-USA) which is a significant original contribution. Unlike prior methods that often focus on isolated artifact types or single-scale modifications, FIA-USA is a dual-artifact collaborative enhancement framework designed for universal detection, addressing both global/local and facial/regional alterations, and specifically introducing USA, which was less emphasized previously.\n\n* Complementary Techniques: The introduction of Automatic Forgery-aware Feature Selection (AFFS) and Region-aware Contrastive Regularization (RCR) as complementary techniques further contributes to the originality. AFFS is described as a \"statistically-driven feature compression paradigm\" that quantifies channel-wise sensitivity to forgery regions, and RCR explicitly models divergence between forged and authentic regions through multi-granularity contrastive learning.\n\nWeaknesses:\n* Building on Existing Blend Paradigm: While FIA-USA injects new vitality, it builds upon an existing blend-based data augmentation paradigm. The originality lies in how they extend this paradigm (multiple masks, USA introduction) rather than inventing blending from scratch. This is a minor point, as the authors clearly explain their advancements over previous blending methods.\n\nQUESTIONS:\nQuestion: The paper states that its core methodology for detecting deepfakes primarily relies on analyzing image-level artifacts, namely Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). While the evaluation includes \"satisfactory video-level detection performance\", the framework acknowledges that \"video-level artifacts present more complex challenges - including temporal inconsistencies and audio-visual asynchrony - which fall outside the current research scope\". How does the current image-level focused approach implicitly capture or respond to these video-level inconsistencies?  Is this a future area of research interest?\n\nProposed Suggestion: For the authors to strengthen the paper's contribution towards a \"universal detection framework\", it would be beneficial to elaborate on the specific temporal or inter-frame cues (if any) that their current image-based approach implicitly leverages to achieve reported video-level AUC scores. If applicable, could the authors provide a brief discussion or hypothesis on how the FIA-USA framework might be extended or complemented in future work to directly address these complex video-level artifacts (e.g., inconsistencies over time or audio-visual discrepancies), rather than solely relying on aggregating frame-level detections? This would clarify the path towards a truly universal solution for all types of deepfakes, including video-specific forgeries.\n\nQuestion: The paper explicitly acknowledges that it has \"not yet explored the integration of data augmentation methods with such [large] models due to practical constraints\". This implies a current boundary to the method's general applicability, especially as larger models increasingly dominate deepfake generation and detection.\n\nProposed Suggestion: While a full exploration of large model integration is noted as future work, could the authors expand on the primary challenges they anticipate when adapting or integrating their FIA-USA framework, AFFS, and RCR with large foundation models (e.g., Vision Transformers, very large Diffusion Models)? Are these challenges primarily computational, architectural, or related to the nature of artifacts produced by and detectable by these larger models? Providing this insight would enhance the paper's forward-looking perspective and set a clearer agenda for future research in this area.\n\nQuestion: The paper states that all experiments were conducted on \"a single NVIDIA 3090\" and indicates that resource requirements are \"minimal\". However, specific metrics like execution time for training/inference or precise memory consumption are not provided in the main text or the appendix beyond this general statement.\n\nProposed Suggestion: To ensure full experimental reproducibility and provide practical insights for researchers and practitioners considering deploying this method, could the authors provide approximate training times (e.g., per epoch or total) and inference times (e.g., frames per second) for their model on a standard dataset (e.g., FF++) using the specified hardware? Additionally, GPU memory consumption (e.g., VRAM usage) would be a useful detail. This specific data would allow others to more accurately gauge the computational feasibility and scalability of the proposed approach.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

comments: SUMMARY:
The paper proposes a universal deepfake detection framework that identifies and exploits two general forgery artifacts—FIA and USA—across various types of deepfake generation. The authors introduce a novel dual-artifact augmentation technique (FIA-USA) for pseudo-deepfake sample synthesis, along with two training enhancements: AFFS and RCR. Extensive experiments demonstrate great generalization and robustness compared to SOTA methods.

STRENGTHS AND WEAKNESSES:
### Strengths
1. The FIA-USA data augmentation strategy is simple yet remarkably effective.
2. Extensive experiments on a wide range of datasets and rigorous ablation studies.
3. The method has good generalizability to unseen methods and robust under perturbations.

### Weaknesses
1. The brackets in Eq. 3 are not closed, making it unclear.
2. Although FIA-USA shows great effectiveness, it may heavily relies on the quality of pseudo-fake generation; performance might drop when faced with adversarial attacks or higher-fidelity forgeries.
3. The paper lacks a formal theoretical justification for the strong cross-domain generalization performance of FIA and USA. Providing such analysis would strengthen the claims.
4. There lacks a detailed analysis about the results in Tab. 6a to 6c.

QUESTIONS:
1. Why you only choose (2.5, 1, 0.25) as your hyperparameters ($\lambda_1, \lambda_2, \lambda_3$)?
2. See Weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWhile this paper contains minor limitations, they do not detract from its overall strengths. The logical flow is exceptionally clear, experimental validation is comprehensive with impressive performance, and supplementary materials (particularly the visual analysis of misclassified samples and systematic evaluation of mainstream deepfake methods) provide substantial value. I therefore recommend a Weak Accept. Should the authors address Concern #1 and #2 raised in my review, I would be more willing to elevate my score.\n\nSTRENGTHS AND WEAKNESSES:\nStrenth:\nThis paper categorizes deepfake artifacts into two types: FIA (Face Inconsistency Artifacts): Arising from detail discrepancies between forged faces and backgrounds (e.g., lighting/texture discontinuities), USA (Up-Sampling Artifacts): Introduced by the up-sampling process in generator decoders (e.g., interpolation traces). By analyzing technical workflows of mainstream forgery algorithms (e.g., Full Face Synthesis and Face Swapping), it demonstrates that all existing methods inevitably produce at least one of FIA/USA artifacts. Building on this foundation, the paper proposes FIA-USA data augmentation, whose three components (Multi-Type Mask Synthesis (MTMS), Multi-Modal Reconstruction (MMR), Random Artifact Combination (RAC)), collectively cover potential artifact types in deepfakes comprehensively, addressing the limitation of prior augmentation methods that only simulate FIA locally or globally. Additionally, the complementary AFFS and RCR techniques fully leverage FIA-USA’s capabilities. Validation across 7 datasets (including traditional and generative deepfakes) encompassing 58 distinct forgery methods proves the approach’s generalizability. Though the framework involves multiple components, the authors’ thorough ablation studies confirm each element’s effectiveness, and the paper’s logical structure remains exceptionally clear.\n\nWeakness:\n1.The FIA-USA framework involves numerous components, and understanding its full operational procedure necessitates referring to the algorithmic details in the appendix. To improve accessibility, the complete algorithm should be moved to the main text. Additionally, the initialization process of AFFS relies on training dataset data—a critical implementation detail currently buried in the appendix that warrants inclusion in the main body.\n\n2.Although the proposed method still relies on the frameworks of SBI and Face X-ray, the improvements derived from the authors' artifact analysis of mainstream deepfake techniques are compelling. As clearly demonstrated in Table 1, their approach exhibits significant distinctions from prior works. However, I am perplexed: the method appears to be an image-domain enhancement—why can it be applied to video-level deepfake detection? More specifically, I wonder whether the video-level experiments use the maximum value or the average value of multi-frame results.\n\n\n3.While the method demonstrates compelling efficacy across most forgery techniques, its detection performance remains notably weak for certain generative approaches, particularly StyleGANXL, DIT, and others. Overall, the performance is still impressive.\n\n4.The framework contains too many components, making the method somewhat complex. Although the author has devoted a lot of effort to elaborating on each component, it still needs to be carefully read in conjunction with the appendix and have a certain understanding of data augmentation work in the RGB field.\n\nQUESTIONS:\n1.  It is recommended to relocate selected content from the appendix to the main body.\n2.  It is necessary to clarify the details of video-level detection.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the growing problem of deepfakes by suggesting a more general way to detect them, rather than trying to identify every specific forgery technique. The authors noticed that most deepfakes, regardless of how they’re created, tend to have two main types of flaws: Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA). \n\nThe author generates \"pseudo-fake\" training images that deliberately include only these two types of artifacts. The idea is that by training a detector on these fundamental flaws, it becomes much better at spotting real deepfakes it hasn't seen before. They also added a couple of supporting techniques: Automatic Forgery-aware Feature Selection (AFFS) helps the detector zero in on the most relevant forgery clues, and Region-aware Contrastive Regularization (RCR) helps it distinguish between genuine and manipulated areas. Essentially, they're showing that a focused approach on universal artifacts can lead to a more robust and adaptable deepfake detection system.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1) This paper seems pretty solid in terms of quality. The authors put in the work to systematically analyze existing deepfake algorithms to pinpoint common artifacts, which is a smart move. They propose a novel pseudo-fake creation framework that's designed specifically to introduce these general artifacts, avoiding less common ones, which sounds really intentional and well-thought-out.\n2) The paper definitely brings some novel ideas to the table. While data augmentation for deepfake detection isn't new, their specific approach of categorizing artifacts into FIA and USA and then explicitly designing a pseudo-fake generation framework (FIA-USA) to introduce both types is novel. They highlight that existing methods typically focus on isolated artifact types or single-scale modifications, which limits their generalization. Their Multi-Type Mask Synthesis for blending and Multi-Modal Reconstruction for USA, leveraging autoencoders and super-resolution, seem quite original in their combination and purpose.\n\nWeaknesses:\n1) While the FIA-USA framework and its components are innovative, the underlying concepts of blending and reconstruction for data augmentation have been explored before in deepfake detection.\n2) While the experiments are extensive, they mention that some results are \"cited from\" or \"our reproduction results\" for other methods.\n\nQUESTIONS:\nIn the limitations, the authors acknowledge \"relatively weaker performance in specific edge cases\" among the 58+ forgery methods tested. Could they provide more specific examples of which forgery methods constitute these \"edge cases\"?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}