Can We Leave Deepfake Data Behind in Training Deepfake Detector?

Authors: jikang  cheng
zhiyuan  yan
ying  zhang
yuhao  luo
zhongyuan  wang
chen  li
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-08 10:32:37

Abstract: The generalization ability of deepfake detectors is vital for their applications in real-world scenarios. One effective solution to enhance this ability is to train the models with manually-blended data, which we termed ''blendfake'', encouraging models to learn generic forgery artifacts like blending boundary. Interestingly, current SoTA methods utilize blendfake $\textit{without}$ incorporating any deepfake data in their training process. This is likely because previous empirical observations suggest that vanilla hybrid training (VHT), which combines deepfake and blendfake data, results in inferior performance to methods using only blendfake data (so-called "1+1<2"). Therefore, a critical question arises: Can we leave deepfake behind and rely solely on blendfake data to train an effective deepfake detector? Intuitively, as deepfakes also contain additional informative forgery clues ($\textit{e.g.,}$ deep generative artifacts), excluding all deepfake data in training deepfake detectors seems counter-intuitive. In this paper, we rethink the role of blendfake in detecting deepfakes and formulate the process from "real to blendfake to deepfake" to be a $\textit{progressive transition}$. Specifically, blendfake and deepfake can be explicitly delineated as the oriented pivot anchors between "real-to-fake" transitions. The accumulation of forgery information should be oriented and progressively increasing during this transition process. To this end, we propose an $\underline{O}$riented $\underline{P}$rogressive $\underline{R}$egularizor (OPR) to establish the constraints that compel the distribution of anchors to be discretely arranged. Furthermore, we introduce feature bridging to facilitate the smooth transition between adjacent anchors.  Extensive experiments confirm that our design allows leveraging forgery information from both blendfake and deepfake effectively and comprehensively. Code is available at https://github.com/beautyremain/ProDet.

Main paper: 
 introduction: 
 In recent years, the development of deepfake 3 has aroused significant concerns regarding privacy and security among the public. Deepfake detection aims to identify whether a face from an unknown source has been manipulated by deepfake techniques. Most detection methods perform promisingly when trained and tested on identical manipulations. However, given the unpredictability and complexity of real-world scenarios, it is increasingly critical for these methods to generalize and identify previously unseen manipulations. One particularly effective and mainstream solution is data synthesis, which involves image blending to create new training forgery samples [7, 28, 39, 27, 55] . Figure 1 : 1a: The detection performance experiences an abnormal decline when naively combining deepfake and blendfake as the negative sample for training, even though the forgery information is enriched in this process. 1b: Illustration Example for illustrating latent space organization. With progressively organized latent space (ours), information in both deepfake and blendfake is effectively leveraged, and deepfake samples become easier to distinguish from the real. See Fig. 4a and 9 for experimental results of actual latent-space distribution.Specifically, these methods aim to synthesize pseudo-fake data that share forgery similarities with the actual deepfake data. The pseudo-fake data is generated by blending a real face with a near-landmark fake face [7] , near-landmark real face [55, 27] , or a real face itself [28, 39] . Considering the "deep" process is not involved during pseudo-fake data generation (without utilizing any deep network), we term these manually-blended pseudo-fake faces as blendfake faces to differentiate them from deepfake faces. The learned information from training on blendfake is similar to that on Deepfake: by simulating the blending operations in the Deepfake generation process, detectors trained on blendfake can identify generalized blending clues. In comparison to deepfake, blendfake necessitates the use of two images with closely matching landmarks (or with itself) for generation, thereby curtailing its practical utility. However, this also means that blendfake contains fewer forgery clues compared with Deepfake and its distribution is closer to that of real images, making blendfake a conveniently obtained hard sample for detector training.In this paper, we commence with a question that many researchers may ponder: Can the blendfake face entirely substitute the actual AI-generated deepfake face in training deepfake detectors? As shown in Fig. 1a , considering the empirically inferior performance of using both deepfake and blendfake for training, exclusively utilizing blendfake as the negative sample becomes the mainstream scheme for data-synthesis methods [39, 27] , where actual deepfake data is deliberately excluded. However, given that actual deepfake data is widely available in real-world scenarios, and it should contain extra forgery clues (e.g., generative artifacts in deep net) absent in blendfake, training a deepfake detector without using any deepfake data seems counter-intuitive. Therefore, we argue that the significance of deepfake samples is underestimated due to insufficient exploration of how to appropriately utilize these samples. As illustrated in Fig. 1b , we attribute the abnormal efficiency of VHT to the unorganized latent-space distribution, while a carefully organized latent-space distribution is proven advantageous to network performance [26, 53, 2] . Specifically, since deepfake and blendfake are entangled while also distinctly containing forgery clues, the direct VHT may fail to disentangle the learned representation in the latent space. Inspired by introducing inductive bias that can reflect the structure of the underlying data (i.e., real, blendfake, and deepfake in our case) is beneficial to feature robustness and generalization ability [19, 13, 23] , we want to organize the latent space based on an inductive observation. As shown in Fig. 2 , it can be observed that the transition from real to fake is a progressive process. Based on the observation, we posit that the latent space distribution from real to blendfake to deepfake should also be conceptualized as a progressive process, where real can gradually become deepfake through a continuous transition. That is, in the latent feature space, the distribution of blendfake and deepfake can be regarded as "oriented pivot anchors" in the progressive transition from real to fake. Achieving this progressive organization can disentangle learned representations of blendfake and deepfake data, thereby enabling more effective use of forgery clues from both data types. Consequently, we propose ProDet (deepfake Detection with Progressive transition) to achieve the progressive transition in the latent space that we envisioned. As shown in Fig. 2 , the oriented anchors used in this paper include four types: real, SBI [39] , CBI [27] , and fake, where both SBI (self-blended image) and CBI (cross-blended image) represent blendfakes. Within our framework, the anticipated progressive transition can be concretely understood as a progressive accumulation of forgery information, encompassing three specific aspects: the presence of blending clues, identity inconsistency, and generative artifacts of deep neural nets. To implement the aforementioned progressive effect, we consider ordering the distribution of these transition anchors by allocating different hard labels. Therefore, we introduce a module named Oriented Progressive Regularizer (OPR) to fully utilize the forgery information in all oriented anchors. OPR contains a triplet one-hot classifier corresponding to the forgery attributes and a projector to re-dimension the output auxiliary attention map, which is aligned with the feature dimensions and serves to guide the final detection results. Based on the inherent forgery attributes of different anchors, the label of the triplet one-hot output can be viewed as a cumulative three-bit binary number. Consequently, we establish the constraints that compel the distribution of anchors to be arranged in a progressive manner. To "build bridges" between the distinct adjacent anchors, we propose feature bridging to simulate continuous transitions between adjacent anchors. Specifically, feature bridging conducts the mixup to the extracted features of two adjacent anchors at random ratios, predicting mixed labels for these blended features to facilitate a smooth and continuous transition process. Additionally, to enhance the progressive transition process between different anchor distributions, we further propose the transition loss, which constrains an extracted feature to transform into its adjacent distribution after undergoing noise addition and specific mapping.2 Related Works 2.1 Deepfake Detection Toward Generalization Ability Deepfake detection aims to determine the authenticity of images potentially manipulated by deepfake approaches. Various detection methods concentrate on distinct facial features, such as movements of the lips [22] and the consistency of facial action units [4] . In parallel, several studies aim to optimize neural network architectures, including MesoNet [1] , Xception [12] , and CapsuleNet [34] , to elevate detection efficacy. Based on the observation of model bias in the detector, many methods [51, 30] are proposed to remove certain general biases found in the forgery samples. Moreover, Chen et al. [20] construct a test sample-specific auxiliary task to improve the detection performance. To capture local-to-global information, Guan et al. [8] propose LTTD to focus on the temporal information within the local sequence. In the spectral domain, techniques like SPSL [31] and SRM [33] utilize phase spectra and high-frequency disturbances to augment the forgery details used in training. These strategies [31, 20, 8, 33, 22, 4, 24, 30, 51, 10, 6] target particular weaknesses found in deepfake technologies, and are highly effective in identifying deepfake images characterized by these flaws. 
deepfake detectors with blendfake faces: 
 Blendfake refers to a series of forgery detection methods that employ data generation schemes. These methods [7, 28, 39, 27, 55] manually create new fake samples that imitate general forgery clues that are similar to deepfake based on real samples from the dataset. Specifically, SLADD [7] dynamically crops a portion of a fake image and blends it onto a real image, thereby generating a fake sample that is more challenging to distinguish. Both CBI [27] and I2G [55] involve randomly selecting a real image as the base image, and then searching for another source real image with closely matching facial landmarks. Following a color transfer, the face of the source real image is attached to the base image according to the landmarks, ultimately producing a new fake sample. The resulting blendfake image is cross-face blended and exhibits inconsistencies in identity and also blending clues. FWA [28] and SBI [39] omit the landmark matching process, instead generating two different versions of a base real image through distinct transformations, and subsequently blending these two versions to create a new fake image. The blendfake images generated in this way contain only blending clues, causing their distribution to resemble real images more closely.Owing to their superior implementation designs, SBI and CBI have demonstrated advanced performance in deepfake detection. Notably, they chose to use the generated blendfake solely instead of deepfake during training, thereby completely excluding deepfake from the detector training process. 
methodology: 
 In this paper, we aim to devise an appropriate scheme to simultaneously leverage the forgery information inherent in both deepfake and blendfake, thereby training a more effective detector termed ProDet. Firstly, we consider constraining the oriented anchors, that is, the real, blendfake, and deepfake features, to be progressively distributed in a discrete manner. Then, we propose feature bridging to simulate the continuous and progressive transition between adjacent anchors. Lastly, a transition loss is introduced further to enhance our simulation of the progressive transition process.As shown in Fig. 3 , our method is simple yet effective in organizing the latent space in a progressive manner.To regularize a progressive distribution in latent space, we design a module termed Oriented Progressive Regularizor (OPR) using multi-attribute classification constraints. Specifically, blendfake images generated by both SBI [39] and CBI [27] are introduced as oriented pivot anchors to construct the process of progressive transition. Since SBI contains no clues of ID inconsistency compared to CBI, it is notably more difficult to identify SBI as fake data. Consequently, in a mini-batch during training, we incorporate four types of samples aligned with the same frame in the same video. They can be organized progressively from real to fake as follows: real (I r ), SBI (I s ), CBI (I c ), and deepfake (I d ), as depicted in Fig. 3) Generative artifacts in I d . As illustrated in Fig. 3 , it is straightforward to assign the ground-truth attribute label groupsEQUATIONto these oriented anchors, thereby enabling the progressive forgery accumulation through multi-labeling. Therefore, each image I ∈ R H×W ×3 will be first extracted a feature F ∈ R h×w×c by the backbone encoder, then predicted with three forgery attributes:EQUATIONTo effectively leverage information from OPR during inference and obtain a conclusive deepfake detection result, the outputs of multi-attribute classification are re-dimensioned to an auxiliary attention map M ∈ R h×w×c as:EQUATIONwhere P is a projector that re-dimensions the classification results. M is aligned and integrated with the extracted feature F and provides supplemental information that improves the final deepfake detection outcome y. Therefore, y can be written as:EQUATIONwhere ⊙ denotes element-wise multiplication, C f denotes the final classifier for deepfake detection.In fact, several alternative strategies can also achieve a similar multi-attribute classification for distribution anchoring, including multi-class, multi-label, and triplet binary (ours) classification. The multi-class classification is actually detecting different types of negative data instead of perceiving their forgery accumulation. Therefore, it cannot organize the latent space in a progressive manner as we anticipate. The multi-label strategy with three classes can be interpreted as using one vector to represent three parallel forgery attributes, which may achieve a comparable progressive accumulation. However, it is constrained by the mapping capability of a single detection head, limiting its effectiveness in distinguishing among different forgery attributes. In contrast, the deployed triplet binary strategy can effectively differentiate the distributions of the oriented anchor with progressive accumulation. The experimental results can be found in Sec. 4.3.By assigning progressively multi-labels to different data distributions, we successfully placed discrete anchors for the real-to-fake progressive transition in the latent space. However, as analyzed in Sec. 1, the progressive transition from real to fake should be continuous rather than discrete. Therefore, we consider simulating this continuous progressive transition by constructing bridges between adjacent anchors, thus further clarifying the orientation. Here, we propose feature bridging to fill the continuous latent space between oriented anchors. Specifically, given four extracted features {F r , F s , F c , F d } and their one-hot label groups {A r ′ , A s ′ , A c ′ , A d ′ } in an aligned mini-batch, we conduct mixup to each pair of adjacent features and labels. We take bridging real to SBI as an example:EQUATIONwhere M ix(a, b) = α×a+(1-α)×b, and α ∈ [0, 1] is a random mixing ratio. It is important to note that, to ensure that the information mixed during the feature bridging process is the desired deepfakerelated context and to exclude interference from deepfake-unrelated content (e.g., background, identity, and pose), the images used for mixing should be aligned to the same video frame. Therefore, we can assign the forgery attribute label of F r,s as A ′ r,s to simulate the continuous transition between two adjacent anchors. Since feature bridging across skip-neighbor anchors introduces short-cut transition paths, thereby disrupting the oriented progressive transition in the "real-SBI-CBI-deepfake" process, we only conduct feature bridging on two adjacent anchors instead of any arbitrary pair.Transition Loss. Given that a real distribution can be progressively transitioned to a fake distribution, features augmented with noise should be mapped to a more fake distribution. Therefore, we take this noise-augmented transition as an additional constraint to further enhance the simulation of progressive transition. Specifically, we first randomly generate random noise N that has the same shape as F. Subsequently, we introduce a group of convolutional layers to merge and transform F and N, producing a new feature F ′ ∈ R h×w×c . We named the above process as feature transition (T ). Since F ′ = T (N, F) is expected to be transitioned to a more fake anchor distribution, we introduce the following transition loss:EQUATIONwhere F i represents the i-th feature in the mini-batch {F r , F s , F c , F d }, thus F i+1 denotes the subsequent feature to F i .Oriented Loss. Oriented loss is deployed to both discrete anchors and continuous bridging features. Given a feature F and its one-hot label groups A ′ = {a 0 ′ , a 1 ′ , a 2 ′ } either directly extracted or mixed, its three attributes A = {a 0 , a 1 , a 2 } predicted by C t are constrained respectively. Formally, the oriented loss can be written as:EQUATIONwhere A T denotes the transpose matrix of A.Detection Loss. We deploy detection loss on the discretely oriented anchors to constrain the final detection result used in both training and inference, which is formulated as:EQUATIONwhere y ′ is the label that takes both blendfake and deepfake as the negative samples.Therefore, the overall loss function in our method is: EQUATIONy ′ = {y ′ r , y ′ s , y ′ c , y ′ d }, A ′ ∼ S do extract features for each data F = {F r , F s , F c , F d } = Backbone(I) conduct feature bridging (FB) between adjacent features F, Â′ = FB(F, A ′ ) anchor distributions via oriented loss L o = BCELoss(C a ({ F, F}), { Â′ , A ′ }) enhance transition simulation via transition loss L t = i,j ||T f (N, F i∈{r,s,b} ) -F j∈{s,b,d} 
experimental setting: 
 Datasets. To extensively explore the generalization ability of deepfake detectors, the most advanced and widely used deepfake datasets are applied in our experiments. FaceForensics++ (FF++) [37] is constructed by four forgery methods including Deepfakes (DF) [15] , Face2Face (F2F) [44] , FaceSwap (FS) [18] , and NeuralTextures (NT) [43] . FF++ with High Quality (HQ) is employed as the training dataset for all experiments in our paper. The base images to generate blendfake images are also from FF++ (HQ) real. For cross-dataset evaluations, we introduce Celeb-DF-v1 (CDFv1) [29] , Celeb-DF-v2 (CDFv2) [29] , DeepFake Detection Challenge Preview (DFDCP) [16] , and DeepFake Detection Challenge (DFDC) [16] . Implementation Details. For preprocessing and training, we strictly follow the official code and settings provided by DeepFakeBench [52] to ensure fair comparison. EfficientNetB4 [42] is employed as the backbone of our detector. The trade-off parameters are set to β = 1 and γ = 10. The Adam optimizer is used with a learning rate of 0.0002, epoch of 20, input size of 256 × 256, and batch size of 24. Feature Bridging is deployed after a warm-up phase of two epochs. Frame-level Area Under Curve (AUC) and Equal Error Rate (EER) [52] are applied as the evaluation metrics of experimental results. All experiments are conducted on two NVIDIA Tesla V100 GPUs.Table 1 : Cross-dataset evaluations (AUC) from FF++ [37] (in-dataset) to CDFv1 [29] , CDFv2 [29] , DFDCP [17] and DFDC [17] (cross-dataset). C-Avg. denotes the average value of cross-dataset results. The best results are highlighted in bold. Cross-dataset improvements compared with the previous best one are written in pink. See Appendix for comparisons with video-based methods.Venue FF++ CDFv1 CDFv2 DFDCP DFDC C-Avg.Xception [12] CVPR'17 0.9637 0.7794 0.7365 0.7374 0.7077 0.7403 Meso4 [1] WIFS'18 0.6077 0.7358 0.6091 0.5994 0.5560 0.6251 FWA [28] CVPRW'18 0.8765 0.7897 0.6680 0.6375 0.6132 0.6771 EfficientB4 [42] ICML'19 0.9567 0.7909 0.7487 0.7283 0.6955 0.7408 Capsule [34] ICASSP'19 0.8421 0.7909 0.7472 0.6568 0.6465 0.7104 CNN-Aug [46] CVPR'20 0.8493 0.7420 0.7027 0.6170 0.6361 0.6745 X-ray [27] CVPR'20 0.9592 0.7093 0.6786 0.6942 0.6326 0.6787 FFD [14] CVPR'20 0.9624 0.7840 0.7435 0.7426 0.7029 0.7433 F3Net [36] ECCV'20 0.9635 0.7769 0.7352 0.7354 0.7021 0.7374 SPSL [31] CVPR'21 0.9610 0.8150 0.7650 0.7408 0.7040 0.7562 SRM [33] CVPR ' In Tab. 1, we provide extensive comparison results with existing state-of-the-art (SoTA) deepfake detectors based on DeepFakeBench [52] , where all methods are trained on FF++ (HQ) and tested on other datasets. The 18 methods provided in the table are within the benchmark with the exact same experimental setting as our method. EfficientB4 can be treated as the baseline of only using deepfake data. X-ray (CBI) and SBI are the baselines of only using one specific type of blendfake data. Since we effectively incorporate SoTA generative methods [39, 27] and actual deepfake data, our method outperforms other detectors in all evaluated cross-data metrics.In Tab. 2, we provide the baseline results in the upper part. Specifically, Blendfake-only (BF-only) represents using only blendfake data (i.e., CBI [27] and SBI [39] ) to train the deepfake detector. Vanilla Hybrid Training (VHT) represents simply incorporates all three data (i.e., Deepfake, CBI [27] , and SBI [39] ) into training. It can be observed that VHT is outperformed by BF-only in all crossdataset metrics, which demonstrates the empirical basis of solely using blendfake data. Overall Ablation. In the middle part of Tab. 2, we provide the results of removing each proposed component sequentially to evaluate their effectiveness. L t can enhance the simulation of the progressive transition and thus improve the overall detection performance. Meanwhile, Feature Bridging (FB) plays a crucial role in filling the continuous space between discrete anchors. In the case of w/o L o , OPR cannot constrain the anchors to organize progressively and can be treated as an operation of feature self-attention, while feature bridging functions as a simple manifold mixup. The inferior performances of w/o L o demonstrate the effectiveness of progressively organizing latent space, rather than relying on simple tricks that might coincidentally improve performance. Effect of Classification Strategies. In Sec. 3.1, we analyze the effect of alternative strategies for multi-attribute classification in OPR. Here, we conduct experiments to validate the performance of three different strategies, that is, Multi-class (M-C), Multi-label (M-L), and Triplet Binary (TB) strategies. The results in the lower part of Tab. 2 show that M-C performs better when in-dataset while cross-dataset performance is only marginally improved. This is because the disentanglement of one-hot values allows deepfake learning independently but does not utilize both datasets for crossdataset generalization. M-L performs similarly to the proposed TB since it can achieve progressive transition. Still, our method exhibits superior effectiveness considering its improved capability in discerning different forgery attributes.Progressively Leveraging Data. To further verify the superiority of the progressive transition distribution, we explore the effects of various hybrid training schemes from an experimental perspective. As shown in Tab. 3, we consider progressively incorporating pivot anchor data into the training samples from real to deepfake, and augmenting anchors with feature bridging (FB). Overall, by learning a progressive transition process, our method is capable of effectively utilizing forgery information contained within both blendfake and deepfake data.To comprehensively evaluate the robustness against unseen perturbations of our method, we conduct experiments from two perspectives, that is, feature robustness (mPD) and detection robustness (AUC). In Tab. 4, we provide the detailed performance on the three types of distinct unseen perturbations, that is, Block-wise masking (Block-wise), Gaussian noise (Noise), and Shifting (Shift). All perturbations are applied with random intensity ten times. Namely, Block-wise is applied with a ratio of 0.1 and 4 × 4 grids. Noise is applied with a random variance selected from the range of 10 to 50. Shift is applied with the range of -50 to 50 on both x and y axes. In Fig. 5 , we further provide the robustness comparisons with multiple perturbation intensities. The superior robustness of our method stems from the organized latent space, which allows the network to leverage the rich information in hybrid training more effectively.To obtain features with dimensions that can be directly visualized, we cleverly construct toy models to facilitate our analysis of the learned feature distribution. The reasons for not applying clustering visualization methods (e.g., t-SNE [45] ) are demonstrated in Appendix. Specifically, we modify the architecture of the backbone to extract features from 512 to 2 dimensions. Subsequently, we train two toy models using VHT and our approach while keeping all other settings consistent. Based on these toy models, we can easily visualize the actual learned features directly in the 2D space. Feature Organization. In Fig. 4a , it can be observed that the distribution of features extracted by our method is aligned with our anticipated progressive transition. Simultaneously, our method enhances the separation between real and fake features, leading to improved clustering and more effective detection. In contrast, the features from VHT are unorganized and highly entangled, which is detrimental to both decision-making and feature generalization ability. Feature Regularity. Feature regularity measures the domain shifting issue in the latent space, while a better regularity implies that the learned features are more accurate, robust, and generalizable [41, 5, 54] . To analyze the regularity of the latent space organization achieved by our progressive transition scheme, we apply random unseen perturbations to each data point and calculate a new metric named Perturbed Distance (PD). A smaller PD suggests that the image with its perturbed versions is mapped more closely in the latent space, indicating superior feature regularity and smoothness. Subsequently, mPD denotes the mean PD in a distribution. See Appendix for the precise formulation and definition of PD and mPD. As shown in Fig. 4b , the result demonstrates that our scheme can organize the latent space with enhanced regularity, which is beneficial for generalization. It also indicates the improved robustness of our method against unseen perturbations.To further establish the superiority of the proposed organization of latent space, we conduct comparisons with other feasible latent space distributions, including 1) Real to Deepfake to Blendfake transition (R2D2B). 2) Like a circle, blendfake and deepfake surround real with similar distances (Surround). The proposed Real to Blendfake to Deepfake transition is denoted by R2B2D. In Tab. 5, it can be observed that R2D2B performs the worst since it violates the inductive observation, that is, the real-to-fake progressive transition. Surround may be treated as another improved distribution organization compared with VHT, while it is still empirically inferior to the proposed organization. 
conclusion and discussions: 
 In this paper, we commence with a question that many researchers may ponder: Can we leave deepfake data behind in training deepfake detectors? We argue that the abnormal data efficacy of vanilla hybrid training may stem from the poor latent space organization. Hence, we propose to formulate the process of "real to blendfake to deepfake" to be a progressive transition. By progressively organizing blendfake and deepfake as oriented anchors and bridging them to simulate the continuous transition, the proposed ProDet effectively utilizes both data and exhibits superior detection performance.Limitations and future work. It is intuitive to observe that the current deepfake is easier to distinguish compared to blendfake. Naturally, we can organize them based on the "real-blendfakedeepfake" to improve the learned representation. However, with the advancement of deepfake techniques, all forgery attributes in deepfake data might be significantly reduced, thus making the intuitive argument that "deepfakes are easier to distinguish than blendfakes" potentially inaccurate. Nevertheless, to simultaneously utilize blendfake and deepfake data, a rational organization of the latent space still plays a crucial role. Therefore, in future work, we may identify more universally distinctive characteristics between blendfake and deepfake to effectively organize the latent space. Ethic Impacts. The proposed method strives to minimize negative social impacts and provide stronger protection for privacy, including the societal harms introduced by AIGC and the implications and challenges posed by detection errors. Specifically, our method improves the effectiveness and generalization ability of the deepfake detector, which could be crucial for reducing errors and mitigating the negative impact of malicious deepfake content. 
appendix: 
 Blendfake (SBI) Blendfake (CBI) Blendfake (CBI) Blendfake (SBI) Real Deepfake Real Deepfake Considering that this paper is primarily based on the observation of a "progressive transition from real to fake", in Fig. 6 , we present additional perceptual examples to provide a comprehensive impression.It is evident that sequentially from real to blendfake to deepfake, they become easier to distinguish as fake and the forgery information is accumulated. Therefore, it is intuitive to improve the latent space organization based on such inductive observation.In this paper, all experiments are conducted based on an open-source benchmark DeepfakeBench [52] . Specifically, for face extraction, we deploy an 81 facial landmarks shape predictor in Dlib [25] and select the detected rectangle with the biggest area in one frame. We use 32 frames in each video for both training and testing. During training, various data augmentation techniques are deployed including JPEG compression, random brightness and contrast, rotation, and median blur. As shown in Fig. 7 , we provide the detailed implementation of three different strategies. For the multi-class strategy with four classes, labels appear to be progressively assigned from 0 to 3. Namely, it can be treated as classifying real, SBI, CBI, and deepfake, respectively. However, the positions of each one-hot value are disentangled and can be rearranged arbitrarily. Therefore, the multi-class strategy is inherently unable to grant different anchors to be organized in a progressive manner. The latter two strategies can achieve similar effects of accumulating forgery attributes, therefore, their experimental performances are relatively closer. Still, we attribute the superior performance of triplet binary classification to its enhanced ability in disentangling forgery attributes.As we discussed in the main paper, we designed a new metric termed mean perturbed distance (mPD) to measure the feature robustness of each data point. Supposing the data point and its extracted feature as I and F, applying perturbation on I can be written as P(I, r), where P() is the perturbation function and r is a randomly generated value for the perturbation intensity. By performing P(I, r) with random P and r and extracting their features, we can obtain a set of perturbed feature {F 1 , F 2 , ..., F n }. Then, we calculate the average Euclidean distances between the original features and ten perturbed features. Finally, we standardize the distances in distribution with different orders of magnitude. Therefore, the PD can be written as:EQUATIONwhere F std denotes the standard deviation of each dimension in one distribution. Finally, the mPD of one distribution {F 1 , F 2 , ..., F m } can be formulated as:EQUATIONComparison with Video-level Methods. Given that the proposed method makes decisions based on one single image frame, the main paper includes comparisons with other frame-level methods and measures their performance using frame-level AUC. Nevertheless, video-based deepfake detection methods can also achieve promising results. Therefore, in Tab. 6, we compare the cross-dataset generalization ability of our method with state-of-the-art video-based methods, taking video-level AUC as the metric. The results of comparison methods are copied from their official papers. By leveraging both blendfake and deepfake effectively, our method achieves the best performance among those methods.Evaluations on More Various and Advanced Datasets. We further enlarge the evaluation scope by considering several key aspects. Firstly, we utilize nine different deepfake datasets to ensure the diversity and comprehensiveness of the testing data in our evaluation. Secondly, we include the latest synthesis methods by using testing data from newly released deepfake datasets in 2024. These datasets incorporate advanced deepfake techniques such as UniFace [48] , E4S [32] , BlendFace [40] ,and MobileSwap [49] from DF40 [50] , as well as DiffSwap [56] from DiffusionFace [9] . Finally, our evaluation design involves different variants, including Deepfake-only (DF-only), Blendfake-only (BF-only), VHT, and our proposed method. These variants are used in ablation studies using both frame-level/video-level AUC. As shown in Tab. 7, we can observe that our method consistently exhibits superiority in almost every testing data, which empirically suggests an improved generalization result.To analyze the regions of interest in the network, we used Grad-CAM [38] to generate saliency maps for visualization. As shown in Fig. 8 , VHT consistently focuses on partial regions of the face and fails to discern different fake data in aligned video frames. In contrast, our method can perceive forgery information from the broader regions of the face and the attention regions are more adaptive to different fake data.Here, we demonstrate through experiments why we chose to construct toy models rather than directly visualizing the features extracted from original models using t-SNE [45] . In Fig. 9 , the t-SNE and the actual feature distribution (see Fig. 4a ) of the toy model are highly similar in terms of clustering. However, unlike the actual distribution, the progressive transitions between anchors are not precisely presented by t-SNE. This indicates that t-SNE is good at clustering [3] but performs limited in visualizing the distribution of progressive transitions, which is crucial for estimating our method. We also provide the t-SNE results of the original model. Based on our prior analysis of the t-SNE-feature pair results on the toy models, it can be anticipated that the progressive transitions in the t-SNE results will not be prominent. Nonetheless, it is still evident that our approach exhibits better clustering while exhibiting progressive transitions to a certain extent.7.7.2 Distributions of the predicted confidence and logit outputs.In Fig. 10 , we provide a further investigation of the learned information of VHT and our method. Specifically, we summarize and analyze the distribution of logits output and confidence from VHT and Ours. We notice that VHT is less confident in both fake and real data. As we discussed, this may be because naively combining DF and BF for training confuses the network, thus limiting its confidence in "understanding" the forgery representation of distinct BF and DF. In contrast, our model can predict both fake and real with high confidence since the model "understands" how real gradually becomes more and more fakes. 
 
 Appendix: Further experiments for learned feature representations.The term "deepfake" hereafter specifically refers to face forgery technology. The entire (or natural) image synthesis is not within our scope. 


Decision: 1x A, 1x BA, and 2x BR. This paper proposes a novel training strategy for Deepfake detection using real, blendfake, and deepfake datasets. The reviewers agree on the (1) clear writing, (2) fresh motivation, and (3) novel and effective method. Most of the concerns, such as the insufficient recent datasets, insufficient experiments on latent distribution, and insufficient robustness evaluation, have been addressed by the rebuttal. The ethical concerns on the proposed deepfake detector's limitations have also been cleared. Therefore, the AC leans to accept this submission.

Reviews: review: paper_summary: paper_summary: The paper explores the utilization of blendfake and pseudo-fake data in training deepfake detectors. It argues that the significance of deepfake samples has been underestimated due to insufficient exploration. To better exploit both pseudo-fake and deepfake data, the paper introduces a progressive transition from "real to blendfake to deepfake" and proposes a hybrid training scheme. This scheme includes an oriented progressive regularizer (OPR) to model the transition and a feature bridging strategy to simulate a continuous transition.The paper explores the utilization of blendfake and pseudo-fake data in training deepfake detectors. It argues that the significance of deepfake samples has been underestimated due to insufficient exploration. To better exploit both pseudo-fake and deepfake data, the paper introduces a progressive transition from "real to blendfake to deepfake" and proposes a hybrid training scheme. This scheme includes an oriented progressive regularizer (OPR) to model the transition and a feature bridging strategy to simulate a continuous transition.
strength_weakness: strength_weakness: 1. Choice of Blend Algorithms: The paper does not provide sufficient explanation or discussion on the choice of blendfake image algorithms (SBI and CBI). As mentioned in Section 2.2, there are many other methods for crafting blendfake images. Would these methods be effective as well?
2. Interpolation Strategy: In Section 3.2, the paper introduces an interpolation strategy to achieve a smoother transition from real to deepfake. Why was interpolation performed at the feature level, and would setting multiple mixing parameters (alpha) for more interpolations further improve performance?
3. Possible Typos: There might be a typo on line 149, $M_a$.
questions: questions: See in weakness
limitations: limitations: See in weakness
paper_summary: paper_summary: This paper investigates the generalization ability of deepfake detectors and proposes a novel training approach using "blendfake" data to enhance the model's learning of generic forgery artifacts. The authors point out that existing state-of-the-art methods do not incorporate deepfake data in their training process, which contradicts previous empirical observations. The paper introduces an "Oriented Progressive Regularizor" (OPR) to establish constraints on anchor distribution and proposes feature bridging to facilitate smooth transitions. Experimental results indicate that the proposed method effectively utilizes forgery information from both blendfake and deepfake.
strength_weakness: strength_weakness: - The attribution of the unorganized latent-space distribution lacks comprehensive experiments.
- There are some minor writing issues, such as the consistency of using SOTA and SoTA.
questions: questions: - In ablation Table 2, the AUC of VHT is lower than that of BF-only in cross-dataset comparison, which is opposite to the statement in line 230.
- Are all the training sets for the SOTA model comparisons the same? How do you control the blendfake and deepfake training datasets for different models?
limitations: limitations: The authors believe that the reason VHT performs worse than blendfake-only is due to the unorganized latent-space distribution. Although the results indicate that the proposed method is effective, there is a lack of detailed experimental validation for attribution.
paper_summary: paper_summary: The authors introduced a method aimed at detecting deepfakes. Their approach, known as Oriented Progressive Regularizor (OPR), employs a progressive transition strategy. This strategy is designed to enable the model to effectively train on a combination of blendfake and deepfake data, ultimately leading to improved performance. The experimental results indicated that this method surpasses current state-of-the-art (SOTA) approaches when tested on deepfake datasets.
strength_weakness: strength_weakness: Argument on Blendfake: The argument that blendfake data alone is sufficient for training deepfake detectors is based on empirical observations on certain datasets or benchmarks with some particular deepfake detection models and may not hold universally. I would suggest toning down that claim or providing the exact conditions when this argument holds.

CDFv1 vs CDFv2: I believe that using CDFv1 for evaluation may not be necessary. It would have been more beneficial to utilize a different deepfake benchmark dataset such as FakeAVCele, DFD from Google/Jigsaw, or RWDF-23 (please refer to this repository for additional information https://github.com/Daisy-Zhang/Awesome-Deepfakes-Detection). The same applies to DFDC and DFDCP. I advocate for incorporating more diversity in the selection of benchmark datasets. In essence, the authors compared against three datasets instead of five, which is still an acceptable number.

Datasets: The authors utilized widely known deepfake datasets from 2019 in their research. However, considering the rapid advancements in deepfake technology since then, I believe these datasets may no longer accurately represent the current landscape. It would be valuable for the authors to include an assessment of their method using real-world deepfake videos sourced from social media and other online platforms. By doing so, they can demonstrate the effectiveness of their proposed solution in addressing contemporary and future iterations of deepfakes.

Progressive transition: Currently, the Progressive transition goes like this "Real --> Blendfake (SBI) --> Blendfake (CBI) --> Deepfake". I could imagine it being further extended to have addition of compression or adversarial artefacts (i.e., "Real --> Blendfake (SBI) --> Blendfake (CBI) --> Deepfake--> compression and other artefacts"). That way one could really see a generalisable pipeline that could incorporate the variance in the types of deepfakes available on social media and will greatly increase the quality of the work.
questions: questions: See the above comments.
limitations: limitations: See the above comments.
paper_summary: paper_summary: This study introduces a novel training strategy for Deepfake detection using real, blendfake, and deepfake datasets. By designing an oriented progressive regularizer and a feature bridging module, the proposed approach effectively extracts forgery information from the training data, resulting in enhanced generalizability.
strength_weakness: strength_weakness: 1. The method employs a progressive transition from real to blendfake to deepfake samples. However, the necessity of continuity in these features remains unclear. The transition from real to fake faces, as depicted in Fig. 2, appears conceptually weird. The rationale behind the feature bridging and transition design is not well-explained. The progressive transition between adjacent anchors seems unusual, and the reasoning for a continuous rather than discrete transition is not justified.
2. Despite the generative artifacts present in deepfake data, it remains ambiguous why directly incorporating blendfake and deepfake data during training degrades performance. The authors suggest that direct VHT may fail to disentangle the learned representation in the latent space, but no experiments support this claim.
3. Fig. 1(b) does not appear to be an experimental result, which is crucial for validating the work's motivation.
4. In Line 44-45, the authors raised a question “Can the blendfake face entirely substitute the actual AI-generated deepfake face in training deepfake detectors?” However, this question has already been addressed by Face X-ray and SBI, which successfully use blendfake data to train general models.
5. The term A^T in Eq. (6) is not explained.
6. It is unclear why features augmented with noise should be mapped to a more fake distribution.
7. More Deepfake datasets, such as WildDeepfake and DeepForensics-1.0, should be included in cross-dataset evaluations.
8. For robustness evaluations in Table 6, the method should be compared with recent state-of-the-art deepfake detection methods, and more severity levels for each perturbation type should be included to mimic complex real-world scenarios.
questions: questions: 1. In Table 4, is R2B2D the same as D2B2R?
2. What is the real/fake label of the mix of F_r and F_s?
limitations: limitations: N.A.

Comments: comment: We are grateful to the reviewer for highlighting a significant concern. We endeavor to address this with a systematic and comprehensive response:

- **Acknowledgment of Raised Ethic Issue**.
    - We acknowledge the potential for error detection by the detector, which may, to some extent, lead to significant implications and challenges by misleading human judgment. 
- **Ethic Impacts and Potential Solutions**
    - First, we must clarify the **positive social impact** that may surpass the negative one. That is, our proposed method aims to further enhance the performance of the DF-Detector, thereby aiding society in more effectively **addressing the ethical impacts associated with AIGC**. 
    - Second, our method is improving the effectiveness of the detector, which could be crucial for reducing errors and mitigating the negative impact.
    - Third, We believe that the issue of error detection cannot be inherently eradicated; thus, the detector’s results may be utilized with caution in practical applications, serving as an **auxiliary tool for human decision-making**. In fact, our method can provide **multiple perspectives of supporting evidence** for human judgment, such as by integrating results from different attribute detectors or identifying patterns through t-SNE clustering.

In conclusion, we are confident that our method strives to **minimize negative social impacts**, including the societal harms introduced by AIGC and the implications and challenges posed by detection errors.
Hi all, the author-reviewer discussion for this paper is extended to Aug 16 11:59pm ET for authors to reply to ethics reviews. -- PCs
Dear Reviewer t2ao,

We are grateful for your comments and feedback. We understand that **your biggest concern is about the evidence of latent distribution**. 

Actually, we have **already provided evidence of the latent distribution** from visualization approaches like **t-SNE.** Please see **Figure 4** and **Figure 6** in the manuscript. In these figures, we can see that the latent spaces of both VHT and Ours obey our expectation. 

If you have **any further questions** about our visualization and validation experiments, we sincerely anticipate further discussing and addressing any concerns you may have.

Best, Authors
Thank you for the authors' response. Some of my minor concerns have been addressed. However, the justifications in responses R1-R4 are still not convincing. The proposed method shows limited robustness and has not been compared with the SOTA methods. Therefore, I will keep my rating unchanged.
I carefulky read through your respnose and apprecite your extra experiments. But the latend distribution concern remains, as evidence or experimental cues are missing, even with your response to R2. So I decidrd to hold my original decision.
Dear Reviewer 6DHe,

Thank you for your thorough evaluation of our work. We are committed to incorporating your feedback comprehensively into our revised paper to improve both its content and overall quality.

As the discussion period draws to a close, we hope our response has sufficiently addressed your concerns.   If there are any additional issues or points that require further clarification, we are more than willing to address them promptly.

Best regards,
The Authors
Dear Reviewer Ssp1,

We are encouraged by your thoughtful feedback and grateful for your interest in our research.

As the discussion period coming to a close, we would like to inquire if our response has adequately addressed your concerns.  If there are any additional issues or points that require further clarification, we are more than willing to address them promptly.

Best regards,
The Authors
Dear Reviewer t2ao,

We deeply appreciate your dedicated efforts and insightful concern regarding our manuscript.

With the discussion period coming to a close, we would like to inquire if our response has adequately addressed your concerns.  If there are any additional issues or points that require further clarification, we are more than willing to address them promptly.

Best regards,
The Authors
Dear Reviewer LWpa,

We greatly appreciate your careful review and the insightful comments you shared regarding our manuscript.

With the discussion period coming to a close, we would like to inquire if our response has adequately addressed your concerns.  If there are any additional issues or points that require further clarification, we are more than willing to address them promptly.

Best regards,
The Authors