Diffusion Facial Forgery Detection

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-01 03:49:55

Abstract: Detecting diffusion-generated images has recently grown into an emerging research area. Existing diffusion-based datasets predominantly focus on general image generation. However, facial forgeries, which pose severe social risks, have remained less explored thus far. To address this gap, this paper introduces DiFF, a comprehensive dataset dedicated to face-focused diffusion-generated images. DiFF comprises over 500,000 images that are synthesized using thirteen distinct generation methods under four conditions. In particular, this dataset utilizes 30,000 carefully collected textual and visual prompts, ensuring the synthesis of images with both high fidelity and semantic consistency. We conduct extensive experiments on the DiFF dataset via human subject tests and several representative forgery detection methods. The results demonstrate that the binary detection accuracies of both human observers and automated detectors often fall below 30%, revealing insights on the challenges in detecting diffusion-generated facial forgeries. 
Moreover, our experiments demonstrate that DiFF, compared to previous facial forgery datasets, contains a more diverse and realistic range of forgeries, showcasing its potential to aid in the development of more generalized detectors.
Finally, we propose an edge graph regularization approach to effectively enhance the generalization capability of existing detectors.

Main paper: 
 introduction: 
 Conditional Diffusion Models (CDMs) have achieved impressive results in the field of image generation [3, 49] . Utilizing simple inputs, such as natural language prompts, CDMs can generate images with a high degree of semantic consistency [14, 24, 71] . However, the precise control over the generation process offered by CDMs has also raised concerns regarding security and privacy. For instance, malicious attackers can mass-produce counterfeit images of victims at a minimal cost, thus engendering negative social impacts.Table 1 : Comparison of DiFF and mainstream diffusion datasets. Existing diffusion datasets primarily focus on general arts and utilize limited conditional input. For generation conditions -T2I: Text-to-Image, I2I: Image-to-Image, FS: Face Swapping, FE: Face Editing. Pertaining to the Real Images column, Source represents that whether there is a real image collection process. To address this problem, recent efforts have been made to collect datasets containing diffusion-generated images, wherein distribution differences [61] or amplitude variations [9] offers important cues for detection. Nevertheless, as shown in Table 1 , these datasets are inadequate when applied to detect facial forgeries, which pose more significant threats than generic fake artifacts. Specifically, most large-scale diffusion-based datasets prioritize generic images [9, 25, 45, 52, 56, 61] , like bedrooms and kitchens [68] . Although some face-related datasets have been introduced [5, 38] , they all yet suffer from their small scales (e.g., only 1.5K facial images in [38] ). Moreover, these facial images are typically collected under restricted conditions with a narrow range of prompts, lacking comprehensive annotations as well. As a result, training a detector with generalizability on these datasets remains less viable.This paper fills the gap by introducing the Diffusion Facial Forgery dataset, dubbed DiFF. There are three notable merits that make our dataset distinct from existing ones. i) To the best of our knowledge, our DiFF is the first comprehensive dataset that exclusively focuses on diffusion-generated facial forgery. It contains more than 500,000 facial forgery images, a scale that significantly surpasses previous facial datasets (as shown in Table 1 ). ii) DiFF is curated using a rich variety of diffusion methods and prompts. Specifically, it encompasses thirteen state-of-the-art diffusion techniques across four different conditions, including Text-to-Image, Image-to-Image, Face Swapping, and Face Editing. These methods are applied to generate high-quality images using over 20,000 carefully collected textual and 10,000 visual prompts, derived from 1,070 selected identities. iii) It is worth noting that each forged image in DiFF is meticulously annotated with the forgery method employed and the corresponding prompt.We conduct in-depth human studies and extensive experiments upon the DiFF dataset. The results show that our DiFF includes forged images with more diversity and better reality than previous mainstream diffusion datasets, achieving over a 30% improvement in Fréchet Inception Distance (FID). Moreover, experiments with several deepfake and diffusion detectors [42, 48, 57, 61] reveal that existing detectors exhibit limited reliability in detecting diffusionsynthesized facial forgeries. For instance, the Xception model [48] , originally designed for deepfake detection, achieves an Area Under the receiver operating characteristic Curve (AUC) of only 60% on DiFF (versus 99% on conventional deepfake datasets). To overcome this issue, we propose a new regularization approach that leverages the edge graph of images to discern high-level facial features, thereby enhancing the generalizability of models. Our approach can be seamlessly integrated into existing detectors, achieving an average of 10% AUC improvements when applied to four popular detectors. This approach establishes an effective benchmark for the task of diffusion facial forgery detection.The contributions of this paper are three-fold:• We construct a diffusion-based facial forgery dataset with more than half a million images. To the best of our knowledge, this is the first large-scale dataset that focuses on high-quality diffusion-synthesized faces1 . • We conduct extensive experiments on this dataset, which demonstrate that DiFF contains a richer and more realistic collection of synthetic images compared to previous diffusion datasets. • We devise an approach based on edge graphs to identify the manipulated faces. Our approach can be seamlessly integrated into existing detection models, enhancing their detection ability and establishing comprehensive benchmarks for diffusion-generated face forgery detection. 
related work 2.1 image generation with diffusion models: 
 Following the paradigm of introducing and then removing small perturbations from original images, diffusion models demonstrate the capability to generate high-quality images from white noise [54] .Early methods require no supervision signals and often perform unconditionally. For instance, Ho et al. [18] proposed a reverse learning process by estimating the noise in the image at each step. Subsequently, researchers have explored several optimization directions, including backbone architectures [4, 11, 49] , sampling strategies [33, 39, 66] , and adaptation for downstream tasks [1, 28, 70] . For example, Sinha et al. [53] proposed mapping latent representations to images using a diffusion decoding model. Song et al. [55] employed a non-Markovian forward process to construct denoising diffusion implicit models, resulting in a faster sampling procedure.In contrast to the above unconditional approaches, recent diffusion models have shifted their focus toward conditional image synthesis [7, 22, 46, 50, 67, 69] . These conditions rely on various source signals, including class labels, textual prompts, and visual information, which generally describe specific image attributes. For instance, Cascaded Diffusion Models [19] initially generate low-resolution images from class labels and then employ subsequent models to increase resolutions. Furthermore, to achieve more detailed control, Text-to-Image Synthesis, which combines visual concepts and natural language, has emerged as one of the most notable approaches in diffusion models. These studies, exemplified by Stable Diffusion [41, 47] , DALL-E [44] , and Imagen [51] , aim to align different modalities through pre-trained vision language models such as CLIP [43] . Additionally, some approaches leverage images as conditional inputs. Zhao et al. [72] utilized an energybased function trained on both the source and target domains to generate images that preserve domain-agnostic characteristics. Lugmayr et al. [35] proposed an inpainting method that is agnostic to mask forms, altering reverse diffusion iterations by sampling unmasked regions from provided images.Detecting generated images has long been a popular research focus in computer vision. Earlier methods concentrate on the detection of specific types of forgeries, such as splicing [23] , copy-move [34] , or inpainting [30] . Thereafter, deep learning-based approaches have been applied to identify high-quality forgeries generated by GANs or diffusion models [59] . For instance, Frank et al. [13] proposed using frequency-domain features to detect forged images, as GAN models inevitably introduce artifacts during up-sampling. Guo et al. [15] presented a hierarchical fine-grained model to learn both comprehensive features and the inherent hierarchical nature of different forgery attributes.Many recent studies have been dedicated to facial forgery detection [6, 60, 62] . Thus far, the majority of them have focused on the detection of swapped faces generated by VAE or GAN, i.e., deepfakes [32] . For example, Masi et al. [36] introduced a two-branch network to extract optical and frequency artifacts separately. Real-Forensics [16] leverages visual and auditory correspondences in real videos to improve detection performance. Huang et al. [21] derived explicit and implicit embeddings using face recognition models, and the distance between these features serves as the foundation for distinguishing real from fake faces. With the rapid development of diffusion models, the risk posed by using them to generate counterfeit faces is gradually increasing [27] . However, research on the detection of diffusion-generated faces remains relatively unexplored. Although preliminary efforts have contributed to the detection of diffusion-generated outputs [9, 45, 61] , they often lack generalizability and do not specifically focus on the detection of facial forgery. 
dataset construction: 
 In this work, our objective is to construct a high-quality dataset for diffusion-based facial forgery. The dataset is composed of three essential components: pristine images, prompts, and forged images. The pristine images constitute the real (original) instances of our dataset. Derived from these pristine images, the prompts serve as textual descriptions or visual cues that guide the diffusion model in generating forged images. We maintain a high degree of semantic consistency between pristine and forged images via these prompts.Our pristine images are sourced from a pool of celebrity identities. Specifically, we manually select 1,070 celebrities from established celebrity datasets such as VoxCeleb2 and CelebA [2, 8, 29] . Figure 2 illustrates that we have ensured a balanced gender distribution and diverse age groups among these identities. In particular, the age distribution of the selected celebrities ranges from 20 to 60 across different subsets. Subsequently, we curate approximately 20 images per identity from public resources, resulting in a pristine collection, denoted as I 𝑝𝑟𝑖 , which encompasses a total of 23,661 images. Prior studies have demonstrated a positive correlation between the quality of conditional inputs and that of diffusion-generated images [40] . As a result, diverse and precise prompts are particularly useful for generating high-quality images in CDMs. Figure 3 illustrates that the construction of our dataset includes three categories of prompts: original textual prompts P 𝑡 𝑜𝑟𝑖 , modified textual prompts P 𝑡 𝑚𝑜𝑑 , and visual prompts P 𝑣 . These prompts serve as conditions to guide the sampling process of diffusion models. The construction processes of these prompts are detailed below.• Original textual prompts P 𝑡 𝑜𝑟𝑖 . We generate diverse and natural textual prompts via a semi-automated approach. Initially, we curate a set of 2,531 high-quality images Î𝑠 ⊂ I 𝑝𝑟𝑖 by selecting the clearest images of the frontal face for each identity. These images are then converted into textual descriptions using prompt inversion tools [10, 37] . These descriptions are reviewed and rewritten by experts to remove irrelevant terms and improve clarity. Consequently, we obtain 10,084 polished prompts, and some frequent words are shown in Figure 4 .• Modified textual prompts P 𝑡 𝑚𝑜𝑑 . To broaden the diversity of prompts and enable the generation of images with specific modifications, P 𝑡 𝑚𝑜𝑑 involves alterations in key attributes of P 𝑡 𝑜𝑟𝑖 . In particular, we randomly modify the salient words that describe identities in P 𝑡 𝑜𝑟𝑖 , such as gender, hair color, or facial expression. For instance, we transform a prompt like 'A man with an emotive face' into 'A woman with an emotive face. ' • Visual prompts P 𝑣 . These prompts comprise comprehensive facial features -such as sketches, landmarks, and segmentations -extracted from each image in Î𝑠 . These features are applied for diffusion models conditioned on visual cues, which is particularly useful in tasks like face editing.As illustrated in Figure 5 , we categorize existing CDMs into four main subsets based on their input types: Text-to-Image (T2I), which operates on textual prompts; Image-to-Image (I2I) and Face Swapping (FS), both of which utilize visual inputs; and Face Editing (FE), which incorporates a combination of text and visual conditions. Figure 5a demonstrates that T2I methods receive textual prompts (e.g., 'A man in uniform') and synthesize images that align with the inputs' semantic content [47] . In contrast, models processing visual input are further divided into I2I and FS categories based on their manipulation processes. Specifically, I2I, as illustrated in Figure 5b , pertains to methods that replicate a single identity. On the other hand, FS models simultaneously handle two identities and perform identity swaps as presented in Figure 5c . Lastly, Figure 5d highlights that FE models utilize multi-modal inputs to modify facial attributes, such as expressions or lip movements, while preserving other attributes. These four subsets achieve comprehensive coverage of the conditions under which existing diffusion models operate. Moreover, to ensure the diversity of generated faces, we utilize multiple cutting-edge techniques within each category. A detailed introduction to these methods is as follows: Text-to-Image. We employ four state-of-the-art methods -Midjourney [37] , Stable Diffusion XL (SDXL) [41] , FreeDoM_T [69] , and HPS [64] -for this subset. The first two are the most influential web services for which we employ official APIs. The latter two are recently released T2I models, and we apply their pre-trained models. These models are guided by textual prompts P 𝑡 𝑜𝑟𝑖 . Image-to-Image. We apply four methods in this context: Low-Rank Adaption (LoRA) [20] , DreamBooth [50] , SDXL Refiner [41] , and FreeDoM_I. Among these approaches, the former two require fine-tuning of diffusion models to capture specific facial features. We employ I 𝑝𝑟𝑖 to train these two models. SDXL Refiner optimizes results from SDXL, whereas FreeDoM_I substitutes the textual encoder in FreeDoM_T with a visual encoder to reconstruct faces. Face Swapping. In this subset, we implement DiffFace [26] and DCFace [27] for the face swapping task. For each image in Î𝑠 , we randomly choose ten targets from other identities to perform face swaps. In particular, to prevent information leakage, we divide the 1,070 identities into disjoint training, validation, and testing sets in an 8:1:1 ratio. Face Editing. This subset involves three approaches. In particular, the modified textual prompt set P 𝑡 𝑚𝑜𝑑 and the pristine image set Î𝑠 are both used in Imagic [25] and Cycle Diffusion (CycleDiff) [63] [22] .In summary, we show the statistics pertaining to the images generated by the aforementioned methods in Table 2 . As can be observed, the total number of generated images is over 500K from thirteen diffusion methods. 
dataset evaluation: 
 Following the methodologies in deepfake detection [48] , we cast the detection of diffusion-generated facial forgeries as a binary classification task.We conducted comprehensive human studies involving 70 participants. In this study, participants are instructed to classify the authenticity of randomly selected images that are generated from varied approaches. The image selection followed a 50:50 split between pristine and fake images, with each identity appearing only once to prevent bias. Each participant is required to carefully examine 200 images, yielding 14,000 human results in total.Table 3 presents the accuracy (ACC) of this experiment across all forgery methods under four conditions. One can observe that human observers struggle to distinguish the vast majority of forgery methods, as accuracy falls below the chance level (50%). For instance, participants achieved an accuracy of 27.78% when identifying images generated by CoDiff. Among the four conditions, FE poses the most challenge for human observers. This result can be attributed to the fact that this subset involves manipulations of modifying a single real image, which allows for a more faithful reproduction of original features.We presented the Fréchet Inception Distance (FID) and Peak Signal-to-Noise Ratio (PSNR) metrics in Table 4. The reported results reveal notable improvements in DiFF. For instance, the FID of DiFF shows an approximate improvement of 30% and 20% over the previous diffusion datasets GFW and Dif-fusionForensics (DFor), respectively, which suggesting that the images in our dataset bear a closer resemblance to reality. 5 indicates that the scores of Area Under the receiver operating characteristic Curve (AUC) of DiFF are relatively lower, highlighting the dataset's higher complexity. This can be attributed to the extensive diversity of conditions and image synthesis methods in DiFF, both of which are three times greater than those in GFW. Moreover, we utilized FF++ (vanilla deepfake dataset, HQ version), DFor (diffusion-generated general forgery dataset), GFW (diffusion-generated facial forgery dataset), and our DiFF as training datasets to compare their generalization capabilities. Additionally, we utilized two widely acknowledged deepfake datasets, DFDC [12] and ForgeryNet [17] , for further evaluation. From Table 6 , it can be seen that the detector trained on DiFF exhibits superior generalization capabilities. It is worth noting that detectors trained on other datasets achieve high accuracy when tested on DiFF. This may be attributed to DiFF's inclusion of a diverse array of image types, effectively encompassing a wide spectrum of distributions.For this experiment, we split our DiFF dataset into training, validation, and testing sets with a 8:1:1 ratio. We tuned the hyperparameters with the validation set, and reported results on the testing set. [61] , we adopted a cross-domain testing methodology to explore the challenges of facial forgery detection. This involves evaluating models that have performed well in related detection domains. Initially, these models are trained on benchmark datasets tailored to their respective tasks. We then evaluated their performance on DiFF. Three widely recognized deepfake detection models are utilized: Xception [48] , F 3 -Net [42] , and EfficientNet [57] . Moreover, we included DIRE [61] , a state-of-the-art detector for general diffusion-generated images, for this experiment. These models are trained on the FF++ dataset [48] and the DiffusionForensics dataset [61] , respectively. Table 7 displays the AUC scores for these detectors. From this table, we can observe that these detectors encounter a significant drop in performance upon domain transfer. For example, DIRE exhibits an AUC drop of over 60%. This sharp degradation indicates the inherent challenge of detecting diffusion-based facial forgeries and suggests the considerable obstacles that pre-trained detectors face when applied to this new task.Given that existing deepfake and general diffusion detectors cannot be seamlessly transferred to detect diffusion facial forgery, one may question the efficacy of re-training these detectors on DiFF. Therefore, we conducted experiments with an in-domain setting. Similar to previous evaluation protocols for the detection of deepfake and general diffusion forgery [31, 61] , detectors are trained on a single subset of DiFF, followed by the test on the remaining ones. Detection on re-training detectors. We presented the re-training results in Figure 6 . It can be observed that detectors perform satisfactorily when trained and tested on the same subset. However, when transferred to different subsets, they exhibit varying degrees of performance degradation. The most significant drop reaches up to 80% (e.g., Xception, trained on FS and tested on FE). This significant drop highlights the challenges in developing a facial forgery detector that effectively generalizes across various conditions.It is worth noting that detectors trained on the T2I and I2I subsets, which both rely on classical diffusion processes, demonstrate a higher degree of similarity in performance. This is evident from mutual benefits observed in the first two subplots of Figure 6 . FEtrained detectors show better generalization capability than those trained in other subsets. This may be attributed to the FE subset's utilization of multi-modal inputs, leading to a wider diversity of images, thereby enabling detectors trained on the FE subset to capture more diffusion artifacts. Detection on linear probing detectors. We introduced the strategy of linear probing as an alternative to the full re-training approach. Specifically, we used Xception pre-trained on the FF++ dataset as described in Section 4.3.1 and optimized its last linear layer to align with the data distribution of DiFF. The results are presented in Figure 7 .One can observe that models using the linear probing strategy significantly outperform the re-training ones in detecting FS and FE forgeries. For instance, when trained on the I2I subset, the linear probing model for detecting FS and FE forgeries surpass the retraining models by 50% and 40%, respectively. A critical reason is that the pre-training dataset, i.e., FF++, encompasses a large number of GAN-based manipulated faces. This diversity enables linear probing models to effectively identify face-swapping and face-editing images. However, it is worth noting that linear probing models show inferior results when trained and tested on the same subset (e.g., both trained and tested on T2I), suggesting insufficient capacity of this strategy. Detection on fine-tuning detectors. In contrast to the linear probing strategy, which updates only the final layer, the fine-tuning approach optimizes all the model parameters. For this experiment, we reduce the learning rate of models for stable training. Figure 7 illustrates that fine-tuning models demonstrate superior performance compared to the re-training ones. For example, fine-tuning models achieve higher AUCs in the detection of FS and FE forgeries, regardless of the training subset used. This can also be attributed to the pre-training on FF++. However, compared to linear probing models, the generalizability of the fine-tuning approaches is somewhat limited. This may be due to a significant discrepancy between diffusion-generated facial forgeries and GAN-based manipulated faces. Such a domain gap could lead to catastrophic forgetting. Detection with post-processing methods. We evaluated retraining detectors under various image quality settings by applying several post-processing techniques. Following previous settings [74] , we processed real and forged images with Gaussian Noise (GN), Gaussian Blur (GB), Median Blur (MB), and JPEG Compression (JPEG). Table 8 reveals that, in most scenarios, applying post-processing methods leads to a degradation in the detection performance. For instance, the use of GB results in a 40% reduction in the AUC for Xception when trained on the T2I subset. 
edge graph regularization 5.1 motivation: 
 Compared to real human faces captured by cameras, generated faces are more likely to evoke anomalies such as frequency transitions or brightness fluctuations [45] . In particular, we extract the edge graphs of pristine and forged images with the Sobel operator [58] . Figure 8 illustrates that the edge graphs of the pristine images are significantly different from those of the synthesized images. Specifically, edge graphs extracted from pristine images often capture intricate facial details, such as fine wrinkles around the cheeks. In contrast, the synthesized images lack these subtle contours and are of considerable contrast. One naive approach to exploit the discriminative capability of edge graphs for facial forgery detection is to train a binary classifier directly. However, our experiments indicate that this approach is less favorable, as demonstrated in Section 5.3.3. Instead, we propose an Edge Graph Regularization (EGR) method, which enhances the discriminative ability of detectors by incorporating edge graphs into the processing of original images.Vanilla deepfake and general diffusion detecors entails fitting the distribution of a specific dataset to discriminate between pristine and forged images. Let S = {(I 𝑖 , 𝑦 𝑖 )} 𝑛 𝑖=1 be the dataset, where I 𝑖 is the 𝑖-th image with respect to the target label 𝑦 𝑖 . For each parameter set 𝜃 ∈ Θ, wherein Θ represents the continuous parameter space, the empirical risk during training is formulated as follows:EQUATIONwhere ℓ (•) is the loss function such that,EQUATIONwhere ŷ𝑖 is the score from the predictive function 𝑓 𝜃 : I 𝑖 → [0, 1] associated with 𝜃 . However, such training approaches are highly susceptible to overfitting [31, 48, 65] . Therefore, many endeavors have been made to improve generalizability using additional features [21, 73] . In light of these studies, our method employs a novel regularization method, which incorporates edge graphs as a regularization term into the empirical risk. This strategy encourages the model to simultaneously focus on the features of both the original and edge graphs, thereby mitigating overfitting. Specifically, we refine the empirical risk as follows:EQUATIONwhere E 𝑖 represents the edge graph of the 𝑖-th image, and 𝜆 ∈ [0, 1] is a regularization parameter that calibrates the influence of edge graphs.In Table 9 , we compared the performance of baseline detectors and our method. Each model is trained on one forgery condition and subsequently evaluated on all four conditions. From the table, one can observe that our EGR method significantly improves the generalizability of the baseline detectors. It is worth noting that even when a model is trained and tested in the same subset, EGR still contributes to performance enhancement, such as improving Xception with 2.2% AUC on T2I.Curves. In Figure 9 , we presented the ROC curves of EfficientNet trained on the FE subset and tested on the T2I subset, the full DiFF, and GFW. It can be observed that EGR significantly enhances the model's performance across various datasets, further demonstrating the effectiveness of the EGR method in developing generalized diffusion detectors.To evaluate the impact of the proposed EGR method, we conducted experiments using edge graphs as the only input. In other words, we removed the RS (𝜃 ) in Equation (3), and optimized the model with E 𝑖 . The results of these tests are presented in Table 10 . We can observe a significant decline in detector performance upon removing the regularization approach. For instance, in the FE subset, the AUC of F 3 -Net drops by 15%. The dominant reason is that relying solely on edge graphs overlooks vital information in original images, such as color and texture. On the other hand, incorporating the EGR enables the model to capture a more broad context, leading to better performance. In a nutshell, the combined utilization of edge graphs and color information achieves optimal results. 
discussion and conclusion: 
 We propose DiFF, a large-scale high-quality diffusion-generated facial forgery dataset, to address limitations of existing datasets that underestimate the risks associated with facial forgeries. Our dataset comprises over 500,000 facial images. Each image maintains high semantic consistency with its original counterpart, guided by diverse prompts. We conduct extensive experiments using DiFF and establish a facial forgery detection benchmark. Moreover, we design an edge graph regularization method that effectively improves detector generalization performance. In the future, we plan to further expand DiFF in terms of generative algorithms and conditions and explore new tasks based on DiFF, such as the traceability and retrieval of diffusion-generated images. Potential Ethical Considerations. The pristine faces in our dataset are sourced from publicly accessible celebrity online videos. We have rigorously reviewed all prompts to ensure that they do not describe specific biometric details. Each generated image has been carefully examined to align with societal values. We will try our best to control the acquisition procedure of our DiFF to mitigate potential misuse.  
 
 Appendix: The dataset will be released upon the acceptance of this paper. 


Decision: The initial ratings of this paper are positive. After the rebuttal by authors, PAYy changed its rating to weak reject, due to the concerns about the comprehensiveness of the experiments. AC agrees that the comparisons and literature reviews contain certain flaws, but the proposed dataset has its value to the community, which is also agreed by all reviewers. Therefore, AC decides to accept this paper, but the authors should seriously revise the paper according to the review comments and the rebuttal.

Reviews: review: paper_summary: paper_summary: This paper collects a large synthetic data set containing 13 generative models.
This paper constructs a wealth of experiments and proposes a method to enhance generalization ability.
strength_weakness: strength_weakness: This paper constructs a diffusion-based facial forgery dataset with more than half a million images. This is a boost for the fake face detection community.
The proposed EGR approach is interesting.
limitations: limitations: 1. Lack of comparative experiments on data sets based on GAN models.
2. For section 5.2, detailed explanation and formula proof are lacking.
3. The motivation of the proposed method is not clear enough to understand the role of edge maps in alleviating overfitting.
4. Use Sobel operator to capture the edge graphs, which is widely used in the field of image forensics and lacks innovation.
5. From the visualization results in Figure 8, it can be observed that the true and false images are very similar, and it is difficult to detect the difference.
6. Lack of 3to1 experiment Settings, such as training in three generation methods and testing in one generation method.
7. There is a lack of cross-generation experiments, such as training in DDPM architecture and testing in LDM architecture.
8. This dataset lacks the exploration of local face tampering localization tasks and the discussion of related papers, such as:
[1] On the detection of digital face manipulation. CVPR 2020
[2] Inconsistency-aware wavelet dual-branch network for face forgery detection. TBIOM 2021
[3] FakeLocator: Robust localization of GAN-based face manipulations. TIFS 2022
[4] Multi-spectral Class Center Network for Face Manipulation Detection and Localization. Arxiv 2023
paper_summary: paper_summary: The paper introduces DiFF, a large-scale diffusion-based facial forgery dataset with over 500k images generated by 13 diffusion methods. These methods are categorized into four types: text-to-image (T2I), image-to-image (I2I), face swapping (FS), and face editing (FE). This dataset significantly enriches the current pool of diffusion-based data, which predominantly focuses on general images. Additionally, the paper proposes a technique for extracting edge graphs to serve as supplementary input alongside RGB images, enhancing the effectiveness of existing deepfake detection methods.
strength_weakness: strength_weakness: 1. The paper provides a large-scale, diverse dataset of facial forgeries created using 13 different diffusion methods across four comprehensive categories.

2. The dataset includes rich annotations with prompts in addition to labels.

3. The dataset appears to be balanced in terms of demographics and manipulation methods.

4. The proposed edge graph regularization method is shown to be effective.
limitations: limitations: 1. The paper claims that "this is the first large-scale dataset that focuses on high-quality diffusion-synthesized faces." However, Țânțaru et al. (WACV 2024) [i] and Chen et al. (pre-print) [ii] have introduced similar datasets.

2. The exclusive use of high-quality images of celebrities may not be ideal. Since deepfakes can vary in quality depending on the attacker's purposes, it would be better to also include images of varying quality and distributions. This could involve using images of everyday people taken with different cameras under various conditions.

3. The use of PSNR for quality measurement is questioned. The paper does not clarify how PSNR was applied, and it may not be an appropriate metric in this context.

4. The paper asserts that the DiFF dataset is challenging, but detectors trained on other datasets perform well on DiFF. It would be more accurate to state that DiFF is more diverse, leading to higher test accuracy on other datasets when detectors are trained on DiFF.

5. There is a discrepancy between the descriptions in section 5.3.3 and Table 10. Did the authors intend to use "only" rather than "w/o" regulation in Table 10?

**References**

[i] Țânțaru, Dragoș-Constantin, Elisabeta Oneață, and Dan Oneață. "Weakly-supervised deepfake localization in diffusion-generated images." In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 6258-6268. 2024.

[ii] Chen, Zhongxi, Ke Sun, Ziyin Zhou, Xianming Lin, Xiaoshuai Sun, Liujuan Cao, and Rongrong Ji. "DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysis." arXiv preprint arXiv:2403.18471 (2024).
paper_summary: paper_summary: This paper introduces a comprehensive dataset dedicated to face-focused diffusion-generated images, i.e., DiFF, which using a variety of diffusion models and carefully collected textual and visual prompts to generate a more diverse and realistic range of forgeries. Extensive experiments are conducted on the proposed dataset, which demonstrate that DiFF contains a richer and more realistic collection of synthetic images compared to previous diffusion datasets. In addition, the author proposes an edge graph regularization approach to enhance the generalization performance for deepfake detection.
strength_weakness: strength_weakness: The structure of this paper is clear, and the content is easy to follow.
To construct diffusion-based facial forgery dataset, the authors utilize c a variety of diffusion models and carefully collected textual and visual prompts to generate more than half a million face forgery images. 
Comprehensive experimental evaluation substantiates the proposed DiFF includes forged images with more diversity and better reality than previous mainstream diffusion datasets.
limitations: limitations: Since this article is concerned with the construction of deepfake's dataset, but the related work only describes the generative model (i.e., diffusion model) and the synthetic image detection, there is no systematic presentation of other deepfake's datasets.

The proposed edge graph regularization approach, which essentially uses the Sobel operator to extract high frequency information, is not novel. This operation of extracting high-frequency information for improving detection performance has actually been widely used in deepfake detection.