DETER: Detecting Edited Regions for Deterring Generative Manipulations

## Abstract

Abstract Generative AI capabilities have grown substantially in recent years, raising renewed concerns about potential malicious use of generated data, or “deep fakes.”
However, deep fake datasets have not kept up with generative AI advancements sufficiently to enable the development of deep fake detection technology which can meaningfully alert human users in real-world settings. Existing datasets typically use GAN-based models and introduce spurious correlations by always editing similar face regions.
To counteract the shortcomings, we introduce DETER , a large-scale dataset for DETE cting edited image R egions and deter ring modern advanced generative manipulations. DETER includes 300,000 images manipulated by four state-of-the-art generators with three editing operations: face swapping (a standard coarse image manipulation), inpainting (a novel manipulation for deep fake datasets), and attribute editing (a subtle fine-grained manipulation). While face swapping and attribute editing are performed on similar face regions such as eyes and nose, the inpainting operation can be performed on random image regions, removing the spurious correlations of previous datasets. Careful image post-processing is performed to ensure deep fakes in DETER look realistic, and human studies confirm that human deep fake detection rate on DETER is 20.4% lower than on other fake datasets. Equipped with the dataset, we conduct extensive experiments and break-down analysis using our rich annotations and improved benchmark protocols, revealing future directions and the next set of challenges in developing reliable regional fake detection models.

## 1 Introduction

“If you know both enemy and yourself, you will fight a hundred battles without danger or failure.”

– Sun Tzu, The Art of War, 300 BC

Generative AI models such as StableDiffusion [] and ChatGPT [] have captured significant attention from both the research community and the general public in recent years, following groundbreaking advances in generative modeling.
The booming of those generative AI techniques brings numerous advantages and conveniences but also raises heightened concerns about the potential malicious usage of their generated fake data, especially within the context of identifiable human faces.
Our primary motivation in this work is to catch up with the rapidly advancing generative techniques for building an effective protective shield against the potential misuse of generative AI, via a *more comprehensive dataset* and *less biased benchmark protocol*.

Figure: Figure 1: In this work, we introduce *DETER* dataset for detecting edited regions manipulated by state-of-the-art generative models. By formalizing the problem as a regional detection task, models can achieve much better performance than human evaluators and popular Large Language Models (LLMs) such as GPT-4 [].
Refer to caption: x1.png

*Existing deep fake datasets are lagging behind in terms of the quality of generators and editing operations during construction.*
Failure to catch up with the fast-developing generative fields in the dataset construction front sets the first gap between the generative and detection teams.
In the upstream generative architecture area, the Diffusion Models (DMs) [, , ] are replacing Generative Adversarial Nets (GANs) [, , ] and become the new state-of-the-art generative models by achieving impressive performance in data generation for images [, , , , , ], audio [, , , ], and videos [, ].
On the downstream front, where generative models are leveraged for versatile and fine-grained applications like data editing and customization, both GANs-based [, , , ] and DMs-based frameworks [, , , , ] continue to share an equal footing.
However, as listed in Tab. [1](#S1.T1), none of the existing deep fake datasets [, , , , , , , , , ] has yet incorporated Diffusion Models as the manipulation sources.
In addition, most manipulations focus on face swapping, with some of them also including attribute editing.
While these used to be representative forgery operations, current generative models can do more than the above.
Particularly, a popular branch of recent works can *achieve very photorealistic and natural effects for image inpainting on arbitrary image regions* [, , , ], which is a novel type of forgery operations that has never been addressed before in the detection side.
It is worth investigating since it presents a different generation mechanism compared to face swapping and attribute editing. While face swapping and attribute editing rely on the information from reference images to “replace” the target region of real images, inpainting techniques leverage the generators’ “intrinsic understanding” of the real images to fill in the missing regions, as illustrated in Fig. [3](#S3.F3).
*To this end*, our *DETER* presents high-quality images edited by four state-of-the-art generators (at most one year long) based on both GANs and DMs backbones [, , , ] for three regional editing operations in different granularities (i.e., face swapping, inpainting, and attribute editing, ordering from coarse to fine), each annotated with binary labels, precise masks, and source generators.
We also employ careful post-processing techniques during the dataset construction to ensure that the synthesized image look realistic, including color matching, Poisson fusion and image sharping. To the best of our knowledge, this is the first large-scale regional deep fake dataset with DMs-based generators.
More details about our *DETER* dataset are presented in Sec. [3](#S3), with samples included in Fig. [1](#S1.F1).

**Table 1: Comparison of basic dataset statistics for deep fake detection. We list the existing popular deep fake datasets in time ordering, with their scales, generators and editing operations. Most existing popular deep fake datasets are video-based, several recent image datasets edit face images with the swapping operation. *DETER* includes the state-of-the-art GANs and DMs-based generators with diverse editing operations and annotations.**
| Datasets | Format | Real | Fake | GANs | DMs | FaceSwap | Attribute | Inpaint | Multiple faces | Masks |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| DF-TIMIT 18’ [] | Videos | 320 | 640 | $\checkmark$ | ✗ | $\checkmark$ | ✗ | ✗ | ✗ | ✗ |
| FaceForensics++ 19’ [] | Videos | 1,000 | 5,000 | ✗ | ✗ | $\checkmark$ | ✗ | ✗ | ✗ | $\checkmark$ |
| Celeb-DF 20’ [] | Videos | 590 | 5,639 | $\checkmark$ | ✗ | $\checkmark$ | ✗ | ✗ | ✗ | ✗ |
| DFFD 20’ [] | Videos | 1,000 | 3,000 | $\checkmark$ | ✗ | $\checkmark$ | $\checkmark$ | ✗ | ✗ | $\checkmark$ |
| DFDC 20’ [] | Videos | 23,564 | 104,500 | $\checkmark$ | ✗ | $\checkmark$ | ✗ | ✗ | ✗ | ✗ |
| ForgeryNet 21’ [] | Videos | 99,630 | 121,617 | $\checkmark$ | ✗ | $\checkmark$ | $\checkmark$ | ✗ | $\checkmark$ | $\checkmark$ |
| DF-Platter 23’ [] | Videos | 764 | 132,496 | $\checkmark$ | ✗ | $\checkmark$ | ✗ | ✗ | $\checkmark$ | ✗ |
| SwapMe/FaceSwap 17’ [] | Images | 4,600 | 2,010 | ✗ | ✗ | $\checkmark$ | ✗ | ✗ | ✗ | ✗ |
| OpenForensics 21’ [] | Images | 45,473 | 115,325 | $\checkmark$ | ✗ | $\checkmark$ | ✗ | ✗ | $\checkmark$ | $\checkmark$ |
| DGM${}^{4}$ 23’ [] | Images&Texts | 77,426 | 152,574 | $\checkmark$ | ✗ | $\checkmark$ | $\checkmark$ | ✗ | ✗ | ✗ |
| DETER (Ours) | Images | 38,996 | 300,000 | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ |

The quality of our dataset is further guaranteed by extensive *Institutional Review Board (IRB) approved* human studies in two layouts, from which we show solid superior visual quality compared to existing deep fake datasets [, , ], as well as the challenge for humans to detect regional manipulations.
In addition, as Large Language Models (LLMs) can now achieve superior or similar performance to humans in several existing multimodal tasks such as image captioning [], we also evaluate the deep fake detection on the state-of-the-art LLMs (i.e., GPT-4 []) by proper prompt engineering.
Details about human studies and LLMs evaluations are included in Sec. [4](#S4).

*In addition, the current image fake detection benchmark suites tend to introduce spurious patterns in regional detection tasks, which makes the existing detection methods achieve seemingly good performance but biased with an undesired high false alarm rate.*
Binary classification formulation [, , ] where a detector classifies the whole image as being “real” or “fake” is an over-simplified situation compared to the real-life malicious scenarios where often part of the real images are modified without changing the personal identity as shown in Fig. [1](#S1.F1).
As an intuitive step forward, OpenForensics [] is the first image dataset that introduces fake regional detection and segmentation benchmark tasks.
However, despite its efforts to bring the fake detection studies closer to a more fine-grained setup, there is a critical gap to fulfill before building reliable fake detection models: the *spurious correlation* problem.
Specifically, with a deeper dive investigation into the current benchmark designs, we note that due to the simplicity of the forgery operation, current detection and segmentation models tend to capture the spurious correlations.
For example, simplicity of face swapping can cause detection models to predict certain parts of images such as faces as fake regions because they are most frequently seen in the training data), instead of learning the true generative patterns.
Moreover, existing evaluation protocols with classic metrics such as the Average Precision (APs) *fail to reflect* such undesired backdoor shortcut.
The existing detection and segmentation models [, ] thus can achieve seemingly good performance but are often biased with a high false alarm rate as verified and explained in our extensive experimental results and break-down analysis in Sec. [5](#S5).
As mitigation, our inpainting operation in *DETER*, which can be deployed on arbitrary image regions, helps to *decouples the spurious correlations* between certain visual cues (e.g., face shapes) and the actual forgery operations.
In addition, we introduce negative examples (i.e., real images) in the current detection and segmentation setup, accompanied with a *region-based image-level classification accuracy* in the evaluation protocol as an extra assessment criterion to supplement the standard metrics, which helps better reflect the true performance of regional fake detection methods.

To sum up, we believe that *DETER* shall help our community to build more robust and reliable fake detection systems with its main contributions summarized as follows:

- •
We present the *DETER*, the first large-scale dataset that incorporates the SOTA diffusion-based generators for better deterring regional generative manipulations.
- •
We introduce a novel type of regional forgery operation, i.e., inpainting, from the generative modeling side that has been not yet addressed in the detection community.
- •
We note the spurious correlations in the current regional fake detection benchmark suites and propose to mitigate them in *DETER* with inpainting operation in the *dataset construction*, an improved *detection task setup*, as well as extra assessment criterion in the *evaluation protocol*.

## 2 Related Work

We present existing literature from domains that are mostly closely related to our work in this section.

Figure: Figure 2: Statistical distributions in *DETER*. Our dataset covers images in diverse resolutions, edited via multiple SOTA generators with different editing operations and versatile mask sizes and shapes. Best viewed in color with zoom-in.
Refer to caption: x2.png

Deep Fake Datasets.
Most existing deep fake datasets can be categorized as either video-based [, , , , , , , , , ] or image-based [, , ], as summarized in Table [1](#S1.T1).
All of these datasets provide true or false labels that enable binary classification benchmark tasks, while few of them integrate more fine-grained box or mask-level annotations for fake region detection or segmentation tasks.
Face swapping with GANs-based generators is the most commonly adopted forgery operation during the construction, with few including attribute editing.
In comparison, *DETER* is the first large-scale dataset that uses the latest state-of-the-art fine-grained methods as generators, and covers editing operations with different granularities.
Notably, inpainting is a forgery operation that has never been addressed before in deep fake datasets.

Generative Models for Image Manipulations.
While diffusion models (DMs) [, , , , , , ] are steadily replacing generative adversarial networks (GANs) [, , , ] and have become the dominating method for image synthesis in the past two years, GANs have not yet been entirely supplanted in the downstream side for more fine-grained data manipulation applications such as face swapping and inpainting.
Among the most recent works that perform fine-grained image manipulations within the past two years [, , , , , , , , , ], we carefully select four methods [, , , ] that cover both GANs and DMs backbones based on their editing quality and versatility as the generators in this work.

Fake Detection Modeling.
Fake detection methods are closely entangled with available benchmarks and evaluation systems.
Many earlier works [, , , , ] tackle the problem against GAN-based generators using Convolutional Neural Networks (CNNs) discriminators and can already achieve very high accuracy (more than 99.9%) in discerning fake/real images.
Even the most recent fake detection works that build upon diffusion models [, , ] still follow the conventional setting and formalize it as a binary classification problem.
However, the demand for fake detection methods has gone beyond a true or false label, especially given the more sophisticated generators.
In this work, we formalize the problem as more fine-grained detection and segmentation tasks, but explicitly note that the improvements in terms of the experimental setup are necessary to reflect the true performance.

## 3 DETER Dataset

In this section, we present the details about our *DETER* dataset and the construction method.

### 3.1 Dataset Overview

Diverse Real-life Scenarios.
Among different real image datasets that include humans, we select CelebA [] and WiderFace [] as the real human face image sources.
The rationales for the above choices is that CelebA [] is one of the most widely adopted datasets in the generative modeling area, and WiderFace [] includes in-the-wild real images that better capture the complex real-life scenarios.
The former includes mostly single-face images while the latter has images with multiple people.
Both datasets are open access to the public under proper license (Creative Common License) for non-commercial research purposes.

Editing Operations.
*DETER* incorporates three image editing operations with different granularities from coarse to fine in terms of editing areas: face swapping, inpainting and attribute editing.
Specifically, *face swapping* involves replacing one person’s face in a real image given a reference image (face).
*Inpainting* fills up a missing part of a given image using generative models without other reference images.
*Attribute editing* is similar to *face swapping* but in a finer grain, with replaced regions being facial regions such as eyes, ears and lips.
These operations include different editing regions grounded by binary masks.
As illustrated in Fig. [1](#S1.F1) and Fig. [3](#S3.F3), their average editing areas are 31192, 6111, and 1625 measured in pixel numbers, corresponding approximately to squares of 176, 78, and 40, respectively.

Particularly, while face swapping and attribute editing are more commonly adopted forgery techniques in existing datasets, inpainting is an *unique* operation in our *DETER*.
Different from the other conventional techniques, inpainting does not rely on reference images and can be applied to arbitrary regions, which presents a novel and important type of forgery that mitigates the spurious correlations introduced in previous dataset constructions.
As revealed in our experimental results and analysis in Sec. [5](#S5), despite having larger editing masks than attribute editing, the inpainted regions are more difficult to detect by current models, *contradicting the common intuition* that “with more pixels being modified, it should be easier to detect”.

SOTA Generators.
We adopt four state-of-the-art generative models as the deep generators for dataset construction after having extensively examined and compared their editing quality.
For *face swapping* and *attribute editing*, we adopt the GANs-based E4S [] and DMs-based DiffSwap []; for *inpainting*, we deploy the GANs-based MAT [] and DMs-based DiffIR [] as the manipulation tools.
All the generators are the most recent state-of-the-art methods for fine-grained image manipulations, dating back to only one year at most.

Interestingly, while DMs [, , , ] are believed to have surpassed GANs [] in unconditional data synthesis, our analysis suggests the current detection methods are more robust against GANs-based generative techniques, as shown in our cross-generator experiments in Sec. [5](#S5).

Overall Statistics.
To sum up, *DETER* presents 300,000 edited images based on 38,996 real images.
The training, validation, and testing splits are partitioned following the 6:1:3 ratio, which includes 180K, 30K, and 90K edited images, respectively.
We incorporate three editing operations via four SOTA generators.
Our images cover diverse real-life scenarios that includes both single and multiple faces.
Fig. [2](#S2.F2) summarizes important statistics about our *DETER* dataset with more details can be found in Appendix [7](#S7).

Figure: Figure 3: Pipeline for our regional fake approach in *DETER* construction. We first pre-process raw images with face detection and mask parsing techniques [, ]. Next, we apply SOTA DMs and GANs-based generators [, , , ] for different editing operations. Finally, we apply a series of post-rendering techniques and paste the edited regions back to original images, ensuring the high visual quality.
Refer to caption: x3.png

### 3.2 Regional Editing Approach

Our regional editing approach for the dataset construction is in Fig. [3](#S3.F3), which mainly include pre-processing, regional editing, and post-processing steps as explained below.

Pre-processing.
We first run face detection and alignment methods [] on the real images and parse the detected face to obtain masks with different levels that include the entire face and detailed features such as eyes, lip, and nose [, ].
The selection of editing masks is based on specific operations.
For face swapping and attribute editing, we adopt the face and feature-level masks, respectively.
As for inpainting, there are two ways to obtain the editing masks: we either dilate the original feature-level masks into arbitrary shapes or randomly pick an image region within the face mask.
The rationale behind our mask generation mechanism for the inpainting operation is to ensure that it has an editing granularity that is between the face swapping and attribute editing, also further decoupling the spurious correlations between low-level face feature characteristics and the editing operations, increasing the difficulties for the model detection as revealed by our experiments in Sec. [5](#S5).

Regional Editing.
After the pre-processing of raw images and the mask selection, we then proceed to the specific editing step.
As previously mentioned, we use GANs-based E4S [] and MAT [], and DMs-based DiffSwap [] and DiffIR [] as the deep generators.
Specifically for face swapping and attribute editing, the deep generators also take a reference image as input in addition to the original face image and binary masks.
In contrast, inpainting models take an image with missing regions grounded by our editing masks as input and output an image completed by generative models, as illustrated in Fig. [3](#S3.F3).

Post-processing.
To better ensure the quality of our *DETER* dataset, we apply a series of post-rendering techniques on the output of various deep generators, which include color matching, Poisson fusion, and image sharping. These operations alleviate the boundary effects (i.e., low-level visual image distortions perceivable by human eyes) in conventional forgery construction pipelines and further boost the quality of our dataset.
We then paste the face regions back into the original images to get the final edited images, which strictly ensures that our mask annotations precisely reflect the actual regions manipulated by generative models.

Better Visual Quality and ID Preservation.
We demonstrate the high quality of our dataset in both qualitative and quantitative assessments.
On the one hand, as illustrated by the samples in Fig. [1](#S1.F1), the edited images from *DETER* can be hardly detected by bare eyes, which is further confirmed by our human studies in the next section.
On the other hand, our dataset has a lower ID-distance [] score of 0.30, compared to the most recent DGM${}^{4}$ [] dataset that has the same score of 0.93 based on 10,000 sampled images, indicating *DETER* has the better identity preservation quality.

## 4 Human Study and LLMs

To validate the visual consistency and harmony of the proposed dataset, we perform human studies and LLMs-based evaluations with the state-of-the-art GPT-4 from OpenAI [].
The human studies and data analysis are conducted under appropriate Institutional Review Boards (IRB) approval and regulations.
While humans are usually believed to be the performance upper-bound in various computer vision tasks to ground the model learning such as object recognition and segmentation [, ], they become *lower-bound* in fake detection. ChatGPT performs even worse than human in this case, which further confirms the quality of our *DETER*, as well as the great potential and necessity of model assistance when deploying responsible Generative AI in real life.

### 4.1 General Quality Assessment

In the first layout of human studies, we investigate the human performance on the general fake detection, which resembles the conventional binary image classification problem, similar to previous works [, , ].

Specifically, we prepare 400 image triplets, with each including two real images and one edited image, and ask the human evaluators to pick the fake one.
In addition to the above three image options, we also include a supplementary *“I am not sure”* option, which allows the evaluators to forfeit instead of forcing them to make a choice when it comes to hard samples.
Among 400 edited images, half of them are randomly selected from our *DETER*, with another half equally sampled from existing deep fake sources including SeqDeepFake [], DGM${}^{4}$ [], OpenForensics [], and DDPMs [].
We report the distribution of picks, as well as the detection rate conditioned on correct picks in Tab. [2](#S4.T2).
Given *an equal population* of fake images, the detection rate on *DETER* is 20.4% lower than the ensemble of other fake sources, which demonstrates the high visual quality of our edited images.

**Table 2: User-study and LLMs results for general quality assessment. “Human picks” indicates humans/LLMs prefer marking the candidate as the fake image, while “detection rate” represents the pick proportion over all picked fake images. Given an equal population, the detection rate is much lower on *DETER* compared to the ensemble of other fake sources, demonstrating *DETER* has *better visual quality* more difficult to be detected by humans.**
| Choices | Real | Others datasets | *DETER* | Unsure | Total |
| --- | --- | --- | --- | --- | --- |
| Human picks | 38.3% | 23.7% | 15.7% | 22.3% | 100% |
| Human detection rate | - | 60.2% | 39.8% | - | 100% |
| LLMs picks | 0% | 3% | 2% | 95% | 100% |
| LLMs detection rate | - | 60% | 40% | - | 100% |

### 4.2 Regional Fake Detection

We conduct another more fine-grained layout of human evaluations for regional fake detection using another 100 triplets.
In this case, each image triplet comprises the same edited image from *DETER*, with each one grounded in different regions. One of these regions represents the ground truth, while the other two are randomly selected as distractors.
Similar to the first layout, we ask the human evaluators to pick among the three candidate regions or choose to forfeit by picking the *“I am not sure”* option.
The results in Tab [3](#S4.T3) show that this is a more challenging task for humans, with the rate to pick the GT regions close to a random guess.

**Table 3: User-study and LLMs results on regional fake selection. Picking the edited region is a more challenging task for human evaluators, and the rate for picking the GT regions is similar to a random guess.**
| Choices | Random regions | GT | Unsure | Total |
| --- | --- | --- | --- | --- |
| Human picks | 59.0% | 30.3% | 11.7 % | 100% |
| LLMs picks | 0% | 0% | 100% | 100% |

### 4.3 LLMs

To comprehensively assess the quality of the *DETER*, we also adopt GPT-4 [], the state-of-the-art LLMs that can process multimodal information including images and texts, for evaluating the fake detection performance.
We follow proper prompt tuning to setup the similar evaluation process as the human studies for general quality assessment and regional fake detection.
The results are integrated into Tab. [2](#S4.T2) and Tab. [3](#S4.T3) based on our 100 queries for each evaluation task.
Based on our evaluations, GPT-4 tends to give an uncertain answer, by picking the option *“I am not sure”*.

## 5 Benchmark for Regional Fake Detection

In this section, we present our benchmark suites based on the *DETER* dataset for the regional deep fake detection task.

**Table 4: Quantitative evaluation results for regional fake detection under *(C)*onventional (i.e., training w/o negative image samples) and *(I)*mporved (i.e., training with negative image samples) settings. We only report the scores calculated with IoU=0.5 in the main paper due to space limit, with more results in Appendix [9](#S9). All metrics are the higher the better; best and worst results are marked in bold and underlined, respectively. Note that the *region-based image-level classification accuracy* is an extra metric in our evaluation protocols that explicitly reflects the *image-level* false alarm rate within the formulation of regional detection and segmentation.**
| Methods |  | Classification<br>(image-level) | Object Detection<br>(box-level) | Instance Segmentation<br>(mask-level) |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Swap | Inpaint | Attribute | Swap | Inpaint | Attribute | Swap | Inpaint | Attribute |  |  |  |  |  |  |
|  | Setup | Accuracy | Precision | Recall | AP | Precision | Recall | AP | Precision | Recall | AP | Mask AP |  |  |  |  |
| MaskR-CNN 17’ [] | Conventional | 0.51 | 0.43 | 0.41 | 0.25 | 0.97 | 0.97 | 0.24 | 0.92 | 0.86 | 0.35 | 0.95 | 0.87 | 0.96 | 0.85 | 0.87 |
| Yolact 19’ [] | 0.52 | 0.45 | 0.45 | 0.08 | 0.97 | 0.96 | 0.06 | 0.89 | 0.77 | 0.10 | 0.91 | 0.77 | 0.96 | 0.75 | 0.77 |  |
| Mask2Former 22’ [] | 0.47 | 0.42 | 0.40 | 0.20 | 0.97 | 0.95 | 0.20 | 0.88 | 0.73 | 0.31 | 0.92 | 0.84 | 0.95 | 0.73 | 0.84 |  |
| FasterR-CNN 15’ [] | 0.53 | 0.43 | 0.41 | 0.27 | 0.97 | 0.97 | 0.25 | 0.90 | 0.83 | 0.37 | 0.93 | 0.85 | - | - | - |  |
| YOLOX 21’ [] | 0.54 | 0.51 | 0.52 | 0.29 | 0.96 | 0.96 | 0.30 | 0.91 | 0.80 | 0.43 | 0.93 | 0.86 | - | - | - |  |
| DINO 22’ [] | 0.44 | 0.38 | 0.41 | 0.11 | 0.97 | 0.96 | 0.11 | 0.93 | 0.84 | 0.19 | 0.96 | 0.87 | - | - | - |  |
| MaskR-CNN 17’ [] | Improved | 0.75 | 0.68 | 0.64 | 0.45 | 0.97 | 0.96 | 0.41 | 0.91 | 0.88 | 0.53 | 0.93 | 0.89 | 0.96 | 0.88 | 0.89 |
| Yolact 19’ [] | 0.85 | 0.78 | 0.74 | 0.47 | 0.97 | 0.96 | 0.35 | 0.88 | 0.85 | 0.45 | 0.88 | 0.83 | 0.96 | 0.83 | 0.83 |  |
| Mask2Former 22’ [] | 0.78 | 0.70 | 0.65 | 0.44 | 0.97 | 0.96 | 0.37 | 0.87 | 0.83 | 0.48 | 0.90 | 0.84 | 0.96 | 0.83 | 0.84 |  |
| FasterR-CNN 15’ [] | 0.77 | 0.69 | 0.65 | 0.50 | 0.97 | 0.96 | 0.43 | 0.89 | 0.86 | 0.55 | 0.91 | 0.87 | - | - | - |  |
| YOLOX 21’ [] | 0.92 | 0.86 | 0.82 | 0.78 | 0.96 | 0.95 | 0.68 | 0.90 | 0.88 | 0.74 | 0.88 | 0.85 | - | - | - |  |
| DINO 22’ [] | 0.74 | 0.67 | 0.67 | 0.28 | 0.97 | 0.97 | 0.22 | 0.93 | 0.88 | 0.36 | 0.96 | 0.92 | - | - | - |  |

### 5.1 Improved Experimental Setup

As we have previously mentioned, current evaluation benchmark suites for regional fake detection tend to introduce spurious correlations during the dataset construction, which leads to biased seemingly good performance under the conventional detection and segmentation task setup and evaluation protocols.

Testing Setting Closer to Practice.
The testing setup aligns with our improved training designs, in which we incorporate another 90K unedited real images, and each operation task comprises 30K distinct images.
The motivation for our testing design again is to create a setup closer to the practical situations where a large population of images should be free of generative manipulations.

Conventional and Improved Training Settings.
In the conventional object detection and instance segmentation training setting, the model learns to distinguish positive and negative regions *within the same image*.
However, relying solely on intrinsic features for self-comparison introduces a strong prior: the models tend to *assume there should always be the presence of target regions* given an image.
This bias is even further amplified in the case of the regional fake detection problem due to relatively fixed edited region patterns and generators, but it contradicts the real-life scenario where a large number of images on the Internet should be real images without any generative manipulations.
To this end, we investigate two training settings in our experiments: the conventional setup with no image-level negative samples (i.e., all the training images are from our *DETER* training split, each including at least one positive edited region), and an improved setting with image-level negative samples (i.e., the mixture of our training split and another 140K unseen real images without any manipulations).

Improved Evaluation Protocols.
Our evaluation protocols include classic metrics for detection and segmentation tasks, as well as an extra *region-based image-level classification accuracy*.
For classic evaluation metrics, we adopt Precision, Recall, the standard COCO-style Average Precision (AP) [] for box-level detection, and the Segmentation AP for instance segmentation.

The objective of the region-based image-level classification accuracy is to *reflect the image-level false alarm rate*, as supplementary to Precision which reveals the box-level false positives.
The detection and segmentation models trained in our improved setup predict regions that are believed to be edited by generative models.
During inference, we first count the detected boxes with an IoU greater than 0.5 with the ground truth ones as positive regions. If there are *no missed detected regions or false positives* in the given image, we then consider it as correctly classified.
Note this is *different from* the conventional image binary classification, but rather an extra metric introduced *within the formulation of detection and segmentation tasks*.

### 5.2 Experimental Details

Baseline Methods.
We experiment with six detection and segmentation models covering the most classic to the state-of-the-art methods: Mask R-CNN [], Yolact [], Mask2Former [], Faster R-CNN [], YOLOX [], and DINO []. Among these methods, Mask R-CNN [] and Faster R-CNN [] stand out as well-known convolutional-based two-stage methods, providing a reliable baseline.
Mask2Former [] and DINO [] build upon the success of DETR [], utilizing the transformer-based architecture to model detection and instance segmentation as a direct set prediction.
The remaining methods are single-stage and aim at real-time performance.

Implementation Details.
All the methods used ResNet50 [] as the backbone for a fair comparison, except for YOLOX [], which utilized DarkNet53 []. The models were initialized with COCO pretrained weights to enhance performance. We adhered to default settings with slight modifications in epochs and trained the models on 8 Nvidia RTX 4090.
Specifically, for the improved training setting, we do not skip the real images with no forgery regions, but use them as abundant negative samples to update the region proposal networks or classifiers in contrast to the default training where data samples with no foreground bounding boxes usually are skipped.

### 5.3 Evaluation Results and Analysis

We present break-down experimental results and analysis below, with more details can be found in Appendix [9](#S9).

**Table 5: Quantitative results in terms of *Inpainting* and *Attribute editing* data in cross-domain experiments with Mask R-CNN []. Scores are calculated with IoU=0.5. Models trained with inpainting data achieve better cross-domain performance.**
| Train<br>Test | Inpaint | Attribute |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Precision | Recall | AP | Precision | Recall | AP |  |
| Inpaint | 0.66 | 0.92 | 0.91 | 0.47 | 0.47 | 0.39 |
| Attribute | 0.07 | 0.23 | 0.08 | 0.50 | 0.95 | 0.90 |

Spurious Correlations and Inpainting.
The editing regions for face swapping, attribute editing, and inpainting operations are approximately squares of 176, 78, and 40, respectively.
While the detection difficulties are seemingly related to the area of edited regions by intuition, i.e., larger areas of modification tend to be easier to detect, we observe that this does not hold for current detection and segmentation models as shown in Tab. [4](#S5.T4).
Specifically, we note the edited regions with *inpainting* are consistently more difficult to predict compared to both *face swapping* and *attribute editing*. For example, the precision on inpainting data is on average 0.11 lower (i.e., 0.30 versus 0.41) than that of attribute editing across all models.
The operation-wise difficulty variance further validates our initial claim on the spurious correlations introduced in the dataset construction stage with oversimplified editing types.
Our proposed *DETER* dataset seeks to mitigate the above by integrating *inpainting* to diversify the editing regions and shapes.

Generalization Ability of Inpainting.
We conduct cross-domain experiments to study the generalization ability of different editing operations.
As shown in Tab. [5](#S5.T5), the model performs much better in in-domain testing (training and testing on the same editing operation) and performs worse in the cross-domain case.
We also observe the model trained on the inpainting data has better cross-domain generalization performance compared to the one trained on attribute-edited data. The main reason is that the flexible inpainting operation in *DETER* can be applied on arbitrary face parts, and thus, the model captures the better intrinsic difference between real and manipulated regions, rather than just memorizing the position prior/bias in the training data.
As a result, the model trained on inpainting data has a precision of 0.47 on attribute-edited test samples, similar to the in-domain test precision of 0.50.
Our take-away message here is that regional fake detection models should consider the inpainting training data to avoid spurious correlation.

Image-level and Region-level False Alarms.
The comparisons among various metrics further reveal the high false alarm rate across existing detection and segmentation methods.
Particularly, the models tend to achieve very high recall (e.g., greater than 0.9) but low precision (e.g., lower than 0.3) in the conventional setup.
This recall-precision contrast indicates that the models’ predictions involve a large number of real regions that have been predicted as fake, as shown in Fig. [4](#S5.F4).
The same issue is further supported by our *region-based image classification accuracy*, through which we find a lot of real images are classified as edited, resulting in low classification accuracies.
This is undesired when deploying a reliable regional fake detection system in practice, where most images on the Internet should still be free of generative manipulations.
We believe that a reliable regional fake detection system should ideally *detect edited regions without giving many false alarms*.

Figure: Figure 4: Qualitative results of regional fake detection from Mask R-CNN []. GT, correct predictions, and false positives are annotated in green, blue, and red boxes, respectively. Current models induce a relatively high false alarm rate. Best viewed in color.
Refer to caption: x4.png

Improved Setup with Negative Samples.
Another dimension of our break-down analysis focuses on the improved task setup with mixed real images in training.
Tab. [4](#S5.T4) also include the evaluation results obtained under both conventional training and improved training setup.
Our improved setup *significantly* boost the classification accuracy and precision by *more than 20%* across operations and methods, demonstrating its effectiveness.

GANs vs. DMs Generators.
As a further deep-dive analysis, we also conduct cross-domain experiments on the generative models, with results shown in Tab. [6](#S5.T6).
We report the Precision, Recall, and AP scores under the *inpainting* operation task trained with the conventional setting as an illustration example (more generator-based cross-domain results in Appendix [9](#S9)).
We observe that detection models trained with the GANs-based generators can generalize well to the DMs-based testing images, while the inverse setting induces a non-trivial performance drop.
The above experiments suggest that the GANs-based generators include more robust features that are perceivable by detection and segmentation models, which have not yet been explicitly revealed in the generative modeling area.

**Table 6: Quantitative results in terms of GANs-based and DMs-based generators in cross-domain experiments with Mask R-CNN []. The scores are calculated with IoU=0.5. Models trained with GANs-based generators can achieve *equally good performance* when tested with DMs-based testing images. In contrast, DMs-based training data *induce performance flip when tested in GANs-based testing images.***
| Train<br>Test | GANs | DMs |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Precision | Recall | AP | Precision | Recall | AP |  |
| GANs | 0.48 | 0.90 | 0.87 | 0.48 | 0.91 | 0.88 |
| DMs | 0.38 | 0.76 | 0.71 | 0.53 | 0.92 | 0.89 |

## 6 Discussion and Conclusion

Broader Social Impact.
We seek to raise awareness of the potential malicious impact of generative models and support future research on building effective and reliable detection systems, by introducing an up-to-date, high-quality, and large-scale comprehensive image benchmark for deterring regional generative manipulations.

Ethical Conduct.
All of the images in *DETER* are derived from existing public open access datasets under proper license (Creative Common License) for non-commercial research purposes.
The human studies and data analysis are conducted under appropriate Institutional Review Board approval and regulations.
None of sensitive or personally identifiable information is collected during our studies.

Conclusion and Future Directions.
We introduce our *DETER* dataset for the regional deepfake detection task, featuring an up-to-date, large-scale, high-quality image dataset.
We ensure the quality of our benchmark to catch up with the fast-developing generative AI techniques,
including SOTA generators, novel forgery operations, deep-dive investigations on current benchmarks and their problematic spurious correlation issues, as well as improved benchmark designs as mitigation.

For future research on the detection methods, we explicitly emphasize the significance of a more comprehensive and less biased evaluation system that reflects the real performance of models, with particular attention on the false alarm rate when deployed in real-life scenarios.

## Acknowledgements

We appreciate the support from Princeton First Year Fellowship for Amaya Dharmasiri; also the Princeton SEAs Howard B. Wentz, Jr. Junior Faculty Award for Olga Russakovsky.
We thank William Yang, Angelina Wang, and Max Gonzalez Saez-Diez for helpful input to this work.
This work was also supported in part by the National Natural Science Foundation of China under Grant 62372341.

## 7 More Details about DETER

*DETER* includes 300,000 edited images in total, obtained with three editing operations, as described in our main paper.
For face swapping, inpainting, and attribute editing, there are 93636, 94253, and 112111 images, which corresponds to 106673, 114066, and 199958 regional manipulation masks, respectively.
The image resolutions vary based on the real images, from smaller than 300 to greater than 2048.
Fig. [5](#S7.F5) and Fig. [6](#S7.F6) show the distributions of editing masks and their detailed box heights and widths.

Figure: Figure 5: Distributions of mask sizes in terms of different manipulation operations.
Refer to caption: extracted/5258803/figs/Supplementary/masks.png

Figure: Figure 6: Details of box sizes. *Face swapping* operation has the largest average editing area, followed by *inpainting*, and *attribute editing*.
Refer to caption: x5.png

Fig. [7](#S7.F7) show more qualitative comparisons between other existing deep fake datasets including DFFD 20’ [], SeqDeepFake 22’ [], DGM${}^{4}$ 23’ [], FaceForensics++ 19’ [], ForgeryNet 21’ [], and OpenForensics 21’ [].

Figure: Figure 7: More qualitative comparisons among samples from different deep fake datasets. Image samples in *upper* rows come from existing deep fake datasets: DFFD 20’ [], SeqDeepFake 22’ [], DGM${}^{4}$ 23’ [], FaceForensics++ 19’ [], ForgeryNet 21’ [], and OpenForensics 21’ [], while the *bottom* rows include samples from our *DETER*.
Refer to caption: x6.png

## 8 More Details about Human Studies

This section describes further details about our human studies.
We organize our human studies in two settings.
The first task: General Quality Assesment is selecting the fake image from a triplet of 2 real photos and a fake.
This task is aimed at evaluating the difficulty of spotting the fake images generated by our method vs other methods used in existing datasets.
We use human error in selecting the fake image, as a proxy for the difficulty of spotting cues of deepfake generation, hence the realistic quality of the fake image sample.

The second task: Regional Fake Detection is to select the edited region of a photo. We create samples with a specific facial feature/ region of the face edited or altered using our method.
Each image triplet for this task involves the same edited image from *DETER* but grounded with different regions, among which one is the ground truth region that has been edited, with the other two untouched regions randomly selected as distractors.
We use human error in grounding the edited regions as a proxy for the realistic and subtle nature of localized feature/attribute alterations achieved in our dataset.

### 8.1 Crowdsourcing and Setup

We hosted the two evaluation tasks as separate web apps and crowdsourced them through Cloud Research.
For the first task, we prepared 400 total image triplets, each including two real images and one edited image.
Fake images for 200 of these triplets were randomly selected from our *DETER*, and another 200 equally sampled from existing deep fake sources including SeqDeepFake [], DGM${}^{4}$ [], OpenForensics [], and DDPMs [].
For the second task, we had 100 triplets assembled using 100 photos from our dataset with random facial features/regions altered.

For both tasks, in addition to three image options, we also included a *“I am not sure”* option, which allows the evaluators to forfeit instead of forcing them to make a choice when it comes to hard samples.
The layout of the survey for one selection is shown in Figures [11](#S8.F11) and [11](#S8.F11) respectively for tasks 1 and 2.

For both tasks, we split our triples into multiple surveys containing 50 image triplets each. Each survey with 50 image triplets was completed by 3 human evaluators. To ensure that crowdsourced human evaluators spend adequate time looking for cues of deepfakes in each selection, we encourage them to spend at least 20 seconds on each selection.
The instructions given to the evaluators for the two tasks are shown in Figure.[9](#S8.F9) and [9](#S8.F9).

Figure: Figure 8: Task instructions for human evaluation - General Quality Assessment
Refer to caption: extracted/5258803/figs/Supplementary/quality-instructions.png

Figure: Figure 10: Task layout for human evaluation - General Quality Assessment
Refer to caption: extracted/5258803/figs/Supplementary/quality1.png

## 9 More Details about Regional Fake Detection

We report the experimental results measured with IoU=0.5 in the main paper, and provide additional results with IoU=0.75 in Tab. [7](#S9.T7).
The additional results further validate and support our break-down analysis and take-away messages presented in the main paper.

As shown in Tab. [7](#S9.T7), the performance of various models uniformly decreases with the increasing stringency of IoU constraints, aligning with the overall conclusion of the main paper. Specifically, among the three tasks, inpainting exhibits the poorest performance. This is primarily attributed to the arbitrary shape of masks, in contrast to the relatively fixed mask transformation ranges in the other tasks, further underscoring the issue of spurious correlations during the dataset construction stage. In attribute editing, the modified regions are more fixed compared to face swapping, focusing on specific facial areas such as the eyes, mouth, and nose. Consequently, attribute editing achieves the highest precision. Despite its elevated recall, the precision across all tasks remains at a relatively low level. This discrepancy indicates that the model has biases in the learning process, where it fails to adequately capture the inherent differences between features in real and fake images, leading to a significant number of false positives. To address this issue, we introduce additional real images, i.e., improved settings in Tab. [7](#S9.T7), during the training process to encourage the model to better discern between real and fake images. This strategy results in a substantial improvement of over 20% in precision and accuracy across all tasks and methods. Therefore, ensuring comprehensive learning of distinctions in features between real and fake images is a crucial focal point for advancing the task of fake regional detection.
Fig. [12](#S9.F12) includes more qualitative samples.

We have also provided additional generator-based cross-domain results for both inpainting and attribute editing tasks. From Tab. [8](#S9.T8), it is evident that the cross-domain performance of models trained with the GANs-based generators significantly surpasses those trained with the DMs-based generators, even outperforming the original DMs domain in inpainting tasks. Specifically, models trained with GANs-based generators exhibit superior performance on GANs-based and DMs-based test data (GANs + DMs), once again highlighting the robustness of features generated by GANs over DMs-based features. Additionally, there is complementary information in the features of GANs-based and DMs-based generators, and joint training further enriches the representation of fake features, leading to better results.

Figure: Figure 12: Additional qualitative results of regional fake detection. GT, correct predictions, and false positives are annotated in green, blue, and red boxes, respectively. Current models induce a relatively high false alarm rate.
Refer to caption: x7.png

**Table 7: Quantitative evaluation results for regional fake detection under *(C)*onventional (i.e., training w/o negative image samples) and *(I)*mporved (i.e., training with negative image samples) settings with IoU=0.75. All metrics are the higher the better, best and worst results are marked in bold and underlined, respectively.**
| Methods |  | Classification<br>(image-level) | Object Detection<br>(box-level) | Instance Segmentation<br>(mask-level) |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Operations | Swap | Inpaint | Attribute | Swap | Inpaint | Attribute | Swap | Inpaint | Attribute |  |  |  |  |  |  |
|  | Setup | Accuracy | Precision | Recall | AP | Precision | Recall | AP | Precision | Recall | AP | Mask AP |  |  |  |  |
| MaskR-CNN 17’ [] | C. | 0.51 | 0.40 | 0.40 | 0.24 | 0.97 | 0.96 | 0.20 | 0.77 | 0.73 | 0.33 | 0.88 | 0.81 | 0.958 | 0.718 | 0.807 |
| I. | 0.75 | 0.65 | 0.62 | 0.45 | 0.96 | 0.95 | 0.35 | 0.77 | 0.74 | 0.49 | 0.86 | 0.82 | 0.954 | 0.738 | 0.818 |  |
| Yolact 19’ [] | C. | 0.52 | 0.41 | 0.42 | 0.08 | 0.96 | 0.96 | 0.05 | 0.67 | 0.60 | 0.08 | 0.78 | 0.68 | 0.955 | 0.564 | 0.655 |
| I. | 0.85 | 0.73 | 0.71 | 0.46 | 0.96 | 0.96 | 0.27 | 0.69 | 0.64 | 0.39 | 0.76 | 0.70 | 0.959 | 0.617 | 0.685 |  |
| Mask2Former 22’ [] | C. | 0.47 | 0.38 | 0.38 | 0.20 | 0.96 | 0.94 | 0.16 | 0.70 | 0.56 | 0.28 | 0.85 | 0.76 | 0.946 | 0.578 | 0.758 |
| I. | 0.78 | 0.65 | 0.63 | 0.44 | 0.96 | 0.95 | 0.28 | 0.67 | 0.61 | 0.44 | 0.82 | 0.75 | 0.953 | 0.638 | 0.735 |  |
| FasterR-CNN 15’ [] | C. | 0.53 | 0.40 | 0.39 | 0.27 | 0.97 | 0.96 | 0.20 | 0.72 | 0.67 | 0.33 | 0.84 | 0.78 | - | - | - |
| I. | 0.77 | 0.66 | 0.63 | 0.50 | 0.96 | 0.95 | 0.35 | 0.73 | 0.69 | 0.50 | 0.83 | 0.78 | - | - | - |  |
| YOLOX 21’ [] | C. | 0.54 | 0.48 | 0.49 | 0.29 | 0.96 | 0.95 | 0.26 | 0.77 | 0.69 | 0.40 | 0.86 | 0.80 | - | - | - |
| I. | 0.92 | 0.82 | 0.79 | 0.77 | 0.95 | 0.95 | 0.58 | 0.77 | 0.74 | 0.69 | 0.81 | 0.79 | - | - | - |  |
| DINO 22’ [] | C. | 0.44 | 0.36 | 0.40 | 0.11 | 0.97 | 0.96 | 0.09 | 0.78 | 0.72 | 0.18 | 0.90 | 0.82 | - | - | - |
| I. | 0.74 | 0.65 | 0.65 | 0.28 | 0.97 | 0.96 | 0.19 | 0.79 | 0.75 | 0.33 | 0.90 | 0.85 | - | - | - |  |

**Table 8: Quantitative results in terms of GANs-based and DMs-based generators in cross-domain experiments with Mask R-CNN []. The scores are calculated with IoU=0.5.**
|  | GANs | DMs | GANs + DMs |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Inpaint | Attribute | Inpaint | Attribute | Inpaint | Attribute |  |  |  |  |  |  |  |  |  |  |  |  |
|  | Precision | Recall | AP | Precision | Recall | AP | Precision | Recall | AP | Precision | Recall | AP | Precision | Recall | AP | Precision | Recall | AP |
| GANs | 0.48 | 0.9 | 0.87 | 0.56 | 0.94 | 0.91 | 0.48 | 0.91 | 0.88 | 0.42 | 0.79 | 0.67 | 0.48 | 0.91 | 0.87 | 0.50 | 0.88 | 0.82 |
| DMs | 0.38 | 0.76 | 0.71 | 0.43 | 0.78 | 0.64 | 0.53 | 0.92 | 0.89 | 0.59 | 0.95 | 0.92 | 0.44 | 0.83 | 0.79 | 0.50 | 0.85 | 0.77 |
| GANs + DMs | 0.52 | 0.91 | 0.88 | 0.58 | 0.95 | 0.91 | 0.55 | 0.93 | 0.91 | 0.59 | 0.95 | 0.92 | 0.53 | 0.92 | 0.89 | 0.58 | 0.95 | 0.91 |

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. In your training setup, to address the issue where “models tend to assume target regions exist in every image,” why didn’t you just use the original, unaltered fake images from the open-source dataset?
2. Considering "another 140K unseen unmanipulated images" in the improved setting, will they be released to the research community?
3. It would be interesting to benchmark AIGC detectors pre-trained on previous SOTA datasets against the DETER dataset.
4. Model performance appears similar across various models, from the 2015 Faster R-CNN to DINO. Does model capacity influence performance in this case, and if not, why?
5. What instructions were fed to GPT-4 for prompting in Section 3.3?
6. Other questions are already mentioned in Weakness.
soundness: 3
strengths: 1. The paper presents a new dataset of 30,000 images, which is larger and incorporates different granularities for image editing. It also includes masks for more accurate evaluations.
2. The paper includes many different models for predictions and detection.
summary: The paper introduces a new dataset, which includes different and new deepfake tasks like inpainting. It includes the images edited by two GANs and two Diffusion models. The size of the dataset is 30,000. Through human studies and GPT-4 evaluation, the paper shows the difficulty of prediction and detection over DETER by human beings and GPT-4. Through further benchmarking, the paper shows how incorporating unseen and unaltered images improves model performance. It also delivers insights like the regions edited by inpainting is counterintuitively hard to predict.
weaknesses: 1. In the user study, it is a bad idea to include "I'm not sure" as an option, which will greatly harm the data quality and amount of information. Instead, binary selections can still be enforced, and you can ask participants to indicate their confidence scores. The same is true for GPT-4.
2. The dataset only includes edited images from four different models, which might be insufficient for a comprehensive dataset, especially considering the current flourishing development of generative AI technologies. The claims of difficulty of predicting regions edited by GANs over Diffusion Models are not well supported. By the way, how many models do other SOTA datasets include, as listed in Table 1?
3. A cutoff of IoU of 0.5 is not very informative in showing the benchmarked performance. Why not show the averaged IoU directly?
4. Many of the insights obtained from Section 4.3 are not very informative and are limited to observations. For example, why are images edited by GANs more difficult to predict? Why do eyes and eyebrows raise more errors? Why do models trained on DETER transfer better to OpenForensics (noted that both are face-swapping)?
5. The paper does not include instruction-based image editing, especially those conducted by commercial platforms. This is, however, probably the most significant threat when considering edited images as deepfakes, which can be used and accessed by any ordinary user on the Internet.
6. The dataset is not yet open-sourced. Nor does the paper mention any plans to open-source the dataset.
7. The biggest novelty of the dataset is that it includes different granularities. However, there are no insights into granularities in human studies. For example, why is DETER harder for human beings to identify? Among FaceSwap, Attribute, and Inpaint, which one is the most difficult? It seems that the only purpose of the human study is to show that "it is hard for human eyes".
8. Although images including "multiple faces" are considered novel, no insights are given about this, either in human studies or the benchmark.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Two minor comments/questions: first, in the first paragraph, the authors wrote “the upstream SOTA generative models and their applications, the midstream existing deep fake datasets, as well as the downstream fake detection formulation and models”. I am just curious why the authors define “upstream, midstream, downstream” this way. Also, the authors consider datasets that include humans in this work, and target human face (either the whole face or part of it) manipulation. What about other datasets of natural scenes, animals, etc.? Can the proposed method be applied there as well?
soundness: 3
strengths: see the above summary.
summary: This paper proposes DETER, a large-scale dataset for detecting edited images and regions. The contributions of this work include: 1) in addition to face swapping and attribute editing, the authors also include image inpainting in the data editing operations; 2) to address the spurious correlation challenge in the current datasets for regional fake detection, the authors included additional 90K unedited images; and 3) the authors introduced region-based image-level classification accuracy as an additional assessment criterion. Interesting and important work. Enjoyed reading.
weaknesses: Two points to clarify, rather than weakness.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: 1. How do you ensure the dataset remains relevant as new generative models emerge?
1. How does the post-processing pipeline affect the natural artifacts that might help in detection?
1. Why not include more diverse manipulation techniques beyond facial modifications?
soundness: 3
strengths: The new dataset introduced by this paper can potentially make valuable contributions by addressing a critical challenge in the era of widespread AI image manipulation. It has the following merits: 
- Incorporates current state-of-the-art generative models
- Comprehensive evaluation framework
- Extensive experiments with multiple detection methods
- Well-documented human evaluation studies
- Thorough cross-domain analysis

Clarity. This paper has clear methodology presentation and well-structured experimental results
summary: This paper introduces a dataset for detecting AI-generated image manipulations. There are 3 main contributions:
1. A new dataset that contains state-of-the-art generation methods, which are not incorporated in previous datasets
1. An improved evaluation framework that considers both whole-image and region-specific manipulations
1. Provides techniques to mitigate spurious correlations in fake detection

The dataset contains 300,000 edited images using three types of manipulations: face swapping, attribute editing, and inpainting, created using both GAN and DM-based generators.
weaknesses: The main concerns about this paper are ethical and legal concerns. 
- There is no clear discussion of whether they have rights to modify/redistribute CelebA and WiderFace images. For example, the CelebA dataset has the following agreement: “You agree not to further copy, publish or distribute any portion of the CelebA dataset. Except, for internal use at a single site within the same organization it is allowed to make copies of the dataset.”
- And there is no discussion of consent from individuals in the images. 
- And there is no discussion of rights clearance for redistributing modified faces.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: See Weaknesses
soundness: 3
strengths: 1. The research problem is useful.
2. Dataset for regional deepfake detection is indeed valuable for deep fake detection tasks.
3. The organization of the paper is good.
summary: The paper presents an image dataset for regional deepfake detection task. Recent images generated by current generative models are included.

I have read the response of the authors and the comments of other reviewers. I would keep my original score.
weaknesses: 1. The size of the dataset is not large, which I am afraid cannot be termed as 'large-scale'.
2. In section 4.2, method DINO (Zhang et al., 2023) is published in 2023, in Table 4, it is referred to as DINO' 22.
3. In Table 4, the methods are relatively not up-to-date.
4. Grammar errors: 
a. we use GANs-based E4S (Liu et al., 2023b) and MAT (Li et al., 2022),
and DMs-based DiffSwap (Zhao et al., 2023) and DiffIR (Xia et al., 2023) as the deep generators.
b. Too long sentence: We posit ourselves in the entire research pipeline of
deep fake detection, present an in-depth and comprehensive study, covering the upstream SOTA
generative models and their applications, the midstream existing deep fake datasets, as well as
the downstream fake detection formulation and models, that motivates us to introduce this novel
large-scale fine-grained deep fake detection dataset.

Meta-review: The paper introduces a new dataset, including different and new deepfake tasks like image editing or inpainting, edited by GANs and Diffusion models. The size of the dataset is 30,000. The authors also do human studies and GPT-4 evaluation to show the difficulties of predictions and detection over DETER. Including this new dataset can further improve the fake detection.

Strength:

1. The research problem is useful. Datasets for regional deepfake detection is valuable for de-fake.

2. The paper is good writing.

Weaknesses:

1. The dataset size is not large enough and may quickly be out-of-date due to the development of GenAI. Considering many fake image detection papers will generate image datasets by themselves, the real impacts be limited.

2. Lack of evaluations on state-of-the-art fake image detection methods.

3. Possible ethical problems when publishing this dataset.

Most reviews raise their concerns about the paper's impacts, evaluation, analysis, and ethical problems. I think instead of publishing a debatable dataset, exploring the failure reasons of state-of-the-art fake image detection methods may be a better way and will also benefit the community more. Therefore, I tend to reject this paper.

Decision: Reject