Real-Time Deepfake Detection in the Real World

## Abstract

Abstract Recent improvements in generative AI made synthesizing fake images easy; as they can be used to cause harm, it is crucial to develop accurate techniques to identify them. This paper introduces "Locally Aware Deepfake Detection Algorithm" ( LaDeDa ), that accepts a single 9 × 9 9 9 9\times 9 9 × 9 image patch and outputs its deepfake score. The image deepfake score is the pooled score of its patches. With merely patch-level information, LaDeDa significantly improves over the state-of-the-art, achieving around 99 % percent 99 99\% 99 % mAP on current benchmarks. Owing to the patch-level structure of LaDeDa, we hypothesize that the generation artifacts can be detected by a simple model. We therefore distill LaDeDa into Tiny-LaDeDa, a highly efficient model consisting of only 4 4 4 4 convolutional layers. Remarkably, Tiny-LaDeDa has 375 × 375\times 375 × fewer FLOPs and is 10 , 000 × 10\text{,}000\times 10 , 000 × more parameter-efficient than LaDeDa, allowing it to run efficiently on edge devices with a minor decrease in accuracy. These almost-perfect scores raise the question: is the task of deepfake detection close to being solved? Perhaps surprisingly, our investigation reveals that current training protocols prevent methods from generalizing to real-world deepfakes extracted from social media. To address this issue, we introduce WildRF , a new deepfake detection dataset curated from several popular social networks. Our method achieves the top performance of 93.7 % percent 93.7 93.7\% 93.7 % mAP on WildRF, however the large gap from perfect accuracy shows that reliable real-world deepfake detection is still unsolved.

## 1 Introduction

Deepfake images are a leading source of disinformation with government and private agencies recognizing them as a grave threat to society (; ). Recent improvements in generative models, such as DALL-E (), StableDiffusion (), and Midjourney (), significantly lowered the bar of creating fake images. Malicious parties are exploiting this technology to spread false information, damage reputations, and violate privacy online. Recent studies (; ; ) showed that although deepfakes are semantically similar to real images, they have subtle, low-level artifacts that are easier to discriminate. This suggests that deepfake methods may gain from focusing on low-level image features.

We therefore introduce LaDeDa, a patch-based classifier that leverages local image features to detect deepfakes effectively. LaDeDa’s algorithm: i) splits an image into multiple patches ii) predicts a patch-level deepfake score iii) pools the scores of all image patches, resulting in the image-level deepfake score. To allow LaDeDa to work on small image patches, we use a variant of ResNet50 () that replaces some of the $3\times 3$ convolutions by $1\times 1$ convolutions. In particular, the best version of LaDeDa uses a receptive field of $9\times 9$, which we find is an effective size for deepfake image detection. This encourages the classifier to focus on local artifacts rather than global semantics. Using only patch-level information, LaDeDa significantly improves over the state-of-the-art (SoTA), achieving around $99\%$ mAP on the most popular benchmarks.

Since LaDeDa focuses on small patches, we postulate that a very simple model may be sufficient for detecting deepfake artifacts. To test this hypothesis, we design Tiny-LaDeDa, a highly efficient model consisting of only $4$ convolutional layers. We train Tiny-LaDeDa by performing logit-based distillation () using the patch-level deepfake scores predicted by LaDeDa (i.e., the teacher). Remarkably, Tiny-LaDeDa demonstrates superior computational efficiency compared to other SoTA methods (see Fig. 1), allowing efficient deepfake detection on edge devices with a minor decrease in accuracy.

With LaDeDa and Tiny-LaDeDa achieving an almost perfect score on current standard benchmarks, we ask whether deepfake detection is close to being solved. Arguably, the most popular source for spreading deepfakes is social media; we therefore test the performance of recent SoTA methods on deepfakes taken from social platforms. Perhaps surprisingly, we found that when using the current standard training protocols, SoTA methods (including ours) fail. Standard protocols attempt to simulate a real-world generalization, by training a detector using a single generative model (typically ProGAN ()), and evaluate it across other generative models. However, the commonly used datasets in this simulation exhibit preprocessing discrepancies (e.g., real images are in lossy JPEG format while fake images simulated directly from a generator and saved in lossless PNG format), making the protocol less applicable for practical scenarios.

The failure in generalization to in-the-wild deepfakes persists even for methods that use post-processing augmentations that should make them robust to distribution shifts. To address these simulation imperfections, we introduce WildRF, a new deepfake detection dataset curated from popular social networks (Reddit, X (Twitter) and Facebook). WildRF serves as a comprehensive and realistic dataset that captures the diversity and complexities inherent in online environments, which includes varying resolutions, formats, compressions, editing transformations, and generation techniques.

We validate the effectiveness of WildRF by retraining current SoTA methods on it. Our method achieves the top performance of $93.7\%$ mAP on WildRF. Notably, it generalizes across social media platforms (e.g., training on Reddit images and evaluating on Facebook images) and is robust to JPEG artifacts, despite not using post-processing augmentations during training. The evaluation on WildRF shows that there is still a large gap from perfect real-world deepfake detection and highlights the importance of using a real-world benchmark.

To summarize, our main contributions are:

- 1.
Introducing LaDeDa, a state-of-the-art patch-based deepfake detector for the real-world.
- 2.
Distilling LaDeDa into Tiny-LaDeDa, a fast and compact, yet accurate student model for deepfake detection on edge devices.
- 3.
Introducing the WildRF benchmark, extending deepfake evaluation to real-world settings, which are currently lacking in popular simulated datasets.

## 2 Related work

Over the past few years, there has been a remarkable progress in generating photo-realistic images unconditionally (; ; ; ) or using text prompts (; ; ). Deepfake detection aims to classify whether a given image was captured by a camera ("real") or generated via a generative model ("fake"). Two main paradigms exist to tackle this challenge.

##### Deepfake detection by supervised learning.

These methods mostly focus on the architectural designs and discriminative features for discerning real and fake images. Wang et al. () proposed using ResNet50 as a deepfake classifier, trained on real and fake images from one GAN method, and evaluate the performance on other GAN methods. PatchFor () extends this idea, by learning a network that takes in a patch and outputs a deepfake score. While our method uses patches, similarly to PatchFor, we use knowledge distillation to estimate patch-level labels while PatchFor simply copies the image-level labels. Ojha et al. () leverages the pre-trained feature space of CLIP (), by performing linear probing on CLIP’s image representations. Reiss et al. () introduced the concept of "fact checking" for detecting deepfakes, while being training-free and relying solely on off-the-shelf features.

##### Artifact-based detection methods

These methods leverage inductive biases in the image preprocessing stage to discriminate between real and fake images. Marra et al. () revealed that GAN-based methods leave fingerprints in their generated images, which can be extracted using noise residuals from denoising filters. DIRE () focused on diffusion-based methods, measuring the error between an input image and its reconstruction, using a pre-trained diffusion model. Tan et al. () introduced the NPR (Neighboring Pixel Relationships) image representation, aiming to capture the local interdependence among image pixels caused by the upsampling layers in CNN-based generators.

## 3 Method

### 3.1 LaDeDa: Locally Aware Deepfake Detection Algorithm

Our core premise is that it is possible to discriminate between real and fake images with high accuracy based on low-level, localized features. This assumption is based on vast supporting literature (; ; ; ; ; ). We therefore introduce LaDeDa (denoted by $\phi$), a model that maps patches $p_{i}$ into a deepfake score $\phi(p_{i})$, where higher values represent fake patches and lower values real ones.

Figure: Figure 2: LaDeDa Training. By limiting its receptive field to $q\times q$ pixels, LaDeDa yields a deepfake score for each $q\times q$ patch. The image-level deepfake score is the global pooling of the patches scores. We use binary cross entropy loss between the image label and its deepfake score.
Refer to caption: x1.png

Average pooling the patch-wise scores of all image patches results in the image-level deepfake score:

$$ $S(I)=\frac{1}{N}\sum_{i=1}^{N}\phi(p_{i})$ (1) $$

where $I$ denotes the suspected image and $p_{1},p_{2}..p_{N}$ its patches.

LaDeDa is a variant of ResNet50 () (similar to BagNet ()), but replaces most of the $3\times 3$ convolutions by $1\times 1$ ones, limiting the receptive field to $q\times q$ pixels (we use $9\times 9$). The small receptive field forces LaDeDa to focus on local artifacts rather then global semantics, which we hypothesize is a good inductive bias for deepfake detection.

Specifically, for each image patch of size $q\times q$, LaDeDa infers a 2048-dimensional feature representation using multiple stacked ResNet blocks. The network applies a linear layer on the final patch-based representation, resulting in a per-patch score. The image-level score is the global pooling of the per-patch scores. Finally, we apply a sigmoid activation on top of the image-level score resulting in the predicted likelihood that the image is fake. We optimize the network parameters using the binary cross entropy loss:

$$ $\mathcal{L}=-\sum_{f\in\mathcal{F}}\log(\sigma(S({f})))-\sum_{r\in\mathcal{R}} \log(1-\sigma(S({r}))).$ (2) $$

where $\mathcal{R}$ and $\mathcal{F}$ denote the real and fake image datasets respectively, $S$ denotes the network output (deepfake score) given an image, and $\sigma$ is the sigmoid function. See Fig. 2 for LaDeDa illustration.

While our default choice is to pool the patch-level scores using global average pooling, it is also possible (and sometimes desirable) to use other pooling operations. For instance, using max pooling effectively classifies the image based on the most fake patch. Using average pooling, results in classifying the image based on the collective characteristics of all patches. These patch-based deepfake scores can yield an interpretable way to visualize the patches that contribute the most for LaDeDa classification decision (see Sec. 7). Additionally, the linearity of the average pooling operation, makes the architecture distillation-friendly, as we will elaborate on in Sec. 3.2.

Unlike many previous approaches that rely on prior knowledge from large-scale datasets (e.g., ImageNet ()), LaDeDa randomly initializes its parameters and does not require pre-training.
For further details on the LaDeDa’s architecture, see App. 9.1.

##### Relation to PatchFor ( 6 ) .

Both LaDeDa and PatchFor are patch-level methods, and must adapt to deepfake detection datasets that only provide image-level labels. PatchFor independently labels each patch based on the image label with equal weights i.e., all patches extracted from real images as equally real, and all patches extracted from fake images as equally fake. However, not all fake patches are easy to discriminate, and indeed, it is not necessary to correctly label the most difficult fake patches to obtain the correct overall image score.
LaDeDa has the extra flexibility to provide adaptive importance to different patches, putting emphasis on the patches that are more discriminative.

Figure: Figure 3: Tiny-LaDeDa Distillation. Pre-trained LaDeDa (teacher) transfers patch-level deepfake score knowledge to train Tiny-LaDeDa (student).
Refer to caption: x2.png

### 3.2 Tiny-LaDeDa

Since LaDeDa operates on small patches, we hypothesize that a very simple model will suffice for detecting deepfakes . To this end, we propose Tiny-LaDeDa, a highly efficient model, obtained by distilling LaDeDa.
As LaDeDa (i.e., the teacher) outputs patch-wise deekfake scores, we can train a simpler model (i.e., the student) to mimic the teacher’s knowledge; aiming for similar performance, while being much more compact. Specifically, we leverage logit-based distillation (), which aims to transfer the knowledge encoded in the logit outputs (deepfake scores) of the teacher model to the student model.

To train Tiny-LaDeDa, we use a trained LaDeDa model to generate a distillation training set comprising of samples in the form of $(p_{i},\phi(p_{i}))$ for each patch $p_{i}$ of each of LaDeDa’s training images. We then train Tiny-LaDeDa to predict a patch-wise logit (deepfake score), using the MSE loss between the student’s prediction and the teacher’s output (see Fig. 3).
In logit-based distillation, the student is not limited by the teacher’s architecture. Consequently, Tiny-LaDeDa uses only 4 convolutional layers with 8 channels each, yielding a model $4$ orders of magnitude smaller than the teacher. Similarly to LaDeDa, at inference time, Tiny-LaDeDa outputs an image deepfake score, by pooling per-patch deepfake scores.
See App. 9.1 for further details on the Tiny-LaDeDa architecture.

**Table 1: Baseline performance on current (simulated) protocol. mean average precision on 16 generative models from conventional benchmarks.**
| Method | mAP |
| --- | --- |
| CNNDet () | 80.6 |
| PatchFor () | 94.9 |
| CLIP () | 96.3 |
| NPR () | 96.7 |
| LaDeDa(Ours) | 98.9 |

## 4 Is the task of deepfake detection close to being solved?

When training and evaluating LaDeDa using commonly used deepfake detection benchmarks (; ), it achieves a near perfect score of $98.9\%$ mAP, significantly outperforming the current SoTA (Tab. 1). This naturally raises the question: is the task of deepfake detection virtually solved? Essentially, deepfake detection methods must be effective in real-world scenarios. As a sanity check, we evaluated them on a small sample of $10$ images ($5$ real, $5$ fake) taken from popular social networks (for a more comprehensive evaluation, see Sec. 5.1). Surprisingly, performance was near random, suggesting that the current evaluation protocol does not correlate with in-the-wild performance. We therefore reexamine the current evaluation protocols and suggest an alternative.

##### Current: simulated deepfake detection protocol.

Most current methods follow a two-stage evaluation protocol. i) training a deepfake classifier using a set of "real" images and a set of "fake" images generated by a single generative model (typically ProGAN). ii) Evaluating the classifier on a set of real images and a set of generative models, most of which were not used for training. Many detection methods also use post-processing augmentations (e.g., JPEG compression, blur) during training to simulate unknown transformations an image may undergo before being encountered in-the-wild, potentially improving generalization. We refer to this as a simulated protocol.

##### The simulated protocol is suboptimal.

Common datasets used in the simulated protocol comprise real images sourced from standard datasets (e.g., LSUN (), ImageNet ()) in JPEG format (lossy compression), whereas the fake images are generated and saved in PNG format (lossless compression). Training a classifier on such datasets can introduce bias towards differences in compression, leading to inflated perceptions of generalization performance when evaluated on test sets with similar biases (see Sec. 5.1). Moreover, simulating real-world artifacts through augmentations may fail to capture the full diversity of corruptions encountered in practice (see Sec. 5.1). In App. 9.3.1 we provide further details on the commonly used datasets.

##### WildRF: Aligning deepfake evaluation with the real-world.

We propose to improve deepfake evaluation and align it with the real-world by introducing WildRF, a realistic benchmark consisting of images sourced from popular social platforms. Specifically, we manually collected real images using keywords and hashtags associated with authentic, non-manipulated content (e.g., #photography, #nature, #nofilter, #streetphotography), and fake images using content related to AI-generated or manipulated visuals (e.g., #deepfake, #AIart, #midjourney, #stablediffusion, #dalle, #aigenerated). Our protocol is to train on one platform (e.g., Reddit) and test the detector on real and fake images from other unseen platforms (e.g., Twitter and Facebook). We denote this protocol as social. As both train and test data contain the type of variations seen in-the-wild, WildRF is a faithful proxy of real-world performance.
See Fig. 4 for a WildRF overview.

Figure: Figure 4: WildRF Overview. A realistic benchmark consisting of images sourced from popular social platforms: Reddit, X (Twitter) and Facebook. WildRF contains high variability in a range of attributes including image resolutions, formats, semantic content, and transformations encountered in-the-wild.
Refer to caption: x3.png

## 5 Experiments

We compare to SoTA baselines e.g., PatchFor (), CNNDet (), CLIP () and NPR () on the standard metrics: classification accuracy (ACC) ($threshold=0.5$), and average precision (AP).

### 5.1 LaDeDa performance under the current (simulated) protocol

We begin by comparing LaDeDa to the baselines using the current (simulated) protocol. For training, we use the standard train set of the ForenSynth dataset (), which contains real images from LSUN (), and fake images from ProGAN. For evaluation, we use 16 different generative models taken from the test sets of ForenSynth () and UFD (). The results in Tab. 2 demonstrate that LaDeDa and Tiny-LaDeDa outperformed the other baselines in terms of mAP ($98.9\%$, $97.5\%$ respectively), and LaDeDa also outperformed the baselines in terms of mean ACC ($92.7\%$). For more details on the ForenSynth and UFD datasets, see App. 9.3.1.

**Table 2: Baseline performance - simulated protocol. All methods are trained on ForenSynth train set (ProGAN), and evaluated on $16$ generative models: (top) ForenSynth test set (), and (bottom) UFD test set ().**
| Method | ProGAN | StyleGAN | StyleGAN2 | BigGAN | CycleGAN | StarGAN | GauGAN | Deepfakes | Mean |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ACC | AP | ACC | AP | ACC | AP | ACC | AP | ACC | AP | ACC | AP | ACC | AP | ACC | AP | ACC | AP |  |
| CNNDet () | 100 | 100 | 73.4 | 98.5 | 68.4 | 98.0 | 59.0 | 88.2 | 80.7 | 96.8 | 80.9 | 95.4 | 79.3 | 98.1 | 51.1 | 66.3 | 74.1 | 92.7 |
| PatchFor () | 99.6 | 99.9 | 93.9 | 99.2 | 94.5 | 99.6 | 74.4 | 89.6 | 85.3 | 93.5 | 76.3 | 90.4 | 64.7 | 92.4 | 89.2 | 92.8 | 84.7 | 94.6 |
| CLIP () | 99.8 | 100 | 84.9 | 97.6 | 75.0 | 97.9 | 95.1 | 99.3 | 98.3 | 99.8 | 95.7 | 99.4 | 99.5 | 100 | 68.6 | 81.8 | 89.6 | 97.0 |
| NPR () | 99.8 | 100 | 96.3 | 99.8 | 97.3 | 100 | 87.5 | 94.5 | 95.0 | 99.5 | 99.7 | 100 | 86.6 | 88.8 | 77.42 | 86.2 | 92.5 | 96.1 |
| LaDeDa(Ours) | 100 | 100 | 100 | 100 | 100 | 100 | 90.1 | 96.5 | 98.9 | 99.8 | 93.7 | 99.7 | 91.0 | 99.2 | 68.1 | 95.6 | 92.7 | 98.9 |
| Tiny-LaDeDa(Ours) | 98.2 | 100 | 99.1 | 100 | 98.8 | 100 | 86.8 | 94.8 | 79.5 | 95.9 | 96.9 | 99.8 | 84.9 | 91.4 | 85.9 | 98.0 | 91.3 | 97.5 |

##### Poor generalization to real-world data.

While methods trained on ProGAN achieve high performance on detecting deepfakes created by a large set of other image generators, they do not perform well when tested on real-world data. Specifically, we evaluate the baselines using our WildRF dataset (Sec. 4). The results in Tab. 3 show much lower performance than the test set of the simulated protocol. This gap highlights the limitations of the simulated protocol in estimating real-world performance.

**Table 3: Poor generalization to real-world data. Performance of SoTA methods trained on ForenSynth-train (ProGAN), evaluated on WildRF. It shows that training on the standard dataset, instead of in-the-wild deepfakes, generalizes poorly to in-the-wild images.**
| Method | Reddit | Twitter | Facebook | Mean |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ACC | AP | ACC | AP | ACC | AP | ACC | AP |  |
| CNNDet () | 51.2 | 49.7 | 50.3 | 50.2 | 50.0 | 43.4 | 50.5 | 47.8 |
| PatchFor () | 63.9 | 74.0 | 47.8 | 51.3 | 68.2 | 75.3 | 60.0 | 66.9 |
| CLIP () | 60.2 | 66.9 | 56.8 | 63.0 | 49.1 | 44.4 | 55.4 | 58.1 |
| NPR () | 65.1 | 69.4 | 51.7 | 52.5 | 77.8 | 86.3 | 64.8 | 69.4 |
| LaDeDa(Ours) | 74.7 | 81.8 | 59.9 | 67.8 | 70.3 | 90.1 | 68.3 | 79.9 |
| Tiny-LaDeDa(Ours) | 72.3 | 77.8 | 59.6 | 64.8 | 70.9 | 86.4 | 67.3 | 76.3 |

##### JPEG compression bias.

Many methods report their results on the simulated protocol (which is biased, see Sec. 4), with two detector variants: i) training with post-processing augmentations (e.g., JPEG compression, blur), and ii) training without these augmentations. We claim that the later variant, can be biased towards compression artifacts. To demonstrate this, we retrained the CNNDet and CLIP baselines on ForenSynth without JPEG and blur training augmentations. For NPR, we used its official released checkpoint that does not include augmentations. We then evaluated these detectors on $3000$ StyleGAN images ($1500$ real, $1500$ fake) from ForenSynth’s test set, where we JPEG-compressed only the fake images. This setup allows us to examine whether compressing fake images impacts their classification as real ones. As shown in Tab. 4, the detectors’ ability to correctly classify fake images decreases as a function of the compression rate, even at a relatively low compression quality of $90$.
Although the augmentation variants attempt to improve generalization by simulating the unknown transformations an image may undergo, in practice, the simulation is suboptimal (see Tab. 3).

**Table 4: JPEG compression bias. Detectors train under the simulated protocol demonstrate lower performance when JPEG-compressing the test fake images. 100 JPEG quality = no compression, 70 JPEG quality = lowest quality.**
| Dataset | Quality | CNNDet () | CLIP () | NPR () | LaDeDa (Ours) |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ACC | AP | ACC | AP | ACC | AP | ACC | AP |  |  |
| StyleGAN | JPEG 100 | 83.3 | 96.2 | 88.0 | 98.5 | 97.6 | 99.8 | 100 | 100 |
| StyleGAN | JPEG 90 | 50.1 | 83.4 | 66.5 | 90.3 | 50.1 | 43.0 | 52.9 | 68.1 |
| StyleGAN | JPEG 80 | 50.0 | 71.3 | 56.8 | 84.6 | 49.9 | 38.0 | 50.3 | 54.5 |
| StyleGAN | JPEG 70 | 50.0 | 46.4 | 53.9 | 79.3 | 49.9 | 36.8 | 50.0 | 44.7 |

### 5.2 Real-world deepfake detection

##### LaDeDa performance under our (social) protocol.

We retrained and evaluate all methods using our proposed WildRF dataset. The train set comprises (1200 real, 1200 fake) images from Reddit, and the test set comprises (750 real, 750 fake) different images from Reddit, (340 real, 340 fake) images from X (Twitter), and (160 real, 160 fake) images from Facebook. The results in Tab. 5 show that training on real data is much more accurate than on simulated data.

##### JPEG Robustness.

Fig. 5(b) shows that LaDeDa is robust to a range of JPEG compression rates. Note that training our method on WildRF made it JPEG-robust without using post-processing augmentations during training.

**Table 5: Baseline performance - social protocol. All methods are trained on WildRF train set (Reddit) and tested on WildRF test set. The results show a remarkable improvement compared to those achieved when trained using the simulated protocol.**
| Method | Train Set | Reddit | Twitter | Facebook | Mean |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ACC | AP | ACC | AP | ACC | AP | ACC | AP |  |  |
| CNNDet () | Reddit | 75.4 | 86.8 | 71.4 | 84.1 | 70.6 | 83.5 | 72.5 | 85.0 |
| PatchFor () | Reddit | 87.8 | 94.3 | 81.6 | 91.4 | 77.1 | 90.3 | 82.2 | 91.9 |
| CLIP () | Reddit | 80.8 | 94.2 | 78.1 | 93.1 | 78.4 | 90.6 | 79.1 | 92.5 |
| NPR () | Reddit | 89.8 | 95.7 | 79.5 | 90.3 | 76.6 | 88.9 | 81.9 | 91.6 |
| LaDeDa(Ours) | Reddit | 91.8 | 96.0 | 83.3 | 92.8 | 81.9 | 92.6 | 85.7 | 93.7 |
| Tiny-LaDeDa(Ours) | Reddit | 84.5 | 92.4 | 82.3 | 91.7 | 80.7 | 90.4 | 82.5 | 91.6 |

### 5.3 Real-time deepfake detection

Some practical scenarios require deepfake detection to not only be accurate, but also computationally efficient for real-time inference at scale and on edge devices. Fast inference will only get more important as deepfake content continues to spread across online platforms. Here, we evaluate computational efficiency in terms of floating-point operations per second (FLOPs) and network latency (seconds). FLOPs are a hardware-independent measure of computational complexity, quantifying the total number of floating-point operations (addition, subtraction, multiplication or division) required for a single forward pass of a given model. Network latency measures the time it takes the model to process an input (e.g., an image) and output its prediction. Clearly, real-time deepfake detection needs low FLOPs and latency.
We simulate a low resource environment, similar to a mid-range smartphone with a single CPU core, and 4GB RAM. Tab. 6 shows that Tiny-LaDeDa with only a mild degradation in performance compared to LaDeDa, is $375\times$ faster and $10\text{,}000\times$ more parameter efficient.

Figure: (a)
Refer to caption: extracted/5665146/plots/CLIP_ensemble.png

### 5.4 Local is all you need?

While LaDeDa achieves SoTA performance by focusing on local artifacts, we ask if global features provide complementary information. We thus linearly ensemble LaDeDa with the CLIP baseline (), which trains a linear classifier on top of the semantic CLIP features. The resulting score is $S(I)=LaDeDa(I)+\alpha*CLIP_{s}(I)$ where $I$ is an input image. The results in Fig. 5(a) show an increase of $\geq 3\%$ mAP on WildRF, when using the combined score, highlighting that global features are also useful for detecting deepfakes.

**Table 6: Real-time deepfake detection. We show number of FLOPs, Parameters and Latency baselines comparison. In (red), we show the number relative to Tiny-LaDeDa, which demonstrates highly computational efficiency.**
| Method | #FLOPs | #Parameters | Latency |
| --- | --- | --- | --- |
| CNNDet () | $5.40\text{B}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 95)$ | $23.51\text{M}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 18\text{k})$ | $0.75\text{ sec}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 37.5)$ |
| PatchFor () | $1.68\text{B}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 30)$ | $0.191\text{M}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 150)$ | $0.21\text{ sec}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 10.5)$ |
| CLIP() | $51.89\text{B}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 920)$ | $202.05\text{M}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 15\text{k})$ | $9.37\text{ sec}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 470)$ |
| NPR () | $2.29\text{B}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 40)$ | $1.44\text{M}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 1.1\text{k})$ | $0.35\text{ sec}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 17.5)$ |
| LaDeDa(Ours) | $21.23\text{B}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 375)$ | $13.64\text{M}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 10\text{k})$ | $2.87\text{ sec}\color[rgb]{0.8046875,0.37109375,0.37109375}(\times 144)$ |
| Tiny-LaDeDa(Ours) | $0.0566\text{B}$ | $0.00129\text{M}$ | $0.02\text{ sec}$ |

## 6 Ablation

##### Is distillation helpful?

To test this, we trained Tiny-LaDeDa without distillation, using exactly the same data as LaDeDa (the teacher). It achieved decent accuracy on WildRF (mAP of $87.4\%$), but was worse than the larger LaDeDa model (mAP of $93.7\%$) and the distilled Tiny-LaDeDa ($91.6\%$). We further trained Tiny-LaDeDa similarly to PatchFor (), labeling each patch with its corresponding image-level label, achieving $84.5\%$ mAP, which is $7\%$ lower than when using LaDeDa’s estimated patch-level labels via distillation.
We conclude that distillation is indeed helpful, however we see that a relatively simple model can already achieve good results.

##### Patch-size.

LaDeDa’s effectiveness stems from its focus on local artifacts in small image patches. We evaluated LaDeDa using different receptive field patch sizes on WildRF. Fig. 6(b) shows that even with patches of size $q=5$, LaDeDa can effectively discern between real and fake images, suggesting that small patches contain sufficient information for accurate deepfake detection.

Figure: (a)
Refer to caption: extracted/5665146/plots/preprocessing_performance_combined.png

##### Gradient inductive-bias.

We investigate the inductive bias in terms of gradient-based patch representations (See Fig. 6(a)).
LaDeDa achieves high performance even using raw pixels without any preprocessing, but a gradient-based representation generally works better. As we discuss in Sec. 7, this improvement can be attributed to gradients pointing up fine details and textures by focusing on changes in pixel values, effectively highlighting local discriminative features.

##### Pooling operator effectiveness.

LaDeDa can use other pooling operations for image-level scoring beyond average pooling.
Tab 7 shows the effectiveness of using max pooling i.e., the image-level deepfake score is the score of the most fake patch in it. It is slightly better in terms of AP on the ForenSynth test set, and worse on all other evaluation sets.

**Table 7: Pooling operator ablation. LaDeDa test performance on ForenSynth, UFD and WildRF.**
| Method | Pooling | Test set |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ForenSynth () | UFD () | WildRF |  |  |  |  |  |
| ACC | AP | ACC | AP | ACC | AP |  |  |
| LaDeDa | Max | 90.1 | 99.1 | 88.9 | 97.0 | 84.9 | 92.4 |
| LaDeDa | Average | 92.7 | 98.9 | 96.3 | 98.8 | 85.7 | 93.7 |

Figure: (a)
Refer to caption: x4.png

## 7 Discussion and limitations

##### Interpretability.

Since LaDeDa maps each $q\times q$ patch into a deepfake score, we can create a heatmap visualizing the most discriminative patches. In Fig. 7(a), we show two such heatmaps of fake images, where areas with notable intensity changes tend to get high deepfake scores. Delving into the most fake patches (patches with maximal deepfake scores), reveals that fake image patches appear smoother than those from real images (Fig. 7(b)). This aligns with studies (; ; ) showing that generative models leave artifacts in high-frequency components due to the upsampling operation, making it difficult to synthesize realistic textures regions, thus smooth patches potentially smoother in fake images, and less smooth in real images, where a natural camera noise can appear.

##### Generalization to near and far deepfakes.

When trained on a social network, our method and the baselines, generalize well to the other platforms. However, when trained on ProGAN/WildRF datasets the methods do not generalize well to the opposite dataset (WildRF/Simulated). To ensure that a single model can succeed on both protocols, we trained LaDeDa on a combination of 4k ProGAN train images and WildRF train set, achieving comparable results to train and evaluate separately on each protocol.

##### Size of WildRF.

Although WildRF is a medium-sized dataset with around 5000 images, it is already sufficient for achieving high accuracy, likely due to the small scale of deepfake artifacts. While WildRF inevitably contains some biases, we expect these biases to reflect those encountered in real-world scenarios. Increasing the size and diversity of WildRF would likely further advance deepfake detection research.

##### Broader Impacts.

As deepfake content continues to spread across online platforms, effective detection methods are crucial for mitigating the risks of malicious usage. Our proposed approach offers a fast and accurate solution to address these threats.

## 8 Conclusion

We propose LaDeDa, a patch-based classifier that effectively detects deepfakes by leveraging local artifacts, and Tiny-LaDeDa, an efficient distilled version. Despite their high accuracy on current simulated benchmarks, we found that existing methods struggle to generalize to real-world deepfakes found on social media. To address this, we introduced WildRF, a new in-the-wild dataset curated from social networks, capturing practical challenges. While our method achieves top performance on WildRF, the considerable gap from perfect accuracy highlights that reliable real-world deepfake detection remains unsolved. We hope WildRF will drive future research into developing robust techniques against online disinformation that generalize to the real-world.

## 9 Appendix

### 9.1 Architecture details

##### LaDeDa architecture.

The LaDeDa architecture is almost identical to the ResNet50 architecture, except for a few changes in the convolutional layers kernel sizes, strides parameters, and the final fully connected layer. We describe the architecture used for $9\times 9$ patch-size receptive field. LaDeDa follows the standard ResNet50 design with four main residual blocks (layer1, layer2, layer3, layer4) consisting of bottleneck residual units. The first convolutional layer has a kernel size of 1x1 and 64 output channels, followed by a 3x3 convolution with the same number of channels. The residual blocks employ bottleneck units with 1x1 convolutions for dimensionality reduction and expansion. The downsampling operation is a $1\times 1$ convolution with stride 2. The number of channels increases from 64 in layer1 to 256, 512, 1024, and 2048 in subsequent layers. Layer1 consists of $3$ parallel bottleneck units, layer2 has $4$ parallel units, layer3 contains $6$ parallel units, and layer4 has $3$ parallel units. Layer2 is the last layers that uses a kernel size of 3 (the layers after uses a kernel size of 1), thus limiting the receptive field of the topmost convolutional layer. After layer4, a global average pooling operation is applied, followed by a fully connected layer with a single output neuron and a sigmoid activation function, for binary classification.

##### Tiny-LaDeDa architecture.

The Tiny-LaDeDa architecture is a compact version of LaDeDa, designed for efficient performance with a reduced parameter count, using only 4 convolutional layers with 8 channels each.
The architecture includes the following convolutional layers:

- •
$1\times 1$ convolution $(3\text{ input channels},8\text{ output channels})$
- •
$3\times 3$ convolution $(8\text{ input channels},8\text{ output channels})$
- •
$1\times 1$ convolution $(8\text{ input channels},8\text{ output channels})$
- •
$3\times 3$ convolution $(8\text{ input channels},8\text{ output channels})$

Tiny-LaDeDa results in a $5\times 5$ receptive field. The final fully connected layer maps the 8 features to a single output, which is passed through a sigmoid activation function for binary classification. This streamlined architecture is lightweight and computationally efficient, making it suitable for real-time deepfake detection.

### 9.2 Implementation details

##### Training LaDeDa.

To train LaDeDa, we use the Adam optimizer () with $\beta_{1}=0.9$, $\beta_{2}=0.999$, batch size 32 and initial learning rate of $2\times 10^{-4}$. Learning rate is dropped by 10$\times$ if after 5 epochs the validation accuracy does not increase by $0.1\%$, which is the same stopping criteria of (; ). During training, to have a uniform size, all images are resized to $256\times 256$ resolution, and then randomly cropped to native size of $224\times 224$ resolution. Note that we do not use post-processing augmentations (JPEG compression and Gaussian blur) as popular works (; ) do. During validation and test time, we directly resize the image to $256\times 256$ resolution. As for the other baselines, we train them according to their official code repository. To train LaDeDa, we used a single NVIDIA RTX A5000 (21g).

##### Training Tiny-LaDeDa.

To train Tiny-LaDeDa, we use LaDeDa’s patch-wise deepfake scores as soft labeling. We then use the Adam optimizer () with $\beta_{1}=0.9$, $\beta_{2}=0.999$, batch size of 729 (which is the number of $9x9$ patches in an input image, and initial learning rate of $2\times 10^{-4}$.
To train Tiny-LaDeDa, we used a single NVIDIA RTX A5000 (21g).

### 9.3 Simulated protocol

#### 9.3.1 Standard benchmarks in the simulated protocol

##### ForenSynth ( 43 ) and UFD ( 32 ) datasets.

In this work, they chose ProGAN () as their single generative model in train set. Specifically, they used real images from LSUN (), and fake images by train ProGAN on 20 different object categories of LSUN, and generate $18K$ fake images per category, resulting in $360K$ real and $360K$ fake images as the train set. As for the test set, they used ProGAN (), BigGAN (), StyleGAN (), GauGAN (), CycleGAN (), StarGAN (), Deepfakes (), SITD (), SAN (), IMLE (), and CRN ().
Another work of () has suggested the UFD dataset, comprising variations of diffusion models: guided (), GLIDE (), LDM (), and DALL-E () as the fake images. The real images sourced from the LAION () and ImageNet () datasets. In this work they also utilize the train set of the ForenSynth dataset to train their detector.
By examining the dataset publication of LSUN () (the real images in the ForenSynth train set), we can see that all images have been resized to $256\times 256$ resolution, and JPEG compressed with quality of 75. Additionally, the real images in the UFD datasets are also in JPEG format.
In 5.1 we show that methods that use the train set of ForenSynth dataset, can become biased towards JPEG compression artifacts, when tested on a test set with the same biases.

##### GenImage Dataset ( 49 )

In this dataset, the real images are all the images in ImageNet (). The fake images was generated using 100 distinct labels of ImageNet. The train set fake images were generated using Stable Diffusion V1.4 (), and the test set fake images were generated using Stable Diffusion V1.4, V1.5 (), GLIDE (), VQDM (), Wukong (), BigGAN (), ADM () and Midjourney (). In total, GenImage contains $1,331,167$ real and $1,350,000$ fake images.
However, also here, we can observe the preprocessing discrepancy mentioned above. ImageNet images (the real images in GenImage) are in JPEG format, while the generated images in GenImage saved in PNG.

#### 9.3.2 Approaches for mitigating the datasets biases

A concurrent work of Grommelt et al. () showed that training a detector on GenImage can cause it functions as a JPEG detector. To overcome this discrepancy, the authors suggested using an unbiased GenImage dataset where the real and fake images have similar resolutions and are JPEG compressed with the same quality factor.
Chai et al. () suggested to preprocess the images to make the real and fake dataset as similar as possible, in an effort to minimize the possibility of learning differences in preprocessing. To do so, they pass the real images through the data loading pipeline used to train the generator.
As these approaches aim to mitigate the preprocessing differences between real and fake images, our approach uses images sampled from the distribution encountered in-the-wild, aiming to capture real-world artifacts differences between real and fake images.

Figure: Figure 8: Another WildRF image examples.
Refer to caption: x6.png

### 9.4 Related datasets extracted from social networking platforms

##### Chen and Zou ( 9 ) .

In this work, they introduce a dataset of $800k$ ai-generated images with metadata from X (Twitter). However, the dataset is not opensourced and they did not provide real images, so we could not tested our method on it.

##### Zi et al. ( 50 ) .

In this work, they introduce a dataset comprising $7300$ face sequences, with more persons in each scene, and more facial expressions, compared to other deepfakes videos datasets. The faces extracted from $700$ deepfake videos collected from video-sharing websites. As we were not able to get access to this dataset, we could not tested our method on it.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Please see weakness above.
soundness: 3
strengths: 1.Innovative patch-based detection method that leverages local features.

2.Development of Tiny-LaDeDa for efficient edge device operation.

3.Creation of WildRF dataset to better simulate real-world deepfake detection challenges.
summary: The paper introduces a novel deepfake detection algorithm named Locally Aware Deepfake Detection Algorithm (LaDeDa), which processes individual 9x9 image patches to assign a deepfake score. LaDeDa significantly outperforms current benchmarks with approximately 99% mAP. The paper also presents a distilled version, Tiny-LaDeDa, which is highly efficient with reduced computational requirements, making it suitable for edge devices. The authors question whether the task of deepfake detection is nearing a solution but find that current training protocols fail to generalize to real-world deepfakes from social media. To address this, they introduce WildRF, a new dataset derived from social networks to better simulate real-world conditions.
weaknesses: 1.According to the results, it seems that training on the datasets from the social media is better than the datasets from the simulated datasets, if training the detectors under more post-processing augmentations, does the performance will be better than training on the social media datasets?

2.Before the ensemble with CLIP, how do you train the CLIP model? All parameters are updated or just the linear classifier head?
3.In the sanity check, what kind of fake images are tested?

4.LaDeDa detects fake images through the extraction of local image patches. However, some deepfake techniques, such as face swapping only change the face region within the real images. Is the method still useful?

5.It is better to describe the details related to other gradient inductive biases in Fig.7(a).

TYPO: In the 772-th row, ‘teset’ should be ‘test’.

---

confidence: 2
contribution: 2
initial_rating: 5
presentation: 3
questions: Although deepfake detection is a relatively well-addressed problem, the new evaluation benchmark proposed in this paper warrants further attention from the community, and I hope the authors can address the aforementioned weaknesses. However, since I am not fully familiar with this field, my final rating will be based on the opinions of other reviewers.
soundness: 3
strengths: I believe the most significant contribution of this paper is the reconsideration of existing deepfake detection protocols. While current methods have reached their performance ceiling on standard benchmarks, the authors revealed shortcomings of existing deepfake detection algorithms when testing on images sourced from social media—the most popular distribution platform for deepfakes. I think this is an important issue that the community should pay attention to. Furthermore, the paper contributes a new deepfake detection method.
summary: This paper introduces the 'Locally Aware Deepfake Detection Algorithm' (LaDeDa), which detects deepfakes using 9 × 9 image patches and outperforms the current method with nearly 99% mAP. The authors further distil LaDeDa into a lightweight model, Tiny-LaDeDa, which significantly reduces computational requirements and is suitable for edge devices. The authors further highlight limitations in current training protocols that hinder generalization to real-world deepfakes sourced from social media. To address this, they present a novel dataset named WildRF. Performance on WildRF, while leading at 93.7% mAP, indicates that reliable deepfake detection in real-world scenarios remains a challenge.
weaknesses: - Although the paper discusses the robustness of the method against JPEG compression and blur perturbation, I believe it would be beneficial to further evaluate the robustness of deepfake detection algorithms against adversarial perturbations, such as those proposed in [1] and basic L-p norm adversarial method like PGD and FGSM. 
- Moreover, the method's design relatively simple. Compared to PatchFor (the primary comparison method in this paper), it mainly improves the patch weighting strategy. It achieves nearly a 10% improvement on WildRF (the new benchmark proposed in this paper) without involving additional training techniques, which raises concerns about the fairness of this comparison. Could the authors further elaborate on the source of their method's effectiveness (either theoretically or through intuitive explanations)? Additionally, could authors provide the training setup, including both the baseline model and LaDeDa?

[1] Exploring Frequency Adversarial Attacks for Face Forgery Detection, CVPR2022

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 2
questions: Can you explain how images were sourced, any filtering criteria applied, and how the balance between real and fake images was maintained?

What would be the most optimal patch sizes and why? 

Whare are some cases where LaDeDa and Tiny-LaDeDa fail to detect deepfakes in the real world? Or one fail and the other succeed?
soundness: 3
strengths: - Interesting Patch-Based Approach: LaDeDa focuses on small image patches (9×9 pixels). By limiting the receptive field, the model emphasizes local artifacts rather than global semantics.
- Tiny-LaDeDa: The distillation of LaDeDa into Tiny-LaDeDa is a valuable contribution. Tiny-LaDeDa maintains high accuracy while being highly efficient, with 375× fewer FLOPs and 10,000× fewer parameters. This makes it suitable for deployment on resource-constrained devices and real-time applications.
- Introduction of WildRF Dataset
summary: This paper addresses the pressing issue of deepfake detection by introducing "Locally Aware Deepfake Detection Algorithm", a  method that operates on 9×9 image patches to produce deepfake scores. The image-level deepfake score is obtained by pooling the scores of all patches. Building on the insight that generation artifacts can be detected through local features, the authors distill LaDeDa into "Tiny-LaDeDa," an efficient model consisting of only four convolutional layers. Tiny-LaDeDa maintains high accuracy while being computationally efficient enough for deployment on edge devices.
The authors also introduce WildRF, a new dataset of deepfake images curated from popular social media platforms. Through experiments, the paper demonstrates that while LaDeDa achieves top performance on WildRF (93.7% mAP), there remains a significant gap from perfect accuracy, highlighting that reliable real-world deepfake detection is still an unsolved challenge.
weaknesses: -Limited Analysis of Failure Cases: The paper mentions that reliable real-world deepfake detection is still unsolved but provides limited analysis of why current methods fail on real-world data and what specific challenges need to be addressed.
-Ablation Studies Could Be Expanded: More extensive analyses of different components (e.g., impact of patch size, pooling methods, and model architecture choices) would strengthen the work.
-Comparison with More Recent Techniques: The paper primarily compares LaDeDa with somewhat classic existing methods but could benefit from including comparisons with the latest advancements in deepfake detection, especially methods that focus on real-world applicability.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: See Weakness.
soundness: 3
strengths: 1. The paper is well-written and easy to follow.
2. The proposed method is feasible, using a patch-level structure to detect deepfakes is quite reasonable. Using a distillation method to obtain a more efficient model is also a good idea.
3. The experiments are comprehensive and the results are convincing.
4. The introduction of WildRF dataset is a valuable contribution to the community, which is the most attractive part of this paper.
summary: This paper proposes a deepfake detection algorithm called LaDeDa, which accepts a single 9 × 9 image patch and outputs its deepfake score. 
The image deepfake score is the pooled score of its patches. 
With merely patch-level information, LaDeDa significantly improves over the state-of-the-art, achieving around 99\% mAP on current benchmarks. 
They then distill LaDeDa into Tiny-LaDeDa, a more efficient model consisting of only 4 convolutional layers.
The authors also introduce WildRF, a new deepfake detection dataset curated from several popular social networks and show that current training protocols prevent methods from generalizing to real-world deepfakes extracted from social media.
weaknesses: 1. The proposed method is not very innovative. The patch-level structure is not new, and the distillation method is also a common technique. The paper is not very appealing.
2. Since the authors claim that they can do patch-level detection, they should analyze and experiment on inpainted images, i.e., images with part real and part synthetic. Can the method detect deepfakes in this case?
3. The necessity of real-time detection is not fully demonstrated. Can the method be applied to detect deepfakes in generated or spliced videos?