<figure><img src="image_1.png" /></figure>

HAL
open science

# Deepfake Detection Using Multiple Feature Fusion

Ya Zhang, Xin Jin, Qian Jiang, Yunyun Dong, Nan Wu, Shaowen Yao, Wei Zhou

<figure><img src="image_2.png" /></figure>

► To cite this version:

Ya Zhang, Xin Jin, Qian Jiang, Yunyun Dong, Nan Wu, et al. Deepfake Detection Using Multiple Feature Fusion. 18th IFIP International Conference on Digital Forensics (DigitalForensics), Jan 2022, Virtual, United States. pp.123-139, 10.1007/978-3-031-10078-9_7. hal-05315728

HAL Id: hal-05315728

https://inria.hal.science/hal-05315728v1

Submitted on 15 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_4.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Chapter 7

DEEPFAKE DETECTION USING
MULTIPLE FEATURE FUSION

Ya Zhang, Xin Jin, Qian Jiang, Yunyun Dong, Nan Wu, Shaowen Yao and Wei Zhou

**Abstract**

It is becoming increasingly easy to generate forged images and videos using pre-trained deepfake methods. The forged images and videos are difficult to distinguish with the human eye, posing security and privacy threats. This chapter describes a deepfake detection method that employs multiple feature fusion to identify forged images. An image is preprocessed to extract high-frequency features in the spatial and frequency domains. Two autoencoder networks are then used to extract the two features of the preprocessed image, following which the extracted features are fused. Finally, the fused feature is input to a classifier comprising three fully-connected layers and a max-pooling layer. The experimental results demonstrate that fusing two features yields an average detection accuracy of 98.62%, much better than using the features individually. The results also show that the method has superior accuracy and generalization capability than state-of-the-art deepfake detection methods.

**Keywords:** Deepfake detection, deep learning, multiple feature fusion

# 1. Introduction

The term deepfake, denoting a synthetic fake image or video created using deep learning, first appeared in 2017 when a Reddit user named “deepfakes” posted a synthesized pornographic video with facial images of celebrities [11]. Conventional image manipulation is done manually using Photoshop, GIMP or other image editing software. However, a deepfake technique uses a large number of images or videos to train a deep learning model over multiple iterations to automatically generate fake images or videos. The forged deepfake images and videos are difficult to distinguish from real images and videos with the human eye.
Deepfake methods can be used to generate films and television clips, virtual fittings and restore portraits of historical figures, which have commercial value. However, deepfakes pose security and privacy threats and seriously pollute cyberspace. Fake news is known to have impacted political discourse in several countries and even undermined the legitimacy of elections [16]. Deepfakes have even greater impacts because the forged images and videos are so realistic that they often generate news stories themselves. Although a number of social media platforms have banned deepfakes, their creation and dissemination are exploding. Several deep-fake pornographic videos have victimized famous female personalities.

Researchers have proposed a number of deepfake detection methods. Bayar and Stamm [2] developed a new type of convolutional layer designed to suppress image content and adaptively learn manipulation features. Zhou et al. [19] proposed a dual-stream detection method that uses a convolutional neural network to learn boundary features and leverages steganographic noise characteristics to identify deepfakes. Durall et al. [7] introduced a novel classification pipeline for artificial face detection using frequency domain analysis; the approach directly extracts image frequency domain features using a discrete Fourier transform to provide classifier inputs. More recently, deepfake detection methods based on Resnet-50, Xception and InceptionV3 that achieve better performance have been developed.

Most deepfake detection methods have problems. Some detection methods extract spatial or frequency domain features of images that, to some extent, ignore the diverse information existing in real and fake images. Other detection methods require large numbers of training samples. Yet other methods do not have a generalization capability or the capability is not documented. Indeed, the lack of generalization is a serious problem in deepfake detection methods.

To address the problems, the proposed deepfake detection method employs multiple feature fusion. It combines spatial and frequency domain features, enabling the effective detection of images and videos generated by a variety of deepfake methods while exhibiting good generalization performance. Additionally, the proposed detection method requires smaller training datasets than competing deepfake detection methods.

## 2. Related Work

This section describes contemporary deepfake generation and detection methods. Also, it discusses the application of multiple feature fusion in image classification.
<figure><img src="image_5.png" /><figcaption>Figure 1. Basic generative adversarial network structure.</figcaption></figure>

## 2.1 Deepfake Generation

Most contemporary deep generative models are based on generative adversarial networks (GANs) or autoencoder networks.

Goodfellow et al. [8] proposed the generative adversarial network concept, which is regarded as a breakthrough in the image generation field. Figure 1 shows the basic generative adversarial network structure. The core idea underlying a generative adversarial network is game theory. The generator is used to create fake images and a discriminator is used to distinguish real images from fake images. During the training phase, the generator and discriminator are optimized iteratively and the parameters are continuously updated until a Nash equilibrium is attained.

Collier et al. [5] proposed the progressively generative adversarial network (ProGAN). The key idea is to grow the generator and discriminator progressively. New layers are added starting with low resolutions and the model increasingly refines details as training progresses to achieve good generation performance. To improve the controllability of forged images, Karras et al. [9] proposed the StyleGAN model that is based on the progressively generative adversarial network. This alternative generator architecture borrows from work on style transfer, which can achieve good image generation effects. However, an obvious flaw with StyleGAN is that the generated images sometimes contain spot-like artifacts. Karras et al. [10] subsequently proposed StyleGAN2, which addresses the flaw in StyleGAN. Recently, Brock et al. [3] proposed BigGAN, a trained generative adversarial network with the largest scale yet attempted, and studied instabilities specific to the large scale. They demonstrated that the maximum performance improvement in a generative adversarial network can be achieved by increasing the number of parameters by two to four times.

An autoencoder comprises an encoder and decoder. Image features are compressed to low-dimensional space by the encoder and are subsequently reconstructed by the decoder. The autoencoder is iteratively trained many times until the output of the decoder is very close to the input of the encoder.
<figure><img src="image_6.png" /><figcaption>*Figure 2.* Deepfake face transformation using an autoencoder.</figcaption></figure>

Figure 2 shows the deepfake face transformation process using an autoencoder. Two encoder-decoder groups are used to swap faces between source and target images, and the parameters are shared between the two encoders. The extracted features of Face A are connected with those of Decoder B to reconstruct Face B from the original Face A.

## 2.2 Deepfake Detection

Deepfake detection is an active area of research. Mo et al. [14] developed a modified convolutional neural network architecture with supervised learning that yielded good detection performance. Afchar et al. [1] proposed a method that automatically and efficiently detects tampered faces in videos. The method employs deep learning and two networks, both with a small number of layers, to focus on the mesoscopic properties of images. Kumar et al. [13] developed a multi-stream network that learns regional artifacts; specifically, a facial image is divided into multiple small patches that are sent to different Resnet-18 networks and ultimately to a fully-connected layer for weight distribution to obtain good detection results.

Other research has focused on preprocessing methods that improve deepfake detection performance. Nataraj et al. [15] extracted co-occurrence matrices on three color channels in the pixel domain and trained a deep convolutional neural network model to detect deepfake images generated by CycleGAN [20] and StarGAN [4]. Younus and Hasan [18] proposed a preprocessing method for deepfake video frames that lever-
ages the Haar wavelet transform to discriminate between different types of edges; the method also determines the extent of blurred images.

Some researchers have attempted to improve generalization performance while enhancing deepfake detection accuracy. Cozzolino et al. [6] analyzed the generalization using a pair of datasets. To quickly adapt their detection network to new manipulation scenarios without retraining the entire network, they added a few images from the target domain to the source domain, which improved the generalization capability. Zhou et al. [19] developed a two-stream tampered face detection technique, where one stream detects low-level inconsistencies between image patches and the other stream explicitly detects tampered faces. The network was trained using FaceSwap and an evaluation using SwapMe demonstrated good performance.

## 2.3 Multiple Feature Fusion

Multiple feature fusion is finding increased applications in image recognition and classification. The approach is often more effective than using a single feature due to the complementary information provided by multiple features. Kikutani et al. [12] employed the weighted fusion of a global feature (color histogram) and local feature (bag-of-features model) for scene image representation and input the fused features into a classifier, significantly improving the recognition rate. Deepfake detection is essentially a two-class classification task. Inspired by the work of Kikutani and colleagues, the proposed method leverages multiple feature fusion for deepfake detection.

# 3. Deepfake Image Detection Method

This section describes the proposed deepfake detection method, including its image preprocessing, image feature extraction and fusion, and classification components. Additionally, it specifies the optimized loss function and deepfake detection algorithm.

## 3.1 Method Overview

Since the fundamental concept is to combine multiple features of a source image, a dual-stream autoencoder network structure is used as the backbone network. Figure 3 provides an overview of the deepfake image detection method. First, high-frequency information and spectrum information of an image $X_i$ are extracted and input to autoencoders. Next, the encoders with squeeze-and-excitation (SE) blocks are trained to extract the features of the preprocessed images, $X_i^{high}$ and $X_i^{freq}$,
<figure><img src="image_7.png" /><figcaption>Figure 3. Deepfake image detection method overview.</figcaption></figure>

respectively. The encoders are subsequently used to fuse the extracted feature maps. Finally, the fused features are input to a classifier through a max-pooling layer.

## 3.2 Preprocessing

The low-frequency information associated with an image conveys its large-scale features whereas high-frequency information conveys the edges and details of the image. Deep image generation models often do not produce perfect image details. Therefore, given a source RGB image $X_i \in R^{W \times H \times 3}$ with width $W$ and height $H$, the high-frequency information map $X_i^{high}$ is extracted as follows:

$$X_i^{high} = X_i - X_i^{low}$$

where $X_i^{low}$ is the low-frequency information map.

A Gaussian linear smoothing filter is very effective at suppressing normally-distributed noise. It is used to scan each pixel in an image $X_i$ using a mask $M$ to yield the low-frequency information map $X_i^{low}$:

$$X_i^{low} = X_i \cdot M$$

where $\cdot$ is the dot product and $M$ is computed using the Gaussian function:

$$G(u, v) = \frac{1}{2\pi\sigma^2}e^{-(u^2+v^2)/(2\sigma^2)}$$

where $\sigma$ is the standard deviation of the normal distribution.
<figure><img src="image_8.png" /><figcaption>Figure 4. Preprocessing results using an Xception network.</figcaption></figure>

The preprocessing operation was tested on an Xception network. Fake images were generated using multiple methods as in [17]. The methods included ProGAN, StyleGAN, BigGAN, StarGAN, GauGAN, StyleGAN2, CRN (cascaded refinement network), IMLE (implicit maximum likelihood estimation) and Deepfake. Figure 4 shows the preprocessing results obtained using an Xception network.

High-frequency information plays a positive role and low-frequency information plays a negative role in deepfake detection. Compared with the feature map extracted in the spatial domain, some image features are more prominent in the frequency domain. In particular, the repetitive nature and frequency features of images can be analyzed using the Fourier transform. The transformation indicates how the energy of a signal is distributed over a range of frequencies. The RGB domain image $X_i$ is transformed to the frequency domain image as follows:

$$X_i^{\text{freq}} = F(X_i)$$

where $X_i^{\text{freq}}$ is the frequency of image $X_i$ and $F(\cdot)$ is the Fourier transform.
<figure><img src="image_9.png" /><figcaption>Figure 5. Autoencoder structure.</figcaption></figure>

For a two-dimensional source image, the Fourier transform is computed as:

$$F(\mu, v) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(x, y)e^{-j2\pi(\mu x + v)} dxdy$$

where $x$ and $y$ are time domain variables, $\mu$ and $\nu$ are frequency domain variables, and $j^2 = -1$.

The high-frequency information is concentrated in the middle and the low-frequency information is distributed in the four corners of the frequency spectrum. To facilitate filter use, the spectrum is centered by the translation property of the two-dimensional Fourier transform. Finally, the centered image in input to autoencoder to extract the features.

## 3.3 Feature Extraction and Fusion

Figure 5 shows the autoencoder structure. A squeeze-and-excitation block (SE-block) can model the dependency between feature channels, which automatically provides the importance of the feature channels. Therefore, a squeeze-and-excitation block is added to an encoder to effectively extract image features.

The encoder comprises six modules. Except for the first module, which has a convolution layer, the remaining five modules all have a squeeze-and-excitation block before the convolution layer and a batch normalization (BN) layer after the convolution layer. A rectified linear unit (ReLU) can address the gradient disappearance problem and speed up training; hence, all the activation functions use a rectified linear unit in the encoder.

The decoder comprises five deconvolution layers. The activation function of each of the first four layers is a rectified linear unit. The tanh activation function is incorporated in the last deconvolution layer because
<table><caption>Table 1. Autoencoder settings.</caption><thead><tr><th></th><th>Type</th><th>Filter Size</th><th>Stride</th><th>Activation Function</th></tr></thead><tbody><tr><td rowspan="5">Encoder</td><td>Conv2D</td><td>3x3</td><td>1</td><td>ReLU</td></tr><tr><td>Conv2D</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td>Conv2D</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td>Conv2D</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td>Conv2D</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td rowspan="5">Decoder</td><td>Deconv</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td>Deconv</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td>Deconv</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td>Deconv</td><td>3x3</td><td>2</td><td>ReLU</td></tr><tr><td>Deconv</td><td>3x3</td><td>2</td><td>Tanh</td></tr></tbody></table>

its range is [-1, 1]. Table 1 provides the details of all the autoencoder settings.

Given a source image $X_i$, features $F_{(i,1)}$ and $F_{(i,2)}$ are extracted from $X_i^{high}$ and $X_i^{freq}$, respectively, as follows:

$$F_{(i,1)} = \text{Encoder}_1 (X_i^{high})$$

$$F_{(i,2)} = \text{Encoder}_2 (X_i^{freq})$$

where Encoder(·) is an encoder used to extract features.

Next, the features $F_{(i,1)}$ and $F_{(i,2)}$ are fused to create the fused features $F_i$:

$$F_i = F_{(i,1)} \cdot F_{(i,2)}$$

where · is the dot product.

Feature map pooling can reduce the feature dimensionality, compress data and parameters, reduce overfitting and improve the fault tolerance of the model. Max-pooling can reduce computations in the upper layers by eliminating the non-maximal values. Therefore, a max-pooling layer is incorporated in the model.

## 3.4 Classification

The classifier has three fully-connected layers with 512, 256 and two neuron nodes in successive layers. The first two layers use a rectified lin-
**Algorithm 1:** Multiple feature fusion deepfake detection.

**Data:** Environment *E*, Detection Image *X* and Label *l*.

**Result:** Trained Model Parameters $\theta$.

Randomly generate environment;

**Image Preprocessing**

for every image do

Obtain preprocessed results $X_i^{high}$ and $X_i^{freq}$;
Resize $X_i^{high}$ and $X_i^{freq}$;
Normalize data;

end

**Network Training**

for every epoch do

Train the network using the stored environment *E*;
for every iteration do

if environment is changed then

Generate network $l'$ and $\tilde{y}$;

Find the global optimal position;

end

Perform action toward the global position

end

end

The unit activation function and the last layer uses the softmax activation function. Images are classified as real (0) or fake (1):

$$X_i = \begin{cases} 0 & \text{if } X_i \text{ is real} \\ 1 & \text{if } X_i \text{ is fake} \end{cases}$$

Finally, the deepfake detection result $l'$ is computed as:

$$l' = F_c(F_i)$$

where $F_c(\cdot)$ denotes the classifier.

Algorithm 1 summarizes the deepfake detection algorithm.

## 3.5 Loss Function

The loss function *L* employed in this work has two parts, a cross-entropy loss $L_{ce}$ and a reconstruction loss $L_{re}$:

$$L = \alpha L_{ce} + \beta L_{re}$$

where $\alpha = \beta = 1$.

Since deepfake detection is a two-class classification problem, cross-entropy can be used to measure the difference between two probability distributions. The cross-entropy loss $L_{ce}$ is computed as:

$$L_{ce} = -(l \log(p) + (1 - l) \log(1 - p))$$
<table><caption>Table 2. Dataset details.</caption><thead><tr><td>Dataset</td><td>Sample Set Size</td><td>Testing Set Size</td></tr></thead><tbody><tr><td>ProGAN</td><td>7,593</td><td>1,820</td></tr><tr><td>StyleGAN</td><td>9,582</td><td>2,400</td></tr><tr><td>BigGAN</td><td>3,200</td><td>800</td></tr><tr><td>StarGAN</td><td>3,200</td><td>800</td></tr><tr><td>GauGAN</td><td>8,000</td><td>2,000</td></tr><tr><td>StyleGAN2</td><td>13,143</td><td>3,192</td></tr><tr><td>CRN</td><td>10,212</td><td>2,552</td></tr><tr><td>IMLE</td><td>10,010</td><td>2,502</td></tr><tr><td>Deepfake</td><td>4,528</td><td>1,132</td></tr></tbody></table>

where *l* is the label of the real image and *p* is the probability of predicting the real image.

The reconstruction loss is the difference between the output data of the decoder and input data of the encoder. In this work, the L1 loss function is used to measure the difference between the input and output data of the autoencoder. The reconstruction loss $L_{re}$ is computed as:

$$L_{re} = L^{l1}(\hat{y}, y) = \sum_{i=0}^{T} |y - \hat{y}|$$

where *y* is the input data to the encoder and $\hat{y}$ is the output data of the decoder.

# 4. Experiments and Results

This section describes the experiments and the experimental results.

## 4.1 Dataset and Experimental Details

Nine datasets, each generated by a different deepfake image generation method, were employed in the experiments. Table 2 provides details about the nine datasets. Note that each dataset is assigned the name of the deepfake method used to generate its images. Real images and fake images account for 50% of each dataset.

In the experiments, each dataset was divided into a training set (80%) and testing set (20%). Figure 6 shows sample fake images (first row) and real images (second row). The two types of images cannot be distinguished by the human eye.
<figure><img src="image_10.png" /><figcaption>Figure 6. Sample fake (first row) and real images (second row) in the testing sets.</figcaption></figure>

The accuracy metric was used to evaluate deepfake detection performance. Model training employed an Adam optimizer with a le-4 learning rate; the number of epochs was 50.

TensorFlow was used to define the layers of the various networks. The experiments employed an Nvidia 2080 Ti GPU with an Intel Core i7-8700K 3.70 GHz processor and 32 GB memory.

<table><caption>Table 3. Ablation experiment setup.</caption><thead><tr><th>Features</th><th>High Frequency</th><th>Frequency Spectrum</th><th>Fused</th></tr></thead><tbody><tr><td>High Frequency Feature</td><td>Y</td><td></td><td></td></tr><tr><td>Frequency Spectrum Feature</td><td></td><td>Y</td><td></td></tr><tr><td>Fused Features</td><td>Y</td><td>Y</td><td>Y</td></tr></tbody></table>

## 4.2 Ablation Experiment Results

An ablation experiment was conducted to validate the proposed deep-fake detection method based on multiple feature fusion. Table 3 shows the experimental setup.

Table 4 shows the results of the ablation experiment. Note that the highest detection accuracy results are underlined. When the preprocessed high frequency and frequency spectrum features were employed as model inputs, average accuracy values of 97.10% and 90.41%, respectively, were obtained. However, after the two preprocessed features were extracted and fused, the average accuracy value increased to 98.62%. This result verifies the efficacy of multiple feature fusion in deepfake image detection.
<table><caption>Table 4. Ablation experiment detection accuracy results.</caption><thead><tr><th>Image Generation Algorithm</th><th>High Frequency Feature</th><th>Frequency Spectrum Feature</th><th>Fused Features</th></tr></thead><tbody><tr><td>ProGAN</td><td>96.25</td><td>93.88</td><td><u>99.34</u></td></tr><tr><td>StyleGAN</td><td><u>99.21</u></td><td>96.92</td><td>98.96</td></tr><tr><td>BigGAN</td><td>95.00</td><td>95.88</td><td><u>97.63</u></td></tr><tr><td>StarGAN</td><td><u>100.0</u></td><td>50.00</td><td><u>100.0</u></td></tr><tr><td>GanGAN</td><td>96.25</td><td>83.95</td><td><u>98.45</u></td></tr><tr><td>StyleGAN</td><td>89.66</td><td>96.37</td><td><u>95.33</u></td></tr><tr><td>CRN</td><td><u>100.0</u></td><td>97.80</td><td><u>100.0</u></td></tr><tr><td>IMLE</td><td>99.88</td><td>99.37</td><td><u>99.92</u></td></tr><tr><td>Deepfake</td><td>97.69</td><td>99.53</td><td><u>97.92</u></td></tr><tr><td>Average</td><td>97.10</td><td>90.41</td><td><u>98.62</u></td></tr></tbody></table>

## 4.3 Comparative Experiment Results

Several classification models, such as InceptionV3, Resnet-50 and Xception, have achieved good deepfake image detection performance. This research implemented and trained several popular models and the model underlying the proposed deepfake detection method, and compared their performance for datasets with deepfake images created by different algorithms. Each model was trained using the training set of each dataset and tested using the testing set of the associated dataset.

Table 5 shows the detection accuracy results. As before, the highest detection accuracy results are underlined. The results reveal that the proposed method performs better than the other methods and its average accuracy over all the datasets reaches 98.62%. In fact, in the case of the StarGAN and IMLE datasets, the proposed method yielded accuracy values of 100%.

## 4.4 Generalization Experiment Results

Several researchers have not considered the generalization capability of their deepfake detection methods. A generalization experiment was conducted to compare the deepfake detection accuracy under generalization.

Table 6 shows the generalization performance results. As before, the highest detection accuracy results are underlined. The results reveal that the proposed method yields the highest average accuracy of 85.69% compared with the other deepfake detection methods when the ProGAN
<table><caption>Table 5. Comparative experiment detection accuracy results.</caption><thead><tr><th>Image Generation Algorithm</th><th>Inception V3</th><th>Resnet-50</th><th>Xception</th><th>Meso [1]</th><th>Meso-Inception [1]</th><th>Proposed Method</th></tr></thead><tbody><tr><td>ProGAN</td><td>53.47</td><td>53.09</td><td>51.05</td><td>55.39</td><td>61.18</td><td><u>99.34</u></td></tr><tr><td>StyleGAN</td><td>49.53</td><td>50.00</td><td>51.95</td><td>74.29</td><td>83.71</td><td><u>98.96</u></td></tr><tr><td>BigGAN</td><td>50.00</td><td>50.00</td><td>50.00</td><td>53.22</td><td>59.62</td><td><u>97.63</u></td></tr><tr><td>StarGAN</td><td>89.09</td><td>50.00</td><td>99.87</td><td>51.50</td><td><u>100.00</u></td><td><u>100.00</u></td></tr><tr><td>GauGAN</td><td>66.00</td><td>64.00</td><td>66.90</td><td>55.96</td><td>81.40</td><td><u>98.45</u></td></tr><tr><td>StyleGAN</td><td>83.18</td><td>80.85</td><td>86.15</td><td>63.31</td><td>90.29</td><td><u>95.33</u></td></tr><tr><td>CRN</td><td>99.88</td><td>99.88</td><td><u>100.00</u></td><td>63.79</td><td>99.73</td><td><u>100.00</u></td></tr><tr><td>IMLE</td><td>99.80</td><td>99.72</td><td>99.82</td><td>84.42</td><td>99.77</td><td><u>99.92</u></td></tr><tr><td>Deepfake</td><td>50.00</td><td>71.23</td><td>47.99</td><td>69.86</td><td>79.91</td><td><u>97.92</u></td></tr><tr><td>Average</td><td>71.22</td><td>68.75</td><td>72.64</td><td>63.53</td><td>83.96</td><td><u>98.62</u></td></tr></tbody></table>

<table><caption>Table 6. Generalization experiment detection accuracy results.</caption><thead><tr><th>Method</th><th>ProGAN</th><th>Mixed</th></tr></thead><tbody><tr><td>InceptionV3</td><td>50.00</td><td>53.83</td></tr><tr><td>Resnet-50</td><td>59.60</td><td>61.56</td></tr><tr><td>Xception</td><td>52.77</td><td>50.00</td></tr><tr><td>Meso [1]</td><td>54.88</td><td>60.31</td></tr><tr><td>Meso-Inception [1]</td><td>59.84</td><td>73.75</td></tr><tr><td>High Frequency</td><td>82.23</td><td>89.93</td></tr><tr><td>Frequency Spectrum</td><td>74.84</td><td>82.23</td></tr><tr><td>Proposed Method</td><td><u>85.69</u></td><td><u>95.42</u></td></tr></tbody></table>

training set was used as the source domain and tested against all the datasets. Additionally, when all the image samples were mixed into the training and testing datasets, the proposed method yields an average accuracy of 95.42%, which is much better than the other deepfake detection methods. Thus, the proposed deepfake detection method is superior to the other methods with regard to generalization capability.

## 5. Conclusions

The deepfake detection method based on multiple feature fusion detects deepfake images in datasets created using several deepfake image generation methods. The fusion of high-frequency spatial domain and frequency spectrum features of images is an innovative concept. Specifically, a dual-stream autoencoder network with squeeze-and-excitation
blocks is trained and the fused features pass to the max-pooling layer before being input to the classifier. This multiple feature fusion strategy enables the deepfake detection method to exhibit superior average detection accuracy as well as generalization capability compared with a number of competing deepfake detection methods. Future research will continue to explore avenues for improving deepfake detection accuracy and generalization capabilities in the face of increasingly sophisticated deepfake image and video creation technologies and tools.

## Acknowledgement

This research was supported by the National Natural Science Foundation of China under Grant nos. 62002313 and 62101481, Key Ar- eas Research Program of Yunnan Province, China under Grant no. 202001BB050076, Science Research Foundation Project of the Yunnan Education Department, Yunnan Province, China under Grant no. 2021Y- 025 and Key Laboratory in Software Engineering of Yunnan Province, China under Grant no. 2020SE408.

## References

[1] D. Afchar, V. Nozick, J. Yamagishi and I. Echizen, MesoNet: A compact facial video forgery detection network, *Proceedings of the IEEE International Workshop on Information Forensics and Security*, 2018.

[2] B. Bayar and M. Stamm, A deep learning approach to universal image manipulation detection using a new convolutional layer, *Proceedings of the Fourth ACM Workshop on Information Hiding and Multimedia Security*, pp. 5–10, 2016.

[3] A. Brock, J. Donahue and K. Simonyan, Large-scale GAN training for high-fidelity natural image synthesis, *Proceedings of the Seventh International Conference on Learning Representations*, 2019.

[4] Y. Choi, M. Choi, M. Kim, J. Ha, S. Kim and J. Choo, StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation, *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 8789–8797, 2018.

[5] E. Collier, K. Duffy, S. Ganguly, G. Madanguit, S. Kalia, G. Shree-kant, R. Nemani, A. Michaelis, S. Li, A. Ganguly and S. Mukhopad-hay, Progressively growing generative adversarial networks for high resolution semantic segmentation of satellite images, *Proceedings of the IEEE International Conference on Data Mining Workshops*, pp. 763–769, 2018.
[6] D. Cozzolino, J. Thies, A. Rossler, C. Riess and L. Verdoliva, Forensic Transfer: Weakly-Supervised Domain Adaptation for Forgery Detection, arXiv: 1812.02510 (arxiv.org/abs/1812.02510), November 27, 2019.

[7] R. Durall, M. Keuper, F. Pfreundt and J. Keuper, Unmasking Deepfakes with Simple Features, arXiv: 1911.00686 (arxiv.org/abs/1911.00686), March 4, 2020.

[8] I. Goodfellow, J. Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville and Y. Bengio, Generative adversarial nets, *Proceedings of the Twenty-Seventh Annual Conference on Neural Information Processing Systems*, pp. 2672–2680, 2014.

[9] T. Karras, S. Laine and T. Aila, A style-based generator architecture for generative adversarial networks, *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 4396–4405, 2019.

[10] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen and T. Aila, Analyzing and improving the image quality of StyleGAN, *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 8107–8116, 2020.

[11] H. Khalil and S. Maged, Deepfake creation and detection using deep learning, *Proceedings of the International Mobile, Intelligent and Ubiquitous Computing Conference*, pp. 1–4, 2021.

[12] Y. Kikutani, A. Okamoto, X. Han, X. Ruan and Y. Chen, Hierarchical classifier with multiple feature weighted fusion for scene recognition, *Proceedings of the Second International Conference on Software Engineering and Data Mining*, pp. 648–651, 2010.

[13] P. Kumar, M. Vatsa and R. Singh, Detecting Face2Face facial reenactment in videos, *Proceedings of the IEEE Winter Conference on Applications of Computer Vision*, pp. 2578–2586, 2020.

[14] H. Mo, B. Chen and W. Luo, Fake face identification via convolutional neural network, *Proceedings of the Sixth ACM Workshop on Information Hiding and Multimedia Security*, pp. 43–47, 2018.

[15] L. Nataraj, T. Mohammed, B. Manjunath, S. Chandrasekaran, A. Flenner, J. Bappy and A. Roy-Chowdhury, Detecting GAN-generated fake images using co-occurrence matrices, *Proceedings of the IS&T International Symposium on Electronic Imaging*, article no. 532, 2019.

[16] A. Pal and A. Chua, Propagation pattern as a telltale sign of fake news on social media, *Proceedings of the Fifth International Conference on Information Management*, pp. 269–273, 2019.
[17] S. Wang, O. Wang, R. Zhang, A. Owens and A. Efros, CNN-generated images are surprisingly easy to spot... for now, *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 8692-8701, 2020.

[18] M. Younus and T. Hasan, Effective and fast deepfake detection method based on Haar wavelet transform, *Proceedings of the International Conference on Computer Science and Software Engineering*, pp. 186-190, 2020.

[19] P. Zhou, X. Han, V. Morariu and L. Davis, Two-stream neural networks for tampered face detection, *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops*, pp. 1831-1839, 2017.

[20] J. Zhu, T. Park, P. Isola and A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, *Proceedings of the IEEE International Conference on Computer Vision*, pp. 2242-2251, 2017.