# Comparison of Deepfake Detection Techniques through Deep Learning
## 4 Preliminary Results
### Algorithms  
The accuracy, precision and recall rates of analyzed models demonstrated in Table 5, the ROC curve demonstrated in Figure 4, the area under the ROC curve (AUC), F1-scores and AP results demonstrated in Table 6 were used to evaluate model performance in terms of separability and their ability to differentiate between classes. The algorithm comparison results revealed that the VGG-19 model had the best performance among all 3 other algorithms, with an accuracy level of 95%. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Generation Method** & **Fake** & **Real** \\ \hline GAN & 70K & 70K \\ \hline Human Expert & 960 & 1081 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Distribution of the used datasets. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Dataset** & **Drawbacks** & **Improvements** \\ \hline UADF & Lack of quantity and Diversity & Suitable baseline \\ DFTMIT & & \\ \hline DFD & Limited size and methods & Extension to FaceForensics dataset \\ \hline DFDC & Distinguishable visual artifacts & \begin{tabular}{l} Large number of clips of varying \\ quality \\ \end{tabular} \\ \hline Celeb-DF & \
\begin{tabular}{l} Low realness score \\ Biased: impractical for face \\ Forgery detection \\ \end{tabular} &
\begin{tabular}{l} Fixed color mismatch \\ Accurate face masks \\ \end{tabular} \\ \hline Deeper Forensics-1.0 & Challenging as a test database & High realness score \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Dataset analysis summary. The results of this study demonstrate that VGG-19 can be a suitable choice not only for partial face images, but also for full-face images confirming the findings of [57]. The better performance of VGG-19 is because it is pretrained on a wide variety of objects. AP was used as an aggregation function to summarize the precision-recall curve into a single value that represents the average of all precisions. VGG-19, even though it had the highest accuracy, had the lowest AP of 95% in comparison to all other analyzed models. The DenseNet architecture on the original dataset and grayscale dataset had a closer performance to VGG-19, with 94% accuracy. Results from DenseNET architecture demonstrates that gray channel-based analysis does not have a huge impact on model accuracy level in classifying images into the two categories of real and fake. The DenseNet architecture, even though was second best in terms of performance, achieved an AP of 99% on both augmented and grayscale datasets, which is slightly in contrast to the results found in [59] in terms of precision rate; however, it aligns with claims regarding detection time. Custom CNN architecture had the lowest accuracy level (89%). The second-highest AP score after DenseNet was the Custom CNN model. Augmented input reduced model performance and accuracy level on both DenseNET and Custom CNN by 5-22%. However, the Custom CNN had a better performance on augmented data in comparison to the DenseNet architecture. Precision and recall rates from DenseNet architecture trained on augmented data suggest that the final dense block that we have coupled with the DenseNet classification layer did not have a positive impact on model behavior. The issue with reduced performance on augmented data might be resolved by training the model for a larger number of epochs, since augmentation results in harder training samples. VGG-19, even though it was great in terms of performance, aligns with results from [60]; it was computationally very expensive, especially if fed with augmented data. DenseNET was computationally more efficient in comparison to VGG-19 and Custom CNN, which aligns with the results from [40]. The F1-score of the DenseNet architecture on grayscale was the highest, reaching 97% suggesting it could be a suitable backbone when dealing with unbalanced class distribution in their dataset. The second-highest F1-score was achieved by VGG-19, as it achieved a 95% F1-score. The lowest F1-score was achieved by the Custom CNN on augmented data, as the F1-score was only 85%.