# Analysis of Score-Level Fusion Rules for Deepfake Detection
## 4 Experimental Results
### Results
#### 4.2.1 Intra-Dataset Scenario  
Table 2 reports the performance values related to the single models and all the fusion techniques. The best values for each metric are highlighted in bold to make it easier to read. We can observe that, in terms of the AUC, the best performing single models are EfficientNetAutoAttB4 and EfficientNetB4 ST; however, the last one appears better in terms of the EER and Cohen's d. Overall, among the non-parametric fusion rules, the best one was the simple Average fusion rule, followed by the Bayesian fusion rule, which also showed a slightly lower False Negative Rate at a False Positive Rate equal to 1%. Regarding the approaches based on the weighted average, we noticed that the three considered methods exhibited similar results. Finally, among the methods based on machine-learning models, the best one was the MLP. In general, the results in the intra-dataset scenario show a performance improvement with respect to the single deepfake detectors through the parametric fusion approaches. This was largely expected by the plethora of papers in other applications (e.g., [49; 50]). The exception is the SVM, which showed similar performance to the most performing non-fused models. The non-parametric fusion rules led instead to similar or even worse performance than the single detection systems, except in the case of the ResNet. In particular, the fusion approach that highlighted the best performance in relation to all the explored metrics was the MLP. In Figure 4, the ROC curves related to the intra-dataset scenario (on the FF++ dataset) are shown for the three adopted fusion typologies (Figure 4a-c) and for the best performing model (in terms of AUC) selected from the single models and the three fusion typologies (Figure 4d). For each of the categories, the results related to the single models are reported to allow a direct comparison. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model/Fusion Method** & **AUC** & **Cohens’s d** & **EER** & **FPR@FNR1\%** & **FNR@FPR1\%** \\ \hline EfficientNetAutoAttB4 ST & 0.943 & 2.742 & 0.111 & 0.624 & 0.750 \\ \hline EfficientNetAutoAttB4 & 0.959 & 2.958 & 0.101 & 0.628 & 0.388 \\ \hline EfficientNetB4 ST & 0.959 & 3.037 & 0.089 & 0.564 & 0.589 \\ \hline EfficientNetB4 & 0.958 & 2.829 & 0.099 & 0.688 & 0.343 \\ \hline Xception & 0.954 & 3.423 & 0.095 & 1.000 & 1.000 \\ \hline ResNet & 0.428 & 0.230 & 0.553 & 1.000 & 1.000 \\ \hline Avg & 0.977 & 3.555 & 0.072 & 0.418 & 0.222 \\ \hline Bayes & 0.958 & 3.210 & 0.079 & 1.000 & 0.212 \\ \hline Prod & 0.955 & 1.413 & 0.079 & 1.000 & 0.318 \\ \hline Max & 0.762 & 0.943 & 0.257 & 0.770 & 1.000 \\ \hline Min & 0.945 & 1.600 & 0.118 & 1.000 & 0.381 \\ \hline Accuracy-based Avg & 0.979 & 3.670 & 0.071 & 0.430 & 0.190 \\ \hline Correlation-based Avg & 0.978 & 3.691 & 0.071 & 0.455 & 0.185 \\ \hline MI-based Avg & 0.978 & 3.65 & 0.072 & 0.456 & 0.196 \\ \hline Complement NB & 0.963 & 3.351 & 0.083 & 0.591 & 0.558 \\ \hline MLP & **0.984** & **4.310** & **0.053** & **0.401** & **0.131** \\ \hline Perceptron & 0.979 & 3.793 & 0.071 & 0.433 & 0.206 \\ \hline SVM & 0.957 & 3.864 & 0.058 & 0.794 & 0.949 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The results on the FF++ in the intra-dataset scenario (the best values related to each performance metric are reported in bold). As we can notice from Figure 4a, the simple average performed better than the best single model. This is not true for all the other non-parametric fusion methods: the max rule, for example, performed noticeably worse than the other methods but still better than the worse single model (ResNet). In the case of the weighted averages (Figure 4b), we can observe that all the three considered fusion methods performed better than the single models, as in the case of the methods based on machine learning (Figure 4c), except for the SVM.