# Authenticity at Risk: Key Factors in the Generation and Detection of Audio Deepfakes +
## 6 Results and Discussion  
### Results of the Classification of Deepfakes  
In this work, we have used the F1 metric to evaluate the performance of our models, which combines precision and recall into a single value. The choice of this metric is based on its usefulness in scenarios with imbalanced data, as is the case of our dataset, where one class is represented in greater proportion than the other. By balancing the consideration of both false positives and false negatives, the F1 metric avoids favoring the prediction of the majority class, thereby capturing the overall performance of the model more accurately. The summary of the F1 score results for each of the classifiers, together with their corresponding standard deviation for each of the two datasets, is presented in Table 2. From this table, it can be seen that, in general, the performance of the classifiers is higher for _WELIVE_ than for _HABLA_. While for _WELIVE_, the Linear SVM classifier achieves the best F1 score with a value of 98.34 and a low standard deviation of 0.36, in the _HABLA_ dataset, the Random Forest classifier stands out with the best F1 score of 94.60, albeit with a relatively high standard deviation of 1.63. On the other hand, the Naive Bayes classifier shows the lowest performance in both datasets, with an F1 score of 85.33 and a standard deviation of 1.29 for _HABLA_ and an F1 of 95.53 with a deviation of 1.24 for _WELIVE_, the highest of all classifiers for that dataset. The results in this table indicate that there is a difference inthe stability of classifier performance between the two datasets, reflected in the respective standard deviations. On the other hand, the difference between the classifier with the highest F1 value and the one with the lowest performance is 2.81 in the _WELIVE_ dataset. In the case of _HABLA_, three of the classifiers obtain similar F1 values; however, the difference between the classifier with the highest F1 and the one with the lowest performance reaches 9.27. Due to these results, we can conclude that the detection of deepfakes is easier in _WELIVE_ than in _HABLA_ and we believe that this is due to the greater difficulty in creating them when the acoustic context is more complex. It should be noted, however, that the results of the two datasets are not directly comparable as both the number of samples used and the recording conditions of the audios are very different. Figures 5 and 6 present the confusion matrix corresponding to each classifier for both datasets. The main difference between these matrices lies in the fact that, for _WELIVE_, all the audios that are not correctly labeled correspond to real audios that were classified as false, with the exception of the linear SVM classifier, which, among the seven misclassified audios, records one false audio, erroneously labeled as real. On the other hand, when analyzing the confusion matrix of _HABLA_, it is observed that the percentage of misclassified audios, both false labeled as real and real labeled as false, is approximately balanced. Even for the KNN classifier, of the 10 misclassified audios, eight are fakes that were labeled as real. Since we have balanced the two classes in terms of the number of audios, these observations suggest that it is more difficult to trick a classifier into labeling fake audios as real when the deepfakes contain acoustic context. Additionally, we have estimated the quality of both _HABLA_ and _WELIVE_, using the model proposed by [54], which is designed to predict speech quality with a focus on distortions specific to communication networks. This model not only predicts overall speech quality but also evaluates specific dimensions such as Noise, Coloration, Discontinuity, and Loudness. The results show that for _HABLA_, the mean opinion score (MOS) of the audio is 4.37, while for _WELIVE_ it is 1.68. This indicates that the audio corresponding to _HABLA_ with a high MOS exhibits a significantly superior quality, which could favor the generation of deepfakes with a higher level of realism.