# The New Paradigm of Deepfake Detection at the Text Level
## 4 Results
### Comparison of CDFTM Architectures  
An initial set of tests was performed by evaluating the four input configurations for each of the proposed algorithms. The LightGbmMulti algorithm provided the best results in all cases, as recorded in Table 1.  
Table 1 shows that the best combination is (_AW, UR, CL_), having an accuracy of 97%. This configuration is advantageous for the deepfake problem, as no false positive (FP) predictions exist. FP classifications are problematic because the model assumes the text is generated by AI when, in reality, it is generated by a human. For these reasons, the absence of such assumptions makes this configuration advantageous. The number of false negatives (FN) is very low, specifically, three. This means the model does not incorrectly identify a text generated by AI by classifying it incorrectly as human. Data from Table 1 show that _AC_ does not assist in the correct identification of a deepfake text. Therefore, the architecture of the CDFTM model will use _AW, UR,_ and _CL_ as inputs. The authors provide the following supplementary files:  
* CDFTM Algorithms, which contains the assessment of ML.NET algorithms;
* CDFTM Dataset, which comprises the training dataset;
* CDFTM Validation Dataset, which includes the validation dataset employed; \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Input Configuration** & **TP** & **TN** & **FP** & **FN** & **Accuracy (\%)** & **Precision (\%)** & **Recall (\%)** & **F\({}_{1}\)-Score (\%)** \\ \hline (_AW, AC, UR, CL_) & 36 & 50 & 0 & 14 & 86 & 100 & 72 & 83.7 \\ (_AW, AC, UIR_) & 35 & 49 & 1 & 15 & 84 & 97.2 & 70 & 81.3 \\ (_AW, AC, CL_) & 24 & 37 & 13 & 26 & 61 & 64.8 & 48 & 55.1 \\ (_AW, UR, CL_) & 47 & 50 & 0 & 3 & 97 & 100 & 94 & 96.9 \\ (_AC, UR, CL_) & 38 & 50 & 0 & 12 & 88 & 100 & 76 & 86.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Confusion matrix and performance metrics for the CDFTM. * CDFTM Pseudo Code, which describes the CDFTM pseudo-code;
* CDFTM Results, which is a section of the combination of the 40,320 hyperparameters. Table 2 presents the proposed evaluation algorithms and the values of micro-accuracy and macro-accuracy. This table was constructed through training with the ML_NET 1.7.1 version, using binary classification algorithms. ML_NET automatically calculates the algorithms' evaluation metrics, allowing for a quick comparison of their performance. In the CDFTM_Algorithms.xlsx file, the results generated in detail during the initial algorithm evaluation phase are presented. The algorithms are automatically tuned in multiple iterations by adjusting the training time and the number of iterations. After this initial step, the best two algorithms are selected, and their hyperparameters refined. The data from the file CDFTM_Dataset.xlsx were used for training. These files contain the values for _AW_ in the first column, the values for _AC_ in the second column, followed by the values for _UR_ and _CL_, and finally, the label, which represents \"Human\" or \"AI\". In Table 2, LighGbmMulti has the best micro and macro evaluation accuracy. The micro-accuracy considers the total number of correct predictions divided by the total number of predictions. In this case, for the CDFTM, 92.93% of all predictions made by the model are correct. Instead of weighting classes based on their frequency in the dataset (as in micro-accuracy), the macro-accuracy calculates the accuracy for each class, \"Human\" or \"AI\", independently. The calculated macro-accuracy of 92.87% indicates that the CDFTM performs similarly well across the two classes without being biased towards more frequent ones. Next, additional parameter tuning tests are performed after selecting the two algorithms with the best values for micro-accuracy and macro-accuracy. From the initial test set, a baseline hyperparameterization is obtained, which is later extended by exploring repetitive structures with values close to the baseline values, as shown in the CDFTM_Pseudo_Code.txt pseudo-code employed by the CDFTM.