# Romanian Fake News Detection Using Machine Learning and Transformer-Based Approaches
## 4 Results
### Performance Evaluation Results  
Tables 1-5 present a comprehensive overview of the testing performance of the machine learning and Transformer-based models on the FakeRom, NEW, and both FakeRom + NEW datasets. The performance metrics accuracy, precision, recall, and F1 score show the differences in the models' effectiveness for Romanian fake news detection. Table 1 shows Naive Bayes achieving moderate performance (84% accuracy) when trained and tested on the same dataset, but its performance drops significantly (32.2%) in cross-dataset scenarios. Table 2 demonstrates Logistic Regression's improved performance, particularly when trained on the NEW dataset (90.9% accuracy). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Trained on** & **Test Set** & **Accuracy (\%)** & **Precision (\%)** & **Recall (\%)** & **F1 Score (\%)** & **ROC AUC (Micro)** & \begin{tabular}{c} **ROC AUC** \\ **(Macro)** \\ \end{tabular} \\ \hline FakeRom & FakeRom & 84.0 & 84.5 & 84.0 & 84.1 & 0.963 & 0.962 \\ \hline NEW & NEW & 83.3 & 83.9 & 83.3 & 83.0 & 0.970 & 0.969 \\ \hline FakeRom & \
\begin{tabular}{c} FakeRom + \\ NEW \\ \end{tabular} & 32.2 & 31.3 & 32.2 & 30.8 & 0.635 & 0.641 \\ \hline NEW &
\begin{tabular}{c} FakeRom + \\ NEW \\ \end{tabular} & 82.9 & 83.5 & 82.9 & 82.6 & 0.969 & 0.969 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results obtained using Multinomial Naive Bayes model. Figure 7: Machine learning vs. Transformed-based model performance (**a**) Accuracy; (**b**) precision; (**c**) recall; (**d**) F1 score. Table 3 highlights SVM's strong performance, especially when trained on the NEW dataset (94.6% accuracy). Table 4 showcases BERT's superior performance across all metrics, achieving the highest accuracy of 96.5% when trained on NEW and tested on the combined dataset. Table 5 illustrates RoBERTa-large's competitive performance (94.4% accuracy) despite some variability across different test sets. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Trained on** & **Test Set** & **Accuracy (\%)** & **Precision (\%)** & **Recall (\%)** & **F1 Score (\%)** & **ROC AUC (Micro)** & \
\begin{tabular}{c} **ROC AUC** \\ **(Macro)** \\ \end{tabular} \\ \hline FakeRom & FakeRom & 92.8 & 93 & 92.8 & 92.8 & 0.989 & 0.989 \\ \hline NEW & NEW & 95.5 & 95.8 & 95.5 & 95.5 & 0.996 & 0.997 \\ \hline FakeRom & FakeRom + NEW & 72.2 & 74.4 & 72.2 & 71.4 & 0.90 & 0.906 \\ \hline NEW &
\begin{tabular}{c} FakeRom + \\ NEW \\ \end{tabular} & 96.5 & 96.7 & 96.5 & 96.5 & 0.998 & 0.998 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results obtained using BERT model. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Trained on** & **Test Set** & **Accuracy (\%)** & **Precision (\%)** & **Recall (\%)** & **F1 Score (\%)** & **ROC AUC (Micro)** & \begin{tabular}{c} **ROC AUC** \\ **(Macro)** \\ \end{tabular} \\ \hline FakeRom & FakeRom & 85.1 & 85.2 & 85.1 & 84.9 & 0.975 & 0.973 \\ \hline NEW & NEW & 90.4 & 90.5 & 90.4 & 90.3 & 0.986 & 0.985 \\ \hline FakeRom & \
\begin{tabular}{c} FakeRom + \\ NEW \\ \end{tabular} & 19.9 & 4 & 19.9 & 6.6 & 0.576 & 0.646 \\ \hline NEW &
\begin{tabular}{c} FakeRom + \\ NEW \\ \end{tabular} & 90.9 & 91 & 90.9 & 90.8 & 0.986 & 0.985 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results obtained using Logistic Regression model. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Trained on** & **Test Set** & **Accuracy (\%)** & **Precision (\%)** & **Recall (\%)** & **F1 Score (\%)** & **ROC AUC (Micro)** & \begin{tabular}{c} **ROC AUC** \\ **(Macro)** \\ \end{tabular} \\ \hline FakeRom & FakeRom & 89.4 & 89.3 & 89.4 & 89.3 & 0.990 & 0.988 \\ \hline NEW & NEW & 94.4 & 94.5 & 94.3 & 94.3 & 0.997 & 0.996 \\ \hline FakeRom & \
\begin{tabular}{c} FakeRom + \\ NEW \\ \end{tabular} & 21.6 & 45.2 & 21.6 & 10.4 & 0.555 & 0.608 \\ \hline NEW &
\begin{tabular}{c} FakeRom + \\ NEW \\ \end{tabular} & 94.6 & 94.8 & 94.6 & 94.6 & 0.997 & 0.996 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results obtained using Support Vector Machine model.