# Automated Detection of Misinformation: A Hybrid Approach for Fake News Detection
## 5 Experiments and Results
### Comparing ML Models across Feature Extraction Methods  
We explore three feature extraction techniques: TF-IDF, Empath, and our innovative hybrid approach, each paired with six models. Our findings reveal that models utilizing TF-IDF and the hybrid method consistently outperform others (see Table 1 for results on the Kaggle 1 dataset). Notably, logistic regression shines with an impressive accuracy of 0.96. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Extract** & **Model** & **ACC** & **Precision** & **Recall** & **F-Score** & **Time** \\ \hline TF–IDF & LR & 0.95 & 0.94 & 0.95 & 0.95 & 12.39 \\ TF–IDF & DT & 0.88 & 0.87 & 0.88 & 0.88 & 42.26 \\ TF–IDF & KNN & 0.81 & 0.76 & 0.84 & 0.8 & 34.11 \\ TF–IDF & GBC & 0.94 & 0.93 & 0.94 & 0.93 & 196.45 \\ TF–IDF & BNB & 0.81 & 0.74 & 0.93 & 0.82 & 10.96 \\ TF–IDF & MNB & 0.86 & 0.99 & 0.70 & 0.82 & 10.89 \\ \hline Empath & LR & 0.69 & 0.75 & 0.49 & 0.59 & 126.95 \\ Empath & DT & 0.67 & 0.64 & 0.67 & 0.65 & 128.1 \\ Empath & KNN & 0.73 & 0.74 & 0.62 & 0.67 & 127.6 \\ Empath & GBC & 0.77 & 0.77 & 0.72 & 0.74 & 150.76 \\ Empath & BNB & 0.62 & 0.58 & 0.59 & 0.59 & 126.79 \\ Empath & MNB & 0.55 & 0.95 & 0.03 & 0.05 & 126.75 \\ \hline Hybrid & LR & 0.96 & 0.94 & 0.95 & 0.95 & 172.28 \\ Hybrid & DT & 0.91 & 0.9 & 0.91 & 0.9 & 173.89 \\ Hybrid & KNN & 0.84 & 0.8 & 0.79 & 0.8 & 173.28 \\ Hybrid & GBC & 0.95 & 0.93 & 0.95 & 0.94 & 256.1 \\ Hybrid & BNB & 0.85 & 0.76 & 0.95 & 0.84 & 171.7 \\ Hybrid & MNB & 0.89 & 0.97 & 0.75 & 0.84 & 169.09 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results of testing six ML models using different feature extraction methods on Kaggle 1 dataset. Regarding efficiency, the gradient boosting model with the hybrid approach exhibits the longest runtime, clocking in at 256.1 s. Conversely, multinomial naive Bayes (MNB) and Bernoulli naive Bayes (BNB) models leveraging TF-IDF demonstrate the shortest runtimes, both averaging around 11 s.  
Furthermore, MNB consistently outperforms BNB across TF-IDF and the hybrid approach, showcasing higher accuracy alongside shorter runtimes. Like the results from the Kaggle 1 dataset, all models tend to perform better with TF-IDF and the hybrid approach, especially the novel hybrid approach, which yields 99% and 98% accuracy for gradient boosting and logistic regression (see Table 2 for results on the Kaggle 2 dataset). Regarding runtime, MNB and BNB are still the fastest among all six models. However, this time, BNB outperformed MNB with a higher value in all four measurements; meanwhile, MNB is still faster than BNB. All models perform exceptionally well with TF-IDF, particularly decision trees and gradient boosting, achieving near-perfect accuracy and F1 scores (see Table 3 for results on the Merged dataset). This indicates that almost all predictions are correct. However, KNN lags slightly behind, with accuracy and F1 scores around 0.83. This could be attributed to KNN's less effective handling of high-dimensional features. BNB and MNB also exhibit strong performance with accuracy and F1 scores in the 0.93-0.94 range. Except for KNN, all models perform exceedingly well with the hybrid method. Decision trees and gradient boosting models achieve near-perfect accuracy and F1 scores. Logistic regression, BNB, and MNB also show strong performance, with accuracy and F1 scores in the 0.94-0.98 range. Yet, as with the TF-IDF results, KNN is slightly weaker, with accuracy and F1 scores around 0.87. Meanwhile, Empath is outperformed by the other two feature extraction techniques as in the previous six experiments. In terms of runtime, gradient boosting takes the longest in all scenarios, particularly with the hybrid feature extraction method, requiring nearly 400 s. This is likely due to its iterative nature, requiring more time to improve its predictions gradually. Conversely,