# RumorLLM: A Rumor Large Language Model-Based Fake-News-Detection Data-Augmentation Approach
## 4 Experiments
### Ablation Study  
As part of the ablation study, we investigated the impact of data augmentation, specifically the use of RumorLLM, on the performance of the proposed model for the BuzzFeed dataset. The results, presented in Figures 2 and 3, show the performance contrast between the model without data augmentation and the augmented data by the proposed model.  
On the BuzzFeed dataset, without data augmentation, the model achieved an accuracy of 75.70%, a precision of 55.36%, a recall of 36.90%, an F1 score of 44.29%, and an AUC-ROC of 82.34%. Subsequently, with the integration of RumorLLM for data augmentation, the proposed model showed significant improvements in all metrics. The augmented proposed model showed an accuracy of 82.55%, a precision of 70.00%, a recall of 58.33%, an F1 score of 63.64%, and an AUC-ROC of 86.75%. Notably, these results underscore the  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** & **AUC-ROC** \\ \hline TEXT-RF & 0.7826 & 1.000 & 0.0769 & 0.1429 & 0.6791 \\ LR-Bias & 0.8522 & 0.7647 & 0.5000 & 0.6047 & 0.8619 \\ Ternion & 0.8609 & 0.7500 & 0.5769 & 0.6522 & 0.8429 \\ EANN & 0.8261 & 0.6154 & 0.6154 & 0.6154 & 0.8879 \\ SpotFake & 0.8696 & 0.9231 & 0.4615 & 0.6154 & 0.8844 \\ Proposed & 0.9391 & 0.8519 & 0.8846 & 0.8679 & 0.9233 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance of baseline models and proposed model on the PolitiFact dataset.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** & **AUC-ROC** \\ \hline TEXT-RF & 0.7383 & 0.5000 & 0.1310 & 0.2075 & 0.6230 \\ LR-Bias & 0.7882 & 0.6600 & 0.3929 & 0.4925 & 0.8176 \\ Ternion & 0.7477 & 0.5146 & 0.6310 & 0.5668 & 0.8051 \\ EANN & 0.7321 & 0.4900 & 0.5833 & 0.5326 & 0.7831 \\ SpotFake & 0.7944 & 0.7250 & 0.3452 & 0.4677 & 0.8568 \\ Proposed & 0.8255 & 0.7000 & 0.5833 & 0.6364 & 0.8675 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performance of baseline models and proposed model on the BuzzFeed dataset.  
significant positive impact of using RumorLLM for data augmentation, contributing to significant improvements in precision, recall, F1 score, and AUC-ROC. The results highlight the effectiveness of data augmentation in refining the model's performance and emphasize its role in capturing nuanced patterns and relationships within the BuzzFeed dataset.  
On the Politifact dataset, the model achieved the following scores without data augmentation: an accuracy of 91.30%, a precision of 86.36%, a recall of 73.08%, an F1 score of 79.17%, and an AUC-ROC of 90.86%. However, after incorporating RumorLLM for data augmentation, the proposed model demonstrated significant improvements across all metrics. The augmented proposed model achieved an accuracy of 93.91%, a precision of 85.19%, a recall of 88.46%, an F1 score of 86.79%, and an AUC-ROC of 92.33%. These results highlight the substantial positive impact of using RumorLLM for data augmentation, resulting in significant improvements in precision, recall, F1 score, and AUC-ROC. The findings underscore the effectiveness of data augmentation in refining the model's performance and emphasize its role in capturing nuanced patterns and relationships within the Politifact dataset.  
Figure 3: Performance comparison of the proposed model on the PolitiFact dataset with and without data augmentation.  
Figure 2: Performance comparison of the proposed model on the BuzzFeed dataset with and without data augmentation.