# Romanian Fake News Detection Using Machine Learning and Transformer-Based Approaches
## 4 Results
### Error Analysis  
The models had varying levels of success in detecting different types of fake news: propaganda proved most challenging, with lower detection rates across models, particularly being confused with fake news content; while misinformation was frequently misclassified as propaganda, indicating significant overlap in linguistic patterns.  
Real news was sometimes confused with propaganda, especially in cross-dataset scenarios. BERT demonstrated the strongest overall performance, achieving 96.5% accuracy on the combined test set, showed exceptional performance in detecting satire (ROC AUC of 1.0), and maintained consistent performance across categories, with ROC AUC scores above 0.99.  
RoBERTa-large exhibited specific category strengths: it excelled at satire detection (ROC AUC of 1.0), showed strong performance on real news (ROC AUC of 0.989), and struggled with propaganda (ROC AUC of 0.994) compared to other categories.  
Considering cross-category confusion, models frequently confused propaganda with fake news, particularly in cases involving political content. Misinformation was often misclassified as propaganda when dealing with complex narratives. Real news was occasionally misclassified as propaganda in cases involving strong opinion-based content.  
Models trained on the NEW dataset showed significantly better discrimination across categories compared to those trained on FakeRom, particularly in distinguishing between propaganda and fake news, identifying subtle differences between misinformation and propaganda and maintaining consistent performance across all categories.  
The error patterns reveal specific challenges in fake news detection across different categories. Propaganda proved to be the most challenging category for detection, with models frequently confusing it with general fake news content. This confusion was particularly evident in cross-dataset scenarios, where models trained on FakeRom showed significant performance degradation when tested on the NEW dataset's content. Misinformation was often misclassified as propaganda, indicating substantial overlap in linguistic patterns between these categories. The analysis also shows that real news was sometimes misclassified as propaganda, especially when dealing with opinion-based content or complex political narratives.  
Models trained on the NEW dataset consistently outperformed those trained on FakeRom in discriminating between categories, achieving better accuracy across all content types. BERT achieved the highest accuracy, at 96.5%, on the combined test set, showing particular strength in detecting satire (ROC AUC of 1.0) and maintaining consistent performance across categories, with ROC AUC scores above 0.99. RoBERTa-large demonstrated strong performance with 94.4% accuracy but showed slightly lower performance in propaganda detection (ROC AUC of 0.994) compared to other categories.  
The error analysis suggests that improving the detection of propaganda and better distinguishing it from other categories should be a priority for future model development, particularly in cross-dataset scenarios, where performance tends to degrade significantly.