## 5 Results and Experimental Discussion
### Analyzing Test Metrics of the Fine-Tuned Models
#### 5.2.1 Testing with a Dataset of Real and Fake News Naturally Sourced from Web Sources  
The new dataset comprises 63 headlines extracted from various websites in July 2023. These headlines were collected from online media outlets that curated statements made by leaders of the two main political parties following a significant electoral debate during the Spanish general elections that month. These media outlets identified the statements as either true or false, providing contextual explanations to justify their classification. This process allowed us to create a non-synthetic dataset containing both real and fake news. A sample of the online media publications used in this process can be found at [[https://elpais.com/espana/elecciones-generales/2023-07-10/mentiras-y-medias-verdades-del-debate-de-atresmedia-entre-sanchez-y-feijoo.html?ssm=whatsapp](https://elpais.com/espana/elecciones-generales/2023-07-10/mentiras-y-medias-verdades-del-debate-de-atresmedia-entre-sanchez-y-feijoo.html?ssm=whatsapp)]([https://elpais.com/espana/elecciones-generales/2023-07-10/mentiras-y-medias-verdades-del-debate-de-atresmedia-entre-sanchez-y-feijoo.html?ssm=whatsapp](https://elpais.com/espana/elecciones-generales/2023-07-10/mentiras-y-medias-verdades-del-debate-de-atresmedia-entre-sanchez-y-feijoo.html?ssm=whatsapp)) (accessed on 1 November 2023). Out of the 63 headlines, 33 were labeled as fake, while 30 were classified as genuine. All the headlines were manually reviewed to ensure that the articles were correctly categorized as either fake or real news. The evaluation was conducted using the RoBERTa-BNE model, which achieved the best performance according to the metrics outlined in Table 6. We tasked this model with classifying the 63 headlines from the new dataset as either fake or real news, and it achieved an accuracy rate of 71%. This represents a notable decline compared to the accuracy of over 98% observed when testing the model on the original synthetic dataset. This drop in accuracy can be explained by several factors related to the differences between the datasets. The initial synthetic dataset, on which the model was trained, generated fake news by altering names, locations, and figures, or using tools like ChatGPT. The synthetic dataset also included true news articles sourced from online newspapers and INE, covering political issues such as education, the economy, employment, healthcare, and culture. However, the real news articles in the synthetic dataset did not comprehensively cover much of the information found in the non-synthetic dataset. The latter primarily consisted of statements made by political leaders during electoral debates leading up to the Spanish general elections in July 2023, as reported by various media outlets, from which we extracted the headlines. In view of the obtained results, the synthetic nature and specific focus of the initial dataset might have provided the evaluated Transformer models with a consistent and predictable pattern to learn from. In contrast, the new (non-synthetic) dataset consisted of small authentic news articles from a variety of web sources, presenting a broader and more realistic range of fake news scenarios. The diverse and less predictable nature of real-world data posed additional challenges for the Transformer-based model, revealing its limitations in handling the complexities and variations inherent in genuine news articles. Furthermore, there were instances of mixed news in the dataset where candidates' statements combined both factual claims and false information within the same declaration. The presence of such mixed news likely introduced an extra challenge for the models, as they had to distinguish between true and false elements within a single piece of news. This complexity probably contributed to the observed decrease in accuracy, underscoring the model's limitations in dealing with more nuanced, real-world data.