## 5 Results and Experimental Discussion
### Assessing Loss and Accuracy Metrics during Model Fine-Tuning  
Our experimental evaluation tracks the evolution of training losses and the evaluation accuracy of each model across multiple epochs, enabling us to identify irregularities that may indicate a performance gap between the training and evaluation sets during cross-validation, potentially signaling overfitting. Figure 4 provides evidence that the training losses of the three fine-tuned BERT models decrease as the number of training epochs increases and stabilizes after reaching convergence. This observation indicates their effective adaptation to the domain-specific training data within our methodology. Notably, the BETO model converges more rapidly  
than the others, which aligns with expectations, given its extensive pre-training on a large Spanish-language corpus and subsequent fine-tuning on our dataset in the same language. An analysis of the trends in evaluation accuracy (Figure 5) over multiple epochs confirms that the BETO model offers the best overall performance. This model consistently achieves the highest and most reliable accuracy. On the other hand, the BERT-multilingual model displays lower losses than the BERT-base model, which records the lowest accuracy, thus confirming its inferior performance in a classification task in a language (Spanish) different from its pre-training language (English). Since the evaluation accuracy values are high and stable as the BERT models train, it is evident that none of them are overfitting the training data, and they generalize well to unseen data during cross-validation. The positive influence of pre-training with Spanish text corpora is also evident when examining the training and evaluation metrics of RoBERTa-based models. In particular, both training losses (Figure 6) and evaluation accuracy (Figure 7) consistently stabilize, which confirms the absence of overfitting. Figure 4: Progression of training loss over training epochs for the 3 fine-tuned BERT-based models using 80% of our Spanish political fake news dataset. Figure 5: Progression of evaluation accuracy over training epochs for the 3 fine-tuned BERT-based models using 80% of our Spanish political fake news dataset and cross validation. The _RoBERTa-BNE_ model consistently achieves the highest evaluation accuracy, with _RoBERTa-BERTIN_ following closely, as depicted in Figure 7. However, the _RoBERTa-multilingual_ model, despite its inclusion of Spanish among the supported languages, does not attain the performance levels of the first two models. Nevertheless, it exhibits a noteworthy enhancement when contrasted with the base model, which was originally pre-trained on English text corpora. To provide a more transparent comparison among the five fine-tuned models from BERT and RoBERTa families that have performed better (multilingual and Spanish-specific models), their training and evaluation metrics are collectively presented in Figures 8 and 9. Figure 6: Progression of training loss over training epochs for the 4 fine-tuned RoBERTa-based models using 80% of our Spanish political fake news dataset and cross validation. Figure 7: Progression of evaluation accuracy over training epochs for the 4 fine-tuned RoBERTa-based models using 80% of our Spanish political fake news dataset and cross validation. The data illustrated in both graphs unveil the superiority of the RoBERTa models over the BERT variants, as anticipated due to the enhancements introduced in RoBERTa building upon the original BERT architecture. Notably, the Spanish adaptations of RoBERTa, such as BNE and BERTIN, outperform BETO, the Spanish BERT. Furthermore, in line with our earlier discussion, _RoBERTa-BNE_ demonstrates the best performance during the fine-tuning process on our dataset, tailored specifically for Spanish political fake news detection.