## 2 Related Work  
Transformer-based models fine-tuned on existing corpora have also been employed for generic NLP text classification tasks with diverse purposes, beyond the particular fake news detection tackled in our research, which can essentially be seen as a categorization of textual input as fake or true. In this domain, we find works like [15], which aims to identify sexist content on social media using a multilingual BERT model to classify tweets as either sexist or non-sexist, employing a dataset of 13,000 texts (half in English and half in Spanish) to achieve an accuracy of 76.83%. Similarly, the properties and performance of fine-tuned BERT models are also explored in [16] for classification tasks in different languages and purposes, including sentiment analysis of IMDB movie reviews (50,000 entries in English, accuracy of 93.87%), hotel reviews in Chinese (6000 reviews, accuracy of 93.81%), classification of informative tweets related to natural disasters (10,000 tweets in English, accuracy of 83.64%), and categorization of news articles in Portuguese across multiple content categories (12,145 headlines, accuracy of 91.20%). Moving already to the particular domain of fake news detection, the identification of such news has been tackled by multiple conventional ML techniques, including RandomForest (RF), Decision Trees (DT), Boosting Trees (BT), logistic regression (LR), Support Vector Machine (SVM), and Naive Bayes (NB) [17]. These techniques, while effective in various contexts, exhibit several limitations when compared to DL models that have achieved better performance in fake news detection tasks. In particular, one key limitation of traditional ML models like NB and SVB is their reliance on manual feature engineering. In these approaches, human experts must define and select features such as word frequencies, n-grams, or specific keywords to capture patterns in the data. This process can be time-consuming and may fail to capture complex patterns or deeper contextual information. Additionally, traditional models like LR often assume linear relationships between the selected features and the target variable, which limits their ability to handle more intricate interactions within the data. These models also tend to fail to generalize across diverse datasets, particularly when there are variations in linguistic or stylistic patterns. In contrast, DL models have the advantage of automatically learning features from raw data, capturing both simple and complex patterns without manual intervention. By utilizing multiple layers of representation, DL approaches can more effectively model intricate relationships and contextual dependencies in large datasets, making them far more adaptable and powerful for tasks like fake news detection [18; 19; 20]. Given the aforementioned weaknesses, traditional ML techniques have sometimes been integrated with DL models to leverage the strengths of both approaches. Specifically, ML methods have been combined with Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) [21; 22; 23; 24], and also with various types of word and document embeddings and Transformer-based models, resulting in notable outcomes [25; 26; 27]. For instance, the study presented in [28] introduces an effective approach for detecting fake news on social media by utilizing three publicly available datasets (WELFake, FakeNewsNet, and FakeNewsPrediction) and integrating FastText word embeddings with various ML and DL techniques. Specifically, a hybrid model of CNN and LSTM achieved high accuracy, while Transformer models like BERT, XLNet, and RoBERTa showed better performance after hyperparameter tuning, achieving accuracy values of 99%. Additionally, explainable artificial intelligence (AI) methods were employed to elucidate the model's decision-making process. The detection of fake news in social media has been tackled using various methods, including graph convolutional networks and attention networks.