# Romanian Fake News Detection Using Machine Learning and Transformer-Based Approaches
## 3 Materials and Methods
### Dataset Preprocessing  
Data preprocessing [25; 26] is crucial for preparing the dataset for training AI models. In this project, several preprocessing techniques were applied to ensure the data were in an optimal format for training and evaluation.  
In our project, data cleaning involves removing elements that could introduce noise into the model training process. The techniques applied include the following:  
1. Lowercase conversion: All text was converted to lowercase to eliminate variations caused by capitalization and ensure data consistent;
2. Removal of non-letter characters: All characters that were not letters or spaces were removed using regular expressions, while retaining specific Romanian characters (e.g., \(\mathtt{\hat{a}}\), \(\mathtt{\hat{a}}\), \(\mathtt{\hat{s}}\), \(\mathtt{\hat{t}}\)).
3. Removal of isolated letters: Single letters appearing between spaces were removed, as they may represent errors or offer little semantic value.
4. Standardization of spaces: Multiple spaces were replaced with a single space to streamline the text and facilitate further processing.
5. Trimming text edges: Leading and trailing spaces were removed to prevent issues during subsequent analysis.  
Text was transformed into numerical representations using TfidfVectorizer, which reduces the text to a set of significant features for the models to identify and differentiate between content types.  
The data were loaded by importing the dataset from a JSON file and transforming it into a DataFrame structure, which facilitates further manipulation and processing.  
To address class imbalance in the dataset, data augmentation techniques were applied, focusing on under-represented categories. Synonym replacement using pre-trained embeddings was chosen for its computational efficiency, being less resource-intensive than methods like back translation.  
The Romanian language embeddings were loaded using pre-trained fastText embeddings, which capture the semantic relationships between words. To optimize memory and resource usage, the number of loaded words was limited to the most frequent and relevant ones in the language, using the 1 million most frequent words.  
The synonym replacement procedure involves selecting unique words from each text that could be replaced with synonyms, identifying possible synonyms based on similarity in the embedding vector space, controlling the replacement by limiting the number of words changed in each text to preserve the original meaning and coherence and randomizingthe replacements to introduce data diversity and avoid repetitive patterns. In synonym replacement, we randomly replaced a maximum of two words per text sample.  
Exception handling and quality assurance mechanisms were implemented to manage cases where a word lacked available synonyms or was absent from the embeddings, ensuring continuity without compromising data quality. The augmented dataset was then combined with the original, resulting in an expanded dataset that included both the initial examples and those generated through augmentation.  
Class balancing through oversampling involved using the RandomOverSampler technique to equalize the number of examples in each class by randomly replicating data from the minority classes. This approach helps prevent bias in machine learning models toward majority classes and enhances their generalization ability.  
The augmentation process maintains semantic meaning while increasing dataset size. The final preprocessed dataset maintains class balance and preserves Romanian language features while ensuring reproducibility through fixed random seeds.  
Additionally, the following steps were taken: the balanced and preprocessed dataset was exported in a format that preserved Romanian diacritics and language specifics, ensuring its usability for future training and validation stages and the processed data were then prepared for machine learning algorithms, providing a solid foundation for achieving optimal performance. Figure 2 presents the FakeRom and NEW datasets after processing.