# Romanian Fake News Detection Using Machine Learning and Transformer-Based Approaches
## 3 Materials and Methods
### Dataset Collection  
Data collection is a crucial step in developing any AI model. For this project, a dataset was created by combining articles collected from Veridica [22], a platform that monitors and identifies fake news and misinformation, with the open-source FakeRom dataset [23]. Custom datasets were created for use with PyTorch, randomly combining the elements of the FakeRom dataset and another 1000+ items obtained from scraping of the Veridica platform.  
For this phase, a combination of web scraping techniques and Python libraries was used to extract the necessary data. The main libraries used for data collection were requests and BeautifulSoup.  
The stages of the data collection process were as follows:  
1. Defining scraping functions: Functions were created to extract article metadata, obtain article content, and save the collected data in a JSON file. We used a function to extract relevant metadata, such as publication date, from the HTML structure of each article.
2. Scraping article content: The implemented function extracts the content of articles by identifying relevant paragraphs and adding them to the article content.
3. Scraping article details: We implemented a function that combines the title, content, and metadata of articles into a dictionary, which is then saved as a JSON file. These data are later combined with the FakeRom dataset [24], improving it with current articles.  
Collecting articles from multiple pages: Since the article source is organized by pagination, a method was built to navigate through this structure and access the corresponding article URLs.
* Main program execution: The main function initializes the scraping process, prompting the user to enter the URL, number of pages to scrape, and the file name for saving the data. These steps ensure efficient and organized data collection for building AI models. The flow in Figure 1 represents the key steps of the web scraping process: 1. Initialization: Program starts and obtains input parameters (URL, page count, filename); 2. Page processing: Iterates through pages to collect article URLs; 3. Content extraction: For found articles, extracts content and metadata using Thread-PoolExecutor; 4. Data storage: Saves processed articles to JSON format; 5. Completion: Finalizes process with article count.  
Figure 1: Web scraping algorithm.  
The algorithm implements parallel processing through ThreadPoolExecutor for efficient article scraping, with proper error handling and data validation at each step. The main functions work together to: achieve the following:  
* Collect and validate article URLs;
* Extract content and metadata in parallel;
* Save structured data to JSON format;
* Track processing progress.  
Part of the dataset was created using the web scraping algorithm detailed above (see Figure 1). This algorithm accesses pages on the Veridica platform, extracts article content, and stores it in a structured JSON format. This ensures that all articles are properly collected and labeled for training and evaluating AI models. The dataset was then randomly combined with the FakeRom dataset.  
The obtained combined dataset includes a variety of information, with each article tagged to indicate its content type, such as fake_news for false news and misinformation for misleading information. Each record from the dataset includes the following fields:  
* Content: The full text of the article;
* Tag: A label indicating the nature of the article (e.g., fake_news, misinformation, propaganda, real_news, satire).