## 3 Dataset Creation Process
### Preprocessing  
It is essential to preprocess the data to effectively apply machine learning techniques to vulnerability detection in source code. This involves tasks such as the following:  
* Identification of malformed code--Code fragments may be incomplete or syntactically invalid. These segments can be recognized using a compiler or static analysis tool and marked appropriately [40].
* Removal of noise and irrelevant features--Code may contain comments, whitespace, and other non-essential elements that can negatively impact the performance of the machine learning models [41].
* Standardization of code formatting--Inconsistent formatting and coding styles can hinder learning. Standardizing the code format can improve the model's ability to detect patterns [42].  
Since we gathered the data from commits that are vulnerability-fix ones, they generally are not complete, mostly involving snippets of code or part of the code that fixes the bugs that the developer intended. Therefore, identification of the code may result in inconsistent analytics. However, removing noises such as comments, blank spaces, and some elements that don't affect the meaning of the code but improve the data for machine learning processes may make the analytics more consistent.  
Creating a balanced dataset is a critical prerequisite for building effective machine-learning models, particularly in the context of vulnerability detection. An imbalanced dataset, where one class dominates the other, can lead to biased models that fail to generalize to unseen data. For this reason, it was essential to ensure that secure (non-vulnerable) code commits were collected in comparable proportions to the already identified vulnerable commits. A balanced dataset allows the model to distinguish between secure and vulnerable code more accurately, reducing the risk of class imbalance skewing the results. This approach enhances the reliability and robustness of the machine learning system when applied to real-world scenarios.  
Identifying and extracting secure (non-vulnerable) code commits was designed as a systematic workflow to complement the collection of vulnerable code. The initial step involved curating a list of repositories from GitHub, stored in a designated file for structured access. These repositories were programmatically traversed using the GitHub API, and commits were iteratively fetched for examination. To filter out vulnerable commits, a predefined set of keywords associated with security patches or fixes--such as \"fix\", \"patch\", \"CVE\", or \"vulnerability\"--were applied to commit messages. Commits lacking these indicators were classified as \"safe\" and stored for further analysis.  
The script implemented a dynamic rate-limiting mechanism to ensure the process adhered to GitHub's API rate limits (5000 requests per hour). If the API limit was reached, the system automatically paused for the required duration before resuming operations. This enabled uninterrupted data collection without exceeding usage quotas. The workflow continued until the number of secure commits exceeded a predefined threshold, slightly surpassing the number of vulnerable commits for dataset balance. All collected data, including repository names, commit hashes, and messages, were systematically stored in a JSON file for subsequent use in machine learning-based vulnerability detection.