## 3 Methodology
### Dataset and Data Preprocessing  
#### 3.1.1 Dataset Selection  
The success of any intrusion detection system (IDS) heavily relies on the quality and representativeness of the datasets used for training and evaluation. In this research, we have diligently chosen three diverse and widely recognized datasets to ensure comprehensive coverage of network traffic scenarios and attack types, enhancing the credibility and generalizability of our findings. Intrusion Detection Evaluation Dataset (CIC-IDS2017)  
The CIC-IDS2017 dataset plays a crucial role in advancing intrusion detection systems (IDSs) and intrusion prevention systems (IPSs). Unlike many existing datasets, CIC-IDS2017 addresses key limitations by providing up-to-date and diverse network traffic data, including both benign and known attack scenarios [24]. To ensure realistic background traffic, the dataset incorporates the B-Profile system, which profiles human interactions to generate naturalistic benign traffic. The captured data spans five days and encompasses various attacks, such as Brute Force FTP, Brute Force SSH, DoS, Heartbleed, Web Attack, Infiltration, Botnet, and DDoS [24]. CIC-IDS2017 fulfills crucial criteria for a reliable benchmark dataset, including a complete network configuration, labeled dataset, complete interaction coverage, complete capture, available protocols, attack diversity, heterogeneity, feature set extraction, and  
Figure 1: Proposed methodology overview. comprehensive metadata [24]. These features make it a robust resource for evaluating and training IDSs in real-world scenarios. NSL-KDDThe NSL-KDD dataset is a curated version of the KDD'99 dataset specifically designed to address its limitations and challenges [1]. While it may not fully represent real-world networks, its availability and refinements make it a valuable benchmark for evaluating intrusion detection methods [14]. One of the notable strengths of the NSL-KDD dataset is its reasonable size, with sufficient records in both the training and test sets. This feature enables researchers to conduct comprehensive experiments without resorting to random subsampling, ensuring consistency and comparability of the evaluation results [25]. Despite its imperfections, the NSL-KDD dataset remains a crucial resource for the intrusion detection community. Researchers can leverage it to develop and validate innovative techniques aimed at bolstering the security and effectiveness of network-based IDSs [14]. By acknowledging its limitations while utilizing its merits, the NSL-KDD dataset continues to play a pivotal role in advancing intrusion detection research and fostering the development of robust and efficient detection methods. The UNSW-NB15 DatasetThe UNSW-NB15 dataset represents a significant and comprehensive resource tailored for network intrusion detection systems (IDSs). Developed to address the limitations of existing datasets, this repository provides a diverse and realistic collection of network traffic data, specifically designed to aid in the evaluation and advancement of IDSs [26]. The generation of the UNSW-NB15 dataset involved the use of advanced tools and methodologies. The IXIA Perfect Storm tool was employed to create a hybrid of authentic modern network activities and synthetic contemporary attack behaviors. Subsequently, the tcpdump tool was used to capture 100 GB of raw traffic, stored in pcap files, resulting in a dataset that reflects real-world network dynamics more accurately [15]. One of the key strengths of the UNSW-NB15 dataset lies in its comprehensive representation of various attack types. It encompasses nine distinct attack categories, including Fizzers, Analysis, Backdoors, DoS, Exploits, Generic, Reconnaissance, Shellcode, and Worms. By encompassing a wide range of attack scenarios, the dataset enables researchers to evaluate IDSs' performance against diverse threats. The dataset incorporates the Argus and Bro-IDS tools to generate a set of 49 features, each associated with a specific class label.