## 3 Methodology
### Rlfoula Framework Description
#### 3.2.3 Employing Functional Agents in RL
It is noteworthy that, unlike many other papers, which concentrate solely on one dataset, our proposed methodology is rigorously evaluated on both CSE-CIC-ID2018 and NSL-KDD datasets, thus showcasing its robustness and generalizability across diverse data scenarios. This comprehensive evaluation substantiates the superior accuracy, precision, recall, and F1 score achieved by our method. As depicted in Table 2 within Section 4, for the \"neptune\" attack type, all four algorithms achieve high precision, recall, and F1 scores, indicating their effectiveness in detecting this attack type. The Extra Trees Classifier achieves perfect recall and F1 score, indicating that it captures all instances of \"neptune\" attacks without any false negatives. The \"nmap\" attack type is also well detected by all algorithms, with the Bagging Classifier and XGB Classifier demonstrating slightly higher recall and F1 scores compared to the Decision Tree Classifier and Extra Trees Classifier. All algorithms accurately classify instances labeled as \"normal\" with high precision, recall, and F1 scores, indicating their ability to generalize well for benign network activity. The Decision Tree Classifier exhibits the lowest performance for the \"rootkit\" and \"multihop\" attack types, particularly in terms of recall and F1 score. On the other hand, the Bagging Classifier, XGB Classifier, and Extra Trees Classifier show better recall and F1 scores for these categories. Overall, the Extra Trees Classifier achieves the highest recall and F1 scores for most attack types, making it particularly effective in detecting a wide range of attacks within the NSL-KDD dataset. The weighted average scores provide an overview of the algorithms' overall performance across all attack types, considering their frequency in the dataset. The Bagging Classifier and XGB Classifier exhibit higher weighted average F1 scores, indicating their balanced performance across the entire dataset. Moreover, referring to Table 4 within Section 4, the application of RLFOUA on the independent NSL-KDD dataset reveals intriguing insights. Particularly, the \"nback\" attack type stands out with a precision of 1.000, indicating the precise classification of all instances associated with this attack type. However, the \"buffer_overflow\" attack type indicates that while the precision is perfect (1), only 80% of actual \"buffer_overflow\" instances were captured (recall of 0.8), resulting in an F1 score of 0.8889. Some attack types, such as \"ftp_write\" and \"loadmodule\", have precision, recall, and F1 score of 0, indicating that the model failed to correctly classify any instances for these attack types. Transitioning to the next segment, a comparative analysis between RLFOUA and AESMOTE is initiated to discern their unique attributes. AESMOTE, considered one of the most recent and high-performing algorithms in the domain, is marked by a set of unique attributes. Both RLFOUA and AESMOTE operate within an RL framework, leverage the NSL-KDD dataset, and adopt accuracy, recall, precision, and F1 score as metrics for assessing performance. The selection of AESMOTE as a benchmark is predicated on its exceptional performance relative to other contemporaneous methodologies. The main differences between RLFOUA and AESMOTE lie in the resampling approach, dataset consideration, algorithm selection, and performance metrics. RLFOUA introduces the innovative TFRSMOTE algorithm, which systematically generates synthetic data by oversampling falsely detected instances and undersampling correctly detected benign activities. In contrast, AESMOTE relies on the widely used SMOTE algorithm for resampling. Another distinction is that RLFOUA integrates a wide range of 23 machine learning algorithms, 8 undersampling techniques, and 3 oversampling methods, in addition to TFRSMOTE. This comprehensive exploration of the solution space allows for a more comprehensive analysis.