## 3 Methodology
### Rlfoula Framework Description
#### 3.2.3 Employing Functional Agents in RL
for machine learning algorithm training by systematically oversampling falsely detected instances and undersampling correctly detected benign activities. We tested the RLFOUA framework on two widely used IDS datasets: CSE-CIC-IDS2018 and NSL-KDD. The application of RLFOUA on the independent set from the CSE-CIC-IDS2018 dataset yielded impressive results, with precision of 0.985%, recall of 0.9846%, accuracy of 0.9846%, and F1 score of 0.9847%. Similarly, when applied to the independent set from the NSL-KDD dataset, RLFOUA achieved precision of 0.9055%, recall of 0.9118%, accuracy of 0.9981%, and F1 score of 0.9061%. These results were compared with the best and latest tested algorithms, AESMOTE, which were applied to the same datasets using the same performance metrics. As outlined in Section 5, there is an average enhancement of 11% in recall, 7.6% in precision, 21.5% in accuracy, and 9.9% in F1 score when compared to AESMOTE. The comparison revealed significant performance improvements obtained by the RLFOUA framework. Overall, the RLFOUA framework presents a promising environment for effectively balancing imbalanced IDS datasets. Through the incorporation of the TFRSMOTE algorithm into the RLFOUA framework, we demonstrated the capability to address the challenges posed by imbalanced datasets in machine-learning-based IDS applications. The results obtained underscore the potential of RLFOUA in enhancing intrusion detection capabilities and contributing to the development of robust cybersecurity measures. Looking ahead, further research avenues could explore the fine-tuning of RLFOUA's algorithms and re-sampling techniques to adapt to evolving threat landscapes. Additionally, investigating the integration of RLFOUA with emerging network security technologies and exploring its performance in real-time scenarios are promising directions for future work. These endeavors aim to continue advancing the field of anomaly detection in intrusion detection systems and fortifying cybersecurity measures. Conceptualization, N.A. and M.J.; Methodology, N.A. ; Software, N.A. ; Validation, N.A. and M.J.; Formal analysis, N.A. and M.J.; Investigation, N.A. and M.J.; Resources, N.A. and M.J.; Data curation, N.A. ; Writing--original draft, N.A. ; Writing--review & editing, N.A. and M.J. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. Supgested Data Availability Statements are available in [[https://www.unb.ca/cic/datasets/index.html](https://www.unb.ca/cic/datasets/index.html)]([https://www.unb.ca/cic/datasets/index.html](https://www.unb.ca/cic/datasets/index.html)). The authors declare no conflict of interest. ## References

* [1] [PERSON]; [PERSON] Using Markov Chain Monte Carlo Algorithm for Sampling Imbalance Binary IDS Datasets. In Proceedings of the 12 th International Workshop on Security, Privacy, Trust for Internet of Things (IoTSPT) at the ICCCN 2022, Honolulu, HI, USA, 25-28 July 2022; pp. 1-6. * [2] [PERSON]; [PERSON] A Survey in Techniques for Imbalanced IDS Datasets. In Proceedings of the ICICCS 2022: 16 th International Conference on Intelligent Computing and Control Systems, Madurai, India, 25-27 May 2022; pp. 1-6. * [3] [PERSON]; [PERSON] AESMOTE: Adversarial Reinforcement Learning with SMOTE for Anomaly Detection. _IEEE Trans. Netw. Sci. Eng._**2021**, \(8\), 790-802. [CrossRef]
* [4] [PERSON]; [PERSON]; [PERSON]; [PERSON] Applying SMOTE for a Sequential Classifiers Combination Method to Improve the Performance of Intrusion Detection System. In Proceedings of the IEEE International Conference on Dependable, Autonomic and Secure Computing, Fukuoka, Japan, 5-8 August 2019; pp. 1-6. * [5] [PERSON]; [PERSON]; [PERSON]; [PERSON] SMOTE: Synthetic Minority Over-sampling Technique. _J. Artif. Intell. Res._**2022**, _16_, 321-357. [CrossRef]
* [6] [PERSON]; [PERSON] Effect of feature selection, synthetic minority over-sampling (SMOTE), and under-sampling on class imbalance classification.