## 3 Methodology
### Rlfoula Framework Description
#### 3.2.3 Employing Functional Agents in RL
The best models obtained from the RLFOUA framework in Section 4.1 are evaluated using the testing set as an independent dataset. The testing results are presented in Table 3, which demonstrates the effectiveness of the RLFOUA framework on the CSE-CIC-IDS2018 dataset. Table 3 provides a comprehensive overview of the classification metrics achieved after applying the RLFOUA framework on the independent testing set of the CSE-CIC-IDS2018 dataset. Macro Avg displays the macro-average metrics, which calculate the average of precision, recall, and F1 score, respectively, across all classes. It can provide insights into the model's performance, considering class imbalances. Weighted Avg metrics consider the class support (number of instances) in the calculation, providing a more representative performance evaluation, particularly when dealing with class imbalances. In the context of intrusion detection, precision reflects the system's ability to accurately flag potential threats without generating unnecessary alarms. Recall, on the other hand, assesses the proportion of true positives correctly identified from all actual positives in the dataset. It is particularly important in security applications to minimize false negatives, ensuring that actual threats are not overlooked. The F1 score strikes a balance between precision and recall, offering a combined metric that considers both false positives and false negatives. A higher F1 score indicates a model that excels in both precision and recall, signifying a robust detection capability. The results indicate an accuracy of 0.9846, a macro average recall of 0.9817, a weighted average recall of 0.9846, a macro average F1 score of 0.9731, a weighted average F1 score of 0.9847, a macro average precision of 0.9651, and a weighted average precision of 0.985. ### Primary Testing on Independent Set for Nsl-KDD

Within this section, an assessment of the RLFOUA framework is conducted using the NSL-KDD dataset. The superior models garnered from the framework, as elucidated in Section 4.2, undergo evaluation on an autonomous testing set. The outcomes of this evaluation, encapsulated in Table 4, substantiate the efficacy of the RLFOUA framework when applied to the NSL-KDD dataset. Table 4 presents the classification metrics for the testing of the RLFOUA framework on the NSL-KDD dataset, further testing its performance on a different dataset. The RLFOUA framework undergoes evaluation across various attack types, providing corresponding precision, recall, and F1 score values. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Attack Type** & **Precision** & **Recall** & **F1 Score** & **Support** & **Frequency** \\ \hline Back & 1 & 0.9873 & 0.9936 & 79 & 956 \\ Buffer\_overflow & 1 & 0.8 & 0.8889 & 5 & 30 \\ Ftp\_write & 0 & 0 & 0 & 1 & 8 \\ Guess\_passwd & 1 & 1 & 1 & 8 & 53 \\ Ipsweep & 0.9945 & 0.9973 & 0.9959 & 365 & 3599 \\ Land & 0.5 & 1 & 0.6667 & 1 & 18 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Classification metrics for testing the RLFOUA framework with TFRSMOTE algorithm on NSL-KDD dataset. Support parameter indicates the frequency of each class in the independent set of the dataset, while frequency represents the occurrence of each class in the entire dataset. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Precision** & **Recall** & **F1 Score** & **Support** \\ \hline Benign & 0.9953 & 0.986 & 0.9907 & 538,865 \\ Infiltration & 0.9348 & 0.9774 & 0.9556 & 110,453 \\ Accuracy & & 0.9846 & 649,318 \\ Macro Avg & 0.9651 & 0.9817 & 0.9731 & 649,318 \\ Weighted Avg & 0.985 & 0.9846 & 0.9847 & 649,318 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Classification metrics on the testing set of CSE-CIC-IDS2018 dataset using RLFOUA framework. The RLFOUA framework demonstrates promising results on the NSL-KDD dataset, effectively identifying and classifying different types of attacks. Comparing the metrics across different attack types, we can observe variations in the model's performance.