## 3 Methodology
### Rlfoula Framework Description
#### 3.2.3 Employing Functional Agents in RL  
In this research, multiagent reinforcement learning (MARL) is used as it offers advantages over single-agent reinforcement learning (SARL) in complex scenarios [15]. MARL allows multiple agents to learn and make decisions concurrently, capturing nuanced interactions and dependencies among agents. Therefore, in the integrated RL environment of RLFOUA, two agents collaborate to optimize the algorithm selection and resampling processes. The functional agents in RL start with a preprocessed dataset. Then, they integrate it within a pipeline combining a machine learning algorithm, an oversampling method, and an undersampling technique. The goal is to produce a model capable of detecting attacks in a general dataset. In each iteration, the first agent chooses a pipeline generated from Algorithm 1, trains the machine learning algorithms after resampling within the pipeline, and calculates the reward based on the resulting performance. Concurrently, the second agent, called TFRSMOTE (proposed resampling algorithm), performs dataset resampling. In the RLFOUA framework, it is essential to define key components of RL to better understand its operation within the context of our proposed method. These components include the following:  
State: In RLFOUA, the \"state\" represents the current status or configuration of the learning environment, which includes the dataset and the algorithmic choices. It encapsulates all the information needed to make decisions regarding algorithm selection and resampling strategies. Action: Actions refer to the choices available to the agents in the RLFOUA framework. Specifically, Agent 1 selects pipeline combinations, machine learning algorithms, and resampling techniques, while Agent 2 utilizes the TFRSMOTE resampling algorithm with varying parameters. Reward: The \"reward\" in RLFOUA serves as a feedback signal that quantifies the quality of decisions made by Agent 1. It is computed based on the performance metrics. Rewards guide the decision-making process, enabling the selection and refinement of algorithm combinations. Transition function: It is considered as the implicit process through which the pipeline combinations evolve. The transitions involve training and evaluating different combinations of machine learning algorithms and resampling techniques to arrive at improved selections. Understanding these RL components within RLPOUA is pivotal for comprehending how the collaborative interaction of two agents, the Q-learning process of Agent 1, and the TFRSMOTE algorithm of Agent 2 jointly lead to optimized algorithm selection, resampling, and dataset balancing, ultimately enhancing intrusion detection system performance. In the first phase, \"Agent1\", each pipeline received from the pool of machine learning, oversampling, and undersampling algorithms is trained and evaluated on the received preprocessed dataset. Evaluation metrics are computed, and IM is calculated to determine the model's performance, as described in Equation (8). Employing this metric as a pivotal guide, Agent 1 is entrusted with the pivotal task of determining whether to proceed with the framework's execution or halt it altogether. Furthermore, this metric empowers Agent 1 to make informed decisions regarding the adjustment of rewards, encompassing the crucial choices of whether to augment or diminish them. IM introduces a new strategy for comparing classification algorithms within the RLPOUA framework. The IM metric incorporates the following parameters: TP, TN, FN, and FP [2]. TP represents cases where the label is an attack, and it is correctly predicted as an attack. TN denotes instances where the label is benign, and it is correctly predicted as benign. FP refers to cases where the label is benign, but it is incorrectly predicted as an attack. FN represents situations where the label is an attack, but it is incorrectly predicted as benign.