## 1 Introduction  
Cyber-attacks are on the rise due to a combination of evolving tactics by malicious actors and an expanding digital landscape [1]. With the proliferation of connected devices and the increasing dependence on online platforms, there are more opportunities for cybercriminals to exploit vulnerabilities. Additionally, the sophistication of attacks has grown, making them harder to detect and defend against. To defend against these threats, intrusion detection systems (IDSs) must continuously improve their ability to detect new types of attacks [2]. However, a major challenge for machine-learning-based IDSs is that the data used for training and testing are often imbalanced. This means there are usually more normal activities than intrusion instances, making the learning models lean towards recognizing regular behaviors. This can result in cyber-attacks going unnoticed or being misclassified. Dealing with imbalanced datasets has been a significant focus for researchers, especially in security applications where attack data are limited [2]. This challenge becomes even more apparent when trying to classify different types of attacks, as some categories may have very few examples or even just one. To tackle this challenge, our research introduces an innovative algorithm to improve training with imbalanced datasets. Our approach includes three main novel parts: The Reinforcement Learning (RL) Framework with Oversampling and Undersampling Algorithms (RLFOUA), the True False Rate Synthetic Minority Oversampling Technique (TFRSMOTE) resampling algorithm, and a new evaluation metric called the Index Metric (IM). In the context of intrusion detection systems (IDSs), a system's response to a detected potential intrusion can be categorized as either passive or active. A passive response involves the IDS solely in the process of detecting and notifying about suspicious activities without taking direct action against them. In this study, our focus is on the development and evaluation of resampling techniques using RL algorithms to enhance the performance of IDS in detecting attacks. Our approach solely focuses on detection, without any influence on the response, whether passive or active, when a potential intrusion is detected by an IDS. The aim is to improve the learning process and model performance in distinguishing between normal and malicious activities, ultimately enhancing the accuracy of intrusion detection. Therefore, our approach is centered around refining the data preprocessing step to provide more effective training data for the machine learning algorithms used in the detection process. Our work does not affect whether or not the IDS takes active response, such as blocking or mitigating detected intrusions. Our primary objective lies in improving the detection accuracy. RL offers a promising approach for intrusion detection due to its ability to adapt and learn from dynamic environments. Traditional rule-based systems or static models may struggle to keep pace with the evolving tactics of cyber-attackers. RL, on the other hand, enables the IDS to continuously refine its strategies based on feedback from its environment. This adaptability is crucial in the ever-changing landscape of cyber threats. In this paper, we introduce the RLFOUA framework, which is designed to balance datasets and train machine learning models for better performance. RLFOUA uses a special learning environment guided by RL principles, which learns from feedback provided by our new evaluation metric. An important component of RLFOUA is the TFRSMOTE resampling algorithm, which systematically generates synthetic data. To measure the effectiveness of our results, we use various metrics like F1 score, accuracy, recall, and precision. We rigorously test the RLFOUA algorithm using two well-known datasets: CSE-CIC-IDS2018 and NSL-KDD. The former focuses on distinguishing between normal and infiltration samples, while the latter classifies 23 different types of attacks.