# Q-Learning Approach Applied to Network Security
## 5 Discussions  
With the obtained results, we provided proof that reinforcement learning can be successfully applied in the IR as a controlling mechanism so our networks can stay protected, depending on the current situation of our decision agent. The application of Q-learning as an intrusion response mechanism is novel, and more research is needed in that area. Based on the results obtained from a broader perspective, Q-learning is a significant tool because the decision agent does not need to know much information to find optimal results. The Q-learning agent learns as it goes and makes decisions based on the feedback it gets from the environment. The agent does not need to know the transition probabilities; the only information he needs is the beginning state of the decision agent. Therefore, this method can be successfully combined with the classical classification ML methods. This model is not an alternative to classification techniques but a complementary instrument. The main idea is for the agent to receive initial information from the classification method so that it knows the beginning state. The proposed mechanism for intrusion response is unique and valuable. The Q-learning algorithm for IR is an excellent tool for an automatic response; there are ways to adapt to streaming data whenever the intrusion detection allows for streaming data classification set-up. This application could turn the Q-learning algorithm and transform the model into a powerful tool in modern security systems. However, this algorithm can be successfully implemented as it is. The first step is to extract the network traffic from the transmission control protocols and send it to the intrusion detection module, where the preferred classification method is applied; see Figure 9. The next step is to let the healthy traffic into the system and isolate the malicious traffic under quarantine. Then we could apply another classification tool that may classify the network as under \"Normal\", \"DoS\", \"Probe\", \"R2L\", and \"U2R \". After that classification, it is time for the Q-learning agent to observe the state that it is currently in and to select an optimal policy, based on the Q-learning algorithm. The reward-penalty system gives administrators a powerful mechanism to mitigate threats successfully and respond effectively to attacks. The proposed method can be further elaborated by adding more available information for the agent's actions. Besides blocking and allowing the traffic in the system, we could add actions related to adjusting the security settings or going beyond protecting. The reward function could be further improved based on the cost function associated with the consequences of selecting a particular action. It is also possible to create a decision path for the agent just as we did in our proposed approach; however, when including more agents in the environment, some of them could be friends, neutral, or foes. Then, the agents will try to maximize their cooperative rewards and simultaneously achieve equilibrium points.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**State** & **Allow (0)** & **Report (1)** & **Return (2)** & **Block (3)** \\ \hline Normal (0) & 6.08946832 & 0.65195162 & 1.41350597 & 0.06186645 \\ DoS (1) & \(-3.91069429\) & 4.11094152 & 1.11089763 & 11.11111108 \\ Probe (2) & 0.0000000 & 0.0000000 & 0.0000000 & 0.00000000 \\ R2L (3) & 0.0000000 & 0.0000000 & 0.0000000 & 0.0000000 \\ U2R (4) & 0.0000000 & 0.0000000 & 0.0000000 & 0.00000000 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Q-Table for Q-learning policy deployed on NSL-KDD dataset.