## 3 Proposed Methodology
### Attack Classification
#### 3.2.1 Reinforcement Learning  
The difference between RL and other ML algorithms is that RL engages the decision-making process for determining the best return or reward by incessantly learning the way the agent and environment interact. The aforesaid interaction process can be designated using the MDP model. A quintuple \(N=(T,B,\Phi,U,S)\) is characterized by MDP. Among these, \(T\) designates the state set of environment, \(B\) suggests the action set the agent is capable of performing, and \(\Phi\) implies the set of state-action pairs that can be executed. A transfer function, represented by \(U\), signifies the probability of transitioning to state \(t^{\prime}\) after executing action \(\beta\) in state \(t\). Moreover, a reward function, designated by \(S\), char acterizes the expected reward value produced while ending action \(b\) in state \(t\) and later transitioning to state \(t^{\prime}\). The specific interaction process between the environment and the agent is categorized into two parts. In the first part, the current state of the external environment \(t_{u}\in B\) is perceived at time \(u\). Then, it decides to choose the suitable action \(b_{u}\in B\) for executing conferring to the strategy \(\pi\). In the second part, the environment responds to the agent as it updates the state \(t_{u}\), produces a return or reward \(s_{u}\in S\), and then moves in the subsequent MDP process. Suppose that after a certain period of time \(u\), the agent interacts with the environment to attempt and gather as many rewards as possible within the environment and incessantly learns to increase the expected return. The below equation illustrates this procedure as follows:  
\[S_{u}=s_{u+1}+\delta\cdot s_{u+2}+\delta\cdot s_{u+3}+\cdots=\sum_{i=0}^{ \infty}\delta^{i}\cdot s_{u+\bar{k}+1} \tag{2}\]  
where \(\delta\) is a discount factor and \(\delta\in(0,1)\). The contribution of future rewards to the anticipated return is generally determined by means of the discount factor. The agent lacks foresight if \(\delta=0\) since it solely deliberates about increasing the present rewards. On the other hand, the agent becomes more foresighted as \(\delta=1\) since it ponders more and more on future rewards. The agent receives a random sequence of variable rewards after time \(u\), which is characterized as \(s_{u+1},s_{u+2},s_{u+3}\ldots\). By offering a value function that is divided into a state-action-value function and a state-value function, RL generates the connection between the optimal policy and optimal criterion. The expected return attained by an agent utilizing policy \(\pi\) in state \(t\) is reflected as the state-value function. Now, the mathematical expression is presented below as follows:  
\[W^{\pi}(t)=E_{\pi}\{S_{u}|t_{u}=t\}=E_{\pi}\{\sum_{l=0}^{\infty}\delta^{l}S_{u +l+1}|t_{u}=t\} \tag{3}\]  
where \(E_{\pi}\{\}\) indicates the expected value under strategy \(\pi\). If the optimal strategy is preferred among every potential state-value function, an optimal state-value function exists. The following describes the equations for the optimal state-value function and the optimal strategy. \[W^{*}(t)=\max_{\pi}W^{\pi}(t) \tag{4}\]  
\[\pi^{*}=\operatorname*{argmax}_{\pi}W^{\pi}(t) \tag{5}\]  
In the same way, the expected return of the agent executing action \(b\) in state \(t\) is demarcated as the state-action-value function. The following equation is used to represent this function. \[R^{\pi}(t,b)=E_{\pi}\{S_{u}|t_{u}=t,b_{u}=b\} \tag{6}\]  
\[=E_{\pi}\{\sum_{l=0}^{\infty}\delta^{l}s_{u+l+1}|t_{u}=t,b_{u}=b\} \tag{7}\]  
Now, the equation of the optimal strategy for obtaining the optimal Q-value function is given as follows. \[\pi^{*}=\operatorname*{argmax}_{\pi}R^{\pi}(t,b) \tag{8}\]The Bellman optimization equation is employed to express the optimal Q-value function further.