## 3 Proposed Methodology
### Attack Classification
#### 3.2.2 Enhanced Deep Reinforcement Learning
The L2 regularization term can be mathematically described as follows:  
Figure 2:0  
where \(L\) specifies the total count of layers in the network, \(n\) is the count of neurons in a layer \(l\), \(k\) is the count of neurons in a layer \(l+1\) and \(\omega_{ji}^{(l)}\) specifies the weight value. Sparsity regularization or a penalty term is added to the loss function using the Kullback-Leibler (KL) divergence. This adds a penalty to the values of \(\delta_{j}\) that diverge from the values of \(\varrho\). The penalty term can be described as follows:  
Figure 2:1  
where \(g_{2}\) specifies the count of hidden neurons in the encoder. Therefore, by adding all the above terms, the overall loss function of the network model can be expressed as:  
Figure 2:2  
where \(\lambda\) and \(\delta\) are the terms that control the strength of the respective regularization terms. Moreover, the feature learning capability of deep AE is potentially higher than that of AE, so the deep AE model is likely to result in an effective approximation of the Q-function. In addition, large or continuous state spaces in RL problems without a Q-table are efficiently handled by utilizing a deep AE to learn a compact representation of the state space and a separate neural network to approximate Q-values. In a complex environment where conventional tabular approaches are impracticable, this approach proves to be quite supportive. Improved Flamingo Search Algorithm (IFSA) for Optimal Policy Selection  
Optimal policy selection is significant in RL for an agent to optimize its long-term reward by building the best decisions in an environment. Numerous algorithms are designed to tackle this issue. However, the existing optimum policy selection algorithms undergo limitations such as high computational costs, which can render them inappropriate for real-time applications. Numerous algorithms also face trouble with trade-offs between exploitation and exploration, which can tend to sub-optimal policies when they have not sufficiently explored the state space. In addition, the existing algorithms can have instability and convergence problems, specifically in continuous or high-dimensional state spaces. The flamingo search algorithm (FSA) [33; 34] is one of the swarm intelligent optimization methods that draws inspiration from flamingo migration and feeding habits. This optimization algorithm utilizes the collective intelligence of flamingo populations,where members exchange information about their positions and food availability, permitting the group to efficiently explore the search area. The capability of FSA to strike a balance between global exploration and local exploitation is one of its main advantages. This progresses the convergence speed and search accuracy. The algorithm has proven to be effective and robust in resolving complicated optimization issues, outperforming alternative optimization methods over a range of test functions. In addition, the adaptability of FSA demonstrates its practicality in real-world applications by making it appropriate for a variety of applications comprising engineering problems like network intrusion detection systems and path planning. However, sometimes the FSA has to local optima issues. The inclusion of chaotic tend mapping [35] at the FSA initialization can support addressing the local optima problem. By employing chaos in the initial positioning of the flamingo population, the method maximizes diversity and exploration ability. This enables it to avoid local optima and upsurges the chances of discovering the global optimal solution. Thereby, in the proposed EDRL approach, the tent chaotic mapping is adapted with FSA for optimal policy selection and termed an improved flamingo search algorithm (IFSA). To accomplish the task, the environment, action, reward, and state are initialized as the initial population, and then the searching procedure is put forth. In FSA, the population members are initialized randomly according to the lower and upper bounds of the problem.