## 4 Experiment Environment
### Case 2: Comparative Analysis of Frequency Control Effects of Different Methods under DoS Attacks  
In the process of reinforcement learning, this paper uses convex neural networks to establish an approximate model of critic networks, and fast optimization of reinforcement learning is carried out in the approximate model established by convex neural networks. Thus, the optimization process of reinforcement learning is approximated as the convex optimization process. In order to verify that the frequency control strategy based on convex neural networks is better than the frequency control strategy based on general neural networks, this section first uses two types of traditional neural networks (radial basis function neural networks (RBF) [38] and recurrent neural networks (RNN) [26]) to approximate critic networks in reinforcement learning. Then, the proposed frequency control strategy is compared with them to analyze the control performance of the frequency control strategies constructed by different neural networks under load change disturbances and DoS attacks. Under three different DoS attack intensities, the IEEE57 bus testing system is used to verify the control performance of three methods on power grid frequency and the adaptability to different DoS network attack intensities. The model parameters of the IEEE57 bus testing system are shown in Table 1, and the initialized parameters of the corresponding ADES controller are shown in Table 4. The actor-critic network learning rate, the sum of attack thresholds, the number of input neurons, and the number of the layers of convex neural networks are re-initialized. The designed parameters meet the sufficient conditions for stability and convergence (32). \(\mathbf{w}_{a,i}\) and \(\mathbf{w}_{c,i}\) meet the Assumption 2. \(\eta_{ji}\in\{0.05,0.1,0.2\}\) is set. The DoS attack period and total attack time are shown in Table 2.  
The experimental results are shown in Figures 13-15, where (a), (b), and (c) are the frequency deviation curves of the IEEE57 bus testing system based on convex neural networks, RBF, and RNN, respectively. The simulation time based on convex neural networks and RBF is set to 8 s, and the simulation time based on RNN is set to 15 s.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter Name** & **Description** & **Value** \\ \hline \(\beta\) & learning rate of critic networks & 0.0025 \\ \(\gamma\) & learning rate of actor networks & 0.0011 \\ \(N\) & number of input neurons & 57 \\ \(\xi\) & sum of attack threshold & 22 \\ \(Q_{N}\) & number of network layers & 59 \\ \(|\lambda_{max}(\mathbf{L_{i}})|\) & the maximum absolute value of \(\mathbf{L_{i}}\) & 0.4 \\ \(\|\mathbf{B}_{i,max}\|^{2}\) & the maximum value of \(\|\mathbf{B}_{i}  
As shown in Figure 13a, the maximum frequency deviation of the bus is about 0.18 Hz, and the frequency deviation of each bus can be within the set stable interval at \(t=3\) s. As shown in Figure 13b, the maximum frequency deviation of the bus is about 0.12 Hz, and each bus can maintain a stable state at \(t=4\) s. As shown in Figure 13c, the maximum frequency deviation of the bus is about 0.2 Hz, and each bus gradually maintains a stable state at \(t=7.5\) s. As the load changes constantly, the frequency deviation of each bus also fluctuates within the stable interval. Figure 14: Frequency deviation curve of the IEEE 57 bus testing system under three different methods, \(\eta_{ji}=0.1\). (**a**) Based on convex neural networks. (**b**) Based on RBF neural networks. (**c**) Based on RNN neural networks. Figure 13: Frequency deviation curve of the IEEE 57 bus testing system under three different methods, \(\eta_{ji}=0.05\). (**a**) Based on convex neural networks. (**b**) Based on RBF neural networks. (**c**) Based on RNN neural networks.