## 5 Experiments and Analysis
### Case I: IEEE 14 Bus Test System  
The IEEE 14 bus test system is carried out under reinforcement learning control and DoS attacks. The initial state of power grid simulation is assumed to be affected by a large disturbance. The simulation time lasts 5 s. Without any control, we can see that the system collapses under serious DoS attacks after a large power grid disturbance occurs as shown in Figure 3. The frequency increases to a large value, which reaches to a maximal value about 40 Hz in 5 s. Therefore, a controller is required to maintain the stability. The parameters of reinforcement learning algorithm are listed in Table 2. The detail of parameter selection principle can be referred to [2].  
The parameter \(\beta_{i}\) decides the result of IEEE 14 bus test system. If \(\beta_{i}\) is too large, the online learning of Equation (20) diverges; otherwise, too small \(\beta_{i}\) may cause the slow convergence rate. The selection of \(\gamma_{i}\) is another key factor. Too large \(\gamma_{i}\) may result in the poor control performance and high sparsity of \(W_{i}\).  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Parameter Name** & **Description** & **Value** \\ \hline \(M_{i}\) & inertia constant & 0.2 \\ \(D_{i}\) & damping constant & 0.26 \\ \(T_{ji}\) & synchronizing constant & 0.5 \\ \(T_{di}\) & governor constant & 5 \\ \(R_{gi}\) & regulation constant & 0.5 \\ \(b_{i}\) & frequency bias gain & 1 \\ \(T_{gi}\) & gas turbine constant & 0.2 \\ \(K_{i}\) & tie-line bias control gain & 0.1 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Parameters of sub-system.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Parameter Name** & **Description** & **Value** \\ \hline \(\alpha\) & damping factor for cost \(p_{i}\) & 0.9 \\ \(N\) & Horizon length for cost & 10 \\ \(\beta_{i}\) & Learning rate of neural network & 0.1 \\ \(\gamma_{i}\) & Weight of group sparse regulation term & 0.012 \\ \(|Li|\) & Norm of damping parameters \(L_{i}\) & 0.1 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Parameter of the controller in the IEEE 14 bus test system.  
Figures 4 and 5 show the results that use the proposed control scheme and the optimal DoS attack strategy. The optimal DoS attack strategy is obtained by solving Equation (26), and the optimal controllers are learned by the online iteration of Equation (20). According to the results in Figure 4, we can see the frequency deviation curves of all sub-systems converge to a steady value around 0. The maximal fluctuant magnitude of these curves is a small value about 1.2 Hz. The swings end in about 4 s.  
For the optimal DoS attack strategy shown in Figure 4(a), the attacks focus on the communication from buses 3, 6, 10, 11, and 13. For the optimal control structure about placement of sensor, actuator, and communication topology shown in Figure 4(b), it shows the F-norm for block \(W_{ij}\) in \(W_{i}\), \(i,j=1,2,...,n\). If the norm value is large, it means sub-system \(j\) needs information from sub-system \(i\). Thus, sub-system \(i\) needs to install sensors, and sub-system \(j\) requires actuators as well as the communication topology from sub-system \(i\) to \(j\). The sensors are mainly installed in buses 2, 3, 6, 10, 11, 13, and 14. According to the solution, those communication lines that are not attacked maintain the system stable. Thus, the obtained solution is optimal for both attacker and defender (control system designer) as Nash equilibrium.  
Figure 3: Frequency deviation of IEEE 14 bus system without control.