## 3 Optimized Load Frequency Control with Convex Neural Network-Based Reinforcement Learning
### Analysis of Power Grid Stability and Convergence of Convex Neural Network Weights
On the one hand, the vectors \(\mathbf{w}_{a,i}\) and \(\mathbf{w}_{c,i}\) are given constant vectors, and they are set as the vectors that satisfy Assumption 2 during the parameter initialization. On the other hand, \(\mathbf{B}_{i}\) is the power input gain of distributed energy sources that affects future frequency deviation changes, and the gain is limited. **Assumption 3**. : _There is a matrix \(\mathbf{P}_{i}\) that makes the projection of matrix \(\mathbf{W}^{\prime}\) on the constraint set \(\Omega\) be expressed as \(\Pi_{\Omega}(\mathbf{W}^{\prime})=\mathbf{P}_{i}\mathbf{W}^{\prime}=\mathbf{W}\), and \(\mathbf{P}_{i}\) satisfies \(\|\mathbf{P}_{i,\min}\|\leq\|\mathbf{P}_{i}\|\leq\|\mathbf{P}_{i,\max}\|\)._  
**Note. ** The above assumption is feasible. Matrix projection can be expressed as projecting the matrix column space composed of matrix column vectors into another matrix column space. Projection from a certain matrix column space to another matrix column space can be completed by linear transformation. Thus, there is a certain matrix that makes the matrix column space project to another matrix column space after linear transformation. Moreover, the dimension of the projection matrix is related to the size of the matrix column space. When the dimension of the projection matrix is limited, the norm of the projection matrix must have an upper bound and a lower bound [54]. Based on the above assumptions, this section presents the sufficient conditions for the stability of the aggregated distributed energy grids and the actor-critic network weight convergence to meet the ultimate uniformly bounded (UUB) stability. This sufficient condition strictly gives the controller parameters and adaptive gain range. Within this range, power grids belong to UUB under the disturbances of DoS attacks and load changes, and the weights of convex neural networks can also converge as shown in Theorem 1. **Theorem 1**. : _If the parameters of aggregated distributed energy grids, actor networks and critic networks satisfy the inequality group (32), then the set of weights \(\tilde{\mathbf{W}}_{a,i}\), \(\tilde{\mathbf{W}}_{c,i}\) and the system state \(\boldsymbol{x}_{i}(k)\) are LUIB and stable._  
\[\left\{ \begin{array}{l}2\Big{[}\|\mathbf{P}_{a,i,\max}\|_{F}^{2}\big{(}\beta^ {2}-\beta\big{)}u_{lim}^{2}+2\Big{]}\times L_{i}^{\prime 2}\|\mathbf{B}_{i,\max}\|^{2 }u_{lim}^{2}<\beta,\\ 4\|\mathbf{P}_{c,i,\max}\|_{F}^{2}{L_{i}^{\prime 2}}^{2}\big{(}\gamma^{2}- \gamma\big{)}-\gamma<0,\\ 2\|\mathbf{L}_{i}\|^{2}-1<0,0\leq|\lambda_{i}(\mathbf{L}_{i})|\leq 1,\\ \beta>0,\gamma>0\end{array} \right. \tag{32}\]  
_where \(\|\mathbf{P}_{c,i,\max}\|_{F}^{2}\) and \(\|\mathbf{P}_{c,i,\max}\|_{F}^{2}\) are the squares of the maximum \(F\) norm of matrix \(\mathbf{P}_{a,i}\) and matrix \(\mathbf{P}_{c,i}\), respectively, and \(|\lambda_{i}(\mathbf{L}_{i})|\) is the absolute value of the eigenvalue of matrix._  
The proof of Theorem 1 is given in the Appendix A. There is still a certain error in using convex neural networks to approximate actor-critic networks. The reason is that a certain error range exists in the training process of convex neural network weights. When the target learning functions \(E_{a,i}(k)\) and \(E_{c,i}(k)\) are within the error range, the network weights are no longer updated. This error does not affect the global stability of aggregated distributed energy grids and the convergence of the weights of actor-critic networks.