###### Abstract
also built a multi-stage hierarchical game with a corresponding hierarchy of decisions that was implemented to achieve a resilient control system [11]. According to optimal control structures, optimal criteria were constructed for DoS attackers and cyber defenders. [PERSON] et al. modeled the remote estimation under DoS attacks by using the strategy of zero-sum stochastic games and presented a monotone structure to solve the proposed model optimistically [12]. At the same time, [PERSON] et al. also formulated the decision-making process of a target channel as a two-player zero-sum stochastic game framework and proposed a Nash Q-learning algorithm to obtain the optimal strategy [13]. [PERSON] et al. built a hybrid game-theoretic framework to improve the robustness of power system security [14]. Since game theory based control strategy can process strategic interactions among multiple decision makers, it is often used in large-scale system security control. Despite the game theory based method being able to be efficient against conventional DoS threats, only using control methods to defense DoS attacks is not sufficient to counter potential and increasingly sophisticated attacks. In large-scale control systems, structure factors, such as distributed energy sources (DES) placements, sensors, and communication link topologies, can critically affect the performance in DoS attacks. Existing work of energy storage systems [15], DES placement [16], sensor scheduling [17] and coverage aims at minimizing the electrical and computational costs [18]. Upon existing research in the optimization of control system structure for cyber security, there are several critical challenges. One critical challenge is the transient and dynamic issues of DES and cyber resources allocation. Most existing research focused on the characteristics of steady state [19]. They ignored the transient and dynamic issues in control systems [20]. Another critical challenge is that most existing algorithms used a precisely analytical model as a premise [21]. However, most large-scale power systems are unable to be precisely modeled. In this case, a model free method is required for control-system-structure optimization. With the consideration of DoS [22; 23] attacks and the control-system-structure optimization problem, we propose a sparse neural network based NCS optimization method that is derived from the Stackelberg game model for the frequency regulation of DES in power systems. A neural controller is trained by reinforcement learning of offline simulation and online processes to improve the system performance. The system performance is also improved under limited cyber resources by optimizing the control structure, which involves the placement of DES, RTU sensors and communication topology. It is optimized by imposing a group sparse regulation on neural network weights. The Stackelberg game model is used, which derives a minimax optimization of system performance, so that the optimized control structure is robust to DoS attacks under the consideration of worst case attacks. The structure consists of the placement of DES, RTU sensors and communication topologies. The contributions of this paper are summarized as follows:  
1. Sparse neural network based reinforcement learning is proposed to improve the frequency regulation of DES in control systems without using a power system analytical model, which involves adaptiveness, performance, and structure. 2. The Stackelberg game model is used to derive the optimal control scheme and structure, so that the proposed frequency regulation system is robust to the worst case of DoS attacks. In addition, the reliability of proposed frequency regulation system is enhanced.