# Network Attack Path Selection and Evaluation Based on Q-Learning
## 4 Security Risk Assessment of Electric Power CPS under Cyber Attack  
This paper selects the function of load control for risk assessment, and its propagation process is as follows:  
1. The attacker uses a certain strategy to launch an attack through the smart terminal to enter the control center, and then randomly or deliberately tamper with the business data according to the knowledge of the physical power grid, so that the load of some physical nodes exceeds the predetermined quota.
2. The control center considers that the load on the node exceeds the capacity, and judges that the node is faulty. Therefore, the control center performs load reduction according to Formulas (5)-(10) with the goal of minimizing load loss, and issues a control command to cut off part of the load of the node and its neighboring nodes to ensure the safe and stable operation of the system.  
\[\text{min}I=\sum\limits_{i=1}^{N}Ls_{i} \tag{5}\]  
where, \(I\) is the total load loss of the physical system, \(N\) is the number of load shedding nodes in the physical system, and \(Ls_{i}\) is the load loss of node \(i\).  
At the same time, considering the power flow constraints of the distribution network and the observable and controllable nodes, the following constraints are obtained:  
\[\left\{\begin{array}{l}P_{i}=G_{ii}U_{i}^{2}+\sum\limits_{j\in s(i)}U_{i}U_{ j}(G_{ij}cos\theta_{ij}+B_{ij}sin\theta_{ij})\\ Q_{i}=-B_{ii}U_{i}^{2}-\sum\limits_{j\in s(i)}U_{i}U_{j}(B_{ij}cos\theta_{ij} -G_{ij}sin\theta_{ij})\end{array}\right. \tag{6}\]  
\[\left\{\begin{array}{l}U_{\text{min}}\leq U_{i}\leq U_{\text{max}}\\ I_{\text{min}}\leq I_{ij}\leq I_{\text{max}}\end{array}\right. \tag{7}\]  
where, \(P_{i}\) and \(Q_{i}\) are the active power and reactive power of node \(i\), respectively, \(s(i)\) is the set of nodes connected to node \(i\), \(G_{ii}\) and \(B_{ii}\) are the self-conductance and self-susceptance of node \(I\), respectively; \(G_{ij}\) and \(B_{ii}\) are the conductance and susceptance between nodes \(i\) and \(j\), respectively; \(U_{i}\) and \(U_{j}\) are nodes, respectively The voltages of \(i\) and \(j\); \(\theta_{ij}\) is the phase angle difference between nodes \(i\) and \(j\); \(U_{\text{min}}\) and \(U_{\text{max}}\) are the lower and upper limits of the voltage of node \(i\), respectively; \(I_{\text{min}}\) and \(I_{\text{max}}\) are the lower and  
CPS security risks under data attacks are related to the threat value of the attack path to the information system and the consequences of cascading failures caused. Section 2.4 discusses the attack benefit \(G\) of the best attack path, that is, the threat to the information system caused by the attack through this path. Based on the above, the risk of CPS under data attack is defined as:  
\[R=G\cdot\sum_{i=1}^{n}Ls_{i} \tag{11}\]  
\[ls_{i}=\sum_{j=1}^{n}ls_{ij} \tag{12}\]  
\[ls_{ij}=\frac{Ls_{ij}}{Load_{j}} \tag{13}\]  
where, \(R_{i}\) represents the risk of power CPS when the load data of node \(i\) is tampered with. \(G\) represents the threat to the information system that an attacker launches an attack on the network through a predetermined path. \(p_{i}\) represents the probability that the data of node \(i\) is modified, which is related to the attacker's familiarity with the operation of the system. \(Ls_{ij}\) indicates that the load change of node \(i\) causes the load loss of node \(j\). \(Load_{j}\) represents the original load of node \(j\). \(ls_{ij}\) represents the load loss rate of node \(j\) caused by the load change of node \(i\). \(ls_{i}\) Indicates the load loss rate of the entire system caused by tampering of the load data of node \(i\), which is the sum of load loss rates of all nodes.