# Network Attack Path Selection and Evaluation Based on Q-Learning
## 5 Simulation Evaluation
### Analysis of Experimental Results
#### 5.2.3 Experimental Results--Cross-Layer Risk Communication Analysis  
Assuming that the attacker only modifies the load of a single node each time, and the offset is 1 MW, the \(ls_{i}\) of each node is obtained as shown in the Figure 8. It can be seen from the figure that when the load shedding amount is the same, the load loss rate caused by changing the data of No. 6 and No. 9 nodes is higher. Therefore, these two nodes are defined as high-risk nodes. Figure 7: (**a**) Comparison with random attack and selective attack and (**b**) Comparison with unimproved Q-Learning algorithm  
In the mode 1, the attacker enters the control master station and masters the operation of the power grid. At this time, the possibility \(p_{i}\) of tampering with data of nodes with high load loss rate is higher. In the mode 2, the attacker does not enter the control center and does not understand the operation of the power grid. At this time, only business data can be modified randomly, so the probability of any physical node data being modified is the same. The risks faced by each node in the two attack modes are shown in Table 1:  
It can be seen from the above table that high-risk nodes (No. 6 and No. 9) face significantly higher risks in attack mode 1 than in attack mode 2. This is because in mode 1, the attacker enters the control master station and masters the operation of the power grid. At this time, the possibility \(p_{i}\) of high-risk node data to be tampered is greater, and the possibility of ordinary node data being tampered is less. However, the opposite is true in mode 2. The attacker does not enter the control center in mode 2, so he does not know the operation of the system and will randomly tamper node data. In this case, the probability \(p_{i}\) of each node's data being tampered is the same. Therefore, the possibility of data tampering of ordinary nodes in mode 2 is greater than that in mode 1, and the risks faced are also increased. However, on the whole, mode 1 poses a greater risk to the system than mode 2, because the attacker in mode 1 has more power grid operating data and can change the operating data in a targeted manner. From the above analysis, we can see that the risks faced by the power CPS system are closely related to the attacker's attack mode. Therefore, for information systems, the protection of confidential data and control centers needs to be strengthened. For physical nodes, high-risk nodes are protected, and the power \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Node**} & \multicolumn{2}{c}{**Risk**} & \multirow{2}{*}{**Node**} & \multicolumn{2}{c}{**Risk**} \\ \cline{2-2} \cline{4-6}  & **Attack Mode 1** & & **Attack Mode 2** & & **Attack Mode 1** & **Attack Mode 2** \\ \hline
1 & 0.05571 & 0.16091 & 8 & 0.28763 & 0.36563 \\ \hline
2 & 0.0402 & 0.13669 & 9 & 1.96169 & 0.95486 \\ \hline
3 & 0.08665 & 0.20068 & 10 & 0.27507 & 0.35756 \\ \hline
4 & 0.17548 & 0.28559 & 11 & 0.30509 & 0.37656 \\ \hline
5 & 0.13683 & 0.25218 & 12 & 0.29032 & 0.36734 \\ \hline
6 & 9.77915 & 2.13194 & 13 & 0.20045 & 0.30523 \\ \hline
7 & 0.15986 & 0.27258 & 14 & 0.24754 & 0.33920 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Node risk under different attack modes. Figure 8: The load loss of every node. load needs to be allocated reasonably to minimize the risk of the power CPS system when the information system is invaded. As can be seen from the above table, the risk to the system caused by the attack mode \(1\) is greater than the attack mode \(2\). This is because the attacker under the attack mode can change the operation data in a targeted manner with more power grid operation data, causing greater risks to the system. At the same time, it can be seen from the table that ordinary nodes (such as No. 6 and No. 9) have greater operational risks under the same load shedding amount. The reason for this is these nodes have less output and are not located in important positions of the system, and the attacker is less difficult to attack.