# Network Attack Path Selection and Evaluation Based on Q-Learning
## 5 Simulation Evaluation
### Analysis of Experimental Results  
#### 5.2.1 Experimental Results  
Security Analysis of the Information Layer  
The attack gain index in this paper is based on the attack reward and attack cost. As a result, the relationship of the three should be studied first. There are 30 attack paths in attack mode 1, and five attack paths in attack mode 2. Figure 4 (its abscissa variables are the attack path number) shows the attack gain, attack reward and attack cost of each attack path in the two attack modes. It can be seen that the value trend of the attack gain is roughly the same as the attack reward, but the attack cost reduces the attack gain to a greater extent. When the attack reward is small, the attack gain may even appear to be smaller than the attack cost (path 5 of attack mode 2), which is not good for the attacker. In order to further illustrate the relationship of the three, a scatter plot between the three under two attack modes is drawn in Figures 5 and 6. Figures 5a and 6a show that the attack gain is closely related to the attack reward, and a high attack reward will bring a high attack gain. Figure 5b,c and Figure 6b,c show that high attack rewards and attack gains often require high attack costs. However, the path with the highest attack cost will not get the highest attack reward and highest attack gain. This is due to the different nature of each security vulnerability. The attack complexity of the security vulnerability that poses the greatest threat to the system may not be high (such as Common Vulnerabilities and Exposures (CVE)-2004-0893, it is a user privilege escalation vulnerability. According to the CVSS, its threat index reaches 7.2 (the highest is 10), but the attack complexity is only moderate.). Therefore, there is an optimal attack path, which reduces the cost of the  
Figure 3: The Institute of Electrical and Electronics Engineers (IEEE) 14 node system.  
attack and obtains a higher attack return. At this time, the attacker obtains the maximum attack gain.  
In the two attack modes, the attacker uses the FPN-Q Learning algorithm to perform attack learning as shown by the solid line in Figure 7. It can be seen that the attack gain  
Figure 4: The relationship between attack gain, attack reward, and attack cost: (**a**) attack mode 1, (**b**) attack mode 2.  
Figure 5: The relationship between attack gain, attack reward, and attack cost in mode 1: (**a**) The relationship of attack gain and attack reward; (**b**) The relationship of attack cost and attack reward; and (**c**) The relationship of attack cost and attack gain.  
Figure 6: The relationship between attack gain, attack reward, and attack cost in mode 2: (**a**) The relationship of attack gain and attack reward; (**b**) The relationship of attack cost and attack reward; and (**c**) The relationship of attack cost and attack gain.  
obtained by the attacker invading the control center is much greater than that obtained by invading the control center. This is because once the information host in the control center is destroyed, the security threat to the system is greater. Therefore, it is necessary to strengthen the monitoring and protection of the control center host.