## 6 Experiment
### Simulation Experiment and Result Analysis
#### 6.3.2 Result Analysis  
In order to verify the influence of different values of \(\varepsilon\) on the convergence, we set the values to 0.15, 0.2, 0.25, and 0.3, respectively. In order to reduce the time spent in training, we set max_episode to 300 for experiments, and the experimental results are shown in Figure 10. The analysis shows that with the increase in the value of \(\varepsilon\), the probability of the agent exploring the system environment also increases, and the probability of falling into the local optimum decreases, which makes the oscillation degree of the step curve decrease, but once the equilibrium path is explored in the later stage of training, a larger value of \(\varepsilon\) will lead to a larger oscillation range of step curves near the equilibrium path, and the convergence effect is relatively poor. For a better convergence effect, we will continue to use the setting of \(\varepsilon\) as 0.1. To verify the convergence of the model, we performed several experiments using the same parameters, and the experimental results of two of them are shown in Figure 11. It can be seen from the figure that in these two experiments, the offensive and defensive sides are in the equilibrium path. It can be seen from the figure that in these two experiments, both sides of the attack and defense fluctuate slightly on the equilibrium path \(NEP_{1}=\{P_{a}=0.5,0.5,0.5\}\), and the defensive side is in the equilibrium path \(NEP_{1}=\{P_{a}=0.5,0.5\}\). Figure 9: Training result using Nash Q-learning. \((0,1,2,7,\cdots,24)\), \(P_{d}=(4,9,8,7)\)}. Compared with the experiment shown in Figure 8a, the experiment shown in Figure 8b has better convergence and only has a large oscillation in the early stage of training, and almost stays on the equilibrium path the rest of the time, which indicates that our proposed model has good convergence. Optimal Strategy  
As shown in Figure 9, across the 500 episodes, the attacker reached the target node 143 times, whereas the defender successfully intercepted the attacker in 357 instances, resulting in a defense success probability of 71.4%. Obviously, the defender has a high defense success probability. Figure 11: Convergence verification. **(a)** Additional experiment 1 for \(\varepsilon=0.1\). **(b)** Additional experiment 2 for \(\varepsilon=0.1\). Figure 10: Effect of different \(\varepsilon\). In order to verify the defense effect of the model, we have done 10 simulation experiments each for the case of \(\varepsilon=0.1\) using Nash Q-learning, Minimax-Q, and random policy, and the obtained defense success probability (dsp) curve is shown in Figure 12. We calculated that the average defense success probability of these three methods is 75.1%, 89.1%, and 79.8%, respectively. Obviously, Minimax-Q algorithm performs better than the other two and this is because the agents in Minimax-Q do not consider Nash Equilibrium strategy, they only choose the action which can bring maximum reward, and there is only one target node for the attacker, but the defender can intercept him in any possible nodes, so the dsp turns out higher than the other two. Also, Minimax-Q has a bad convergence, the random policy does not converge at all, and both Minimax-Q and random policy cannot reach the Nash Equilibrium. So, we use Nash Q-learning not only because it can converge well, but also because it has a good dsp, which can also prove that the model has a good defense effect. Analysis of the data shows that the optimal equilibrium path of both sides is \(NEP_{1}=\{P_{a}=(0,\cdots,7,\cdots,24),P_{d}=(4,9,8,7)\}\), as shown in Figure 13. The attacker can obtain the maximum profit on \(P_{a}=(0,\cdots,7,\cdots,24)\) and this path is the shortest path, while the defender can obtain the maximum profit and the shortest path on \(P_{d}=(4,9,8,7)\) when the attacker takes the above path. Furthermore, we give a comparison with existing deception asset deployment methods, as shown in Table 3.  
Figure 12: Convergence verification. Figure 13: Optimal equilibrium path.