## 3 Experiments
### Simulation Results
As shown in the above scenario, the decisions of the fortieth agent are obviously better than that of the first agent, which means that the proposed method is effective. Figure 6 shows the results of the fortieth agent and the fifteenth agent. Agent 2 represents the fifteenth agent. In simulation 1, the initial yaw angle of Agent 1 is \(-99.1^{\circ}\), and the initial yaw angle of Agent 2 is \(-10.4^{\circ}\). Agent 2 does not launch the missile at the beginning, indicating that the agent gradually learns when to launch the missile during the training. Agent 1 first approaches Agent 2 from the side and then launches the missile. Finally, the missile of Agent 1 hits Agent 2. In simulation 2, the initial yaw angle of Agent 1 is \(123.3^{\circ}\), and the initial yaw angle of Agent 2 is \(125.0^{\circ}\). Agent 1 launches the missile after targeting Agent 2, but Agent 2 fails  
Figure 6: Air combat results of the fortieth agent and the fifteenth agent. (**a**) The 3D trajectory of simulation 1; (**b**) Top view of the trajectory of simulation 1; (**c**) The 3D trajectory of simulation 2; (**d**) Top view of the trajectory of simulation 2; (**e**) The 3D trajectory of simulation 3; (**f**) Top view of the trajectory of simulation 3.  
to target Agent 1 when launching the missile, resulting in missing the target. Finally, due to Missile 1's flight time exceeding 27 s, Missile 1 misses the target. In simulation 3, the initial yaw angle of Agent 1 is 100.3\({}^{\circ}\), and the initial yaw angle of Agent 2 is \(-80.7^{\circ}\). Agent 1 quickly flies away from [PERSON] 2 to avoid being hit by the missile launched by Agent 2. After launching the missile, Agent 2 hardly changes its flight direction and only flies upwards to increase its altitude, so it is hit by Missile 1. As shown in the above scenario, the decisions of the fifteenth agent are better than that of the first agent and still worse than that of the fortieth agent, which means that the agent's ability gradually improved during training. Figure 7 shows the results of the fortieth agent and the thirtieth agent. Agent 2 represents the thirtieth agent. In simulation 1, the initial yaw angle of Agent 1 is \(-155.5^{\circ}\), and the initial yaw angle of Agent 2 is \(8.6^{\circ}\). Agent 1 and Agent 2 are facing away from each other at the beginning. Both agents try to maneuver to make the other's azimuth angle less than the off-axis angle before launching missiles. Agent 1 changes its flight direction after launching the missile to avoid being hit by Agent 2's missile. Finally, Agent 2 is hit by the missile launched by Agent 1, so the simulation ends. Figure 7: Air combat results of the fortieth agent and the thirtieth agent. (**a**) The 3D trajectory of simulation 1; (**b**) Top view of the trajectory of simulation 1; (**c**) The 3D trajectory of simulation 2; (**d**) Top view of the trajectory of simulation 2; (**e**) The 3D trajectory of simulation 3; (**f**) Top view of the trajectory of simulation 3. In simulation 2, the initial yaw angle of Agent 1 is \(2.5^{\circ}\), and the initial yaw angle of Agent 2 is \(66.1^{\circ}\). Agent 2 is in a status of being pursued by Agent 1. Agent 2 flies forward and launches the missile after some time, but the missile misses the target. Agent 1 first tracks Agent 2 and flies away after launching the missile. Finally, Agent 2 is hit by the missile of Agent 1. In simulation 3, the initial yaw angle of Agent 1 is \(-63.7^{\circ}\), and the initial yaw angle of Agent 2 is \(96.9^{\circ}\). Both agents turn to each other at first, launch missiles, and then stay away from each other after launching. Finally, the missile of Agent 2 hits Agent 1. As shown in the above scenario, the decisions of the thirtieth agent are much better than that of the first and fifteenth agents, and its behavior is very similar to the fortieth agent.