## 3 Experiments
### Simulation Results  
In this section, the maneuvering decision-making ability of the proposed method is verified. We categorize simulations as four different scenarios to emphasize the effectiveness of the proposed method: 1. Two untrained agents. 2. The fortieth agent and the first agent. 3. The fortieth agent and the fifteenth agent. 4. The fortieth agent and the thirtieth agent. The results of untrained agents, i.e., the results of random decisions, are shown in Figure 4, which shows the three-dimensional trajectory and its top view. The initial distance range is (4000 m, 16,000 m), the initial angle range is (\(-180^{\circ}\), \(180^{\circ}\)), and the initial velocity  
Figure 3: Wins, losses, and draws. range is (250 m/s, 400 m/s). In Figure 4, the solid line represents the flight trajectory of the aircraft, and the dashed line represents the flight trajectory of the missile. The blue and yellow dots represent the starting points of the two agents, respectively. In simulation 1, both agents adopt random decisions. The initial yaw angle of Agent 1 is \(-138.0^{\circ}\), and the initial yaw angle of Agent 2 is \(-74.4^{\circ}\). From Figure 4b, it can be seen that due to the untrained nature of the agents, they do not make any effective decisions. Agent 1 launches the missile at the beginning of the simulation, while Agent 2 launches the missile after flying forward for some time. However, both missiles miss the target; thus, the simulation ends. In simulation 2, the initial yaw angle of Agent 1 is \(67.6^{\circ}\), and the initial yaw angle of Agent 2 is \(-60.0^{\circ}\). From Figure 4d, it can be seen that both Agent 1 and Agent 2 choose to launch missiles at the beginning of the simulation, and Missile 2 misses the target. Finally, Missile 1's flight time exceeds 27 s; thus, the simulation ends. In simulation 3, the initial yaw angle of Agent 1 is \(-3.3^{\circ}\), and the initial yaw angle of Agent 2 is \(36.1^{\circ}\). From Figure 4f, it can be seen that  
1 misses the target because of the maximum flight time. As shown in the above scenario, without training, agents maneuver randomly; therefore, they cannot make any rational and effective decisions to hit the target. Figure 5 shows the results of the fortieth agent and the first agent. Agent 1 represents the fortieth agent, and Agent 2 represents the first agent. In simulation 1, the initial yaw angle of Agent 1 is \(56.5^{\circ}\), and the initial yaw angle of Agent 2 is \(-66.4^{\circ}\). Agent 1 launches the missile after flying for some time. Then, Agent 1 quickly moves away from Agent 2, as this decision can reduce the probability of being hit by Agent 2's missile. After some time, Agent 2 launches the missile. Finally, Agent 2 is hit by Agent 1's missile; thus, the simulation ends. In simulation 2, the initial yaw angle of Agent 1 is \(19.2^{\circ}\), and the initial yaw angle of Agent 2 is \(-50.2^{\circ}\). Agent 1 adopts similar decisions as in simulation 1, which involves first approaching and aiming at the target, then launching the missile and moving away from the target. Meanwhile, Agent 2 is unable to make effective decisions. It launches the missile at the beginning of the simulation, resulting in missing the target. In simulation 3,  
Figure 5: Air combat results of the fortieth agent and the first agent. (**a**) The 3D trajectory of simulation 1; (**b**) Top view of the trajectory of simulation 1; (**c**) The 3D trajectory of simulation 2; (**d**) Top view of the trajectory of simulation 2; (**e**) The 3D trajectory of simulation 3; (**f**) Top view of the trajectory of simulation 3.  
the initial yaw angle of Agent 1 is \(-92.0^{\circ}\), and the initial yaw angle of Agent 2 is \(101.1^{\circ}\). Agent 1 first turns back and then tracks Agent 2. Due to the long distance between the two agents, Agent 1 does not launch the missile, but Agent 2 keeps flying forward instead of making any effective decisions, and launches the missile without aiming at the target; thus, it misses the target.