## 5 Simulation and Analysis
### Maneuvering Strategy Training and Testing
#### 5.2.2 PRED-RS-MADDPG Strategy vs. Simple Script Strategies  
In this section, the intelligence degree of the 1-vs.-1 WVR air combat strategy obtained while using the PRED-RS-MADDPG algorithm is evaluated with respect to a few simple script strategies. Three basic scripting strategies were proposed: the \"uniform linear motion\" strategy, the \"decelerate left turn\"  
Figure 7: Curve of the normalized average reward of MADDPG (basic MADDPG algorithm), RS-MADDPG (MADDPG algorithm with only reward shaping), and PRED-RS-MADDPG (the MADDPG algorithm with reward shaping and prediction of the target position) with the episodes.  
strategy, and the \"S-type maneuver\" strategy. The performance parameters of UCAVs on both sides are kept at \"advantage\" to ensure that the performance of the two UCAVs is the same. Meanwhile, the initial position of each UCAV is set to face each other; the initial speed is the same, which results in an absolutely equal initial state. The \"uniform linear motion\" strategy is to keep the speed of UCAV at 180 km/h. The \"decelerate left turn\" strategy is to keep the UCAV accelerating to the left. The \"S-type maneuver\" strategy means the UCAV turns from left to right every 15 s and then from the right to left every 15 s. \"Decelerate left turn\" strategy and \"S-type maneuver\" are often used in real air combats.  
Figures 8-10 show the fly trajectory and the change in air combat situation of the 1-vs.-1 WVR air combat strategy obtained by using the PRED-RS-MADDPG algorithm against the \"uniform linear motion\" strategy, \"decelerate left turn\" strategy, and 'S-type maneuver\" strategy. In general, a UCAV using the PRED-RS-MADDPG strategy can achieve termination and win the air combat against UCAVS having any of the three basic scripting strategies. In Figure 9, at 5 s, UCAV\({}_{1}\) realizes that UCAV\({}_{2}\) will turn left, and then immediately turns right and, at 20 s, it turned left to enter the turning circle of UCAV\({}_{2}\), then stable tracking was conducted after 35 s. This indicates that UCAV\({}_{1}\) had a reasonable prediction of UCAV\({}_{2}\)'s maneuvers at 5 s, which is very similar to the maneuvering of manned aircrafts in 1-vs.-1 WVR air combat. In Figure 10, an interesting situation was observed. At 10 s, UCAV\({}_{1}\) did not turn right with UCAV\({}_{2}\), but turned left. This situation is not easy to understand from a bystander's perspective. It is assumed that at 10 s, UCAV\({}_{1}\) predicts that UCAV\({}_{2}\) will turn left in the future to cause UCAV\({}_{1}\) overshoot. Hence, UCAV\({}_{1}\) also turned left to respond. This situation occurs when the target UCAV is not completely rational, which will cause the error in predicting the target position to be very large, which results in erroneous maneuvering. However, overall, UCAV\({}_{1}\) corrected its erroneous maneuver in time and, finally, won the air combat. Table 2 shows the evaluation metrics of the air combat strategy.  
In Table 2, the offensive time accounts for more than 50% of the total intercept time, which indicates that the UCAV has a very efficient attack capability against the basic scripted air combat strategy.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Evaluation Metrics** & **Uniform Linear Motion** & **Decelerate Left Turn** & **S-Type Maneuver** \\ \hline Intercept time \(T_{I}\) & 40 s & 45 s & 50 s \\ Defensive time \(T_{D}\) & 0 s & 1 s & 0 s \\ Offensive time \(T_{\text{O}}\) & 26 s & 32 s & 26 s \\ \hline \end{tabular}
\end{table}
Table 2: Evaluation metrics of the air combat strategy.  
Figure 10: (a) Curve of fly trajectory against “S-type maneuver” strategy. (b) Curve of the change of air combat situation against “S-type maneuver” strategy.  
Figure 9: (a) Curve of the fly trajectory against “decelerate left turn” strategy. (b) Curve of the change of air combat situation against “decelerate left turn” strategy.