## 3 1-vs.-1 WVR Air Combat Engagement Modelling
### Air Combat Reward Function and Termination Condition Designing  
In real 1-vs.-1 WVR air combat scenes, both UCAVs maneuver at the same time and, therefore, one UCAV will be located at the tail of the other UCAV. This ensures stable locking on the other party,then the opponent UCAV is difficult to get rid of the lock and is shot down finally. The combat geometry and parameters shown in Figure 2 are employed.  
Figure 1: The schematic diagram of MADDPG’s “centralized training, distribution execution”.  
In Figure 2, \(\lambda_{1}\) is the antenna train angle (ATA) of UCAV\({}_{1}\), which is the angle between the LOS vector and UCAV\({}_{1}\)'s velocity vector. \(\epsilon_{1}\) is the aspect angle (AA) of UCAV\({}_{1}\), which is the angle between the LOS vector, and UCAV\({}_{2}\)'s velocity vector. \(\epsilon_{1}\) can be obtained in terms of the velocity of vector \(\mathbf{V_{1}}\) and the line-of-sight (LOS) vector \(\mathbf{\rho}\), where \(\mathbf{\rho}\) denotes the LOS vector between UCAV\({}_{1}\) and UCAV\({}_{2}\). The ATA and AA are allowed to take any value between \(\pm 180^{\circ}\). ATA and AA can be obtained from Equations (6) and (7).  
\[\lambda_{1}=\cos^{-1}\left[\frac{\mathbf{V_{r}}\cdot\mathbf{\rho}}{|\mathbf{V_{2}}||\mathbf{ \rho}|}\right] \tag{6}\]  
\[\epsilon_{1}=\cos^{-1}\left[\frac{\mathbf{V_{b}}\cdot\mathbf{\rho}}{|\mathbf{V_{1}}||\mathbf{ \rho}|}\right] \tag{7}\]  
As can be seen from Figure 2, a smaller value of \(|\lambda_{1}|\) means that UCAV\({}_{1}\) or the gun of UCAV\({}_{1}\) is more accurate at aiming at UCAV\({}_{2}\), so \(|\lambda_{1}|\) is inversely proportional to the shooting chance or offensive advantage. Similarly, a smaller \(|\epsilon_{1}|\) indicates a smaller probability of being hit by UCAV\({}_{2}\); therefore, \(|\epsilon_{1}|\) is inversely proportional to the survival advantage. A termination condition of 1-vs.-1 WVR air combat and rewards can be designed through the abovementioned quantitative analysis of the air combat objectives and related parameters.  
This article considers a 1-vs.-1 WVR air combat as a ZSMG problem. Therefore, the rewards of both the UCAVs should be opposite. According to the objective of the 1-vs.-1 WVR air combat, the reward function can be designed using Equation (8), as follows:  
\[\mathcal{R}_{1}=-\mathcal{R}_{2}=\left\{\begin{array}{cl}1.0,&|\lambda_{1}(s )|<30^{\circ}\wedge|\epsilon_{1}(s)|<60^{\circ}\\ -1.0,&|\lambda_{2}(s)|<30^{\circ}\wedge|\epsilon_{2}(s)|<60^{\circ}\\ 0,&\textit{otherwise}\end{array}\right. \tag{8}\]  
According to the air combat geometry that is shown in Figure 2, the geometric relationship between AA and ATA of the two UCAVs can be obtained, as follows:  
\[\begin{split}|\lambda_{1}(s)|+|\epsilon_{2}(s)|&=180^{\circ} \\ |\lambda_{2}(s)|+|\epsilon_{1}(s)|&=180^{\circ}\end{split} \tag{9}\]  
When a UCAV (assumed to be UCAV\({}_{1}\)) satisfies the condition \(|\lambda_{1}(s)|<30^{\circ}\wedge|\epsilon_{1}(s)|<60^{\circ}\) and satisfies this condition for a period of time, it is believed that UCAV\({}_{1}\) can maintain the condition and complete the attack and destruction of UCAV\({}_{2}\).  
Figure 2: Aircraft relative geometry showing aspect angle and antenna train angle.