# Error Dynamics Based Dual Heuristic Dynamic Programming for Self-Learning Flight Control
## 4 Numerical Simulation and Discussion
### Example 2  
To further verify the effectiveness and robustness of the proposed control scheme, the guidance command tracking with morphing process at variable operating points is considered, as well as the model uncertainty.  
The initial tracking errors of the three channels are 1 degree, and the control parameters of the fast loop and slow loop are all the same as in Section 4.1. Three cases with different parameters perturbation are carried into the numerical simulation, shown in Table 1. The switch of the air vehicle's configuration occurs between 20 s-23 s. To ensure the stability of initial learning, TD starts to act on the system at 5 s.  
As is shown in Figure 8, the whole duration of the simulation is 30 s, and the guidance commands are not concrete expressions. The controller makes the vehicle track the reference value of the attack angle of 2 degrees and maintains a sideslip angle and bank angle of around 0 degrees. After that, the vehicle is lifted at an attack angle of 8 degrees, morphing and maintained at the current angle of attack. Additionally, in this simulation condition, the flight speed of the vehicle is fast, and the dynamic pressure changes dramatically within the simulation time.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Case 0** & **Case 1** & **Case 2** \\ \hline \(\Delta I=0\%\) & \(\Delta I=-5\%\) & \(\Delta I=5\%\) \\ \(\Delta m=0\%\) & \(\Delta m=-5\%\) & \(\Delta m=5\%\) \\ \(\Delta\rho=0\%\) & \(\Delta\rho=10\%\) & \(\Delta\rho=-10\%\) \\ \(\Delta\mathbb{C}_{m}=0\%\) & \(\Delta\mathbb{C}_{m}=30\%\) & \(\Delta\mathbb{C}_{m}=-30\%\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: The parameter uncertainty for 3 cases.  
Figure 8: Tracking performance under three cases. In case 1, the dynamic pressure and moment coefficients are larger. So the system owns stronger control efficiency, causing higher requirements for the servo.  
\(\alpha\), \(\beta\), and \(\gamma_{V}\) can follow the reference signals after about 2 s. In Figure 9, the max tracking errors of \(\alpha\), \(\beta\), and \(\gamma_{V}\) are both within 0.5 degrees before 2 s. The tracking errors of the attack angle are shown in Table 2.  
Figure 10 shows that the drastic changes in control surfaces are mainly concentrated in the first two seconds of the learning phase. In the lifting process, the symbol change of the pitch channel's equivalent control surface has occurred, which means the desperate transformation of the operating points. This can be furthermore reflected in the sharp change of critic NN parameters in Figure 11, while the parameters of actor NN are only slightly adjusted.  
The parameters of critic NN are obviously affected during the morphing process. However, the actor NN is adjusted slightly. It shows that the updating of NNs is mainly related to the tracking error. When actor NN owns strong control ability, that is, the tracking errors are very small during the morphing, the network parameters can adapt to the process only by fine-tuning.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Time** & **Case 0** & **Case 1** & **Case 2** \\ \hline
2 s\(\sim\)10 s & \(<\) 0.005\({}^{\circ}\) & \(<\) 0.01\({}^{\circ}\) & \(<\) 0.0025\({}^{\circ}\) \\
10 s\(\sim\)20 s & \(<\) 0.19\({}^{\circ}\) & \(<\) 0.22\({}^{\circ}\) & \(<\) 0.21\({}^{\circ}\) \\
20 s\(\sim\)25 s & \(<\) 0.08\({}^{\circ}\) & \(<\) 0.1\({}^{\circ}\) & \(<\) 0.06\({}^{\circ}\) \\
25 s\(\sim\)30 s & \(<\) 0.02\({}^{\circ}\) & \(<\) 0.005\({}^{\circ}\) & \(<\) 0.04\({}^{\circ}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Tracking errors of attack angle at different time periods.  
Figure 10: Control signals under three cases.  
Figure 9: Tracking errors under three cases.