# Error Dynamics Based Dual Heuristic Dynamic Programming for Self-Learning Flight Control
## 3 Online Tracking Controller Based on ED-DHP  
### Control Structure of the System  
Based on the multiple-timescale features, the dynamics can be divided into the slow loop subsystem and the fast loop subsystem. So the controller for each system can be designed separately. As illustrated in Figure 1, the slow loop subsystem control law is designed by dynamic inversion with the tracking differentiator (TD) in this article to get the virtual control input of the slow loop subsystem. The ED-DHP is applied to angular rate control of the pitch, yaw, and roll rate, augmented with the slow outer loop. Rate control exhibits the lowest learning complexity because the angular rates and control surfaces are under direct dynamic relation. The control surfaces consist of the rough trim value, the output of actor NN, and the uncertainty compensator proposed in Section 3.2.  
The virtual control vector of the slow loop is designed by dynamic inversion:  
\[\omega_{ref}={g_{s}}^{-1}\Bigg{(}\begin{bmatrix}\dot{a}_{ref}+k_{\alpha}(\alpha _{ref}-\alpha)\\ \dot{\beta}_{ref}+k_{\beta}(\beta_{ref}-\beta)\\ \dot{\gamma}_{Vref}+k_{\gamma_{V}}(\gamma_{Vref}-\gamma_{V})\end{bmatrix}-f_{s }C_{b}^{*}\begin{pmatrix}a_{x}\\ a_{y}\\ a_{z}\end{pmatrix}+C_{I}^{b}\begin{bmatrix}g_{x}\\ g_{y}\\ g_{z}\end{pmatrix}\Bigg{)} \tag{4}\]  
where \(\omega_{ref}=[\omega_{x,ref}\ \ \omega_{y,ref}\ \ \omega_{z,ref}]^{\rm T}\) is the reference trajectory vector of the fast loop subsystem. \([a_{x}\ \ a_{y}\ \ a_{z}]^{\rm T}\) is the acceleration vector, which can be measured by an accelerometer. \([g_{x}\ \ g_{y}\ \ g_{z}]^{\rm T}\) is the gravity vector. \(C_{b}^{a}\) and \(C_{I}^{b}\) are the coordinate transformation matrix calculated by attitude angles with pitch angle \(\varphi\), yaw angle \(\psi\), and roll angle \(\gamma\), as well as attack angle \(\alpha\) and sideslip angle \(\beta\).  
\[C_{I}^{b}=\begin{bmatrix}\cos\psi\cos\varphi&\cos\psi\sin\varphi&-\sin\psi\\ -\cos\gamma\sin\varphi+\sin\gamma\sin\psi\cos\varphi&\cos\gamma\cos\varphi+ \sin\gamma\sin\psi\sin\varphi&\sin\gamma\cos\psi\\ \sin\gamma\sin\varphi+\cos\gamma\sin\psi\cos\varphi&-\sin\gamma\cos\varphi+ \cos\gamma\sin\psi\sin\varphi&\cos\gamma\cos\psi\end{bmatrix}\]  
\[C_{b}^{a}=\begin{bmatrix}\cos\alpha\cos\beta&\sin\beta&\sin\alpha\cos\beta\\ -\cos\alpha\sin\beta&\cos\beta&-\sin\alpha\sin\beta\\ -\sin\alpha&0&\cos\alpha\end{bmatrix}\]  
And the matrix function \(f_{s}\) can be written as:  
\[f_{s}=\frac{1}{V}\begin{bmatrix}0&-\sec\beta&0\\ 0&0&1\\ 0&\tan\beta+\cos\gamma_{V}\tan\theta&-\sin\gamma_{V}\tan\theta\end{bmatrix}\]  
Figure 1: Schematic of flight controller with ED-DHP.  
As we can see from the virtual control vector (4), the derivative of guidance commands is necessary to improve the dynamic performance of the system. To alleviate the \"items of explosion\" problem, the tracking differentiator was introduced [26], which is described as:  
\[\left\{\begin{array}{c}\dot{x}_{1}=x_{2}\\ \dot{x}_{2}=-r^{2}\text{sign}(x_{1}-\zeta)|x_{1}-\zeta|^{\tau}-rx_{2}\end{array}\right. \tag{5}\]  
where \(\zeta\) is the guidance command, \(x_{2}\) is the differential value. \(r\) and \(\tau\) are positive parameters to be designed.