# Error Dynamics Based Dual Heuristic Dynamic Programming for Self-Learning Flight Control
## 3 Online Tracking Controller Based on ED-DHP
### The Fast Loop Error Dynamics of Attitude System and Model Identification  
The dynamical equations of the fast loop can be rewritten as:  
\[\dot{x}=f_{1}+f_{2}+gu+\Delta f \tag{6}\]  
The symbols in Equation (6) are given as:  
\[x=\omega,\,f_{1}=-\left(\overline{T}^{r(t)}\right)^{-1}R\overline{T}^{r(t)}\omega,\]  
\[f_{2}=\left(\overline{T}^{r(t)}\right)^{-1}\begin{bmatrix}\overline{c}_{mx}^ {\omega^{\prime}(t)}(Ma,\alpha,\beta)qSl\\ \overline{c}_{my}^{\omega^{\prime}(t)}(Ma,\alpha,\beta)qSl\end{bmatrix},\,gu =\left(\overline{T}^{r(t)}\right)^{-1}\begin{bmatrix}\overline{c}_{mx}^{ \omega^{\prime}(t)}(Ma,\alpha,\beta,\delta)qSl\\ \overline{c}_{my}^{\omega^{\prime}(t)}(Ma,\alpha,\beta)qSl\end{bmatrix}\]  
The disturbance term \(\Delta f\) expressed parameter uncertainty and modeling error, which causes the inaccurate trim value and external disturbance.  
The trim surface is set as \(u_{p}\approx-g^{-1}f_{2}\), which can be roughly obtained by the newton iteration method offline, and the standard control input \(u\in\mathbb{R}^{m}\) is given by:  
\[u=u_{p}+u_{a} \tag{7}\]  
where \(u_{a}\) is the output vector of actor NN in ED-DHP.  
Substitute Equation (7) into Equation (6) and subtract the reference trajectory dynamics \(\dot{x}_{ref}\), we can obtain:  
\[\begin{array}{c}\dot{e}=\dot{x}-\dot{x}_{ref}=f_{1}+f_{2}+gu+\Delta f-\dot{ x}_{ref}\\ =f_{1}+f_{2}+g(u_{p}+u_{a})+\Delta f-\dot{x}_{ref}\\ =f_{1}(x)+gu_{a}+\Delta f-\dot{x}_{ref}\end{array} \tag{8}\]  
And the problem can be written in a discrete form by [PERSON]'s method:  
\[\left\{\begin{array}{c}e(t+1)\approx e(t)-Z(t)+\Delta tf_{1}(x(t))+\Delta tg (t)u_{a}(t)+\Delta t\Delta f\\ x_{ref}(t+1)=\phi(x_{ref}(t))\\ Z(t)=x_{ref}(t+1)-x_{ref}(t)\end{array}\right. \tag{9}\]  
where \(f_{1}\) is known. \(g\), \(\Delta f\) and reference trajectory function \(\phi\) are unrevealed, however.  
The first equation in Equation (9) can be rewritten as follows:  
\[Y(t)=\varsigma(t)^{\text{T}}H(t) \tag{10}\]  
where  
\[Y(t)=e(t)-e(t-1)+Z(t-1)-\Delta tf_{1}(x(t-1)),\]  
\[\varsigma(t)=\begin{bmatrix}\Delta tg(t-1)\\ \Delta t\Delta f\end{bmatrix}=\begin{bmatrix}\varsigma_{1}(t)\\ \varsigma_{2}(t)\end{bmatrix}\in\mathbb{R}^{(m+1)\times n},\,H(t)=\begin{bmatrix} u_{a}(t-1)\\ 1\end{bmatrix}\in\mathbb{R}^{(m+1)\times 1}.\]\(Y(t)\) can be directly calculated from the measured vehicle attitude. \(H(t)\) is the information matrix, which is only related to the previous output vector of the actor NN. \(\varsigma(t)\) is the parameter matrix, and it can be identified in real-time by the RLS method:  
\[\varepsilon(t)=Y(t)-\hat{Y}(t)=\varsigma(t)^{\mathrm{T}}H(t)-\xi(t)^{\mathrm{T} }H(t) \tag{11}\]  
\[\xi(t)=\xi(t-1)+\frac{Cov(t-1)H(t)\varepsilon(t)^{\mathrm{T}}}{\gamma_{\mathrm{ RLS}}+H(t)^{\mathrm{T}}Cov(t-1)H(t)} \tag{12}\]  
\[Cov(t)=\frac{1}{\gamma_{\mathrm{RLS}}}\bigg{(}Cov(t-1)-\frac{Cov(t-1)H(t)H(t)^{ \mathrm{T}}Cov(t-1)}{\gamma_{\mathrm{RLS}}+H(t)^{\mathrm{T}}Cov(t-1)H(t)} \bigg{)} \tag{13}\]  
where \(\varepsilon(t)\in\mathbb{R}^{n\times 1}\) is the prediction error and \(\gamma_{\mathrm{RLS}}\) is the forgetting factor. \(Cov(t)\in\mathbb{R}^{(m+1)\times(m+1)}\) is the estimation covariance matrix.