## 2 Reinforcement Learning and the Longitudinal Aircraft Model
### Aircraft Model  
The state of the aircraft model is composed of the following: the Cartesian position, \(\vec{r}=(x,y,z)\); the body-frame velocity, \(\vec{v}_{b}=(u,v,w)\); the vehicle attitude as a non-normalised quaternion, \(\mathbf{q}=q_{i}\mathbf{i}+q_{j}\mathbf{j}+q_{k}\mathbf{k}+q_{w}\) (denoted in vector form in the order \((q_{i},q_{j},q_{k},q_{w})\)); the body-axis angular rates, \(\vec{\omega}=(p,q,r)\). Using this state representation, the full six degree-of-freedom equations of motion of the aircraft are  
\[\dot{\vec{r}}=\left( \begin{array}{c}\dot{x}\\ \dot{y}\\ \dot{z}\end{array} \right)=D(\mathbf{q})^{T}\vec{v}_{b}, \tag{2}\]  
\[\dot{\vec{v}}_{b}=\left( \begin{array}{c}\dot{u}\\ \dot{v}\\ \dot{w}\end{array} \right)=\vec{v}_{b}\times\vec{\omega}+\frac{1}{m_{b}}\Big{(} D(\mathbf{q})\vec{F}_{w}+\vec{F}_{b}\Big{)}, \tag{3}\]  
\[\dot{\mathbf{q}}=\left( \begin{array}{c}\dot{q}_{i}\\ \dot{q}_{j}\\ q_{k}\\ q_{w}\end{array} \right)=\frac{1}{2}\left( \begin{array}{cccc}0&r&-q&p\\ -r&0&p&q\\ q&-p&0&r\\ -p&-q&-r&0\end{array} \right)\mathbf{q}, \tag{4}\]  
\[\ddot{\vec{\omega}}=\left( \begin{array}{c}\dot{p}\\ \dot{q}\\ \dot{r}\end{array} \right)=I_{b}^{-1}(D(\mathbf{q})\vec{\tau}_{w}+\vec{\tau}_{ b}-I_{b}\vec{\omega}\times\vec{\omega}), \tag{5}\]  
\[D(\mathbf{q})=\frac{1}{\|\mathbf{q}\|^{2}}\left( \begin{array}{cc}q_{w}^{2}+ q_{i}^{2}-q_{j}^{2}-q_{k}^{2}&2(q_{i}q_{j}+q_{w}q_{k})&2(q_{i}q_{k}-q_{w}q_{j})\\ 2(q_{i}q_{j}-q_{w}q_{k})&q_{w}^{2}-q_{i}^{2}+q_{j}^{2}-q_{k}^{2}&2(q_{j}q_{k}+ q_{w}q_{i})\\ 2(q_{i}q_{k}+q_{w}q_{j})&2(q_{j}q_{k}-q_{w}q_{i})&q_{w}^{2}-q_{i}^{2}-q_{j}^{2}+ q_{k}^{2}\end{array} \right). \tag{6}\]  
For the work presented here, only the longitudinal terms, \(x,z,u,w,q_{j},q_{w},q\), are non-zero. Follow on work will consider the lateral motion of the aircraft. The forces \(\vec{F}_{b,w}\) and torques \(\vec{\tau}_{b,w}\) on the vehicle are computed from parameterised curves representing the aerodynamic coefficients. These aerodynamic coefficients are resolved into dimensionalised forces and torques through equations of the forms:  
\[F_{X} =\frac{1}{2}\rho V_{\infty}^{2}SC_{X},\] \[\tau_{Y} =\frac{1}{2}\rho V_{\infty}^{2}SC_{Y}.\]  
These equations use the dynamic pressure of the fluid from the \(\frac{1}{2}\rho v_{\infty}^{2}\) term, where \(\rho\) is the air density and \(V_{\infty}\) is the freestream velocity. Alongside this is the coefficient itself \(C_{X}\), and scaling terms, \(S\) and \(\bar{c}\). Typical choices in aerospace, and those used for this model, are the main wing area for \(S\) and the main wing mean aerodynamic chord for \(\bar{c}\). The coefficient terms are, typically, a function of the aerodynamic state of the aircraft and current control inputs. Figure 1 is a graphical representation of the various angles used in aircraft dynamics. The vehicle's centre-of-mass is travelling along the green arrow, describing the flight path. The angle \(\gamma\) is the angle between this direction of travel, and the horizon is the flight path angle. The angle between the horizon and the aircraft's reference axis is the pitch angle, denoted by \(\theta\), and the difference between the pitch and flight path angles provides the angle of attack, denoted by \(\alpha\). The angle of attack describes the angle between the oncoming airflow and the aircraft reference axis and is key for the modelling of the aerodynamic forces and moments. The elevator angle, \(\eta\), is also shown. The model used in this work is based on wind-tunnel data for the FMS Models MXS2 airframe ([[https://www.fmshobby.com/products/fms-1100](https://www.fmshobby.com/products/fms-1100) mm-mxs-v2-pnp-with-reflex]([https://www.fmshobby.com/products/fms-1100](https://www.fmshobby.com/products/fms-1100) mm-mxs-v2-pnp-with-reflex), accessed on 24 November 2022), pictured in Figure 2. The airframe was mounted in the University of Bristol's \(7^{\prime}\times 5^{\prime}\) wind tunnel, with a load cell mounted internally (see Figure 2a).