# Efficient Neural Network Modeling for Flight and Space Dynamics Simulation

Authors: Ayman Hamdy Kassem
Journal: International Journal of Aerospace Engineering
 (2011)

---

## Abstract

This paper represents an efficient technique for neural network modeling of flight and space dynamics simulation. The technique will free the neural network designer from guessing the size and structure for the required neural network model and will help to minimize the number of neurons. For linear flight/space dynamics systems, the technique can find the network weights and biases directly by solving a system of linear equations without the need for training. Nonlinear flight dynamic systems can be easily modeled by training its linearized models keeping the same network structure. The training is fast, as it uses the linear system knowledge to speed up the training process. The technique is tested on different flight/space dynamic models and showed promising results.


---

## Body

## 1. Introduction

An artificial neural network (ANN), usually abbreviated as neural network (NN), is a mathematical model that is inspired by the structure of the brain biological neural networks. A neural network consists of an interconnected group of elements called neurons, arranged in different layers, and it processes information by propagating data through these elements. Neural networks use “weights” to change the parameters of the connections to the neurons. These weights are calculated through a learning process of the available data. Simply speaking, neural networks are nonlinear statistical data modeling tools used to model complex relationships between inputs and outputs. Two challenges face neural network designer: the selection of neural network topology (i.e., number of layers and number of neurons) and the initialization of the weights. It is well recognized that a perfect way to decide an appropriate architecture and assign initial values to start neural network training has yet to be established. A common criticism of neural networks came from those two challenges where wrong selection of topology or initial values for weights leads to slow operation [1–5].A lot of work has already been devoted to the application of neural networks (NN) in dynamics and control, both with respect to controller design and to system modeling and identification [6–10]. Modeling dynamic systems or controllers by neural networks consumes a lot of the network designer time in selecting the appropriate structure and in training the network and had to be done offline. The network size also affects the execution time and has high impact on network design and selection especially for real-time problems.The network design technique given in this paper is systematic to relieve the network designer from the intensive task of finding an appropriate structure for the neural model. For linear systems, based on the work done by Kaiser [11] and Kassem and Sameh [12], the paper can find the network weights and biases analytically so that the neural network can work directly without the need for training.Nonlinear systems can be modeled by training its linearized models about specific operating point without the need to change the size or the structure of the network. The training is fast and can be done online, as it uses the linear system knowledge to speed up the training process [12].A modified version of time-delay neural network is presented in this work, and when it is combined with the analytical modeling, it can fast and accurately model linear and nonlinear dynamic systems. The technique is tested on different flight and space dynamics models and showed good match with the linear and nonlinear equations.


## 2. NN Analytical Model with One Hidden Neuron

This section represents the mathematical equations for linearizing a neural network model consisting of an input neuron, a hidden neuron, and an output neuron (Figure1). The input and output neurons have linear activation functions, and the hidden neuron has a sigmoid function. The ability of this neural network model to approximate the straight-line equation y=ax+b is an essential feature to allow direct representation of difference equations. During the following calculations, σ(x) denotes a continuous nonlinear transfer function applied in the neurons such as the sigmoid function σ(x)=1/(1+e-x).Figure 1
The model used for approximating the functiony=ax+b.Assuming thaty=ax+b is to be approximated for x∈[x0,x1], x1>x0 within an error limit of ε, ε>0, select z0,z1 on the sigmoid horizontal axis with z1>z0 such that |σ̇(z)-qa|<ε/|x1-x0| for z∈[z0,z1] with a constant q∈R. In other words, this step selects the interval on the horizontal axis of the sigmoid, where the sigmoid function can be seen as linear.Define a functionf, f:[x0,x1]→[z0,z1] as (1)f(x)=z0+z1-z0x1-x0(x-x0)=(z0-z1-z0x1-x0x0)+z1-z0x1-x0x=p1+p2x.
Finally, further modifications (see Kaiser 1994 [11] for details) yield(2)ax+b=w(p2x+p1)+w0,
where (3)w=1p2q,w0=-ap1p2-σ(0)p2q+b.Takingp2 as the weight to the corresponding input unit, p1 as the bias of the hidden neuron, w as the weight of the link between the hidden neuron and an output unit, and w0 as the bias of the output unit results in the desired approximation. This analytic approximation forms the base for the algorithm that will be used to model the dynamics of linear systems. Figure 1 shows the model used in this paper for approximating the linear function.


## 3. Modified Time-Delay Neural Networks

Time-delay neural networks (TDNNs), introduced by Waibel et al. [13], form a class of neural networks with a special topology. They are used for position-independent recognition of features within a larger pattern. In order to be able to recognize patterns place or time invariant, older activation and connection values of the feature units have to be stored. This is performed by making a copy of the feature units with all their outgoing connections in each time step before updating the original units. The total number of time steps saved by this procedure is called delay.In this work, the TDNN is modified to fit exactly the general difference equation for a linearnth order dynamic system with constant coefficients shown as follows [14]: (4)y(n+k)+an-1y(n+k-1)+⋯+a1y(k+1)+a0y(k)=f(k),
where y(i), i=k,k+1,…,k+2 denotes the discrete variable y at the ith instant and f(k) denotes the input value which could be another difference equation.The outputy(n+k) is separated to one side of the equation, while its delayed values and the input are moved to the other side of the equation as follows:(5)y(n+k)=-an-1y(n+k-1)-⋯-a1y(k+1)-a0y(k)+f(k).
Now, the output value is just a summation of linear relations in the form y=ax which can be modeled analytically as an NN with one hidden neuron. This way, the one hidden neuron NN is used as a building block, and the complete network is built systematically as shown in Figure 2.Figure 2
NN for modeling difference equation.


## 4. The Algorithm

First, the nonlinear system is linearized about an operating point (trim condition) using any linearization technique such as Taylor series or small disturbance theory. Second, an analytic NN model is build using this linear model after discretization. Finally, the model is trained with the nonlinear model data points and with initial values for weights and biases taken from the linear analytic NN model. The detailed modeling algorithm is shown in Algorithm1. For linear systems steps 2, 3, and 4 only apply, as there is no linearization, and there is no need for training.Algorithm 1:NN algorithm for modeling nonlinear systems.
(1) Starting with the nonlinear model in the form ofẋ(t)=f(x,u,t),wherex are states vector and u is input vector.(2) Linearize the nonlinear model around an operating point and represent it in state space form as follows:ẋ(t)=Ax+Bu,whereA, B are the system and input matrices, respectively.(3) Discretize the continuous linearized system using finite difference, the equations will beẋi+1=(A+IΔt)xi+BΔtui,whereI is the identity matrix and Δt is the time step.(4) Build an analytic NN to replicate the linear model using the proposed linear system given in section two and Figure2.(5) Train the linear analytic NN with the nonlinear model data and with initial weights and biases taken from the linear analyticNN.(6) If the final NN model is not satisfactory, then increment model size by adding another linear model at different operatingpoint, repeat steps 2 to 5, and concatenate the two models to get a better approximation. The number of final concatenatedmodels depends on the required accuracy and the acceptable execution time.(7) Stop


## 5. Test Cases

Three test cases will be given in this section illustrating the application of the proposed modeling approach using NN. The first two test cases in Sections5.1 and 5.2 show the result of modeling of unstable linear dynamics without details of the proposed algorithm. The third nonlinear test case presented in Section 5.3 will be accompanied with more details.


### 5.1. Helicopter Longitudinal Dynamics

The state space linear model of longitudinal motion of a helicopter [15] is given by (6)[q̇θ̇u̇]=[-0.40-0.01100-1.49.8-0.02][qθu]+[6.309.8]δ,
where q, θ, and v are pitch rate, pitch angle of the fuselage, and horizontal forward velocity, respectively, and the control input δ is the rotor tilt angle. The system is unstable with both zeros and poles in the right hand plane, for example, system poles
(7)p1,2=0.3776±0.3445j;p3=-0.375.
Using the linear algorithm, the NN can fit the system’s three states very accurately for unit step δ input as shown in Figure 3.NN versus state-space models for helicopter dynamics described by (6).
(a)(b)(c)


### 5.2. F-16 Longitudinal Dynamics

The equations for longitudinal motion of the F-16 fighter are considered for a trimmed flight condition withU=502 ft/s, α=0.0393 rad, θ=0.0393 rad, and q=0 rad/s. The state vector is x=[Δu,Δα,Δθ,Δq],  where Δv, Δα, Δθ, and Δq are the longitudinal velocity, angle of attack, pitch angle, and pitch rate, respectively. The single control input is the elevator deflection δE in degrees. The state equation in matrix form becomes [16](8)ẋ=Ax+BδE,
where(9)A=[-1.9311e-28.8157-32.17-0.575-2.5389e-4-1.018900.905100012.9465e-120.82230-1.0774],B=[0.1737-2.14990-0.1756].
Using the linear algorithm, the NN can fit the system’s four states very accurately for unit step input δE, as shown in Figure 4.NN versus state-space models for F-16 longitudinal dynamics.
(a)(b)(c)(d)


### 5.3. Generic Fighter Nonlinear Flight Dynamics

Consider the flight dynamics model of a high-performance aircraft for the short-period mode [17–19], where the motion involves rapid changes to the angle of attack and pitch attitude at roughly constant airspeed. The model covers both linear and nonlinear flight behavior through an extensive range in angle of attack. The model is mathematically described as [18](10)α̇=q+9.168Cz-1.834(δe+7)+7.362,q̇=5.73(-α-1.5δE)+2.865,
where(11)Cz={-0.07378αα≤14.36,0.09722α2-2.865α+20.0414.36<α≤15.6,-0.01971α2+0.7439α-7.80815.6<α<19.6,-0.4733-0.01667αα≤19.6,andα is the angle of attack in degrees, q is the pitch rate in degrees per second, δe is the elevator control surface deflection in degrees, and CZ(α) is the normal force coefficient and the sole nonlinearity in the model. The system dynamics can be summarized as(a)
with angle of attack <14.36, the aircraft has stable dynamics,(b)
with angle of attack between 14.36 and 19.6 degrees, the aircraft has oscillatory instability,(c)
onset of limit cycle oscillations (LCO) at high angles of attack.It is obvious that a single linear model cannot represent this nonlinear behavior. So, this example will show the strength of the proposed algorithm. It will start with a single linear NN model and then train it to capture the nonlinear behavior, then adding a second model at different operating point to improve the accuracy.The linearized model forα<14.36 deg is given by (12)[α̇q̇]=[-0.7671-5.730][αq]+[-1.8348.595]δE+[-5.4762.865].The discretized system (12) using Δt=0.004 s is given by(13)[αi+1qi+1]=[9.973e-014.0e-03-2.292e-021.00][αiqi]+[-7.336e-03-3.438e-02]δEi+[-2.1904e-021.1460e-02].
Using the linear algorithm, the NN can fit the system’s two states very accurately for the unit step input δE, as shown in Figure 5. This assumes that the angle of attack does not exceed value 14.6 deg.Linear NN model versus state-space model for fighter linearized dynamics.
(a)(b)We can see the difference when using the linear NN model for predicting behavior of the nonlinear system when angle of attack exceeds value of 14.6 deg., as shown in Figure6. Therefore, the NN needs some training. A step input of δE=-11 for five seconds is used to train the system, and a fifteen seconds step input with same value (δE=-11) is used for testing, and the results are shown in Figure 7. Initial weights and biases were taken from the linear NN to speed up the convergence.NN trained model versus nonlinear models for fighter dynamics.
(a)(b)NN trained model 1 versus nonlinear model for fighter dynamics.
(a)(b)When initiating the NN learning with zero or random weights and biases, it has been noticed that it does not converge to the correct values or in best case converges after 5 times the required number of iterations compared with the case of using linear NN weights and biases.It is clear that the NN works relatively good compared with its size (4 sigmoid elements). The model can be improved by adding another model (at different operating points).Linearizing the system forα>19.6, we get the following state space model:(14)[α̇q̇]=[-0.1531-5.730][αq]+[-1.8348.595]δE+[8.7152.865].
Equation (14) is discretized using Δt=0.004s to give (15)[αi+1qi+1]=[9.994e-014.0e-03-2.292e-021.00][αiqi]+[-7.336e-03-3.438e-02]δEi+[3.486e-021.146e-02].
Same procedure is used to produce the NN model for this linear system. Adding the two NN models, we get double the size NN (8 sigmoid elements). After training the concatenating model, we can get an improved model which produces a much better results as shown in Figure 8. It is clear that this improved model can catch the limit cycle very well, so there is no need to add another NN model in the region between 14.36 and 19.6 degrees.NN trained model 2 versus nonlinear model for generic fighter dynamics.
(a)(b)


### 5.4. Satellite Attitude Dynamics

This final case study shows a space-related problem and also illustrates the effectiveness of the proposed NN for systems with sparse dynamic matrix(A). In this case study, the linearized attitude dynamics for a symmetric satellite for small angles and gravity gradient torques approximation is given by [20](16)ẋ=Ax+Bu,
where(17)x=[θϕψθ̇ϕ̇ψ̇],A=[000100000010000001δ1000000δ2000δ300δ40δ50],B=[00000000001Iy01Ix00001Iz],u=[TxTyTz],δ1=-3ω02Iy(Ix-Iz),δ2=-4ω02Ix(Iy-Iz),δ3=-ω0Ix(Iy-Iz-Ix),δ4=-ω02Iz(Iy-Ix),δ5=-ω0Iy(Iz+Ix-Iy).
The satellite altitude = 700 km, its moments of inertia Ix=80, Iy=82, and Iz=9 kg-m2, and an initial attitude is represented by θ(0) = 5°, ϕ(0) = 5°, and ψ(0) = 5°. Figure 9 shows initial condition response of the satellite using the state-space and NN models. The NN model is built by direct substitution without the need to train the network, and the results show good match. Also, the NN model is very efficient, as it uses only eight (one hidden neuron NN) for matrix A representing only the nonzero elements.NN versus state-space models for satellite dynamics.
(a)(b)(c)


## 5.1. Helicopter Longitudinal Dynamics

The state space linear model of longitudinal motion of a helicopter [15] is given by (6)[q̇θ̇u̇]=[-0.40-0.01100-1.49.8-0.02][qθu]+[6.309.8]δ,
where q, θ, and v are pitch rate, pitch angle of the fuselage, and horizontal forward velocity, respectively, and the control input δ is the rotor tilt angle. The system is unstable with both zeros and poles in the right hand plane, for example, system poles
(7)p1,2=0.3776±0.3445j;p3=-0.375.
Using the linear algorithm, the NN can fit the system’s three states very accurately for unit step δ input as shown in Figure 3.NN versus state-space models for helicopter dynamics described by (6).
(a)(b)(c)


## 5.2. F-16 Longitudinal Dynamics

The equations for longitudinal motion of the F-16 fighter are considered for a trimmed flight condition withU=502 ft/s, α=0.0393 rad, θ=0.0393 rad, and q=0 rad/s. The state vector is x=[Δu,Δα,Δθ,Δq],  where Δv, Δα, Δθ, and Δq are the longitudinal velocity, angle of attack, pitch angle, and pitch rate, respectively. The single control input is the elevator deflection δE in degrees. The state equation in matrix form becomes [16](8)ẋ=Ax+BδE,
where(9)A=[-1.9311e-28.8157-32.17-0.575-2.5389e-4-1.018900.905100012.9465e-120.82230-1.0774],B=[0.1737-2.14990-0.1756].
Using the linear algorithm, the NN can fit the system’s four states very accurately for unit step input δE, as shown in Figure 4.NN versus state-space models for F-16 longitudinal dynamics.
(a)(b)(c)(d)


## 5.3. Generic Fighter Nonlinear Flight Dynamics

Consider the flight dynamics model of a high-performance aircraft for the short-period mode [17–19], where the motion involves rapid changes to the angle of attack and pitch attitude at roughly constant airspeed. The model covers both linear and nonlinear flight behavior through an extensive range in angle of attack. The model is mathematically described as [18](10)α̇=q+9.168Cz-1.834(δe+7)+7.362,q̇=5.73(-α-1.5δE)+2.865,
where(11)Cz={-0.07378αα≤14.36,0.09722α2-2.865α+20.0414.36<α≤15.6,-0.01971α2+0.7439α-7.80815.6<α<19.6,-0.4733-0.01667αα≤19.6,andα is the angle of attack in degrees, q is the pitch rate in degrees per second, δe is the elevator control surface deflection in degrees, and CZ(α) is the normal force coefficient and the sole nonlinearity in the model. The system dynamics can be summarized as(a)
with angle of attack <14.36, the aircraft has stable dynamics,(b)
with angle of attack between 14.36 and 19.6 degrees, the aircraft has oscillatory instability,(c)
onset of limit cycle oscillations (LCO) at high angles of attack.It is obvious that a single linear model cannot represent this nonlinear behavior. So, this example will show the strength of the proposed algorithm. It will start with a single linear NN model and then train it to capture the nonlinear behavior, then adding a second model at different operating point to improve the accuracy.The linearized model forα<14.36 deg is given by (12)[α̇q̇]=[-0.7671-5.730][αq]+[-1.8348.595]δE+[-5.4762.865].The discretized system (12) using Δt=0.004 s is given by(13)[αi+1qi+1]=[9.973e-014.0e-03-2.292e-021.00][αiqi]+[-7.336e-03-3.438e-02]δEi+[-2.1904e-021.1460e-02].
Using the linear algorithm, the NN can fit the system’s two states very accurately for the unit step input δE, as shown in Figure 5. This assumes that the angle of attack does not exceed value 14.6 deg.Linear NN model versus state-space model for fighter linearized dynamics.
(a)(b)We can see the difference when using the linear NN model for predicting behavior of the nonlinear system when angle of attack exceeds value of 14.6 deg., as shown in Figure6. Therefore, the NN needs some training. A step input of δE=-11 for five seconds is used to train the system, and a fifteen seconds step input with same value (δE=-11) is used for testing, and the results are shown in Figure 7. Initial weights and biases were taken from the linear NN to speed up the convergence.NN trained model versus nonlinear models for fighter dynamics.
(a)(b)NN trained model 1 versus nonlinear model for fighter dynamics.
(a)(b)When initiating the NN learning with zero or random weights and biases, it has been noticed that it does not converge to the correct values or in best case converges after 5 times the required number of iterations compared with the case of using linear NN weights and biases.It is clear that the NN works relatively good compared with its size (4 sigmoid elements). The model can be improved by adding another model (at different operating points).Linearizing the system forα>19.6, we get the following state space model:(14)[α̇q̇]=[-0.1531-5.730][αq]+[-1.8348.595]δE+[8.7152.865].
Equation (14) is discretized using Δt=0.004s to give (15)[αi+1qi+1]=[9.994e-014.0e-03-2.292e-021.00][αiqi]+[-7.336e-03-3.438e-02]δEi+[3.486e-021.146e-02].
Same procedure is used to produce the NN model for this linear system. Adding the two NN models, we get double the size NN (8 sigmoid elements). After training the concatenating model, we can get an improved model which produces a much better results as shown in Figure 8. It is clear that this improved model can catch the limit cycle very well, so there is no need to add another NN model in the region between 14.36 and 19.6 degrees.NN trained model 2 versus nonlinear model for generic fighter dynamics.
(a)(b)


## 5.4. Satellite Attitude Dynamics

This final case study shows a space-related problem and also illustrates the effectiveness of the proposed NN for systems with sparse dynamic matrix(A). In this case study, the linearized attitude dynamics for a symmetric satellite for small angles and gravity gradient torques approximation is given by [20](16)ẋ=Ax+Bu,
where(17)x=[θϕψθ̇ϕ̇ψ̇],A=[000100000010000001δ1000000δ2000δ300δ40δ50],B=[00000000001Iy01Ix00001Iz],u=[TxTyTz],δ1=-3ω02Iy(Ix-Iz),δ2=-4ω02Ix(Iy-Iz),δ3=-ω0Ix(Iy-Iz-Ix),δ4=-ω02Iz(Iy-Ix),δ5=-ω0Iy(Iz+Ix-Iy).
The satellite altitude = 700 km, its moments of inertia Ix=80, Iy=82, and Iz=9 kg-m2, and an initial attitude is represented by θ(0) = 5°, ϕ(0) = 5°, and ψ(0) = 5°. Figure 9 shows initial condition response of the satellite using the state-space and NN models. The NN model is built by direct substitution without the need to train the network, and the results show good match. Also, the NN model is very efficient, as it uses only eight (one hidden neuron NN) for matrix A representing only the nonzero elements.NN versus state-space models for satellite dynamics.
(a)(b)(c)


## 6. Conclusions

The procedure and the test cases described in this paper show that the linear neural network based on the analytic one hidden neuron NN is best suited as a starting point for fast learning nonlinear networks. It also shows that the modified Time-delay neural network combined with the analytical network form a very efficient algorithm for flight dynamics equations approximations. The presented algorithm relieves the network designer from the work intensive task of finding a suitable structure for the neural network, given that the designer has at least some qualitative knowledge about the modeling task. It also shows a systematic way of increasing the size of the neural network to improve the accuracy without the need of guessing.  It is clear that the algorithm gives some guidance to the learning process and helps the network escaping local minima by using initial guess from the linear model. This illustrates the importance and usefulness of this algorithm in the field of neural network modeling of nonlinear flight dynamics.



---

*Source: 247294-2011-10-01.xml*