# A Deep Neural Network-Based Fault Detection Scheme for Aircraft IMU Sensors

Authors: Yiming Zhang; Hang Zhao; Jinyi Ma; Yunmei Zhao; Yiqun Dong; Jianliang Ai
Journal: International Journal of Aerospace Engineering
 (2021)

---

## Abstract

A new fault detection scheme for aircraft Inertial Measurement Unit (IMU) sensors is developed in this paper. This scheme adopts a deep neural network with a CNN-LSTM-fusion architecture (CNN: convolution neural network; LSTM: long short-term memory). The fault detection network (FDN) developed in this paper is irrelative to aircraft model or flight condition. Flight data is reformed into a 2D format for FDN input and is mapped via the net to fault cases directly. We simulate different aircrafts with various flight conditions and separate them into training and testing sets. Part of the aircrafts and flight conditions appears only in the testing set to validate robustness and scalability of the FDN. Different architectures of FDN are studied, and an optimized architecture is obtained via ablation studies. An average detecting accuracy of 94.5% on 20 different cases is achieved.


---

## Body

## 1. Introduction

Inertial Measurement Unit (IMU) is a key sensor to aircraft control system as it measures angular speeds and accelerations in flight. Faults in IMU may result in serious consequences as flight control algorithms are highly dependent on feedback of angular speeds and accelerations. Hence, it is of immense significance to achieve fault detection of IMU in order to improve fault tolerance of the control systems [1].Hardware redundancy (HR) is a commonly adopted approach for sensor fault detection on aircrafts [2, 3]. HR scheme adopts redundant sensors and a voting system to eliminate erroneous data measured by fault sensors. Despite the fact that HR scheme usually improves fault tolerance of flight control systems, defects including high cost and vulnerability to generic fault are nonnegligible [4].Analytical redundancy (AR) is another kind of approach, which can be divided into two categories: model-based AR and data-driven AR.Model-based AR is a more traditional technique. Sensors are modeled mathematically, and an output estimator is proposed [5]. Real output of a certain sensor is monitored and compared with the estimated output, yielding a residual, to identify sensor faults. A majority of studies on model-based analytical redundancy for fault detection were carried out by applying Kalman filtering (KF), e.g., extended Kalman filtering (EKF) [6–8], unscented Kalman filtering (UKF) [9, 10], two-step EKF [6], fuzzy logic KF [11], and hidden-Markov-based KF [12]. Other model-based methods were found using H∞ synthesis [13–15], set-value observer [16–19], and moving horizon estimator [20]. Model-based methods are highly dependent on aircraft dynamics and kinematics, which differ significantly among different aircrafts at diverse flight conditions. Thus, scalability is not promised by using model-based schemes.Data-driven AR is an alternative to model-based AR. As data-driven AR maps sensor outputs to fault cases directly, which means aircraft dynamics are not directly referred to in the fault detection process, it would potentially be a scalable scheme for different aircrafts and flight conditions. Neural network (NN) is widely used in recent years for data-driven AR as it is a powerful nonlinear fitting tool. Fully connected cascade neural network is used to detect IMU failure in [21]. Adaptive EKF is compensated with online updating NN to reduce the computation time in [22]. NN-based observer is developed instead of KF for residual generating in [23, 24]. In [25], fuzzy interval models and NN are used for IMU sensor prediction. Deep neural network (DNN) is an advanced technique in fault detection as an upgraded version of NN, including recurrent neural network (RNN), long short-term memory (LSTM), and convolution neural network (CNN). Fault detection was achieved with RNN in [1, 27]. LSTM-RNN was used to denoise IMU outputs in [27]. CNN is a deep feedforward NN with representation learning capability usually used in computer vision. In [28], flight data were reformed into a 2D image-like format to use CNN for sensor fault detection. And in [29], a CNN-LSTM fusion scheme was developed for fault detection on air sensors. Other researches using NN approaches to deal with fault detection were found in [30–32].Most existing works study fault detection problems of a certain aircraft flying a certain flight condition, e.g., cruising and land and take over (LTO). Robustness is not guaranteed. In this paper, we expect to develop a data-driven scheme to detect IMU faults, which is more robust and scalable for diverse aircrafts and flight conditions. To be specific, a CNN-LSTM-fusion fault detection network (FDN) is developed. Measured flight states reformed into a 2D image-like format is the input to the FDN and fault case classification the output. We prepared data sets of various aircrafts/flight conditions for FDN training and testing. Studies were carried out on FDN architectures starting from a “full” version that adopts all 12 flight states measured by different sensors as input. CNN and RNN are both applied for every single measured channel. Two different kinds of aircrafts at LTO and cruise stage are used for network training, and one other kind is used for testing. Robustness is justified by our diverse test sets.


## 2. Problem Definition

Aerodynamic is written as(1)V̇=ax−gSθCαCβ+ay+gSϕCθSβ+az+gCϕCθSαCβ,α̇=−axSα+azCα+gCϕCθCα+gSθSαVCβ+q−pCα+rSαSβCβ,β̇=−ax−gSθCαSβ+ay+gSϕCθCβ−az+gCϕCθSαSβV+pSα−rCa,wherein S∗=sin∗ and C∗=cos∗. V,α,β are airspeed, angle of attack, and sideslip angle, respectively. ϕ,θ,ψ and p,q,r are angles and angular speeds, respectively. ax,ay,az denote accelerations along the x,y,z axes in the body frame of the aircraft. External forces along body axes are defined as Fk=akmk=x,y,z which are functions of flight states, control inputs, and geometric parameters:(2)Fx=Fxδth,V,α,S,⋯,Fy=Fyδr,δa,V,α,S,b,c,⋯,Fz=Fzδe,V,α,S,c,⋯,wherein δth,δe,δa,δr are control inputs of throttle, elevator, aileron, and rudder. S,b,cdenote wing area, wing span, and mean aerodynamic chord, respectively.Kinematics in rotational channel are described as(3)ϕ̇=p+qSϕSθCθ+rCϕSθCθ,θ̇=qCϕ−rSϕ,ψ̇=qSϕ+rCϕCθ.Dynamics in rotational channel yield [33](4)ṗ=c1rq+c2pq+c3Mx+c4Mz,q̇=c5pr−c6p2−r2+c7My,ṙ=c8pq−c2rq+c4Mx+c9Mz,wherein Mx,My,Mz are moments exerted on the aircraft and c1~c9 are inertial parameters of the aircraft (refer to [33] for detailed definitions). External moments are functions of flight states, control inputs, and geometric parameters:(5)Mx=Mxδr,δa,V,α,S,b,⋯,My=Myδr,δa,V,α,S,b,⋯,Mz=Mzδe,V,α,S,c,⋯.


## 3. Flight Data Sets

### 3.1. Basic Flight Data with Disturbances and Measurement Noises

To guarantee robustness and performance of the DNN-based fault detection scheme, it is vital to prepare a diverse data set. Thus, we generate a data set of 3 different aircrafts (i.e., Y, D, and B). To be specific, Y is a large cargo aircraft, D is a general aviation, and B is a passenger aircraft. We simulated 4 configurations of different flight conditions, including the following:(1)
Landing and take-off (LTO) with manual control (aircraft Y)(2)
High-altitude cruise with autopilot (AP) (aircraft D)(3)
High-altitude cruise with autopilot (AP) (aircraft B)(4)
Low-altitude free flight with manual control (aircraft B)We intended to fully cover the whole flight envelope (e.g., LTO and cruise) by introducing the 4 configurations mentioned above. More detailed information of the configurations is concluded in Table1.Table 1
Aircrafts and flight conditions.
AircraftTypeEnginesWeighSpanFlight conditionUsageYLarge cargoQuadruple piston41.0 t38.0 mLow altitude, LTO, manualTrainDGeneral aviationDouble piston3.12 t19.8 mHigh altitude, cruise, APTrainBLarge passengerQuadruple turbo174 t59.6 mHigh altitude, cruise, APTestBLarge passengerQuadruple turbo174 t59.6 mLow altitude, free flight, manualTestHigh-altitude cruise of aircraft Y and low-altitude LTO of aircraft D are simulated for network training; high-altitude cruise and low-altitude free flight of aircraft B are simulated for network testing.To simulate the atmospheric turbulences, Dryden model is adopted, which injects perturbances to “clean” flight states. A speed model of Dryden wind can be defined by spectrum functions:(6)ϕuΩ=σu22Luπ11+LuΩ2,ϕvΩ=σv2Lvπ1+3LvΩ21+LvΩ22,ϕwΩ=σw2Lwπ1+3LwΩ21+LwΩ22,wherein Lu,Lv,Lw represent the turbulence scale lengths. σu,σv,σw represent the turbulence intensities.Measurement noises are considered by introducing Gaussian noises. See Table2 for noise configuration in each channel of flight states measured.Table 2
Measurement noise configuration.
ChannelStandard deviationUnitV0.5m/sα,β0.0035radax,ay,az0.02 gm/s2ϕ,θ,ψ0.002rad/sp,q,r0.0087radSee Figure1 for simulated flight trajectories of the 4 configurations.Figure 1
Flight trajectories of the 4 configurations in Table1.
(a)
Y, low altitude, LTO, manual(b)
D, high altitude, cruise, AP(c)
B, high altitude, cruise, AP(d)
B, low altitude, free flight, manual


### 3.2. Flight Data with IMU Fault Cases Injected

Typical faults of IMU include drift, noise, and scale factor. As scale factor is produced during manufacturing, which can be calibrated beforehand, we focus on noise and drift, i.e., angle random walk (ARW) and rate random walk (RRW), which behave randomly and are more difficult to detect in flight.We introduced 4 cases with IMU faults by adding randomized faulty values to basic flight data described in Section3.1. To be specific, 6-DoF IMU composed of triaxial accelerometer and three-axis gyroscope is studied. Although there are 3 channels in a triaxial accelerometer and three-axis gyroscope, respectively, they function as 2 measurement units. Thus, we concern about whether there are faults in the accelerometer or gyroscope, rather than in a specific acceleration or angular speed channel, to provide information for further determination if a redundant measurement unit is to take over. Fault cases are listed below. One and only one type of fault occurs simultaneously or individually in acceleration or angular speed channels.Case 1.
Noise to angular speed (i.e.,p,q,r).Case 2.
Drift to angular speed (i.e.p,q,r).Case 3.
Noise to acceleration (i.e.ax,ay,az).Case 4.
Drift to acceleration (i.e.ax,ay,az).We have 5 different flight cases (including cases without IMU faults, Case 0) to be classified for each flight configuration. Fault cases happen at a random time and last for a random period (but less than 60 s) in every 60 s during flight.Figure2 illustrates how fault cases are injected, where red dashed lines represent basic flight data and black solid lines with existence of IMU fault. We repeat this randomized fault-injecting procedure multiple times on each flight configuration depicted in Figure 1.Figure 2
Example of fault cases.


### 3.3. Data Structure for Fault Detection Scheme

In real flights, IMU faults last for a certain length of time period with some “pattern,” as the way we insert them in Section3.2. Therefore, it is more reasonable to observe flight data fragments rather than flight data points. A time window of length ΔTslides on each flight data fragment. Data sets composed of flight data matrices are formed. Each column of a data matrix is a vector of measured flight state at certain time t. To be specific, we downsample data fragments at sample rate of 1 Hz and use a time window of ΔT=30s to form data matrices, yielding inputs with a dimension of 31 columns and 12 rows (representing all 12 variables measured by sensors: 3 ADS states V,α,β, 3 Euler angles ψ,θ,ϕ, 3 angular speeds p,q,r, and 3 accelerations ax,ay,az). Figure 3 illustrates how a flight data matrix is formed in angular speed and acceleration channel, wherein the left plot is part of the flight data sequences with fault inserted, and the right plot is the flight data matrix extracted by a time window of length ΔT at time t from the sequence in the left plot.Figure 3
Formation of flight data matrix.Data sets are divided into training and testing sets. To be specific, 2/3 data of aircraft Y and D is extracted for training. 2/3 of Y and D and all data of aircraft B are for testing. Fault cases are randomly added. See Table3 for detailed distribution of the data sets.Table 3
Data sets for DNN training and testing.
ConfigurationNo. of data setsCase distributionTrainingY LTO manual916026.9%, 18.7%, 19.9%, 17.7%, 16.7%D cruise AP1942031.4%, 17.2%, 16.3%, 17.7%, 17.4%TestingY LTO manual458026.0%, 18.7%, 19.9%, 17.8%, 16.7%D cruise AP971033.5%, 20.5%, 10.5%, 19.4%, 16.1%B cruise AP1942029.7%, 16.4%, 19.4%, 16.9%, 17.5%B free flight manual532831.5%, 16.9%, 20.1%, 19.4%, 12.1%


## 3.1. Basic Flight Data with Disturbances and Measurement Noises

To guarantee robustness and performance of the DNN-based fault detection scheme, it is vital to prepare a diverse data set. Thus, we generate a data set of 3 different aircrafts (i.e., Y, D, and B). To be specific, Y is a large cargo aircraft, D is a general aviation, and B is a passenger aircraft. We simulated 4 configurations of different flight conditions, including the following:(1)
Landing and take-off (LTO) with manual control (aircraft Y)(2)
High-altitude cruise with autopilot (AP) (aircraft D)(3)
High-altitude cruise with autopilot (AP) (aircraft B)(4)
Low-altitude free flight with manual control (aircraft B)We intended to fully cover the whole flight envelope (e.g., LTO and cruise) by introducing the 4 configurations mentioned above. More detailed information of the configurations is concluded in Table1.Table 1
Aircrafts and flight conditions.
AircraftTypeEnginesWeighSpanFlight conditionUsageYLarge cargoQuadruple piston41.0 t38.0 mLow altitude, LTO, manualTrainDGeneral aviationDouble piston3.12 t19.8 mHigh altitude, cruise, APTrainBLarge passengerQuadruple turbo174 t59.6 mHigh altitude, cruise, APTestBLarge passengerQuadruple turbo174 t59.6 mLow altitude, free flight, manualTestHigh-altitude cruise of aircraft Y and low-altitude LTO of aircraft D are simulated for network training; high-altitude cruise and low-altitude free flight of aircraft B are simulated for network testing.To simulate the atmospheric turbulences, Dryden model is adopted, which injects perturbances to “clean” flight states. A speed model of Dryden wind can be defined by spectrum functions:(6)ϕuΩ=σu22Luπ11+LuΩ2,ϕvΩ=σv2Lvπ1+3LvΩ21+LvΩ22,ϕwΩ=σw2Lwπ1+3LwΩ21+LwΩ22,wherein Lu,Lv,Lw represent the turbulence scale lengths. σu,σv,σw represent the turbulence intensities.Measurement noises are considered by introducing Gaussian noises. See Table2 for noise configuration in each channel of flight states measured.Table 2
Measurement noise configuration.
ChannelStandard deviationUnitV0.5m/sα,β0.0035radax,ay,az0.02 gm/s2ϕ,θ,ψ0.002rad/sp,q,r0.0087radSee Figure1 for simulated flight trajectories of the 4 configurations.Figure 1
Flight trajectories of the 4 configurations in Table1.
(a)
Y, low altitude, LTO, manual(b)
D, high altitude, cruise, AP(c)
B, high altitude, cruise, AP(d)
B, low altitude, free flight, manual


## 3.2. Flight Data with IMU Fault Cases Injected

Typical faults of IMU include drift, noise, and scale factor. As scale factor is produced during manufacturing, which can be calibrated beforehand, we focus on noise and drift, i.e., angle random walk (ARW) and rate random walk (RRW), which behave randomly and are more difficult to detect in flight.We introduced 4 cases with IMU faults by adding randomized faulty values to basic flight data described in Section3.1. To be specific, 6-DoF IMU composed of triaxial accelerometer and three-axis gyroscope is studied. Although there are 3 channels in a triaxial accelerometer and three-axis gyroscope, respectively, they function as 2 measurement units. Thus, we concern about whether there are faults in the accelerometer or gyroscope, rather than in a specific acceleration or angular speed channel, to provide information for further determination if a redundant measurement unit is to take over. Fault cases are listed below. One and only one type of fault occurs simultaneously or individually in acceleration or angular speed channels.Case 1.
Noise to angular speed (i.e.,p,q,r).Case 2.
Drift to angular speed (i.e.p,q,r).Case 3.
Noise to acceleration (i.e.ax,ay,az).Case 4.
Drift to acceleration (i.e.ax,ay,az).We have 5 different flight cases (including cases without IMU faults, Case 0) to be classified for each flight configuration. Fault cases happen at a random time and last for a random period (but less than 60 s) in every 60 s during flight.Figure2 illustrates how fault cases are injected, where red dashed lines represent basic flight data and black solid lines with existence of IMU fault. We repeat this randomized fault-injecting procedure multiple times on each flight configuration depicted in Figure 1.Figure 2
Example of fault cases.


## 3.3. Data Structure for Fault Detection Scheme

In real flights, IMU faults last for a certain length of time period with some “pattern,” as the way we insert them in Section3.2. Therefore, it is more reasonable to observe flight data fragments rather than flight data points. A time window of length ΔTslides on each flight data fragment. Data sets composed of flight data matrices are formed. Each column of a data matrix is a vector of measured flight state at certain time t. To be specific, we downsample data fragments at sample rate of 1 Hz and use a time window of ΔT=30s to form data matrices, yielding inputs with a dimension of 31 columns and 12 rows (representing all 12 variables measured by sensors: 3 ADS states V,α,β, 3 Euler angles ψ,θ,ϕ, 3 angular speeds p,q,r, and 3 accelerations ax,ay,az). Figure 3 illustrates how a flight data matrix is formed in angular speed and acceleration channel, wherein the left plot is part of the flight data sequences with fault inserted, and the right plot is the flight data matrix extracted by a time window of length ΔT at time t from the sequence in the left plot.Figure 3
Formation of flight data matrix.Data sets are divided into training and testing sets. To be specific, 2/3 data of aircraft Y and D is extracted for training. 2/3 of Y and D and all data of aircraft B are for testing. Fault cases are randomly added. See Table3 for detailed distribution of the data sets.Table 3
Data sets for DNN training and testing.
ConfigurationNo. of data setsCase distributionTrainingY LTO manual916026.9%, 18.7%, 19.9%, 17.7%, 16.7%D cruise AP1942031.4%, 17.2%, 16.3%, 17.7%, 17.4%TestingY LTO manual458026.0%, 18.7%, 19.9%, 17.8%, 16.7%D cruise AP971033.5%, 20.5%, 10.5%, 19.4%, 16.1%B cruise AP1942029.7%, 16.4%, 19.4%, 16.9%, 17.5%B free flight manual532831.5%, 16.9%, 20.1%, 19.4%, 12.1%


## 4. IMU Fault Detection Network (FDN)

### 4.1. Basic Architecture of IMU FDN

The IMU fault detection problem discussed in this paper can be regarded as a classification issue of sequential data, which is composed of 2D flight-data matrices. A deep neural network with both CNN and LSTM modules is designed to deal with this kind of issue.With CNN, we intend to extract semantic information of 2D data matrices. Feature maps are generated each time an input passes through a convolutional layer. With the increase of network layers, deeper semantic information is expected to be extracted. We use leaky ReLU as activation function which can reduce the appearance of silent neurons.With LSTM, we aim to extract temporal information. Gate operations are used in the LSTM module to improve its performance.


### 4.2. Experiment Environment and Training Method

The computational platform in this work is of one i7-2600 CPU and 16 GB RAM, with one Nvidia GTX 1070 GPU (driver version 456.71). The training and testing backends are based on Keras v2.4.3 and TensorFlow v2.3.0, running on Python 3.8 (Windows 10).For network training, “crossentropy” is used to generate training loss and “Adam” is adopted as optimizer. An exponential-decaying learning rate (lr) was adopted: lr=0.001×e−epoch/200, wherein one epoch means all training data is used once.


### 4.3. Evaluation of Fault Detection Performance

To evaluate the classification performance of the fault detection network, a confusion matrix is computed each time the network is tested with test sets. Each one of diagonal elements of a confusion matrix is the percentage of cases correctly predicted; i.e., the closer the diagonal elements are to 100, the better the network performs.An example of confusion matrix is presented in Table4. As described in Section 3.2, we took into concern both drift and noise in angular speed and acceleration channels, forming 4 fault cases and 1 clean case with no fault. Test sets (for example: aircraft Y with flight condition “low altitude, LTO, manual,” see Table 1) are sent into a fault detection network with a certain structure. Every one element of the confusion matrix is a statistical result of predictions; e.g., the 2nd row of the matrix indicates that 6% of the “angular speed drift” cases were predicted “clean,” 93 correctly predicted “angular speed drift,” 0% “acceleration drift,” 1% “angular speed noise,” and 0% “acceleration noise.” In this case, prediction precision of case “angular speed drift” is 93% for aircraft Y with flight condition “low altitude, LTO, manual.” A diagonal vector 9593959494 of this confusion matrix is extracted to evaluate prediction performance of the network.Table 4
Example of confusion matrix.
Predicted as:CleanAngular speed driftAcceleration driftAngular speed noiseAcceleration noiseActual caseClean950112Angular speed drift693010Acceleration drift409501Angular speed noise420940Acceleration noise402094With the diagonal vectors of confusion matrices discussed above, we obtain a tool to evaluate prediction performance of a certain architecture of FDN. For each network architecture to be discussed in the next section and aircrafts/flight conditions in Table1, we send all test sets into the FDN to come to a diagonal vector of confusion matrix, generating Table 5 for comparison.Table 5
Prediction performance of different FDN architectures.
ArchitectureFlight configuration(see Table 1)Predicted asCleanAngular speed driftAcceleration driftAngular speed noiseAcceleration noiseFDN-FULLCNN+LSTM in ADS, angle, angular speed, accelerationModel sizeY LTO manual9298879691122.0 MBD cruise AP9699919491B cruise AP9597769596B free-flight manual9597899595FDN-FULL/ADSCNN+LSTM in angle, angular speed, accelerationModel sizeY LTO manual939592969679.7 MBD cruise AP9699919596B cruise AP9597839595B free-flight manual9699869495FDN-FULL/ANGLECNN+LSTM in ADS, angular speed, accelerationModel sizeY LTO manual949482959698.8 MBD cruise AP9599959595B cruise AP9798819697B free-flight manual9496849595FDN-CNNCNN in ADS angle, angular speed, accelerationModel sizeY LTO manual959878979684.0 MBD cruise AP9599799596B cruise AP9594717675B free-flight manual9696759495FDN-RNNLSTM in ADS angle, angular speed, accelerationModel sizeY LTO manual678065789246.7 MBD cruise AP7184707683B cruise AP7679606488B free-flight manual8365664374FDN-AB1CNN in ADS, angle, acceleration;CNN+LSTM in angular speedModel sizeY LTO manual939862979493.7 MBD cruise AP9698919592B cruise AP9395729589B free-flight manual9697869590FDN-AB2CNN in ADS, angle;CNN+LSTM in angular speed and accelerationModel sizeY LTO manual979688929660.4 MBD cruise AP9599899294B cruise AP9797809498B free-flight manual9697849395FDN-AB3CNN in angle;CNN+LSTM in angular speed and accelerationModel sizeY LTO manual9398799996103.0 MBD cruise AP9697839795B cruise AP9596689793B free-flight manual9698789695FDN-AB4CNN in ADS, angle, angular speed;CNN+LSTM in accelerationModel sizeY LTO manual969680969593.7 MBD cruise AP9699899592B cruise AP9898769692B free-flight manual9698749392FDN-OPTCNN+LSTM in angular speed, accelerationModel sizeY LTO manual939888979455.8 MBD cruise AP9599949695B cruise AP9398849793B free-flight manual9698929595∗In this table, ADS: V,α,β; angle: ϕ,θ,ψ; angular speed: p,q,r; acceleration: ax,ay,az.


### 4.4. Architecture Studies of the IMU FDN

As the inputs we send into the network are not images but data sequences of aircraft dynamics, general rules for architecture optimizing in image recognition networks may not work. We adopt ablation studies beginning with a full architecture to find a “best” network for the fault detection problem. The structure of the full network is depicted in Figure4.Figure 4
Full architecture of the IMU fault detection network. (a) DNN-FULL architecture. ADS: airspeed, AOA, and sideslip angle measured by air data sensors; ANGLES: Euler angles; AS: angular speeds; ACC: accelerations; Conv2D: convolution layer; Dense: fully connected layer.2×2 kernels are also used in Maxpool layers. Flatten layers reshape 2D data to 1D for Dense layers. Concatenate layer merges all the data from previous layers. (b) Detailed configurations of CNN and LSTM modules in acceleration channel, kernel size, and bias are depicted. Configurations of other channels are the same as those of the acceleration channel.In the full network, namely, FDN-FULL, we sendV,α,β (air data sensors, “ADS”),ψ,θ,ϕ (Euler angles, “ANGLES”), p,q,r (angular speeds, “AS”), and ax,ay,az (accelerations, “ACC”) into the input layer, which means all the measured variables are used for fault detection. And for all channels, both CNN and LSTM layers are adopted.


#### 4.4.1. FDN-FULL/ADS

A general goal of architecture study is to maximize predict precision while minimizing net size. The FDN is essentially a nonlinear function that maps input (flight states) to output (fault cases). Notice that equation (1) indicates that accelerations (in which we are interested) are mapped to ADS data explicitly. Sending both ADS and ACC into FDN may lead to data redundancy and overfitting. So, we cut the ADS channel out so that the size of the network blocks before the Concatenate layer was reduced to 3/4 that of FDN-FULL. Prediction performance is slightly improved as expected, see Table 5.


#### 4.4.2. FDN-FULL/ANGLES

Equation (3) indicates that Euler angle may be redundant due to the explicit mapping relationship between it and angular speed. So, we cut ANGLE channel out just like what we did to ADS in Section 4.4.1. Prediction performance is also slightly improved while the size is reduced, as shown in Table 5.


#### 4.4.3. FDN-CNN and FDN-RNN

We tried to cut either CNN or LSTM branches out to examine the performance of a pure CNN or RNN network. FDN-CNN performs well in some cases, but results are not satisfying when dealing with aircraft B, testing data of which is strictly separated from training set (only Y and D are used for training). That means FDN-CNN is not robust enough. FDN-RNN performs even worse, totally unable to predict faults in acceleration channel. This study led to a conclusion that both CNN and RNN are needed to construct an effective FDN for IMU fault detection.


#### 4.4.4. FDN-OPT and Other Ablation Studies

Several other ablation studies were carried out based on FDN-FULL, trying different combinations of input data and CNN/RNN architecture, as listed in Table5 (FDN-AB1~FDN-AB 4, FDN-OPT).Among all the ablated FDN, FDN-OPT (with both CNN and LSTM in AS and ACC channel, ADS and ANGLES cut out) performs best and shows equivalent ability in fault detection to FDN-FULL/ADS and FDN-FULL/ANGLE and is obviously smaller than them.


### 4.5. Net Hyperparameters

A major issue of constructing the FDN is to determine hyperparameters: number of CNN filters, number of LSTM nodes, and size of CNN kernel. We tried 16 different combinations of these hyperparameters based on FDN-OPT and trained 30 times with 2000 epochs for each combination to eliminate randomness in the training process. Hyperparameters are listed below:Hyperparameter 1. Number of CNN filters: {8,16,32,48,64,80,96}.Hyperparameter 2. Size of CNN kernel: {1×1, 2×2, 3×3, 1×1+2×2}.Hyperparameter 3. Number of LSTM nodes: {32,48,64,80,96,112,128}.Number of CNN filters has the greatest impact on computational cost and net size. Final (at epoch 2000) validation accuracy and training loss are depicted in Figure5. Validation accuracy increases and loss decreases with the number of filters, which means better performance. We chose 64 filters for a balance of performance and net size. Then, studies on hyperparameters 2 and 3 are carried out on a fixed number of 64 CNN filters.Figure 5
Val. accuracy and training loss of different Nos. of CNN filters: (a) final (epoch=2000) accuracy on validation set; (b) loss at the end of training (epoch=2000). Each data point is an average value of 30 trainings.
(a)(b)The training history of different sizes of CNN kernels is depicted in Figure6. As a common view in CNN, the size of receptive field and computational cost are proportional to the size of CNN kernels. Figure 6 shows that FDN-OPT with 2×2 or 3×3 performs better on validation sets than 1×1 or 1×1+2×2. A model size of FDN-OPT with 2×2 CNN kernels is 55.8 MB and 3×3 59.1 MB. So 2×2 kernel is determined for balancing model size and performance.Figure 6
Training history of different sizes of CNN kernel; (a) accuracy on the validation set; (b) loss curve during training. Each curve is an average of 30 trainings.
(a)(b)The training history of different numbers of LSTM nodes is depicted in Figure7. The FDN performs worst in training when the number of LSTM nodes is equal to the number of CNN kernels (64) and best when the number of LSTM nodes is half (32) or twice (128) the number of CNN kernels. As 128 nodes are a commonly used configuration in a majority of LSTM practices, it is finally adopted in our work. This result implicates that the number of LSTM nodes should not be equal to the number of CNN kernels when composing a DNN with both CNN and LSTM.Figure 7
Val. accuracy and training loss of different Nos. of CNN filters: (a) final (epoch=2000) accuracy on the validation set; (b) loss at the end of training (epoch=2000). Each data point is an average value of 30 trainings.
(a)(b)


### 4.6. Brief Summary of FDN for IMU Fault Detection

As studied in Sections4.4.1 and 4.4.2, discarding ADS and ANGLE channel does not make the performance of FDN worse (in some cases even better). Meanwhile, Section 4.4.3 shows that both CNN and RNN are necessary. Predicting result of FDN-OPT proved these two conclusions: for IMU fault detection problem, we need only AS and ACC information, and a CNN-LSTM fusion deep network provides satisfying performance.Starting from FDN-FULL, we obtained an equivalent-in-performance but smaller-in-size deep network for IMU fault detection problem. We used both manual and AP flight data of 2 aircrafts (Y & D) to train the network and another aircraft (B) to test it. Results show that accuracy of prediction is over 84% for all aircrafts and flight configurations, which means the fault detection scheme is robust and potentially effective for other aircrafts/flight configurations.To eliminate stochastic effects during NN training, we trained each architecture 20 times till loss converged (3000 epochs for FDN-RNN, 1000 epochs for other architectures). Figure8 depicts the training history of different architectures discussed in the previous sections. In Figure 5, FDN-OPT (black lines) claims the lowest training loss on average, as shown in (b). FDN-FULL/ADS (blue lines) and FDN-FULL/ANGLE (magenta lines) perform alike, as well as FDN-FULL. Loss curves of those 4 architectures are similar in Figure 8, which strengthen the argument that prediction performances are almost of the same order as shown in Table 5. Average loss of FDN-CNN is greater than those 4 architectures but obviously much better than FDN-RNN. The reason performance of FDN-CNN being close to FDN-FULL might be that input data was arranged in a form temporal information had been included (see Section 3.3). Loss of FDN-RNN decreases much slower than any other architectures as is expected, as its prediction performance is worst in Table 5.Figure 8
FDN training history: (a) training history of different architectures (20 times of training for each architecture); (b) average loss of each architecture in 20 times of training.
(a)(b)Both Figure8 and Table 5 show that FDN-OPT is the most size-performance balanced one among all studied in this work; we adopt this architecture for IMU for detection task. Architecture of the optimized FDN-OPT is depicted in Figure 9.Figure 9
Architecture of FDN-OPT.


## 4.1. Basic Architecture of IMU FDN

The IMU fault detection problem discussed in this paper can be regarded as a classification issue of sequential data, which is composed of 2D flight-data matrices. A deep neural network with both CNN and LSTM modules is designed to deal with this kind of issue.With CNN, we intend to extract semantic information of 2D data matrices. Feature maps are generated each time an input passes through a convolutional layer. With the increase of network layers, deeper semantic information is expected to be extracted. We use leaky ReLU as activation function which can reduce the appearance of silent neurons.With LSTM, we aim to extract temporal information. Gate operations are used in the LSTM module to improve its performance.


## 4.2. Experiment Environment and Training Method

The computational platform in this work is of one i7-2600 CPU and 16 GB RAM, with one Nvidia GTX 1070 GPU (driver version 456.71). The training and testing backends are based on Keras v2.4.3 and TensorFlow v2.3.0, running on Python 3.8 (Windows 10).For network training, “crossentropy” is used to generate training loss and “Adam” is adopted as optimizer. An exponential-decaying learning rate (lr) was adopted: lr=0.001×e−epoch/200, wherein one epoch means all training data is used once.


## 4.3. Evaluation of Fault Detection Performance

To evaluate the classification performance of the fault detection network, a confusion matrix is computed each time the network is tested with test sets. Each one of diagonal elements of a confusion matrix is the percentage of cases correctly predicted; i.e., the closer the diagonal elements are to 100, the better the network performs.An example of confusion matrix is presented in Table4. As described in Section 3.2, we took into concern both drift and noise in angular speed and acceleration channels, forming 4 fault cases and 1 clean case with no fault. Test sets (for example: aircraft Y with flight condition “low altitude, LTO, manual,” see Table 1) are sent into a fault detection network with a certain structure. Every one element of the confusion matrix is a statistical result of predictions; e.g., the 2nd row of the matrix indicates that 6% of the “angular speed drift” cases were predicted “clean,” 93 correctly predicted “angular speed drift,” 0% “acceleration drift,” 1% “angular speed noise,” and 0% “acceleration noise.” In this case, prediction precision of case “angular speed drift” is 93% for aircraft Y with flight condition “low altitude, LTO, manual.” A diagonal vector 9593959494 of this confusion matrix is extracted to evaluate prediction performance of the network.Table 4
Example of confusion matrix.
Predicted as:CleanAngular speed driftAcceleration driftAngular speed noiseAcceleration noiseActual caseClean950112Angular speed drift693010Acceleration drift409501Angular speed noise420940Acceleration noise402094With the diagonal vectors of confusion matrices discussed above, we obtain a tool to evaluate prediction performance of a certain architecture of FDN. For each network architecture to be discussed in the next section and aircrafts/flight conditions in Table1, we send all test sets into the FDN to come to a diagonal vector of confusion matrix, generating Table 5 for comparison.Table 5
Prediction performance of different FDN architectures.
ArchitectureFlight configuration(see Table 1)Predicted asCleanAngular speed driftAcceleration driftAngular speed noiseAcceleration noiseFDN-FULLCNN+LSTM in ADS, angle, angular speed, accelerationModel sizeY LTO manual9298879691122.0 MBD cruise AP9699919491B cruise AP9597769596B free-flight manual9597899595FDN-FULL/ADSCNN+LSTM in angle, angular speed, accelerationModel sizeY LTO manual939592969679.7 MBD cruise AP9699919596B cruise AP9597839595B free-flight manual9699869495FDN-FULL/ANGLECNN+LSTM in ADS, angular speed, accelerationModel sizeY LTO manual949482959698.8 MBD cruise AP9599959595B cruise AP9798819697B free-flight manual9496849595FDN-CNNCNN in ADS angle, angular speed, accelerationModel sizeY LTO manual959878979684.0 MBD cruise AP9599799596B cruise AP9594717675B free-flight manual9696759495FDN-RNNLSTM in ADS angle, angular speed, accelerationModel sizeY LTO manual678065789246.7 MBD cruise AP7184707683B cruise AP7679606488B free-flight manual8365664374FDN-AB1CNN in ADS, angle, acceleration;CNN+LSTM in angular speedModel sizeY LTO manual939862979493.7 MBD cruise AP9698919592B cruise AP9395729589B free-flight manual9697869590FDN-AB2CNN in ADS, angle;CNN+LSTM in angular speed and accelerationModel sizeY LTO manual979688929660.4 MBD cruise AP9599899294B cruise AP9797809498B free-flight manual9697849395FDN-AB3CNN in angle;CNN+LSTM in angular speed and accelerationModel sizeY LTO manual9398799996103.0 MBD cruise AP9697839795B cruise AP9596689793B free-flight manual9698789695FDN-AB4CNN in ADS, angle, angular speed;CNN+LSTM in accelerationModel sizeY LTO manual969680969593.7 MBD cruise AP9699899592B cruise AP9898769692B free-flight manual9698749392FDN-OPTCNN+LSTM in angular speed, accelerationModel sizeY LTO manual939888979455.8 MBD cruise AP9599949695B cruise AP9398849793B free-flight manual9698929595∗In this table, ADS: V,α,β; angle: ϕ,θ,ψ; angular speed: p,q,r; acceleration: ax,ay,az.


## 4.4. Architecture Studies of the IMU FDN

As the inputs we send into the network are not images but data sequences of aircraft dynamics, general rules for architecture optimizing in image recognition networks may not work. We adopt ablation studies beginning with a full architecture to find a “best” network for the fault detection problem. The structure of the full network is depicted in Figure4.Figure 4
Full architecture of the IMU fault detection network. (a) DNN-FULL architecture. ADS: airspeed, AOA, and sideslip angle measured by air data sensors; ANGLES: Euler angles; AS: angular speeds; ACC: accelerations; Conv2D: convolution layer; Dense: fully connected layer.2×2 kernels are also used in Maxpool layers. Flatten layers reshape 2D data to 1D for Dense layers. Concatenate layer merges all the data from previous layers. (b) Detailed configurations of CNN and LSTM modules in acceleration channel, kernel size, and bias are depicted. Configurations of other channels are the same as those of the acceleration channel.In the full network, namely, FDN-FULL, we sendV,α,β (air data sensors, “ADS”),ψ,θ,ϕ (Euler angles, “ANGLES”), p,q,r (angular speeds, “AS”), and ax,ay,az (accelerations, “ACC”) into the input layer, which means all the measured variables are used for fault detection. And for all channels, both CNN and LSTM layers are adopted.


### 4.4.1. FDN-FULL/ADS

A general goal of architecture study is to maximize predict precision while minimizing net size. The FDN is essentially a nonlinear function that maps input (flight states) to output (fault cases). Notice that equation (1) indicates that accelerations (in which we are interested) are mapped to ADS data explicitly. Sending both ADS and ACC into FDN may lead to data redundancy and overfitting. So, we cut the ADS channel out so that the size of the network blocks before the Concatenate layer was reduced to 3/4 that of FDN-FULL. Prediction performance is slightly improved as expected, see Table 5.


### 4.4.2. FDN-FULL/ANGLES

Equation (3) indicates that Euler angle may be redundant due to the explicit mapping relationship between it and angular speed. So, we cut ANGLE channel out just like what we did to ADS in Section 4.4.1. Prediction performance is also slightly improved while the size is reduced, as shown in Table 5.


### 4.4.3. FDN-CNN and FDN-RNN

We tried to cut either CNN or LSTM branches out to examine the performance of a pure CNN or RNN network. FDN-CNN performs well in some cases, but results are not satisfying when dealing with aircraft B, testing data of which is strictly separated from training set (only Y and D are used for training). That means FDN-CNN is not robust enough. FDN-RNN performs even worse, totally unable to predict faults in acceleration channel. This study led to a conclusion that both CNN and RNN are needed to construct an effective FDN for IMU fault detection.


### 4.4.4. FDN-OPT and Other Ablation Studies

Several other ablation studies were carried out based on FDN-FULL, trying different combinations of input data and CNN/RNN architecture, as listed in Table5 (FDN-AB1~FDN-AB 4, FDN-OPT).Among all the ablated FDN, FDN-OPT (with both CNN and LSTM in AS and ACC channel, ADS and ANGLES cut out) performs best and shows equivalent ability in fault detection to FDN-FULL/ADS and FDN-FULL/ANGLE and is obviously smaller than them.


## 4.4.1. FDN-FULL/ADS

A general goal of architecture study is to maximize predict precision while minimizing net size. The FDN is essentially a nonlinear function that maps input (flight states) to output (fault cases). Notice that equation (1) indicates that accelerations (in which we are interested) are mapped to ADS data explicitly. Sending both ADS and ACC into FDN may lead to data redundancy and overfitting. So, we cut the ADS channel out so that the size of the network blocks before the Concatenate layer was reduced to 3/4 that of FDN-FULL. Prediction performance is slightly improved as expected, see Table 5.


## 4.4.2. FDN-FULL/ANGLES

Equation (3) indicates that Euler angle may be redundant due to the explicit mapping relationship between it and angular speed. So, we cut ANGLE channel out just like what we did to ADS in Section 4.4.1. Prediction performance is also slightly improved while the size is reduced, as shown in Table 5.


## 4.4.3. FDN-CNN and FDN-RNN

We tried to cut either CNN or LSTM branches out to examine the performance of a pure CNN or RNN network. FDN-CNN performs well in some cases, but results are not satisfying when dealing with aircraft B, testing data of which is strictly separated from training set (only Y and D are used for training). That means FDN-CNN is not robust enough. FDN-RNN performs even worse, totally unable to predict faults in acceleration channel. This study led to a conclusion that both CNN and RNN are needed to construct an effective FDN for IMU fault detection.


## 4.4.4. FDN-OPT and Other Ablation Studies

Several other ablation studies were carried out based on FDN-FULL, trying different combinations of input data and CNN/RNN architecture, as listed in Table5 (FDN-AB1~FDN-AB 4, FDN-OPT).Among all the ablated FDN, FDN-OPT (with both CNN and LSTM in AS and ACC channel, ADS and ANGLES cut out) performs best and shows equivalent ability in fault detection to FDN-FULL/ADS and FDN-FULL/ANGLE and is obviously smaller than them.


## 4.5. Net Hyperparameters

A major issue of constructing the FDN is to determine hyperparameters: number of CNN filters, number of LSTM nodes, and size of CNN kernel. We tried 16 different combinations of these hyperparameters based on FDN-OPT and trained 30 times with 2000 epochs for each combination to eliminate randomness in the training process. Hyperparameters are listed below:Hyperparameter 1. Number of CNN filters: {8,16,32,48,64,80,96}.Hyperparameter 2. Size of CNN kernel: {1×1, 2×2, 3×3, 1×1+2×2}.Hyperparameter 3. Number of LSTM nodes: {32,48,64,80,96,112,128}.Number of CNN filters has the greatest impact on computational cost and net size. Final (at epoch 2000) validation accuracy and training loss are depicted in Figure5. Validation accuracy increases and loss decreases with the number of filters, which means better performance. We chose 64 filters for a balance of performance and net size. Then, studies on hyperparameters 2 and 3 are carried out on a fixed number of 64 CNN filters.Figure 5
Val. accuracy and training loss of different Nos. of CNN filters: (a) final (epoch=2000) accuracy on validation set; (b) loss at the end of training (epoch=2000). Each data point is an average value of 30 trainings.
(a)(b)The training history of different sizes of CNN kernels is depicted in Figure6. As a common view in CNN, the size of receptive field and computational cost are proportional to the size of CNN kernels. Figure 6 shows that FDN-OPT with 2×2 or 3×3 performs better on validation sets than 1×1 or 1×1+2×2. A model size of FDN-OPT with 2×2 CNN kernels is 55.8 MB and 3×3 59.1 MB. So 2×2 kernel is determined for balancing model size and performance.Figure 6
Training history of different sizes of CNN kernel; (a) accuracy on the validation set; (b) loss curve during training. Each curve is an average of 30 trainings.
(a)(b)The training history of different numbers of LSTM nodes is depicted in Figure7. The FDN performs worst in training when the number of LSTM nodes is equal to the number of CNN kernels (64) and best when the number of LSTM nodes is half (32) or twice (128) the number of CNN kernels. As 128 nodes are a commonly used configuration in a majority of LSTM practices, it is finally adopted in our work. This result implicates that the number of LSTM nodes should not be equal to the number of CNN kernels when composing a DNN with both CNN and LSTM.Figure 7
Val. accuracy and training loss of different Nos. of CNN filters: (a) final (epoch=2000) accuracy on the validation set; (b) loss at the end of training (epoch=2000). Each data point is an average value of 30 trainings.
(a)(b)


## 4.6. Brief Summary of FDN for IMU Fault Detection

As studied in Sections4.4.1 and 4.4.2, discarding ADS and ANGLE channel does not make the performance of FDN worse (in some cases even better). Meanwhile, Section 4.4.3 shows that both CNN and RNN are necessary. Predicting result of FDN-OPT proved these two conclusions: for IMU fault detection problem, we need only AS and ACC information, and a CNN-LSTM fusion deep network provides satisfying performance.Starting from FDN-FULL, we obtained an equivalent-in-performance but smaller-in-size deep network for IMU fault detection problem. We used both manual and AP flight data of 2 aircrafts (Y & D) to train the network and another aircraft (B) to test it. Results show that accuracy of prediction is over 84% for all aircrafts and flight configurations, which means the fault detection scheme is robust and potentially effective for other aircrafts/flight configurations.To eliminate stochastic effects during NN training, we trained each architecture 20 times till loss converged (3000 epochs for FDN-RNN, 1000 epochs for other architectures). Figure8 depicts the training history of different architectures discussed in the previous sections. In Figure 5, FDN-OPT (black lines) claims the lowest training loss on average, as shown in (b). FDN-FULL/ADS (blue lines) and FDN-FULL/ANGLE (magenta lines) perform alike, as well as FDN-FULL. Loss curves of those 4 architectures are similar in Figure 8, which strengthen the argument that prediction performances are almost of the same order as shown in Table 5. Average loss of FDN-CNN is greater than those 4 architectures but obviously much better than FDN-RNN. The reason performance of FDN-CNN being close to FDN-FULL might be that input data was arranged in a form temporal information had been included (see Section 3.3). Loss of FDN-RNN decreases much slower than any other architectures as is expected, as its prediction performance is worst in Table 5.Figure 8
FDN training history: (a) training history of different architectures (20 times of training for each architecture); (b) average loss of each architecture in 20 times of training.
(a)(b)Both Figure8 and Table 5 show that FDN-OPT is the most size-performance balanced one among all studied in this work; we adopt this architecture for IMU for detection task. Architecture of the optimized FDN-OPT is depicted in Figure 9.Figure 9
Architecture of FDN-OPT.


## 5. Conclusion

A CNN-LSTM-fusion fault detection network (FDN) is proposed for aircraft IMU fault detection. Flight data measured by Inertial Measurement Unit (IMU) including angular speed (AS) and acceleration (ACC) are used as inputs to the FDN and fault cases as output. We simulated different aircrafts with various flight conditions to guarantee data diversity, and testing sets were data extracted from flight data that was not adopted in the training process. Testing results were satisfying for 3 different aircrafts (trained and validated with large cargo Y and general aviation D, tested with large passenger B) simulated in different flight conditions (low-altitude LTO, high-altitude AP cruise, and low-altitude manual free flight), which means the FDN developed in this paper is robust to flight conditions and potentially scalable for different types of aircrafts.



---

*Source: 3936826-2021-09-30.xml*