# Air Data Sensor Fault Detection with an Augmented Floating Limiter

Authors: Fabio Balzano; Mario L. Fravolini; Marcello R. Napolitano; Stéphane d’Urso; Michele Crispoltoni; Giuseppe del Core
Journal: International Journal of Aerospace Engineering
 (2018)

---

## Abstract

Although very uncommon, the sequential failures of all aircraft Pitot tubes, with the consequent loss of signals for all the dynamic parameters from the Air Data System, have been found to be the cause of a number of catastrophic accidents in aviation history. This paper proposes a robust data-driven method to detect faulty measurements of aircraft airspeed, angle of attack, and angle of sideslip. This approach first consists in the appropriate selection of suitable sets of model regressors to be used as inputs of neural network-based estimators to be used online for failure detection. The setup of the proposed fault detection method is based on the statistical analysis of the residual signals in fault-free conditions, which, in turn, allows the tuning of a pair of floating limiter detectors that act as time-varying fault detection thresholds with the objective of reducing both the false alarm rate and the detection delay. The proposed approach has been validated using real flight data by injecting artificial ramp and hard failures on the above sensors. The results confirm the capabilities of the proposed scheme showing accurate detection with a desirable low level of false alarm when compared with an equivalent scheme with conventional “a priori set” fixed detection thresholds. The achieved performance improvement consists mainly in a substantial reduction of the detection time while keeping desirable low false alarm rates.


---

## Body

## 1. Introduction

The Air Data System (ADS) [1] is a critical component of the conventional suite of sensors for both manned and unmanned aircraft. The ADS provides direct measurements of the aircraft true airspeed; typically connected with the ADS are the vanes for measuring the important aerodynamic angles known as angle of attack and angle of sideslip. Since the Pitot probes and α-β vanes are installed on the fuselage and/or wings, they are exposed to external atmospheric agents; in general, these devices are robust to even extreme weather conditions. However, there are very specific weather conditions which can lead to peculiar ice crystals obstructing the tiny conducts of the ADS, thus inducing faulty sensor measurements. Incorrect measurements of airspeed could lead to unrecoverable flight conditions, such as in the cases of Air France Flight 447 [2] and the NASA X-31 experimental aircraft [3]. Additional causes of crashes included erroneous covering of the Pitot tubes (AeroPerù 757) [4] and formations of insect nests inside the static taps (Birgenair Flight 310) [5] as well as icing of the angle-of-attack vane (XL Airways Germany Flight 888T) [6]. At present, the conventional approach to provide fault tolerance for flight sensors is based on Hardware Redundancy (HR) [7] which basically consists in the installation of multiple sensors (initially 4, later reduced to 3) measuring the same parameter. These sensors are continuously monitored to check their status, and their values are continuously compared. The use of Built-In-Testing (BIT) voting schemes allows the detection and the isolation of a faulty sensor. An alternative approach is given by Analytical Redundancy (AnR) [8]. AnR basically involves the use of virtual sensors, such as conventional state estimators, neural networks, Kalman Filters, special cases of Kalman Filters (such as Extended Kalman Filters and Unscented Kalman Filters), and/or other predictive models, to provide alternative estimates of the parameter from the faulty sensor. Recently, an AnR scheme based on machine learning techniques has been proposed in [9] for the failure detection and correction of the airspeed sensor of an aircraft.Through data from other functional sensors (measuring parameter highly correlated to the parameter associated with the faulty sensor) used as inputs, the virtual sensor can estimate the target measurement which when compared with data provided by the sensor under scrutiny produces the residual signal. The residual signal is clearly very important for fault detection purposes; in fact, an alarm status can be declared when the residual signal exceeds some detection thresholds. It should be emphasized that the approach proposed in this paper is completely data-driven [10, 11] and does not require any specific aircraft-dependent dynamic mathematical model. In summary, the innovative aspects of this research effort are the complete independence from any model-based requirement along with the introduction and the validation of “ad hoc” designed adaptive thresholds (floating limiters) that are applied to the whitened and filtered residuals with the goal of improving the performance in terms of fault detectability, fault detection delay, and low false alarm rate when compared with conventional fixed-threshold schemes [12–15].Recently, the authors of this paper have also proposed in [16] another data-driven approach for the design of robust fault detection filters for aircraft air data sensors. In [16], the robustness of the scheme is achieved through the adoption of parametric interval prediction models that are derived by the solution of a convex optimization problem. Instead, in the present study, the robustness of the failure detection scheme was achieved through time-varying fault detection thresholds that are derived online by a simple floating limiter filter that is based on the windowed mean and variance of the whitened residual.The paper is organized as follows. Section2 describes the identification model for the neural networks used in this study. Section 3 briefly introduces the P92 Tecnam aircraft from which the flight data were recorded. Section 4 outlines the data-driven modelling of the true airspeed, angle-of-attack α, and angle-of-sideslip β sensors. Sections 5 and 6 are dedicated to the analysis of the residuals and to threshold computation, respectively. Finally, Section 7 provides the results along with an assessment of the performance of the proposed scheme in terms of false alarm rate and detection delay compared with a conventional fixed-threshold fault detection system.


## 2. Prediction Model: Feedforward Neural Network

This section describes the neural network-based model used for the estimation of airspeedTASk, αk, and βk as functions of other available measurements. Analytical Redundancy approaches are based on the use of online estimators that are supposed to provide an estimation of the sensor measurement. Several types of estimators can be found in the literature, such as Kalman Filters [17], Unscented Kalman Filters [18], Extended Kalman Filters [19], FIR filter estimator [20], and neural networks adaptive and nonadaptive [21, 22]. Specifically, in this effort, a nonautoregressive model [23–25] has been used. The model is essentially a nonlinear time series model, also known as nonlinear regression [26] modelled as follows:
(1)y~k=fNRXk+εk,where y~k is the yk (target) estimation, Xk is the final set of regressors (see Sections 4 and 5), and εk is a modelling error. Note that the NR model does not depend on previous values ofyk; this guarantees that the estimation is not affected by the occurrence of a fault on yk. Thus, the NR model is able to provide a reliable long-term (multistep ahead) estimation independently of the occurrence of the fault; on the contrary, an adaptive model (i.e., NARX model) estimation is influenced by the previous occurrence of a fault on k.


## 3. P92 Aircraft and Flight Data

The proposed scheme has been validated using flight data from the semiautonomous P92 Tecnam aircraft shown in Figure1.Figure 1
Semiautonomous P92 Tecnam.The P92 is a general aviation single-engine light aircraft with a classic high-wing configuration [27]. During the test flights, the aircraft was remotely controlled through a data link from a ground station. The on-board pilot was operative only during the take-off and landing phases. The nominal aircraft mass is 600 kg (1324 lb); the propulsion is provided by a Rotax 912 ULS, 74 kW (68 hp) with a two-blade fix pitch propeller, capable of providing a max cruise speed of 219 km/h (118 kts) and an operational ceiling of 4267 (14,000 ft). A total of 8 flights were recorded for approx. 3.5 hours of flight time. The available flight data were divided into two groups, that is, training data (used for the goal of training a pseudo-online learning neural network used as AnR-based virtual sensors) and validation data (used for the assessment of the performance of the estimation scheme). Since the recorded flight data were not completely homogenous, a different number of flights were used for the robust estimation of the TAS and of the aerodynamic angles α and β.For theTAS and β analysis, two flights were used for training purposes and five flights for validation purposes. Instead, for the α analysis, two flights were used for training purposes and six flights for validation purposes. Data statistics for all the flights are shown in Tables 1(a), 1(b), and 1(c).Table 1
(a)
TAS training and validation flight statistics
MaxV(k) (m/s)
MeanV(k) (m/s)
StdV(k) (m/s)
Flight duration (min)
TR-FLIGHT-1
37.1
31.3
2.6
34.8
TR-FLIGHT-2
35.5
30.5
1.1
26.8
VAL-FLIGHT-1
37.3
33.4
2.8
16.5
VAL-FLIGHT-2
37.1
32.1
2
17.3
VAL-FLIGHT-3
33.8
31.1
0.7
18
VAL-FLIGHT-4
34.4
31
1.02
28.3
VAL-FLIGHT-5
38
32
1.5
30.8
(b)
α training and validation flight statistics
Maxα(k) (deg)
Meanα(k) (deg)
Stdα(k) (deg)
Flight duration (min)
TR-FLIGHT-1
14.1
6.4
1.9
34.8
TR-FLIGHT-2
12.1
6.3
0.9
26.8
VAL-FLIGHT-1
11.4
4.9
1.7
16.5
VAL-FLIGHT-2
14
5.6
1.4
17.3
VAL-FLIGHT-3
9.3
6.4
0.5
18
VAL-FLIGHT-4
10.3
4.6
1.5
20.3
VAL-FLIGHT-5
9.1
6.2
0.7
28.3
VAL-FLIGHT-6
10.5
5.8
0.9
30.8
(c)
β training and validation flight statistics
Maxβ(k) (deg)
Meanβ(k) (deg)
Stdβ(k) (deg)
Flight duration (min)
TR-FLIGHT-1
2.9
−0.1
1.5
34.8
TR-FLIGHT-2
8.3
0.3
1.4
26.8
VAL-FLIGHT-1
1.6
−0.7
1.1
17.3
VAL-FLIGHT-2
9.8
0.6
1.1
18
VAL-FLIGHT-3
7.1
1.3
1.5
20.3
VAL-FLIGHT-4
6.2
1.5
0.7
19.9
VAL-FLIGHT-5
6.5
0.9
1.2
30.8For all flights (both training and validation), only the portion of the flight at “cruise” altitude was considered. In fact, the take-off and the initial climbing, as well as the descent and the landing, were not considered since they are associated with specific aerodynamic configurations (including, for example, the use of flaps and/or high angles-of-attack conditions). This decision was made because it was clear that not enough data were available for training the neural estimators at these conditions since only a few minutes for each flight are available. Therefore, only flight data with altitude higher thanhk>250m were considered for the purpose of this study. The flight parameters were recorded with a sampling time of TC=0.1s.


## 4. Data-Driven Modeling of the Aircraft Airspeed Velocity, Angle of Attack, and Angle of Sideslip

The true airspeed (TAS), the angle of attack (α), and the angle of sideslip (β) are all estimated as functions of a combination of measurements from all other sensors (excluding, of course, TAS, α, and β). Furthermore, the scheme does not require any information, data, or functional relationships such as the aircraft equations of motion. In other words, the approach can be applied without any loss of generality to any type of aircraft. Thus, the proposed approach can be classified as “data-driven” instead of “model-driven.” It should be clarified that in all cases, the GPS velocity was not considered. In fact, the GPS velocity and the TAS are strongly related through the relationship gpsk+Vwindk=Vk. Therefore, a wind gust could result as an additive fault ft of the same amplitude.Thus,TAS, α, and β are estimated as function of other measurements using the model structure described in Section 2. The initial vector of regressors Xk for TAS, α, and β has been set simply selecting the signals that characterize the aircraft kinematic and dynamic response as well as considering the correlation coefficient with the target signal. The resulting regressors are
(2)XTAS0k=φk,ϑk,ψk,αk,βk,Pk,Qk,Rk,Nxk,Nyk,Nzk,δEK,δAK,δRK,TTHk,hk,TAIRK,Xα0k=ϑk,hk,Nxk,Nyk,Nzk,Pk,Qk,Rk,TASk,βk,ψk,δEK,δAK,δRK,TTHk,CRk,Xβ0k=hk,Nxk,Nyk,Nzk,Pk,Qk,Rk,TASk,αk,φk,ϑk,ψk,δEK,δAK,δRK,Trk,Wk,CRk,where ϕ, ϑ, and ψ are the aircraft roll, pitch, and yaw angles, respectively; α and β are the angles of attack and sideslip, respectively; P, Q, and R are the aircraft angular rates; Nx, Ny, and Nz are the axial load factors; δE, δA, and δR are the elevator, aileron and rudder deflections, respectively; TTH is the throttle; h is the altitude; TAIR is the air temperature; TAS is the true aircraft airspeed; CR is the aircraft climbing rate; Tr is the track angle; and W is the wind direction. Note that in more general cases, additional control surface deflections could be introduced.To cope with possible nonlinearities and dynamic effects, each vectorXk of potential regressors was augmented with the quadratic terms (only pure quadratic terms were added and not mixed terms), resulting in
(3)XTAS0kAUG=XTAS0k,XTAS02k,(4)Xα0kAUG=Xα0k,Xα02k,(5)Xβ0kAUG=Xβ0k,Xβ02k.


## 5. Regressor Selection

A Matlab-based automated procedure known as Stepwise Selection [28] has been successfully used in this effort for the selection of the regressors. This scheme iteratively adds and removes variables relying on the t-statistics of the estimated regression coefficients. Starting from the initial set of regressors in (3), the final set of selected regressors provided by the Stepwise Selection procedure is given by
(6)XTASk=αk,Qk,Rk,Nxk,Nzk,TTHk,hk,ψ2k,α2k,δE2K,Xαk=ϑk,hk,Nxk,Nzk,Pk,Qk,TASk,δRK,TTHk,CRk,Nz2,TAS2k,TTH2k,Xβk=Nxk,Nyk,Pk,Qk,Rk,TASk,φk,δAK,δRK,Trk,Wk,Ny2k,TAS2k,δR2K,δE2K.Specifically, Figures2(a)–2(c) show the variable selection process by the stepwise selection method for the three sensors. The index of the selected variables is represented by the red dots in the upper section of each figure while in the lower section the RMSE prediction error (evaluated on the training data) as function of the algorithm iteration is reported.Figure 2
(a)
TAS variable selection process
(b)
Alpha variable selection process
(c)
Beta variable selection processThe correlations between each regressor selected by the stepwise process and the target signal are reported, for each of the sensors, in Tables2(a), 2(b), and 2(c):Table 2
(a)
Correlation between selected regressors and TAS
α
Q
R
Nx
Nz
TTH
h
ψ2
α2
δE2
−0.9
0.05
0.13
0.73
0.2
0.35
0.24
−0.83
−0.15
0.36
(b)
Correlation between selected regressors andα
ϑ
h
Nx
Nz
P
Q
TAS
δR
TTH
CR
Nz2
TAS2
TTH2
0.58
−0.005
−0.64
0.05
0.09
0.15
−0.9
−0.01
−0.4
−0.16
0.06
−0.88
−0.32
(c)
Correlation between selected regressors andβ
Nx
Ny
P
Q
R
TAS
φ
δA
δR
Tr
W
Ny2
TAS2
δR2
δE2
0.1
0.9
0.1
−0.1
−0.02
0.14
0.1
0.5
0.07
0.007
0.1
−0.4
0.1
0.04
0.12


## 6. Multilayer Perceptron Models

The estimation model introduced in Section2 is given by
(7)y~k=fNRXk.For each sensorTAS, α, and β, the fNRk function was mapped through a neural network architecture. After considering different options, the neural topology selected for this study is the well-known multilayer perceptron with one hidden layer. The specific architecture for each neural approximator is shown in Table 3.Table 3
Neural network architectures.
Number of neurons in the hidden layer
Number and type of output neurons
TASK
7 sigmoidal neurons
1 linear neuron
αK
7 sigmoidal neurons
1 linear neuron
βk
8 sigmoidal neurons
1 linear neuronTable4 summarizes the statistical results achieved for the training flight data of the residual signals which are defined as the difference between the real values of the measured signal and its estimation rk=yk−y~k.Table 4
Training performance results. Statistics of residual signals based on TR-FLIGHT-1 and TR-FLIGHT-2.
Mean (rk)
Std (rk)
Max (rk)
TASK
3.6∗ 10−3
0.1129
1.9240
αK
2.1∗ 10−4
0.1030
1.2362
βk
−7.5∗ 10−6
0.1992
1.8095In a recent study [16], the authors compared the performance of neural network models with those of simple linear regression models using the same set of regressors. In that study, it was verified that the mapping performances of the linear regression models are worse than those provided by the neural network models. This was the main motivation of the use of the neural network models in the present study. A possible motivation of this fact could be that the unknown identifying functions are not linear in the selected regressors; therefore, neural estimators work better than linear regression models.


## 7. Failure Modeling and Residual Whitening

Failure modeling is a very important task in the overall design of fault tolerant schemes [29–31]. In the literature, several fault modeling approaches can be found, such as bias, freezing, drift, loss of accuracy, and calibration error [32]. In this effort, without any loss of generality, only two specific classes of failures have been considered, that is, step failure and ramp failures to simulate the injection of hard or soft failures. For each of these cases, the magnitude of the failure was also varied to test the fault sensitivity. Clearly, hard and soft failures provide different challenges in terms of detectability. The residual signal is defined as the difference between the sensor output and the estimate of the same signal, that is,
(8)rk=yk−y~k.At nominal conditions (no fault), therk signal should theoretically approach the statistical zero. Unfortunately, this is not true in practice due to unavoidable system and measurement noise as well as estimation errors. Thus, the goals of the fault detection scheme are to detect the occurrence of the failure, as soon as possible after its occurrence, along with the minimization of false alarm rates. From a detailed statistical analysis, it became evident that the residual signal is characterized by a high level of autocorrelation caused by uncertainties at low frequencies and noise. Therefore, in its raw form, the residual signal is not a suitable tool since it contains high levels of colored noise. In fact, the residual signal should be uncorrelated for optimal FD. Therefore, a whitening filter has been designed to remove as much correlation as possible from the residual. For our purposes, it was assumed, as shown in (9), that the correlated sequence can be modeled as an autoregressive (AR) process of order N [33]:
(9)∑i=1Ndi⋅rk−i+εrk,where di are AR parameters and εrk is white noise. The term ∑i=1Ndi⋅rk−i can be considered as a one-step-ahead prediction of rk and εrk=rk−r~k as the prediction error. The di parameters were estimated through the Least Squares method to the mean squared error E2=L−1∑Lεk2 of the training data. Then, after the computing of the di parameters, the εrk can be simply derived as output of the linear filter:
(10)εrz=WFIRzrz,where the discrete time transfer function can be easily obtained as
(11)WFIRz=1−d1z−1−d2z−2−d3z−3−−dNz−N.Theεrz signal is therefore labeled as “whitened residual” rWk.The correct length of the whitening filter was experimentally determined by evaluatingrWk for increasing values of the filter length N, applying the Ljung-Box test [34] to verify whether the whitened residual exhibits a significant autocorrelation. Table 5 shows the final values of N selected by this design process.Table 5
Filter length and whitening filter design.
N length
TASk
25
αk
55
βk
30A comparison between the residuals before and after the whitening filtering is shown in Figures3(a), 3(b), 4(a), 4(b), 5(a), and 5(b).Figure 3
(a)
TAS residual and its autocorrelation
(b)
TAS whitened residual and its autocorrelationFigure 4
(a)
Alpha residual and its autocorrelation
(b)
Alpha whitened residual and its autocorrelationFigure 5
(a)
Beta residual and its autocorrelation
(b)
Beta whitened residual and its autocorrelation


## 8. Design of the EWMA Filter

Following the design of the whitening filter, the design of the fault detection filters was performed using tools from the statistical process control (SPC). In this context, the exponentially weighted moving average (EWMA) filter is a well-known chart used to detect the presence of small shifts in process variables. The EWMA chart tracks the exponentially weighted moving average mean of all the previous samples, so that the most recent samples are weighted more heavily than the previous ones. Even though normality of the distribution is usually the basic assumption in many EWMA charts, this chart has exhibited enough robustness even with non-normally distributed data [35]. The output of the filter zk is the output of a first-order linear digital filter taking as input the whitened residual, that is,
(12)zk=1−λ⋅zk−1+λ⋅rWk0<λ<1.Clearly, the selection of the appropriate value of the smoothing parameterλ is critical. Larger values of λ are associated with a larger weight of the current input on the current output; conversely, smaller values of λare associated with larger weights of all the previous input samples on the current output. The output mean and standard deviation can be computed, at stationary condition, using (14).
(13)μz=μr,(14)σz2=σr2λ2−λ,where μ is the mean and σ is the standard deviation.A standard commonly adopted procedure for the selection ofλ was not found in the literature. In this effort, the Least Squares method [36] was used, with the goal of finding an optimized value of λ so that the sum of the sample squared errors (SSE) of the one-step prediction error ezk=rwk−zk−1 is minimized.Figure5 shows sample squared errors as function of λ for the zk EWMA signals produced by the rwk signals that originate from the three AD models evaluated on the training data (SSE are all shifted so that SSE1=0 for all the three filters).Based on Figure6, the tuning of the EWMA filter was performed selecting the value of λ such that the SSE is minimized. Thus, the minimum values of SSE were obtained with the values of λ in Table 6:Figure 6
EWMA tuning.Table 6
Values of theλ parameter for an optimized design of the EWMA filter.
λ
TAS
0.002
α
0.001
β
0.0009


## 9. Data-Driven Threshold Setting

For FD purposes, conventional approaches include the use of fixed thresholds or varying (floating) thresholds. Several studies have been conducted using both the approaches [12, 13, 37, 38], typically resulting in a trade-off between them. Fixed thresholds are easier to set since they depend on a lower number of parameters. On the other hand, fixed thresholds may not allow detection following the occurrence of small amplitude failures. The setup of floating thresholds is, of course, more complex; however, they provide in general more robustness and better performance in terms of detectability, detection delay, and false alarm rate. Clearly, their success requires a careful design. A typical problem of floating (adaptive) thresholds can be given by an excessively fast adaptation to the failure that could result in a missed fault detection. In this effort, floating (adaptive) thresholds [39] are defined, as follows:
(15)FTUP=z¯+η⋅σz+b,FTLOW=z¯−η⋅σz−b,where z¯ and σz are the mean value and the standard deviation, respectively, of the EWMA-filtered residual, both computed through a sliding window of length 1800 (180 sec), while η and b are design parameters. Once again, a standard procedure for setting η and b could not be found in the literature; therefore, an “ad hoc” sensitivity study was conducted using the training data to find the optimal values with the goal of minimizing false alarms. Specifically, a grid search was performed over the two parameters, with η in the range 0 to 10 (with a 0.5 step increment) and b in the range 0 to 3 (with a 0.05 step increment). For each grid point, the experimental PFA was computed on the training data retaining only the couples of values (η and b) leading to an experimental PFA<0.0001. Next, between these couples, the best was selected as the one minimizing the following index:
(16)j=ησz+b.Clearly a small value forJ implies a small band for the floating limiter. This is positive from a fault detection point of view. Table 7 shows the optimized values of η and b achieved applying the proposed procedure:Table 7
Optimal design of the floating thresholds.
η
b
PFA
TAS
6
0.005
0.0001
Α
7
0.003
0.0001
Β
7
0.0075
0.0001The results of this design are summarized in Figures7(a)–7(c), where the experimental probability plots derived from the training data are provided.Figure 7
(a)
Threshold design forTAS: probability plot
(b)
Threshold design forα: probability density
(c)
Threshold design for beta: probability densityIn each of the Figures7(a)–7(c), there are three lobes. The central lobe is the probability distributions of EWMA outputs, while the blue lobes are lower threshold probability distributions and the red lobes are upper threshold probability distributions. Reducing false alarms requires an intersection between the probability distribution of the EWMA filter and the probability distributions of the thresholds as small as possible. If the EWMA output does not exceed significantly the thresholds, this entails that those high false alarm rates are avoided.


## 10. Results

Fault detection during validation tests was performed using 5 flights for the true airspeed (TAS), 6 flights for the angle of attack, and 5 flights for the angle of sideslip. The testing was conducted in simulated real time using a Simulink model. For each flight, an artificial additive fault was injected at t=700s with the detectors set using the values of λ, η, and b reported in Tables 6 and 7. Without any loss of generality, two failures were considered, that is, a “fast” step and a “slow” ramp (the duration of the ramp is 10 sec, then for t>10, the amplitude remains constant at the value for t=10; in this case, for magnitude (or “size”) of the fault, we conventionally mean the steady state value reached by the fault signal for t=10s) with modeling parameters shown in Table 8.Table 8
Step magnitude and ramp slope for fault modeling.
Step (magnitude)
10 s ramp (magnitude)
TAS (m/s)
2
0.2
α (deg)
3
0.3
β (deg)
3
0.3A critical parameter for a FD scheme is the number of false alarms. Clearly, a lower false alarm level implies higher robustness against system and measurement noise. The evaluation of the FD scheme was conducted in two steps; in the first step, the scheme was tested on every validation flight in a fault-free condition (without injecting any fault). This is due to the fact that false alarms are independent from the fault injection; therefore, false alarm rate was evaluated on the whole length of the flight assuming no fault. During the second step, a fault was injected to evaluate the detection delay. Tables9(a), 9(b), and 9(c) show, for the three sensors, the number of false alarms in terms of number of samples along with the “longest continuous stay out of thresholds” time in seconds.Table 9
(a)
Performance for TAS: robustness against false alarms
Airspeed
Flight duration (min)
False alarms (number of samples)
Maximum stay out of thresholds (sec)
Flight-1
17
7
0.4
Flight-2
17.3
0
0
Flight-3
18
0
0
Flight-4
28.3
0
0
Flight-5
30.8
22
0.6
(b)
Performance forα: robustness against false alarms
Angle of attack
Flight duration (min)
False alarms (number of samples)
Maximum stay out of thresholds (sec)
Flight-1
17
35
0.4
Flight-2
17.3
8
0.3
Flight-3
18
0
0
Flight-4
20.3
34
0.6
Flight-5
28.3
0
0
Flight-6
30.8
56
0.7
(c)
Performance forβ: robustness against false alarms
Angle of sideslip
Flight duration (min)
False alarms (number of samples)
Maximum stay out of thresholds (sec)
Flight-1
17.3
0
0
Flight-2
18
0
0
Flight-3
20.3
40
1.5
Flight-4
19.9
42
1.6
Flight-5
30.8
120
2.2Other critical indexes for assessing the performance of a FD scheme are the fault detection delay and the minimum amplitude detectable fault. These indexes are defined as follows:(i)
Fault Detection Delay. This is the time elapsed from the injection of the fault to the moment when the detector detects a “true fault.”
(ii)
Minimum Amplitude Detectable Fault. This is the minimum amplitude of the fault that produces the detection of a “true fault.”In this study, a fault detection is considered a “true fault” if, considering an observation period of 60 s following the failure, the fault detection thresholds are crossed for a time duration longer than 4 s during the observation period; otherwise, the detection is considered not reliable and the true fault is not declared. This logic implies that the minimum decision time for a fault declaration is 60 seconds.Tables10(a), 10(b), and 10(c) show the performance for TAS, α, and β in terms of detection delay, while Figures 8(a), 8(b), 9(a), 9(b), 10(a), and 10(b) show the threshold evolution during the injection of the fault.Table 10
(a)
Detection delay for step and rampTAS fault: ΔV=2m/s
Airspeed
Step fault detection delay (sec)
Ramp fault detection delay (sec)
Flight-1
0.4
3.8
Flight-2
0.4
4.2
Flight-3
0.5
4.7
Flight-4
0.1
4
Flight-5
0.4
3.5
(b)
Detection delay for step and rampα fault: Δα=3deg
Angle of attack
Step fault detection delay (sec)
Ramp fault detection delay (sec)
Flight-1
0.3
4
Flight-2
0.1
7.1
Flight-3
0.1
8
Flight-4
0.1
8.3
Flight-5
0.1
6.9
Flight-6
0.1
7.2
(c)
Detection delay for step and rampβ fault Δβ=3deg
Angle of sideslip
Step fault detection delay (sec)
Ramp fault detection delay (sec)
Flight-1
2.5
7.5
Flight-2
2.4
7.6
Flight-3
0.8
6.5
Flight-4
1.7
6.9
Flight-5
1.6
7.7Figure 8
(a)
TAS, zoom on fault detection, step fault at time 700 s injected in Flight-1
(b)
TAS, zoom on fault detection, ramp fault at time 700 s injected in Flight-1Figure 9
(a)
α, zoom on fault detection, step fault at time 700 s injected in Flight-1
(b)
α, zoom on fault detection, ramp fault at time 700 s injected in Flight-1Figure 10
(a)
β, zoom on fault detection, step fault at time 700 s injected in Flight-1
(b)
β, zoom on fault detection, step fault at time 700 s injected in Flight-1The minimum amplitude detectable fault was computed by incrementally decreasing the amplitude of the fault injected until it was no longer possible to declare a “true fault” within the predefined observation period.The results for the minimum amplitude detectable faults are shown in Table11:Table 11
Minimum amplitude detectable faults.
Step (magnitude)
10 s ramp (magnitude)
TAS (m/s)
0.7
0.07
α (deg)
0.9
0.1
β (deg)
1.3
0.13Tables12(a), 12(b), and 12(c) summarize the performance of the FD scheme with the minimum amplitude detectable along with the associated fault detection delay.Table 12
(a)
Detection delay for step and rampTAS fault
Airspeed
Step fault detection delay (sec)
Ramp fault detection delay (sec)
Flight-1
3.7
15.4
Flight-2
1.9
10.1
Flight-3
10.2
11.3
Flight-4
1.9
9.4
Flight-5
1.4
13.8
(b)
Detection delay for step and rampα fault
Angle of attack
Step fault detection delay (sec)
Ramp fault detection delay (sec)
Flight-1
15.5
15
Flight-2
2.8
8.8
Flight-3
10.4
10.8
Flight-4
6.2
10.6
Flight-5
5.5
9.6
Flight-6
5.7
18.2
(c)
Detection delay for step and rampβ fault
Angle of sideslip
Step fault detection delay (sec)
Ramp fault detection delay (sec)
Flight-1
17.3
18.3
Flight-2
13.1
18.6
Flight-3
15
25.5
Flight-4
7.6
13.3
Flight-5
9.8
16.4


## 11. Comparison between Adaptive- and Fixed-Threshold Approaches

As discussed in the literature, a very interesting area of research is the performance comparison in terms of specific FD parameters between “fixed” and “adaptive” thresholds. Considering the fixed-threshold computation approach discussed in [37] as baseline, in this study, the comparison was performance using the same training and validation data for both approaches, reporting the performance in terms of false alarms and detection delay. For the purpose of being completely objective and conducting a fair comparison, fixed thresholds have been computed starting from the residual signals using the same neural estimators and the same whitening filters to obtain a whitened residual. The difference between the two approaches starts after the EWMA filtering (designed with the same λ parameters). In fact, for the fixed-threshold approach, the EWMA output was processed to find optimized values for upper and lower limits. In fact, during the training, the UCL and LCL were derived from data depending on the desired probability of false alarm PFA, where the PFA is defined as the probability that zk exceeds the detection thresholds in a fault-free condition. Defining Fz as the experimental cumulative PDF of zk, the fault detection thresholds are defined as follows:
(17)UCL=minzFz≥1−PFA,LCL=maxzFz≤PFA.PFA=0.0001 was set the same way in both approaches. Also, the validation tests were conducted on the same flights used in Section 10, using the same fault amplitude and the same injection time for the failure. Tables 13, 14, and 15 show validation results in terms of false alarms and detection delay:Table 13
(a)
Comparison between fixed-threshold and adaptive-threshold false alarms forTAS
TAS
Flight duration (min)
False alarms (num. of samples) (adaptive)
Max stay out of thresholds (sec) (adaptive)
False alarms (num. of samples) (fixed)
Max stay out of thresholds (sec) (fixed)
FL-1
17
7
0.4
5
0.3
FL-2
17.3
0
0
2
0.2
FL-3
18
0
0
0
0
FL-4
28.3
0
0
3
0.2
FL-5
30.8
22
0.6
18
0.5
(b)
Comparison between fixed-threshold and adaptive false alarms delay forα
α
Flight duration (min)
False alarms (num. of samples) (adapt)
Max stay out of thresholds (sec) (adapt)
False alarms (num. of samples) (fixed)
Max stay out of thresholds (sec) (fixed)
FL-1
17
35
0.4
40
0.7
FL-2
17.3
8
0.3
12
0.4
FL-3
18
0
0
2
0.2
FL-4
20.3
34
0.6
40
0.5
FL-5
28.3
0
0
5
0.5
FL-6
30.8
56
0.7
60
1.2
(c)
Comparison between fixed-thresholds and adaptive false alarms delay forβ
β
Flight duration (min)
False alarms (num. of samples) (adapt)
Max stay out of thresholds (sec) (adapt)
False alarms (num. of samples) (fixed)
Max stay out of thresholds (sec) (fixed)
FL-1
17.3
0
0
5
0.2
FL-2
18
0
0
4
.01
FL-3
20.3
40
1.5
45
1.3
FL-4
19.9
42
1.6
46
1.2
FL-5
30.8
120
2.2
80
0.5Table 14
(a)
Comparison between fixed-threshold and adaptive-threshold detection delay forTAS
TAS
Step fault detection delay (adaptive)
Ramp fault detection delay (adaptive)
Step fault detection delay (fixed)
Ramp fault detection delay (fixed)
FL-1
0.4
3.8
0.7
4.8
FL-2
0.4
4.2
0.7
5.7
FL-3
0.5
4.7
0.8
6
FL-4
0.1
4
0.7
6
FL-5
0.4
3.5
0.8
5.6
(b)
Comparison between fixed-threshold and adaptive-threshold detection delay forα
α
Step fault detection delay (adaptive)
Ramp fault detection delay (adaptive)
Step fault detection delay (fixed)
Ramp fault detection delay (fixed)
FL-1
0.3
4.0
3.6
9.3
FL-2
0.1
7.1
1.6
7.2
FL-3
0.1
8.0
0.6
5.5
FL-4
0.1
8.3
1.8
7.4
FL-5
0.1
6.9
1.2
6.8
FL-6
0.1
7.2
1.1
6
(c)
Comparison between fixed-threshold and adaptive-threshold detection delay forβ
β
Step fault detection delay (adaptive)
Ramp fault detection delay (adaptive)
Step fault detection delay (fixed)
Ramp fault detection delay (fixed)
FL-1
2.5
7.5
3
9
FL-2
2.4
7.6
3.6
9
FL-3
0.8
6.5
4.4
10
FL-4
1.7
6.9
6.5
6.8
FL-5
1.6
7.7
3.5
10Table 15
(a)
Comparison between fixed-threshold and adaptive-threshold detection delay forTAS: soft failures
TAS
Step fault detection delay (adaptive)
Ramp fault detection delay (adaptive)
Step fault detection delay (fixed)
Ramp fault detection delay (fixed)
FL-1
3.7
15.4
7.6
19.4
FL-2
1.9
10.1
15.3
19.2
FL-3
10.2
11.3
31.5
30
FL-4
1.9
9.4
16.3
17
FL-5
1.4
13.8
27.8
28
(b)
Comparison between fixed-threshold and adaptive-threshold detection delay forα: soft failures
α
Step fault detection delay (adaptive)
Ramp fault detection delay (adaptive)
Step fault detection delay (fixed)
Ramp fault detection delay (fixed)
FL-1
15.5
15
Not detected
40.9
FL-2
2.8
8.8
86.2
85.6
FL-3
10.4
10.8
10.7
12.6
FL-4
6.2
10.6
41.2
40.8
FL-5
5.5
9.6
Not detected
51.4
FL-6
5.7
18.2
26.7
26
(c)
Comparison between fixed-threshold and adaptive-threshold detection delay forβ: soft failures
β
Step fault detection delay (adaptive)
Ramp fault detection delay (adaptive)
Step fault detection delay (fixed)
Ramp fault detection delay (fixed)
FL-1
17.3
18.3
19
20
FL-2
13.1
18.6
21
23
FL-3
15
25.5
43.8
48.1
FL-4
7.6
13.3
5.9
11.2
FL-5
9.8
16.4
46.7
42.8As far as soft failures are concerned, the detection delay analysis was performed using the minimum amplitude detectable fault used in Section10 and reported in Table 11. The achieved results are shown in Tables 15(a), 15(b), and 15(c):Analyzing the results, it is evident that detecting fault featuring adaptive limiters could save from 0% to 50% of the time, with the same false alarm rate in the case of hard failures. Substantially different results were achieved when comparing the two approaches in the case of soft failures. In fact, adaptive thresholds allow a much lower detection delay versus fixed thresholds (up to a ratio of 1 : 30). In some cases, the occurrence of a soft failure was not even detected using fixed thresholds.Note: the study presented in this paper is specifically focused on the design of experimental FD filters. Therefore, the problem of fault isolation is not considered at all in the study. Indeed, if the fault signature matrix for the residuals forTAS, α, and β are computed, it can be immediately seen that the three faults are not strongly isolable since each residual depends on TAS, α, and β. Different approaches can be used to deal with the fault isolation problem, such as
(1)
to perform a fault sensitivity analysis to investigate whether the fault has a different effect on the amplitudes of three residuals and then exploit this information for discriminating the faulty signal.
(2)
to exclude, a priori, from the set of regressors the reaming two signals (for instance the regressor set used for approximating theTAS signal does not include the α and β signals). In this case, we achieve, by construction, experimental models that produce a strong isolable failure signature matrix. Interested readers are referred to the recent paper by the authors in [16] where this last approach has been applied.


## 12. Conclusions

The goal of this study was the design of a robust adaptive threshold scheme able to deal with failures involving air data sensors. A completely data-driven approach for the design and the tuning of the FD schemes for true airspeed, angle-of-attack, and angle-of-sideslip sensors of an aircraft has been proposed. It has been shown that, through the use of this scheme, it is possible to improve significantly the robustness in terms of false alarms and failure detection delay. The data-driven design philosophy was used in many parts of the overall FD procedure that is in the regressor selection phase for the considered sensors, in the design of the withering and EWMA filters as long as for the computation of detection thresholds. Validation studies based on actual flight data have shown good overall performance in the presence of hard and soft failures and the possibility to detect very small faults. Additionally, a comparison with a fixed-threshold approach was performed. The results of the comparison revealed that for hard failures, there is a limited difference between the approaches. In contrast, a different trend has been observed in the case of soft failures. Specifically, a significant detection delay has been observed in the case of fixed thresholds, leading also to the “no detection at all” situation in some cases.The capabilities of this data-driven FD approach coupled with the flexibility of the adaptive threshold provided by the floating limiters have proven to be substantially better than fixed-threshold methods in terms of FD performance. This last aspect makes the proposed scheme appealing for the application to a variety of sensor failures for systems where physical redundancy in the sensors is not available.



---

*Source: 1072056-2018-11-25.xml*