# Hybrid Machine Learning-Statistical Method for Anomaly Detection in Flight Data
###### Abstract  
This paper investigates the use of an unsupervised hybrid statistical-local outlier factor algorithm to detect anomalies in time-series flight data. Flight data analysis is an activity carried out by airlines primarily as a means of improving the safety and operation of their fleet. Traditionally, this is performed by checking exceedances in pre-set limits to the flight data parameters. However, this method highlights single events during a flight, making this analysis laborious. The process also fails to establish trends or reflect potential unknown hazards. This research took advantage of machine learning techniques to recognize patterns in large datasets by implementing the local outlier factor (LOF). In order to minimize human input, a statistical approach was adopted to establish the threshold value above which the flights are considered to be anomalous and interpret the scores. This paper shows that LOF quantifies the degree of outlier-ness of an outlier rather than binary categorizing a point into inlier or outlier, as in the case of clustering algorithms. Thus, with LOF, for the first time, we demonstrated that in the aviation industry, anomalous flights could not only be identified but also be given an anomaly score to compare two anomalous flights in an unsupervised manner. Furthermore, LOF helps to track anomalous behavior in time during the flight. This is insightful when a flight is abnormal, only for some seconds or short duration. For the first time, we attempted to detect flight parameters responsible for anomalous behavior or at least give direction to human experts looking for the cause of abnormal behavior. This was all analyzed with real-life flight data in an unsupervised manner in contrast to simulated data.  
unsupervised anomaly detection; time series anomaly detection; local outlier factor; Tukey's method; flight data analysis +
Footnote †: journal:  
human operator to check each exceedance from every individual flight of the airline fleet, making flight data analysis laborious.  
Flight data analysis involves the processing of massive amounts of high-dimensional or unstructured data. While the minimum requirements of flight data recorders are to be able to store 88 flight parameters for 25 h, modern aircraft manufacturers have been taking advantage of data storage technology to capture more parameter data, which can then be fed into the safety and design of their product. For example, the digital flight data recorder onboard the Boeing 787 can record approximately 2000 parameters for 50 h. Therefore, while the data recording aspect has improved dramatically, the data analysis aspect has been lagging, making it very difficult for a human to establish patterns in exceedances and thus receive further benefit from such events. Moreover, the parameter pre-defined maxvals have been based on historical lessons. Therefore, they also impose an intrinsic limitation in the system, which is unable to discover new anomalies.  
This paper addressed this by adapting a modern machine learning algorithm to analyze flight data more effectively and efficiently and making it suitable to establish useful trends. The paper made use of an unsupervised technique--the local outlier factor to establish anomalies in flight data. The technique is added with a hybrid statistical element, thereby reducing the continuous reliance on a human expert to decide if an outlier is anomalous or not. In order to achieve this, the paper is organized in the following manner: Section 2 provides a brief literature review of some machine learning techniques that have been applied to flight data analysis. Section 3 introduces the local outlier factor technique, which is the focus of this paper. This section describes the various tweaks that were required for the successful implementation of the technique in this domain. Section 4 provides the results for a scenario of flights with the same tail number landing at three different airports, and a discussion follows. A conclusion is finally provided in Section 5.