# Hybrid Machine Learning-Statistical Method for Anomaly Detection in Flight Data
## 3 Local Outlier Factor
### Implementation
#### 3.2.4 \(k\) Nearest Neighbor
The local reachability densities are then compared with those of the neighbors using  
\[\text{LOF}_{\text{k}}(\text{A})=\frac{\sum_{\text{B}\in\text{N}_{\text{k}}( \text{A})}\frac{\text{LRD}_{\text{k}}(\text{B})}{|\text{N}_{\text{k}}(\text{ A})|}}{|\text{N}_{\text{k}}(\text{A})|}=\frac{\sum_{\text{B}\in\text{N}_{\text{k}}( \text{A})}\text{LRD}_{\text{k}}(\text{B})}{|\text{N}_{\text{k}}(\text{A})|* \text{LRD}_{\text{k}}(\text{A})} \tag{5}\]  
LOF is the average local reachability density of the neighbors divided by the object's own local reachability density. LOF values of point A can be interpreted as follow:  
1. LOF value approximately equal to one means that the density of point A is comparable to its neighbors, and thus A is not an outlier;
2. LOF value less than one means that point A has a higher density than its neighbors, and thus A is an inlier;
3. LOF value greater than one means that point A has a lower density than its neighbors, and thus A is an outlier. A small value of \(k\) is not preferred as the algorithm becomes sensitive to noise, and a too-large value does not detect local anomalies. There cannot be one definite value for \(k\) in finding the anomalous flights, as each dataset given is unique in the number of total flights (samples) and the number of flight parameters. As such, there are no pre-defined statistical methods to find the most optimal value of \(k\). In this present technique, before implementing the LOF algorithm, the optimal \(k\) value was determined by calculating the LOF score values for each possible \(k\) value. As discussed in detail in the original paper [18], the least value for parameter \(k\) should be 10. This helped to remove unwanted statistical fluctuations. In the aviation domain, we could not fix the lower bound for \(k\) as we did not know how many minimum similar objects a cluster had (other objects can be outliers relative to this cluster), or we did not know the exact number of normal flights. Similarly, we could also not decide the total number of anomalous flights in each dataset, and hence the upper bound for \(k\) could not be fixed. Since the algorithm uses unsupervised learning and we did not have labeled data or know in advance the normal or abnormal flights, \(k\) can be any number. For every \(k\) value, we observed the LOF score calculated. The \(k\) value,which gives the highest LOF score, is the optimal \(k\) value. The \(k\) value corresponding to the highest LOF score was chosen to catch the instance at which the object is the most outlying. The lowest LOF score was not chosen as it erases the outlying nature of an object completely. Figure 5 highlights three values for parameter \(k\) taken at three instances. The \(k\) value, which gives the highest LOF score, is 13. The \(k\) value of 49 gives the mean LOF score, and the lowest LOF score is represented by a \(k\) value of 80. In each of these three instances, the LOF score was calculated, and the results for the three flights are presented in Table 1.  
Table 1 shows that all three flights are most outlying for a \(k\) value of 13, which corresponds to the highest LOF score in the dataset. Flight 2 and Flight 3 even tend to become normal (with respect to [PERSON]'s threshold of LOF score value of 1.1.69) when LOF is calculated at \(k\) value corresponding to the mean LOF score or the lowest LOF score in the dataset. Thus, the optimal value for \(k\) is at the instance which captures the maximum LOF score in the dataset. The distance measure is another important parameter of the \(K\) nearest neighbor algorithm. Choosing the right type of distance measure is another challenge involved. Euclidean distance was not chosen as the distance measure because as the dimensionality increases, the curse of dimensionality impacts Euclidean distance measure [22]. In the case of flight data, there are 132 parameters recorded at different time steps. The dimensions run into thousands. Cosine similarity as the distance measure is suitable for high-dimensional datasets.