# Detecting Cyber Threats in UWF-ZeekDataFall22 Using K-Means Clustering in the Big Data Environment
## 2 Related Works
### Enhanced Partition-Based and Hybrid Clustering (Grid-Based + Density-Based)  
[PERSON] et al. (2016) [PERSON] et al. (2016) explored a clustering approach with industrial intrusion detection data from an attack in 2010 that was able to bypass security measures and cause infections on SCADA systems through thumb drives and network shares. This study applied partition-based clustering methods of fuzzy c-means (FCM) and K-means clustering to group data into tight, distinct clusters. The focus of this paper was to enhance detection rates by introducing a fuzzy inference system to identify anomaly signatures. The initial results of this system were poor due to its inability to detect attacks that appear normal; however, the reduction of clusters and analysis of data structures created a more effective approach for evaluating attacks on SCADA systems. Similar to [PERSON] et al. (2016) [PERSON] et al. (2016), [PERSON] et al. (2009) [PERSON] et al. (2009) uses a hybrid clustering model aimed at optimizing alert management in network intrusion detection systems (NIDS). The authors used K-means clustering combined with Expectation-Maximization (EM), and FCM with unit range (IUR) and principal component analysis (PCA) to reduce alert redundancy and false positives. The utilization of the Intrusion Detection Message Exchange Format allowed data preprocessing, with EM particularly effective for sorting data into models based on local maximum likelihood estimates. The results of this study showed the significance of alert correlation and the benefits of integrating multiple clustering techniques to improve the efficiency and accuracy of network intrusion detection. However, because one of the main components of this algorithm is preprocessing to label the data, it can not only have high computational costs, but it is also a supervised method, making it not useful against evolving attacks. [PERSON] (2000) [PERSON] (2000) and [PERSON] and [PERSON] (2005) [PERSON] and [PERSON] (2005) address these limitations by employing unsupervised techniques in IDS. [PERSON] (2000) [PERSON] (2000) uses purely unsupervised clustering-based anomaly detection to detect intrusions, with K-means used to detect deviations from normal network traffic patterns in the KDD-Cup99 dataset. The algorithm demonstrated good performance, with a high detection rate (40-55%) and a low false positive rate of 1.3-2.3%. However, the authors note that the performance of their algorithm is highly dependent on the training set used, with different datasets leading to significantly varied results. Their reliance on the outdated and redundant KDD-Cup99 data for optimal performance limits the applicability of their approach to modern and evolving network traffic environments. [PERSON] and [PERSON] (2005) [PERSON] and [PERSON] (2005) propose a new grid-based and density-based algorithm called fpMAFIA, also designed to address limitations of supervised methods like misuse detection, which rely on labeled data and cause high labeling costs. To mitigate this, [PERSON] and [PERSON] (2005) [PERSON] and [PERSON] (2005) also employed unsupervised anomaly detection, leveraging existing normal data from the KDD-Cup99 dataset to detect new intrusions based on any deviation from it. Although fpMAFIA showed a high detection rate while reducing computational costs, it also had a high false positive rate, and further research is needed to improve its accuracy. Another paper that aims to optimize alert management is [PERSON] and [PERSON] (2013) [PERSON] and [PERSON] (2013) [PERSON] and [PERSON] (2013), which uses a two-stage technique in order to improve the accuracy of an IDS and utilizes the DARPA 1999 dataset instead of the KDD-Cup99. The first stage classifies the generated alerts based on some similar features to form partitions of alerts using a self-organizing map (SOM) with the K-means algorithm and Neural-Gas with the fuzzy c-means algorithm.