# Detecting Cyber Threats in UWF-ZeekDataFall22 Using K-Means Clustering in the Big Data Environment  
[PERSON]  
1 Department of Computer Science, The University of West Florida, Pensacola, FL 32514, USA; 1 [EMAIL_ADDRESS] (G.C.S.D.C. ); [EMAIL_ADDRESS] (A.M.); [EMAIL_ADDRESS] (S.E.) 1  
[PERSON]  
1 Department of Computer Science, The University of West Florida, Pensacola, FL 32514, USA; 1 [EMAIL_ADDRESS] (G.C.S.D.C. ); [EMAIL_ADDRESS] (A.M.); [EMAIL_ADDRESS] (S.E.) 1  
[PERSON]  
1 Department of Computer Science, The University of West Florida, Pensacola, FL 32514, USA; 1 [EMAIL_ADDRESS] (A.M.); [EMAIL_ADDRESS] (S.E.) 1  
[PERSON]  
2 Department of Cybersecurity, The University of West Florida, Pensacola, FL 32514, USA; 1 [EMAIL_ADDRESS] (A.M.  
newer malware threats that become increasingly complex and sophisticated. Such methods are known as supervised methods since they require data to be labeled before data can be used for detection. Thus, unknown attacks may go undetected because there are no labeled examples of them in the training data. The development of efficient and modern methods for IDS is critical. Historically, failures in intrusion detection have brought severe consequences to companies and users, including financial losses, reputational damage, and data breaches. These limitations in IDSs highlight the need for unsupervised or even semi-supervised techniques, which do not rely totally on labeled data or existing attack patterns. Attacks like Advanced Persistent Threats (APTs) and zero-day exploits, for example, continuously change their behavior and signatures, making them difficult to detect using signature-based approaches. Clustering allows IDSs to identify unusual data in network traffic without having prior knowledge of attack signatures, making it useful in today's cybersecurity landscape. Most machine learning algorithms work well with labeled data. But real-world data is naturally unlabeled. Moreover, there is also a major time commitment as well as computational cost involved in labeling data. Clustering is an unsupervised machine learning technique widely and efficiently used to characterize and group large unlabeled datasets, ranging from financial to medical data and beyond. In this work, our objective is to use clustering to characterize or group large network traffic data into attack vs. non-attack data. Since the objective of any good clustering algorithm would be to have high intra-cluster similarity and low inter-cluster similarity [3], the goal in developing our clusters would be to develop groups of attack clusters and groups of non-attack clusters, each group having similar characteristics within themselves while having low attack cluster and non-attack cluster similarity, that is, low inter-cluster similarity. Hence, in this study, the aim is to achieve intrusion detection utilizing K-means clustering, a popular clustering algorithm [3], widely implemented due to its low computational complexity [3]. K-means clustering, a distance-based clustering algorithm [4], is applied to a modern newly-created network attack dataset, UWF-ZeekDataFall22 [5]. Finding the ideal number of clusters using K-means clustering can be challenging, especially in large datasets [3], hence the uniqueness of this paper is in presenting three different labeling heuristics to find the ideal number of clusters using connection-based features. After K-means clustering is performed using unlabeled data, the labels from the labeled dataset are used to determine which cluster points were attacks and this is used to predict the class of the cluster. K-means clustering has also not been used on connection-based data in any previous work. Though this dataset contains labeled Zeek logs [6], this dataset was used as an unlabeled dataset, that is, the labeled Zeek logs from UWF-ZeekDataFall22 [5] were de-labeled before using this data for this study.