# Detecting Cyber Threats in UWF-ZeekDataFall22 Using K-Means Clustering in the Big Data Environment
### Background  
In order to categorize cyber attacks and understand the different techniques and stages at which attacks take place, for example, adversary behavior, tactical approaches, and systematic malicious actions, a modern cybersecurity behavioral framework, the MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) framework [10] was used for creating the UWF-ZeekDataFall22 dataset. Created in 2013 and maintained by the MITRE Corporation, a non-profit organization, ATT&CK is a \"globally accessible knowledge base of adversary tactics and techniques based on real-world observations\" [10]. This framework is based on certain core components: tactics, techniques, procedures, and mitigations. ATT&CK contains 14 types of tactics, which contain many techniques and sub-techniques under them. As attacks evolve and emerge, the framework is regularly updated, containing over 200 techniques and 435 sub-techniques as of its latest 2024 update [11]. The UWF-ZeekDataFall22 parquet files [5] contain various types of attacks and their occurrences, over several dates in the fall semester of 2022. The attacks are categorized under 12 different tactics, as presented in Table 1.  
While a number of studies have employed the KDD-Cup99 dataset to analyze IDSs, our paper differs from others through the use of a more recent dataset. The survey paper, by [PERSON] et al. (2020) [13], confirms that KDDCup99 remains the most commonly used dataset for anomaly detection in IDS due to its volume, as it features millions of training and test datasets containing 41 features and 24 types of attacks. However, it must be noted that the KDD-Cup '99 dataset comprises data from almost 25 years ago. In contrast, our research leverages Zeek data from Fall 2022, which is a significantly more contemporary dataset in comparison to the KDD Cup 1999 dataset that most other studies use in clustering  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Tactic** & **The Adversary is Trying to** & **Count** \\ \hline Collection & Gather data of interest to their goal & 1 \\ \hline Command and Control & Communicate with compromised systems to control them & 17 \\ \hline Credential Access & Steal account names and passwords & 1 \\ \hline Defense Evasion & Avoid being detected & 3064 \\ \hline Discovery & Figure out your environment & 16,819 \\ \hline Execution & Run malicious code & 30 \\ \hline Initial Access & Get into your network & 19 \\ \hline Lateral Movement & Move through your environment & 11 \\ \hline Persistence & Maintain their foothold & 10 \\ \hline Privilege Escalation & Gain higher-level permissions & 3066 \\ \hline Reconnaissance & Gather information they can use to plan future operations & 51,492 \\ \hline Resource Development & Establish resources they can use to support operations & 275,471 \\ \hline \hline \end{tabular}
\end{table}
Table 1: MITRE ATT&CK Tactics included in UWF-ZeekDataFall22 dataset [12].  
algorithms. The UWF-ZeekDataFall22 [5] dataset captures a variety of attack types, including Reconnaissance, Credential Access, Execution, and others listed in Table 1. All of these align with the MITRE ATT&CK framework, thus providing a comprehensive representation of current attack tactics. Our use of the UWF-ZeekDataFall22 dataset overcomes another drawback of the KDD Cup '99 dataset, which is known for containing a large number of duplicate records in the training and test sets, thus complicating the categorization of network traffic with redundant records [14].  
This study can be considered semi-supervised as it uses unlabeled data to build clusters and labeled data only to determine which clusters contain attacks. The labeled data was used for comparison and verification of results only.