# Detecting Cyber Threats in UWF-ZeekDataFall22 Using K-Means Clustering in the Big Data Environment
## 5 K-Means Clustering Results and Discussion
### Finding the Best Set of Features Within the Connection-Based Features
#### 5.5.1 First Method: Clusters with at Least One Attack as Attack Clusters  
For this analysis, the same 12 initial connection-based features were considered: duration, resp_pkts, conn_state, proto, orig_ip_bytes, missed_bytes, orig_pkts, resp_ip_bytes, dest_port_zeek, orig_bytes, resp_bytes, and src_port_zeek. Set 1 was run with all these 12 connection-based features, as shown in Table 11. For the set 2 runs, the first feature, duration, was removed. If the statistical metrics, that is accuracy, precision, recall, etc. went down, that means that it negatively impacted the results, so this feature was considered important and was kept in the pool of features. If the results were the same or went up, then that feature was removed. In this manner, all the features were tested. The feature sets tested in the First Method are presented in Table 11. The corresponding performance evaluations for these feature sets are provided in Table 12. The best-performing cluster size of 350 and optimally performing seed value of 600, as shown in Table 8, were used to run the various feature set combinations. Figure 4: Accuracy vs. Number of Clusters Using Optimal Seed Values. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline
**Feature** & **Set 1** & **Set 2** & **Set 3** & **Set 4** & **Set 5** & **Set 6** & **Set 7** & **Set 8** & **Set 9** & **Set 10** & **Set 11** & **Set 12** & **Set 13** \\ \hline duration & X & & X & X & X & X & X & X & X & X & X & X & X \\ \hline resp\_pkts & X & X & & X & X & X & X & X & X & X & X & X & X \\ \hline conn\_state & X & X & X & & X & X & X & X & X & X & X & X \\ \hline proto & X & X & X & X & & X & X & X & X & X & X & X & X \\ \hline orig\_ip\_bytes & X & X & X & X & X & & X & X & X & X & X & X \\ \hline missed\_bytes & X & X & X & X & X & X & & & & & & \\ \hline orig\_pkts & X & X & X & X & X & X & & X & X & X & X & X \\ \hline resp\_ip\_bytes & X & X & X & X & X & X & X & X & & & & \\ \hline dest\_port\_zeek & X & X & X & X & X & X & X & X & & & & \\ \hline orig\_bytes & X & X & X & X & X & X & X & X & X & & & \\ \hline resp\_bytes & X & X & X & X & X & X & X & X & X & & & \\ \hline src\_port\_zeek & X & X & X & X & X & X & X & X & X & X & & \\ \hline Memory  Estimation (MB) & 64.1 & 58.8 & 58.8 & 58.8 & 58.8 & 58.8 & 53.4 & 53.4 & 48.1 & 42.8 & 42.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Connection-based Features Used in the First Method. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Set** & **Clusters** & **Seed** & **Accuracy** & **Precision** & **Recall** & **F1 Score** & **AUC** & \ \begin{tabular}{c} **Computation** \\ **Time** \\ \end{tabular}  & \begin{tabular}{c} **Confusion** \\ **Matrix** \\ \end{tabular}  \\ \hline
1 & 350 & 600 & 97.45\% & 97.58\% & 97.45\% & 97.45\% & 97.47\% & 18.41 s & \begin{tabular}{c} [[332577, 17762], \\ [0, 346933]] \\ \end{tabular}  \\ \hline
2 & 350 & 600 & 66.91\% & 80.13\% & 66.91\% & 62.91\% & 67.07\% & 8.09 s & \begin{tabular}{c} [[119582, 230757], \\ [0, 346933]] \\ \end{tabular}  \\ \hline
3 & 350 & 600 & 97.01\% & 97.18\% & 97.01\% & 97.01\% & 97.03  
Analysis of the First Method:
* Impact of Feature Selection on Performance
* The choice of features significantly influenced the clustering results, affecting precision, recall, and overall detection accuracy. * Some feature sets gave significantly better results, demonstrating that certain features are essential for effective attack detection. * Best Performing Feature Sets
* Sets 9 and 11 yielded the highest precision (97.70%) and recall (97.59%), making these two sets the most effective feature combinations. * Set 9 performed best when \"missed_bytes\" and \"resp_ip_bytes\" were excluded, suggesting that these attributes did not contribute significantly to distinguishing attack patterns in the First Method. * Set 11 performed best when \"missed_bytes\", \"resp_ip_bytes\", and \"orig_bytes\" were excluded, suggesting that these attributes did not contribute significantly to distinguishing attack patterns in the First Method.