# Multi-Resolution Analysis with Visualization to Determine Network Attack Patterns
## 6 Results and Discussion
Similarly, a high recall score was determined when analyzing the Kyoto2006 dataset with SVM because the number of attack events is relatively small. From the extensive evaluation of the DWT features with various window and step sizes, we found a performance difference depending on the datasets. Figure 6 represents average F1 scores of classification algorithms based on different window and step sizes. Because of the highly imbalanced dataset, specifically the CICIDS2017 dataset, we could not find a major difference in the F1 scores with the DWT and PCA features (see orange-colored polylines). For the Kyoto2006 dataset, we found a distinguishable result. With the DWT features, we observed the average F1 scores of 0.86 \(\pm\) 0.02. However, with the PCA features, the average F1 scores jumped to 0.95 \(\pm\) 0.002. It is because the normal and attack events in the Kyoto2006 dataset maintain high similarities even if MRA has been applied. With PCA, similar characteristics were removed as it only determined highly significant attributes from the data. This finding indicates that PCA would be a good technique for determining \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & \multicolumn{3}{c}{**DWT Features**} & \multicolumn{3}{c}{**PCA Features**} \\ \hline Metrics & Datasets & SVM & NB & KNN & SVM & NB & KNN \\ \hline \multirow{4}{*}{Accuracy} & NSLKDD & 0.521 & 0.987 & 0.951 & 0.520 & 0.528 & 0.938 \\ \cline{2-7}  & Kyoto2006 & 0.892 & 0.632 & 0.971 & 0.893 & 0.892 & 0.951 \\ \cline{2-7}  & CICIDS2017 & 0.970 & 0.998 & 0.998 & 0.970 & 0.970 & 0.994 \\ \hline \multirow{4}{*}{Precision} & NSLKDD & 0.521 & 0.987 & 0.951 & 0.520 & 0.528 & 0.938 \\ \cline{2-7}  & Kyoto2006 & 0.892 & 0.969 & 0.984 & 0.893 & 0.892 & 0.968 \\ \cline{2-7}  & CICIDS2017 & 0.940 & 0.998 & 0.998 & 0.940 & 0.940 & 0.994 \\ \hline \multirow{4}{*}{Recall} & NSLKDD & 0.521 & 0.987 & 0.951 & 0.520 & 0.528 & 0.938 \\ \cline{2-7}  & Kyoto2006 & 0.998 & 0.621 & 0.984 & 0.998 & 0.999 & 0.978 \\ \cline{2-7}  & CICIDS2017 & 0.970 & 0.998 & 0.998 & 0.970 & 0.970 & 0.994 \\ \hline \multirow{4}{*}{F1 score} & NSLKDD & 0.521 & 0.987 & 0.951 & 0.520 & 0.528 & 0.938 \\ \cline{2-7}  & Kyoto2006 & 0.775 & 0.916 & 0.974 & 0.943 & 0.943 & 0.973 \\ \cline{1-1} \cline{2-7}  & CICIDS2017 & 0.955 & 0.998 & 0.998 & 0.955 & 0.955 & 0.994 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance differences between all DWT features and PCA features with multiple classification algorithms. For the PCA features, the first two principal components are used to run the classification. Figure 5: Detailed attack information with the DWT features (db3, \(\alpha=150\) and \(\beta=10\)) in the PCA projections. (**A**) NSLKDD; (**B**) Kyoto2006; (**C**) CICIDS2017. attack events from the network traffic data, especially when the attack events maintain highly similar characteristics compared to the normal events. For the NSLKDD dataset, we found lowered F1 scores with the PCA features than with the DWT features. As explained above, the DWT features are good for differentiating the normal and attack events in the NSLKDD dataset. However, this result explains that applying PCA does not benefit the performance in classifying them (normal vs. attack) if the difference between them has already been discovered through wavelet transformation. Agglomeration hierarchical clustering is applied to understand the effectiveness of performing a visual analysis within the PCA projection space as a part of MRA. It aggregates data until \(k\) clusters are formed. For determining similarities, different distance measurements are utilized as Euclidean distance (L\({}^{2}\)), Chebyshev distance (L\({}^{\infty}\)), City-block distance (L\({}^{1}\)), and Pearson correlation coefficient (R\({}^{2}\)). Figure 7 shows the results when the hierarchical clustering technique is applied. Clustering results are presented with solid lines in the visualizations.