## 5 Experiments
### Multi-Class Classification and Binary Benign-Class Case  
In the first phase of experiments, we focused on testing the overall quality of cross-validated classifiers, measuring the multi-class measures for all combinations of training and test sets. In addition, the fifth case was introduced--the classifier was trained using a sum of all four training sets. In each case, the final measure is the mean value of two tests (2-fold cross-validation). Table 4 presents the results. In this table, some combinations of train-test sets generate higher metric values than others. Based on F1 and accuracy (which are highly correlated--their correlation equals 0.99), one may observe that classifiers trained on ToN perform pretty well when tested on IDS18 and NB15 (obviously, it obtained the highest score for ToN-origin test set). In contrast, its performance on the BoT test set is dramatically low. Similar patterns may be seen when training on NB15 and IDS18--also, the testing on BoT results in measures close to 0. High \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Train** & **Test** & **P** & **R** & **F1** & **A** \\ \hline BoT & BoT & 0.985 & 0.985 & 0.985 & 0.985 \\ BoT & ToN & 0.075 & 0.038 & 0.050 & 0.038 \\ BoT & NB15 & 0.931 & 0.243 & 0.380 & 0.243 \\ BoT & IDS18 & 0.908 & 0.446 & 0.584 & 0.446 \\ \hline ToN & BoT & 0.000 & 0.004 & 0.000 & 0.004 \\ ToN & ToN & 0.955 & 0.955 & 0.955 & 0.955 \\ ToN & NB15 & 0.920 & 0.896 & 0.908 & 0.896 \\ ToN & IDS18 & 0.769 & 0.788 & 0.778 & 0.788 \\ \hline NB15 & BoT & 0.036 & 0.003 & 0.000 & 0.003 \\ NB15 & ToN & 0.133 & 0.350 & 0.187 & 0.350 \\ NB15 & NB15 & 0.990 & 0.989 & 0.989 & 0.989 \\ NB15 & IDS18 & 0.771 & 0.843 & 0.805 & 0.843 \\ \hline IDS18 & BoT & 0.082 & 0.003 & 0.000 & 0.003 \\ IDS18 & ToN & 0.138 & 0.268 & 0.155 & 0.268 \\ IDS18 & NB15 & 0.922 & 0.958 & 0.940 & 0.958 \\ IDS18 & IDS18 & 0.992 & 0.992 & 0.992 & 0.992 \\ \hline All & BoT & 0.976 & 0.947 & 0.954 & 0.947 \\ All & ToN & 0.953 & 0.953 & 0.952 & 0.953 \\ All & NB15 & 0.990 & 0.989 & 0.989 & 0.989 \\ All & IDS18 & 0.988 & 0.988 & 0.988 & 0.988 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Multiclass classification resultsmetrics values imply relatively high similarity of the network traffic between sets NB15 and IDS18 and lower similarity between each of them and the ToN. It may imply that they may be considered generalizable. The BoT set is, however, a different case, as indicated by the low values of measures. When, in turn, this set is used as a training one, the measures obtained when the remainder of sets as test ones are remarkably lower (F1: 0.584,0.380) or even close to 0 (0.050). This behavior confirms that the BoT differs from other sets and is poorly generalizable. To dive deeper into this issue, one must look into all the sets, performing quantitative and qualitative investigations. Looking at Table 1, one may see that the principal difference between BoT and the remaining sets, apart from the origin of data and classes, relies on the class balance. In all but BoT datasets, the _Benign_ traffic class dominates, in IDS18 and NB15, it dominates totally (88.05% and 96.02%, respectively), while in ToN, it is still the most frequent class, gathering 36.01 % of samples. This is because the share of classes in the dataset tries to reflect the one in the typical network flow, where intrusions and attacks are relatively rare and what dominates is the benign network traffic. From this point of view, BoT looks completely different; the share of the _Benign_ class is less than 1%, and the set is almost equally divided into two harmful classes: _DoS_ and _DDoS_. All of the above raises suspicion that the high values of metrics obtained in the cross-validation of IDS18, ToN, and NB15 are caused by the significant share of the benign traffic in the datasets.