## 5 Experiments
### Multi-Class Classification and Binary Benign-Class Case
To validate this hypothesis, the binary classification setup is applied, focusing on the _Benign_ traffic class and measuring the ability of classifiers to correctly classify samples of this class (true-case), treating the rest of the classes as the false-case. Table 5 shows the classification metrics in this setup. Following the previous considerations, when analyzing the _Benign_ traffic class, we focus on precision and false positive rate (FPR) measures, which are the most adequate in this case. The F1 measure is used as the supplementary one. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Train** & **Test** & **A** & **P** & **R** & **F1** & **FNR** & **FPR** \\ \hline BoT & BoT & 1.000 & 0.956 & 0.988 & 0.972 & 0.012 & 0.000 \\ BoT & ToN & 0.536 & 0.209 & 0.104 & 0.139 & 0.896 & 0.221 \\ BoT & NB15 & 0.271 & 0.970 & 0.248 & 0.395 & 0.752 & 0.188 \\ BoT & IDS18 & 0.548 & 0.963 & 0.507 & 0.663 & 0.493 & 0.146 \\ \hline ToN & BoT & 0.190 & 0.005 & 0.974 & 0.009 & 0.026 & 0.813 \\ ToN & ToN & 0.984 & 0.967 & 0.990 & 0.978 & 0.001 & 0.019 \\ ToN & NB15 & 0.897 & 0.959 & 0.934 & 0.946 & 0.067 & 0.975 \\ ToN & IDS18 & 0.793 & 0.873 & 0.895 & 0.884 & 0.105 & 0.957 \\ \hline NB15 & BoT & 0.468 & 0.006 & 0.922 & 0.012 & 0.078 & 0.534 \\ NB15 & ToN & 0.351 & 0.354 & 0.971 & 0.518 & 0.029 & 0.999 \\ NB15 & NB15 & 0.970 & 0.999 & 0.998 & 0.998 & 0.002 & 0.037 \\ NB15 & IDS18 & 0.843 & 0.876 & 0.957 & 0.915 & 0.043 & 0.997 \\ \hline IDS18 & BoT & 0.041 & 0.003 & 0.867 & 0.006 & 0.134 & 0.962 \\ IDS18 & ToN & 0.294 & 0.304 & 0.743 & 0.431 & 0.257 & 0.959 \\ IDS18 & NB15 & 0.958 & 0.960 & 0.998 & 0.979 & 0.003 & 0.999 \\ IDS18 & IDS18 & 0.992 & 0.995 & 0.996 & 0.996 & 0.004 & 0.036 \\ \hline All & BoT & 0.962 & 0.080 & 0.930 & 0.148 & 0.071 & 0.038 \\ All & ToN & 0.982 & 0.960 & 0.990 & 0.975 & 0.010 & 0.023 \\ All & NB15 & 0.997 & 0.999 & 0.998 & 0.998 & 0.002 & 0.040 \\ All & IDS18 & 0.989 & 0.994 & 0.993 & 0.994 & 0.007 & 0.045 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Binary classification results—class _Benign_The investigation confirms the influence of the _Benign_ class on the multi-class classification results. The high precision (P) and low false positive rate (FPR) values are present for the same train and test set combinations as in the multi-class case. In particular, the IDS18 and NB15 datasets are very close to one another regarding cross-validated classification results. It implies the similarity of samples from both sets belonging to the _Benign_ traffic class. The ToN dataset performs worse; however, the results of the C class are lower than in the multi-class case. This property may come from differences in the size of the _Benign_ class. The number of samples for IDS18 and NB15 is approximately 2.5 times higher than in the ToN case (see Table 1). Looking, in turn, on the BoT dataset, one may observe dramatically weak results. However, not only qualities make the difference. In the case of NB15 and IDS18, the captured benign traffic consists of the natural transaction data based on standard application protocols, like HTTP, HTTPS, FTP, SSH, and email. In both cases, such a user's normal behavior profile was selected. The BoT data set, however, was constructed to mimic the usage of adversarial behavior of compromised IoT networks as Botnet malware. Therefore the regular traffic is substantially reduced. One may notice that the _Benign_ class network traffic in the BoT consists only of a realistic smart home network, as the five IoT devices. Several instances are relatively small and do not include the normal user's behavior like in the NB15 and IDS18 datasets [6]. The t-SNE visualization of _Benign_ class is shown in Figure 2. The area covered by samples belonging to particular datasets overlaps. Observation of distributions of points in reduced 2D space confirms previous conclusions--areas covered by IDS18 and NB15 are very close to one another--practically, they fully overlap; the ToN dataset only partially overlaps both.