## 5 Experiments
### Harmful Traffic Classes Detection  
The first phase of experiments revealed that the relatively good results that may confirm the generalizability of three of four datasets are mostly driven by the good classification of the _Benign_ class that dominates over other classes and consists of samples that exhibits some intra-dataset resemblance. In the case of the BoT dataset, even multi-class measures do not confirm the generalizability. Figure 2: The t-SNE visualization of benign traffic class with labeled original sets and their share in class samples ((**left**)—all classes in a single chart; (**right**)—separated charts per class). A _Benign_ class in the network traffic dataset is necessary to properly train the ML models to detect the remaining classes referring to harmful traffic, finally allowing them to distinguish between regular (benign) traffic and various dangerous ones. From this point of view, correctly detecting _Benign_ traffic by models trained on datasets other than the one on which the model is tested would eventually allow for separating benign from harmful traffic. One need, however, to consider two additional issues. At first, it is restricted to the NB15 and IDS18 datasets, containing the enterprise network traffic. In the case of the IoT traffic, results are not that optimistic. Secondly, in most cases, the primary goal of NIDS exceeds simple differentiation between benign and harmful classes--the system should also detect the type of danger. In other words, it should correctly classify harmful data samples into one of the pre-defined classes. To investigate this issue, we conducted experiments similar to the one with _Benign_ class binary classification but on classes representing harmful traffic. Such experiments should be conducted on co-occurring classes, i.e., classes in more than one dataset. Looking at Table 1, one may easily observe that the collection of such classes is relatively small. It consists of just four classes: _DoS_, _DDoS_, _Reconnaissance_, and _Backdoor_. All the co-occurring classes are analyzed using the previously mentioned measures. However, following again our previous considerations, in this case, we focus on the Recall measure, which should converge to 1 when the performance of classifiers grows, and the false negative rate (FNR) should converge to 0. The measures obtained for all co-occurring harmful classes are shown in Table 6.  
The only harmful traffic class in all four sets is _Denial of Service (DoS)_. It is one of two principal classes of the BoT dataset, so it is no wonder that the number of samples belonging to this set far exceeds the number of samples in the remainder of the sets. It is more than ten times higher than in the second most frequent class--ToN. This situation resembles the case of the _Benign_ class--some datasets contain more samples of a particular class than others. Nevertheless, contrary to the _Benign_ class case, this does not directly imply that the classifier trained on these sets performs well on others, which may be observed in metrics values. Here, despite high values of precision, recall, and F1 measures and low value of FNR for the case BoT-BoT (train-test), the model trained on BoT and tested on three other datasets perform poorly--the metric values are close to 0, while error value (FNR) is close to 1. It is because samples of the _DoS_ class from different original datasets occupy disjoint regions in the feature space. It is visible on the t-SNE visualization shown in Figure 3 (left). For the same reason, other cross-validated combinations of train-test sets perform poorly. To understand why this happens, one needs to look carefully at the characteristics of the _DoS_ class, which, in general, is differentiated in particular sets. These datasets differ mainly in the types of systems targeted (enterprise networks vs. IoT) and the techniques employed in the attacks. In NB15, the DoS attacks involve traditional network-based denial-of-service attempts targeting system resources, typically from a single host.