## 4 Methodology
### Quality Measures  
Our research investigates the behavior of classifiers working with all classes and single ones. The latter case refers to binary classification, where the selected class is confronted with the remainder. Consequently, the performance of classifiers should be measured using overall measures considering all classes (multiclass case) and using per-class ones (binary case). The usual per-class measures are accuracy, recall (sensitivity, detection rate), precision, and F1, defined as, respectively:  
\[A_{c}=\frac{TP_{c}+TN_{c}}{TP_{c}+TN_{c}+FP_{c}+FN_{c}}=\frac{TP_{c}+TN_{c}}{n}, \tag{1}\]  
\[R_{c}=\frac{TP_{c}}{TP_{c}+FN_{c}}\,\ P_{c}=\frac{TP_{c}}{TP_{c}+FP_{c}}\,\ F1_{c}=\frac{2 \cdot P_{c}\cdot R_{c}}{P_{c}+R_{c}}, \tag{2}\]  
where lower index \(c\) stands for the class for which the measure is computed, \(TP\), \(FP\), \(TN\), \(FN\) for numbers of true positive, false positive, true negative, and false negatives samples, respectively, and \(n\) stands for the total number of samples.  
Apart from the above measures, there are other measures that focus on classification errors. Two of them, useful when measuring the performance of ML-based NIDS, are false negative rate (miss rate) and false positive rate (false alarm rate), which are defined respectively as:  
\[FNR_{c}=\frac{FN_{c}}{FN_{c}+TP_{c}}\,\ FPR_{c}=\frac{FP_{c}}{FP_{c}+TN_{c}}. \tag{3}\]  
An important question concerning the various measures is: Which is the best choice to properly validate the classifier for NIDS? To answer this question, one must first define some principles the classifier used in the NIDS must follow and apply them in the correct order. There are, in fact, two such principles. At first, the classifier must provide the highest possible level of protection, which means that as many harmful traffic samples as possible must be correctly classified, even if some benign ones are misclassified as harmful. Having met this requirement, the second principle should be fulfilled--one needs to take care of false alarms, i.e., the number of benign traffic samples misclassified as harmful should be low. The first principle directly implies that, in the case of classes of harmful traffic, the \(FN\) must be as low as possible. Only in the second place should the \(FP\) be taken into account. However, the order should be inverted in the _Benign_ traffic class case. In this case, the number of \(FP\) should be as small as possible to fulfill the first requirement. It is because the sample incorrectly classified as _Benign_ is, in fact, a harmful one that cannot be accepted. Only in the second place should \(FN\) be taken into account. Considering the above, one must differentiate measures to validate harmful and benign classes. In the first case, recall \(R_{c}\) and \(FNR_{c}\) are more suitable based on \(FN_{c}\). In the second case, the benign traffic, one should use precision \(P_{c}\) and \(FPR_{c}\) based on \(FP_{c}\).  
Only the accuracy measure is equivalent to the binary case in the multi-class case. It is computed as the ratio of correct classifications to the total number of samples. In the case of other measures, the binary classifiers must be evaluated before computing the final score regarding precision and recall. In our experiments, we applied weighted measures, defined as:  
\[M=\sum_{c\in Classes}\frac{n_{c}}{n}M_{c}, \tag{4}\]  
where \(M_{c}\) stands for measure of class \(c\), which can either be precision (\(P_{c}\)) or recall (\(R_{c}\)), \(n_{c}\) for the number of samples in class \(c\) and \(n\) is the total number of samples.