# Intelligent Cyber Attack Detection and Classification for Network-Based Intrusion Detection Systems
## 3 Materials and Methods
### Evaluation Metrics  
Several metrics, such as Accuracy, Recall, Precision and F1-Score, can be used to evaluate the performance of a given classifier [35]. However, these metrics' employment should not be made carelessly, particularly in intrusion detection systems, where high-class imbalance situations are relatively common. This section will briefly describe the metrics used in this research, their mathematical representation [36], and how they can be interpreted to better understand the results obtained from applying the previously enumerated methods.  
Accuracy is one of the most common metrics used in classification problems and usually gives a reliable measure of the model's performance. It can be calculated through the following equation [28]  
\[accuracy(y,\hat{y})=\frac{1}{n_{samples}}\sum_{i=1}^{n_{samples}}I(y_{i}=\hat{y }_{i}) \tag{16}\]  
where \(n_{samples}\) is the number of samples of a given dataset, \(\hat{y}=\hat{y}_{1},\hat{y}_{2},...,\hat{y}_{n-1},\hat{y}_{n}\) is the vector of predicted values, \(y=y_{1},y_{2},...,y_{n-1},y_{n}\) is the corresponding vector of true values and \(I\) is the indicator function, which returns 1 if \(y_{i}\) matches \(\hat{y}_{i}\) and 0 otherwise.  
In this particular case, due to the fact that CIDDS-001 is a very unbalanced dataset, accuracy is not a good metric to use. In the selected sample, 82.53% of the flows are benign traffic. Even if the classifier fails to classify every other attack class and only predicts the benign flows correctly, it would still achieve an accuracy value of 82.53%. When working with unbalanced data, the accuracy is biased towards the majority class. To overcome this situation, other metrics were considered.  
In terms of the number of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN) reported by the confusion matrix, the macro-averaged precision can be expressed as follows  
\[P_{macro}=\frac{1}{n_{classes}}\sum_{i=1}^{n_{classes}}\frac{TP_{i}}{TP_{i}+ FP_{i}} \tag{17}\]  
Precision measures the number of labels a model has incorrectly predicted to be positive that were actually negative. \(TP_{i}\) is the number of flows correctly classified as class  
Figure 7: Anomaly Detection Approach.  
\(c_{i}\) and \(FP_{i}\) is the amount of flows whose true value corresponds to a given class \(c_{j}\), where \(j\in[1,n_{classes}]\wedge j\
eq i\) that were incorrectly labeled as \(c_{i}\).  
On the other hand, recall expresses the ability of a model to find relevant instances in a dataset. Macro-averaged recall can be defined as  
\[R_{macro}=\frac{1}{n_{classes}}\sum_{i=1}^{n_{classes}}\frac{TP_{i}}{TP_{i}+ FN_{i}} \tag{18}\]  
where \(FN_{i}\) stands for the number of flows of a class \(c_{i}\) that were incorrectly labeled as a class \(c_{j}\).  
F1-score is a metric that considers both precision and recall through the computation of their harmonic mean. The macro-averaged f1-score is well suited for unbalanced datasets such as CIDDS-001, and it can be mathematically described as  
\[F1_{macro}=2\frac{p_{macro}\cdot R_{macro}}{P_{macro}+R_{macro}} \tag{19}\]  
False Positive Rate (FPR), or Fall-Out, expresses the probability of false alarm and can be determined by the following mathematical statement  
\[FPR_{macro}=\frac{1}{n_{classes}}\sum_{i=1}^{n_{classes}}\frac{FP_{i}}{FP_{i} +TN_{i}} \tag{20}\]  
In this research, results are expressed through all the above measures. Understanding each evaluation metric and each employed model's nature is crucial to making sense of the obtained results.