# Machine Learning for Network Intrusion Detection--A Comparative Study
## 6 Results and Discussion
### Classification Report  
The evaluation results of the models in the form of classification reports are shown in Figure 4. A classification report shows the precision, recall, and F-1 score for every class, as well as the macro-average and weighted average of these metrics across the classes. The overall accuracy of the predictions on the test data is also part of the report. These are defined as follows.  
* Accuracy = \(\frac{TP+TN}{TP+TN+FP+TN}\), the percentage of the total number of correct classifications.
* Precision = \(\frac{TP}{TP+FP}\), the proportion of positives that are correctly identified.
* Recall (sensitivity) = \(\frac{TP}{TP+FN}\), the percentage of actual positives that are classified as attacks.
* F1-score= \(2*\frac{precision*recall}{precision+recall}\), the harmonic mean of the precision and recall.  
Here, TP = true positives, TN = true negatives, FP = false positives, and FN = false negatives. A high F1-score means that the precision and recall are both high, while a low F1-score indicates that one or both of the precision and recall are low. A high precision value means that the model can correctly identify most of the positive cases, while a high recall value means that the model can correctly identify most of the actual positive cases.  
Figure 3: Models’ performance as confusion matrices. The classes are denoted as follows: Benign \(\to\) 0, DoS \(\to\) 1, Port Scan \(\to\) 2, Bot \(\to\) 3, Infiltration \(\to\) 4, Web Attack \(\to\) 5, Brute Force \(\to\) 6.  
Accuracy of RF: 0.9998036242462711 ```
Precision of RF: 0.977980242622729373
```

Recall of RF: 0.998035355765757 ```
F1-score of RF: 0.97419293292042185
```  
precision recall 1-score support  
accuracy  
macro avg 0.98 0.97 0.87 0.87 25280
```
macro avg 0.14 0.14 0.14 1.25280
```
weighted avg 0.71 0.71 0.71 25280  
(a) Random Forest (RF)  
Accuracy of RF: 0.00324246868142027
```
Precision of LR: 0.2408540761341285
```
Recall of LR: 0.2533994287272827
```
F1-score of LR: 0.24095199534008
```
precision recall 1-score support  
1 0.93 0.96 0.94 200928 1 0.83 1.00 0.91 209928 2 0.76 0.81 0.73 23228 2 0.00 0.00 32028 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 5 5 0.00 0.00 0.00 218 6 0.00 0.00 0.00 218 7 0.00 0.00 0.00 923 accuracy 0.00 0.90 252080
```
precision recall 1-score support

1 0.93 0.96 0.94 200928 1 0.83 1.00 0.91 209928 2 0.76 0.81 0.79 23228 2 0.00 0.00 0.00 32028 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 5 6 0.00 0.00 0.00 0.00 218 7 0.00 0.00 0.00 923 7 0.00 0.00 0.00 923 accuracy 0.00 0.90 252080
```  
Once again, using the classification reports, the RF model clearly emerges as the best-performing model, with a macro-average F1-score of 97%. The macro-average is calculated by taking the average of the F1-scores of each class. Therefore, it is a good measure to express the model's performance in a single number. As can be seen by examining the classification reports, for the other models, the macro-average is significantly lower than that of the RF model, indicating the superiority of the RF model in predicting different attacks.