# Intelligent Cyber Attack Detection and Classification for Network-Based Intrusion Detection Systems
## 4 Results and Discussion
### Multi-Flow  
From the multi-flow viewpoint, the CIDDS-001 preprocessed data were transformed as multiple sequences, and several window sizes from 0 to 70 were used. Figure 9 describes the evolution of the f1-score for each model as the window size increases.  
The LSTM's f1-score increases alongside the window size while the corresponding score for the remaining models consistently decreases. Despite this trend, the RF model for a window of 10 flows presented an f1-score of 89.82. This result is considerably close to the best recorded value, 91.66, exhibited by the LSTM for the maximum window size.  
A possible explanation for the decrease in the f1-score value for the RF model after reaching such a relevant result and the consistent increase in the values presented by the LSTM can be related to the nature of these techniques. While the LSTM model has a cell state carefully regulated by gates that allow it to learn only meaningful short-term relationships across a given sequence of flows (independently of its size), the RF model was not designed to deal with sequential data. With the increase in window size, the feature space of each entry for the RF substantially increases as well. Its inability to learn sequence-based relationships from data can potentially lead to worse results with the  
Figure 8: Single-Flow f1-score values of the LSTM, RF and MLP models for the Brute Force, DoS, No Attack, Ping Scan and Port Scan classes.  
Regarding the RF, for the 10 flow window, the precision value increased from 79.43% to 96.83% while the recall decreased from 95.68% to 85.65%. These results contributed to the better f1-score result of the RF, 89.82%. As the window size increased, both precision and recall values became smaller, resulting in consistently lower f1-scores until reaching the value of 65.40% for a window size of 70. On the other hand, in the LSTM, for a window of 10 flows, precision dropped from 98.37 to 89.74 and recall increased from 71.40 to 80.21. This trade-off resulted in a better balance between the values of each metric, which translated into a greater f1-score value. Although casual decreases have occurred for some sequence lengths, both precision and recall were consistently better for further window sizes.  
Macro-averaged f1-score can also be detailed by understanding how the individual score for each class evolved. Figure 11 expresses the RF and LSTM model's behaviour for the minority classes.  
The most prevalent classes were not represented because there were no significant changes in their values. For the RF, at window size 10, the f1-score for the Brute Force class reaches its peak at 91.01%, explaining the increase in the overall f1-score of the model. For larger window sizes, the f1-score of all classes, Brute Force, Ping Scan and Port Scan, rapidly decreased. Concerning the LSTM, for the DoS class, the score remained at 99.99, for attack absence at 99.77, and for Port Scan it increased from 98.48 to 99.12. Regarding the minority classes, despite the considerable drop in Brute Force's f1-score for the 10 flow window, it steadily increased from 70.84 to 71.76 for further sizes. On the other hand, the Ping Scan class presented the most relevant increase, from 01.86 to 87.46.  
The presented results can lead to the interpretation that the LSTM is able to detect consistently more attacks with less false-positive errors as the flow sequences grow larger. This general improvement does not seem to imply favoring one class performance over another, as the f1-score of every class either remained the same or increased.  
To ensure that the model has been trained properly, the learning curves, represented in Figure 12, were carefully studied.  
The loss value decreases over each epoch for both training and validation sets without any evidence of the occurrence of underfitting or overfitting. The accuracy score for both sets is also quite similar and it slowly improves as loss value is minimized.  
Figure 11: F1-score changes for the minority classes for the RF and LSTM models.