## 2 Theoretical Background and Related Works  
As illustrated in Figure 1, a typical SIDS has three major components: (1) Decoder and pre-processing, which is responsible for collecting the information (packet, log file, etc.), decoding it and carrying out the pre-processing required by the detection engine; (2) detection engine, which is responsible for the discrimination between normal event and an intruder event. To this end, it uses matching with a set of signatures of known attack or intrusion patterns; (3) alert and logging, this component releases an alert if an event is considered as an intrusion. Optionally, some SIDS include output modules to process the alarm and include further information in the output log file.  
The performance of SIDS can be evaluated by keeping track of the number of events (e.g., log event or traffic packet) correctly and incorrectly classified. There are four basic scores that determine the content of the confusion matrix:  
* True Positives (_TP_): The number of attack events classified as attack.
* True Negatives (_TN_): The number of normal events classified as normal.
* False Positives (_FP_): The number of normal events classified as attack.
* False Negatives (_FN_): The number of attack events classified as normal.  
In this paper, we use two metrics derived from the confusion matrix: Recall, and Precision. The Recall (R)--or True Positive Rate (TPR), Sensitivity or Detection Rate [7]--indicates the proportion of actual positives correctly classified and can be calculated using the following equation:  
\[R=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FN}} \tag{1}\]  
Figure 1: Main components of a typical SIDS.  
The Precision (P)--or Positive Predicted Value (PPV) [7]--indicates the proportion of predicted positive that is truly positive. Precision is defined as the ratio of correctly predicted attacks to the predicted size of the attacks and can be calculated using the following equation:  
\[P=\frac{TP}{TP+FP} \tag{2}\]  
There is an inverse relationship between \(R\) and \(P\). When signatures define generic attack patterns we will obtain high recall but poor precision. On the contrary, very specific attack patterns will not match similar attacks so we will obtain high recall but poor precision. As such, there is a trade-off between these two factors that requires careful consideration of the Receiver Operating Characteristic (ROC) curve. It is worth highlighting that SIDS deals with a strongly unbalanced classification problem since the share of attacks in the events analyzed is very low by nature. This implies that \(FP\) is the dominant factor in the optimal SIDS operation point since even a low \(FP\) rate can generate more alerts in practice than scarce real attacks.  
To evaluate and compare SIDS performance, public datasets--which are usually artificially generated [7]--are used. In those datasets, each event is marked as attack or normal, so we can obtain all the above-mentioned metrics. Nevertheless, when using a real-life dataset, with millions of events, manual classification is not feasible since we do not have a priori knowledge, and expert-driven revision of alerts should be performed in order to classify each alert as \(TP\) or \(FP\). In real-life cases with millions of events, the computation of recall can be very difficult because the classification of events that do not trigger alerts should be carried out by an expert in order to determine \(TN\) or \(FN\).