# A Local Feature Engineering Strategy to Improve Network Anomaly Detection +
## 6 Strategy Validation
### Dataset  
The main real-world dataset we used in the validation process is the NSL-KDD dataset ([[https://github.com/defcom17/NSL_KDD](https://github.com/defcom17/NSL_KDD)]([https://github.com/defcom17/NSL_KDD](https://github.com/defcom17/NSL_KDD))). It essentially represents an improved version of the KDD-CUP99 dataset widely used in the past, which suffered from some issues [101]. In more detail, the main improvements of the NSL-KDD dataset, with regard to the original KDD version, are the following:  
* The redundant records in the training set have been removed to avoid that a classifier can be biased by the frequency of them. * The duplicate records in the test sets have been removed to avoid that a classifier performance can be biased when the better detection rate of it is related to these records. * The new number of records in the training and test sets allows a classifier to perform the validation process without the need to select small portions of them, randomly. The NSL-KDD dataset was selected since it is considered as a benchmark in the intrusion detection research area, allowing the researchers to compare their solutions performance with an ever-increasing number of works in this field, unlike other recent datasets. Moreover, we chose this dataset for the network scenario it provides, as we aimed to assess our performance in the context of canonical networks. The NSL-KDD dataset involves events related to the UDP, TCP, and ICMP protocols, allowing to validate approaches and strategies in a real-world network scenario, in terms of both number of events and involved protocols and attacks. We adopted this dataset to perform an independent experimental evaluation of the proposed strategy by intrusion typology, separately. On the other hand, in Section 7.1, we take into account two additional datasets, with the aim to evaluate such a strategy in heterogeneous scenarios (i.e., which include all types of attacks). The numerical relevance of the NSL-KDD dataset in terms of normal and intrusion events (respectively, \(E_{+}\) and \(E_{-}\)) is reported in Table 6. Note that the original dataset consists of two different files, i.e. KDDTrain+ and KDDTest+, respectively, related to the original training and test sets, as defined by the dataset authors. However, since some normal/intrusion unique events can be found in one file only, we discarded this original separation, and joined the two files as a unique full dataset for the rest of this section, thus only considering the global values IN Table 6 (here, the original distinction between KDDTrain+ and KDDTest+ is reported for information purposes). The distribution of the network events is provided in Table 7, while their classification in the five classes (i.e., PEA, DSA, RSA, RAA, and NNA) is detailed in Table 8. Finally, Table 9 presents a numerical overview of the aforementioned classes of dataset events. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Dataset** & **Total Events** & **Normal** & **Intrusion** & **Features** & **Unique** \\  & \(|E|+|\hat{E}|\) & \(|E_{+}|\) & \(|E_{-}|\) & \(|F|\) & **Events** \\ \hline \(KDDTrain+\) & \(125,973\) & \(67,343\) & \(58,630\) & \(41\) & \(23\) \\ \(KDDTest+\) & \(22,543\) & \(9,710\) & \(12,833\) & \(41\) & \(38\) \\ \hline \(Global\) & \(148,516\) & \(77,053\) & \(71,463\) & & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Numerical relevance of the original dataset events.