# Intelligent Cyber Attack Detection and Classification for Network-Based Intrusion Detection Systems
## 3 Materials and Methods
### Data Preprocessing and Sampling  
In order to train the Machine Learning methods, the CIDDS-001 data first had to be preprocessed. The data were first analysed in order to detect errors, duplicated values and inconsistent data. Some abnormalities were found, such as the _Flows_ column, with the same value for each dataset entry and the _Bytes_ columns representation for its numerical values being 1K instead of 1000. Thereby, the _Flows_ column was removed as well as three other columns that correspond to labels not considered in our study, _Class_, _AttackID_ and _AttackDescription_. Only _AttackType_ was considered because this research is focused on the evaluation of different machine- and deep-learning approaches for attack recognition and specification. Some additional transformations were also done to correct the _Bytes_ column, such as replacing \"\(K\)\" for \(10^{3}\) and \"\(M\)\" for \(10^{6}\) and then converting the corresponding result to its correct numeric representation. The _Date first seen_ feature was used to index the data in order to preserve the flow sequence. After these operations seven categorical features and three numerical features remained. The resulting feature vector is composed of the following features: Src IP, Src Port, Dest IP, Dest Port, Proto, Flags, ToS, Duration, Bytes and Packets. Since the input value of the employed algorithms is expected to be a numerical matrix, all non-numerical features, such as _Src IP_ and _Dst IP_, were encoded into a representative numerical form using the ordinal encoding method. Finally, every feature was normalized between 0 and 1 using min-max normalization to enhance the performance of the mentioned techniques. Min-max scales look at each feature individually according to the following equation \begin{table}
\begin{tabular}{c c c} \hline \hline
**N\({}^{o}\)** & **Name** & **Description** \\ \hline
1 & Src IP & Source IP Address \\
2 & Src Port & Source Port \\
3 & Dest IP & Destination IP Address \\
4 & Dest Port & Destination Port \\
5 & Proto & Transport Protocol (e.g., ICMP, TCP, or UDP) \\
6 & Date first seen & Start time flow first seen \\
7 & Duration & Duration of the flow \\
8 & Bytes & Number of transmitted bytes \\
9 & Packets & Number of transmitted packets \\
10 & Flags & OR concatenation of all TCP Flags \\
11 & Tos & Type of Service \\
12 & Flows & Not specified \\
13 & Class & Class label (Normal, Attacker, Victim, Suspicious and Unknown) \\
14 & AttackType & Type of Attack (PortScan, DoS, Bruteforce, PingScan) \\
15 & AttackID & Unique Attack id. Allows attacks which belong to the same class carry the same attack id \\  & & & \\  & & Provides additional information about the set attack parameters (e.g., the number of attempted password guesses for SSH-Brute-Force attacks) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Specification of the CIDDS-001 dataset features. Adapted from [8]. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Open Stack**} & \multicolumn{4}{c}{**External Server**} \\ \hline
**Week** & **DoS** & **Brute-Force** & **Ping Scan** & **Port Scan** & **Dos** & **Brute-Force** & **Ping Scan** & **Port Scan** \\ \hline
1 st & Yes & Yes & Yes & Yes & No & No & No \\
2 nd & Yes & Yes & Yes & Yes & Yes & No & No & Yes \\
3 rd & No & No & No & No & Yes & No & Yes \\
4 th & No & No & No & No & Yes & No & Yes \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Attack distribution over time. \[x^{\prime}=\frac{(x-x_{min})}{(x_{max}-x_{min})} \tag{1}\]  
where \(x^{\prime}\) represents the scaled value of \(x\). As previously stated, the dataset comprises a total of nearly 33 million flows. Due to this considerable number of flows, high hardware requirements are needed to conduct this study. In order to solve this problem and substantially decrease the required amount of memory, processing power and time, only a portion of the data were used.