## 4 Methodology
### Feature Engineering (FE)  
Information from network traffic usually comes in the shape of huge (binary) files containing highly heterogeneous information. This issue makes it impossible to directly apply ML techniques to identify the different kinds of attacks. However, it is possible to overcome this issue by processing the data in order to build a more suitable input for automatic classifiers. In this sense, the application of some feature engineering technique is proposed to build a well-structured dataset, suitable for ML-based NIDSs. Thus, Feature as a Counter (FaaC) [32] is used as a functional solution to the problem of learning from large heterogeneous datasets. It consists of  
Figure 1: Diagram of the methodology followed in this work. merging and transforming different data sources (structured or not) of information and their variables into new variables, which are just counters of the original ones, in a given interval of time. For instance, it could be interesting to count the number of accesses to port 22 in a given time lapse, because a high number might mean a brute force SSH attack. The FaaC approach has been successfully tested in unsupervised ML-based NIDS approaches [12; 29; 33]. In this work, authors make use of FaaC to transform the TEST part of UGR'16 dataset, a structured data source with \(N\sim 4000M\) observations of \(P=11\) features of Netflow traces, into \(N^{\prime}\sim 47,000\) observations of \(P^{\prime}=132\) new features that extend the original feature space. Table 3 shows the new variables. According to [29], a time-lapse of 1 min is enough for anomaly detection in network communications. Note that the application of this technique leads to a reduction in the number of observations, as can be seen in Table 4 and Figure 2. In that figure, an example of how the FaaC approach works is shown. For each minute time interval, FaaC transforms raw samples (traffic flows) falling into that corresponding time slot (ts) into new observations which will comprise valuable information from the original variables in the form of counters. For example, the protocol raw variable is transformed in two new ones: protocol_tcp and protocol_udp. These new variables counts how many times UDP and TCP protocol, highlighted in bold green and red in the figure respectively, are seen in a minute. In the case of packets raw variable we binning it into what we consider a _low, very low, medium, high_ or _very high_ value according to its distribution. It allows transforming the original variable in five new ones which are npackets_verylow, npackets_low, npackets_medium, npackets_high and npackets_veryhigh respectively. Figure 2 show an example of this transformation where several packets in the interval \([0,4)\) are considered _very low_ while those in the interval \([4,21]\) are _low_. Same approach is used for the bytes variable but with different magnitude. Of course, such intervals can be customized on demand. Regarding the class identifiers, authors will also create new variables containing the number of times every class has happened during the considered time interval. Therefore, it may happen to have situations in which more than one class took place in the same observation, a new issue not present in the original dataset. This fact is illustrated in Figure 2 where observation output-20160727t1343 comprises three different class labels: background, dos and nerisbotnet. In that case, the most \begin{table}
\begin{tabular}{c c c} \hline \hline
**Class** & **\# of Observations** & **\%** \\ \hline Background & \(30,091\) & 63.65 \\ Botnet & \(594\) & 1.26 \\ DoS & \(417\) & 0.88 \\ SSH scan & \(176\) & 0.06 \\ Scan & \(27\) & 0.37 \\ Spam & \(15,961\) & 33.76 \\ UDP scan & \(9\) & 0.02 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Number of observations per class of the derived dataset after FaaC application.