# Intelligent Cyber Attack Detection and Classification for Network-Based Intrusion Detection Systems
## 3 Materials and Methods
### Models
\(h_{t-1}\) is the hidden state of previous timestep and \(\theta\) comprises the weights and biases of the network. RNNs' main issue is that they have difficulties in learning long-term relationships between the elements of the input sequence due to the vanishing and exploding gradient problem. LSTM cells were designed to overcome this problem through the use of cell memory and gating units. Therefore, an input \(x_{t}\) at a given timestep \(t\) can change or even override the cell state. This process is carefully regulated by three gating units, the _input gate_, the _forget gate_ and the _output gate_[34]. Figure 4 describes the basic architecture of an LSTM cell. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Epoch & 50 \\ Batch Size & 1024 \\ Optimizer & Adam \\ Learning Rate & 0.001 \\ Objective Loss & Categorical Cross-Entropy \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Summary of MLP configuration. Figure 3: Standard to many RNN in unfold form (influenced by [33; 34]). The _forget gate_ is a _sigmoid_ layer that outputs a result between 0 and 1 for each value of the cell state \(C_{t-1}\). This results determines which information should be erased or kept. It can be mathematically represented as  
\[f_{t}=\sigma(W_{f}\cdot[h_{t-1},x_{t}]+b_{f}) \tag{10}\]  
_The input gate_ determines which values should be updated and the _tanh_ layer creates a vector of candidate values \(\hat{C}_{t}\). The current cell state, \(C_{t}\), results from the addition of the forget computation, obtained by the multiplication of \(C_{t-1}\) and \(f_{t}\), with the scaled candidate values. This process can be represented by the following equations  
\[i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i}) \tag{11}\]  
\[\hat{C}_{t}=tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C}) \tag{12}\]  
\[C_{t}=f_{t}*C_{t-1}+i_{t}*\hat{C}_{t} \tag{13}\]  
The _output gate_ establishes the output information. The _sigmoid_ layer determines what cell status information will be output. The cell state \(C_{t}\) is processed by _hyperbolic tangent_ in order to scale its values between \(-1\) and \(1\). Finally the multiplication between these outputs is the LSTM cell output value. The process can be mathematically defined as  
\[o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \tag{14}\]  
\[h_{t}=o_{t}*tanh(C_{t}) \tag{15}\]  
The employed LSTM network model has one input layer of shape (1, 10), two hidden layers with 100 nodes each, and one output layer with five nodes, one for each class. Changes in the length of the flow window implies alterations in the input shape from (1, 10) to (Window Size, 10). Each hidden layer has Dropout set to 0.2 to prevent the network to overfit the training data. Table 7 describes the employed LSTM network architecture. The LSTM network configurations are the same as the ones configured for the MLP. Categorical Cross-Entropy remains as the loss objective function and an Adam optimizer with a default 0.001 learning rate was used. The number of epochs was set to 50 and batch size to 1024. Table 8 summarizes the main configurations of the LSTM model. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer** & **Size** & **Activation** & **Dropout** \\ \hline Dense & (Window, 10) & - & - \\ Dense & 100 & Tanh & 0.2 \\ Dense & 5 & Softmax & - \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Employed LSTM network architecture. Figure 4: Basic LSTM cell architecture (adapted from [34]).