# Enhancing Machine Learning Prediction in Cybersecurity Using Dynamic Feature Selector
## 5 Experimental Analysis
### Meta-Learner
#### 5.7.1 CNN-LSTM  
The hybrid of CNN and LSTM was developed in two steps namely, feature extraction based on CNN followed by a feature fusion part based on LSTM. This specific hybrid approach proved effective on NSL-KDD dataset without any feature reduction [54]. CNN is used since it has a proven track record for accurate classification in several domains such as image classification and prediction on multivariate datasets. Using LSTM modules enabled us to incorporate Recurrent Neural Network (RNN). An RNN is able to perform sequence modelling. Unlike a CNN, an RNN can help to create interaction between input sequences. To predict the next element of a series or a batch, RNN uses the previous features or elements thereby creating a loop which helps to keep track of this information [61, 62, 63, 64, 65]. This makes RNN extremely suitable for finding patterns in time series datasets [66], especially in the case of network intrusions since the datasets have multiple features like _sessions_.  
The first part of this approach is feature extraction based on CNN. This is followed by a feature fusion part based on LSTM as shown in Figure 3. In the first stage, the forward propagation process is applied as it assumes the \(l\) layer is a convolution layer and the \(l-1\) layer is a pooling layer or another input layer for the next extraction process. The entire process can be represented using Equation (3). The \(f\) here denotes the activation function, \(b\) represents the offset, \(k\) represents the convolution kernel, \(M_{j}\) represents the input feature map from the previous layer and variable \(x_{ji}\) represents the \(j\)th feature map of the \(l\)th layer. The summation represents the total result obtained from a convolution operation of all the feature maps in \(l-1\) layer and the \(j\)th convolutional kernel of layer \(l\).  
\[x_{ji}=f(\sum i\epsilon M_{j}x_{il-1}\times k_{ijl}+b_{ji}). \tag{3}\]  
In a CNN, information may get lost as the network gets deeper due to the vanishing gradient problem. This is referred to as having a short term memory. LSTM modules is comprised of gates and a cell state. The cell state can be designated as a highway of information where features from a certain layer can be directly passed down several layers without any modification. The gates behave like neural networks and are responsible for knowledge discovery during training. These two features overcome the drawbacks of short term memory found in a CNN.  
In the feature extraction stages, the sequential CNN model was comprised to 4 convolution layers with a max-pooling layer of size 2 placed after every 2 convolution layers. Relu was used as  
Figure 3: Illustration of the Hybrid Convolutional Neural Network (CNN) and Long Short term Memory (LSTM) architecture.  
an activation function for both convolutions. For the first convolution and pooling layer, there were 48 convolutional kernels with with a size of 3. The feature fusion consisting of LSTM was used where the output size of the LSTM part as 100. Finally, the classification results of the attack types were obtained through the Softmax function for better stochastic gradient descent. Number of epochs was set to 10. To prevent over-fitting issue, we used dropout as 0.1.