# Machine Learning-Based Adaptive Synthetic Sampling Technique for Intrusion Detection
## 3 Material & Methods
### Model Architecture  
Two different model architectures were trained to predict the sorts of assaults for the three sub-problems using this dataset. An MLP method was used in the first architecture. MLPs are a type of feed-forward neural network made up of several layers of interconnected nodes (neurons). They are well-known for capturing complex non-linear relationships between features and target variables. MLPs are widely used for a variety of machine learning tasks, including classification problems such as the one investigated in this study. As the initial model, the authors chose an MLP architecture to evaluate its performance in predicting the types of assaults in the given dataset.  
A hybrid model combining CNN-1D and LSTM was used in the second architecture. This hybrid model aimed to improve performance by combining the strengths of CNN-1D and LSTM. CNN-1D is a convolutional neural network that is designed to process one-dimensional sequences such as time series or sequential data. It excels at capturing local patterns and identifying relevant data features. The authors hoped to extract meaningful features from the input data by incorporating a CNN-1D component into the model.  
The hybrid model utilized LSTM, a type of RNN, to address the vanishing gradient problem that traditional RNNs can encounter. The problem arises when the gradients of the error become too small while backpropagating through time, making it difficult for the network to learn long-term dependencies. LSTM networks overcome this issue by incorporating memory cells that can retain information over extended periods. This approach enables the hybrid model to effectively remember and utilize prior inputs to capture temporal dependencies in the data. Using CNN-1D and LSTM in combination in the hybrid model allows for local feature extraction and capture of long-term dependencies  
Figure 8: Feature importance scores.  
in the sequential data. This architecture was selected to enhance the model's ability to learn and represent complex patterns present in the dataset, ultimately improving the prediction performance for various types of assaults.  
As shown in Figure 9, the initial neural network design utilized for training was an MLP. There were 38 inputs for the features in the input layer, 64 units in the dense layer, and the ReLU activation function. Then, a dropout layer with a ratio of 0.4 was implemented to avoid overfitting by randomly discarding nodes during training. A thick layer of 128 units followed the ReLU activation function in the second hidden layer. Next, a dropout layer with a ratio of 0.4 followed. Next, a dropout layer of 0.4 was placed after a ReLU activation function, and 512 units made up the third hidden layer. Finally, a ReLU activation layer and a dropout layer with a unit count of 0.4 came after the fourth layer, which had 128 units. Following from the model type, either binary, four-category, or five-category, the output layer was a dense layer with one, four, or five units.  
The second neural network design used for training was a hybrid model that combined CNN-1D and LSTM, as shown in Figure 10. The input layer consisted of 38 inputs for the features, and the next layer was a CNN-1D with 64 filters, a kernel size of 9, and a ReLU activation function. After that, there was a max-pooling layer with a pool size of two. Applying a dropout layer with a dropout rate of 0.4 was the next step, to prevent overfitting caused by randomly removing nodes during training. The second hidden layer was a CNN-1D layer with 64 filters and a kernel size of 6, followed by the ReLU activation function. Then, a max-pooling layer with a pool size of 2 and a dropout layer of 0.3 was added. Then, an LSTM layer with 64 units and a dropout layer of 0.2 made up the third hidden layer. Finally, the output layer was dense, with one, four, or five units, depending on the model type--binary, four-, or five-category.  
Figure 9: Architecture of the MLP model.