# Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey
## 3 Common Machine Learning Algorithms in IDS
### Machine Learning Models
#### 3.1.2 Deep Learning Models  
Deep learning models consist of diverse deep networks. Among them, deep brief networks (DBNs), deep neural networks (DNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs) are supervised learning models, while autoencoders, restricted Boltzmann machines (RBMs), and generative adversarial networks (GANs) are unsupervised learning models. The number of studies of deep learning-based IDSs has increased rapidly from 2015 to the present. Deep learning models directly learn feature representations from the original data, such as images and texts, without requiring manual feature engineering. Thus, deep learning methods can execute in an end-to-end manner. For large datasets, deep learning methods have a significant advantage over shallow models. In the study of deep learning, the main emphases are network architecture, hyperparameter selection, and optimization strategy. A comparison of various deep learning models is shown in Table 4.  
**Autoencoder**. An autoencoder contains two symmetrical components, an encoder and a decoder, as shown in Figure 3. The encoder extracts features from raw data, and the decoder reconstructs the data from the extracted features. During training, the divergence between the input of the encoder and the output of the decoder is gradually reduced. When the decoder succeeds in reconstructing the data via the extracted features, it means that the features extracted by the encoder represent the essence of the data. It is important to note that this entire process requires no supervised information. Many famous autoencoder variants exist, such as denoising autoencoders [14; 15] and sparse autoencoders [16]. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Algorithms** & **Suitable Data Types** & **Supervised or Unsupervised** & **Functions** \\ \hline
**Autoencoder** & Raw data; Feature vectors & Unsupervised & Feature extraction; Feature reduction; Denoising \\ \hline
**RBM** & Feature vectors & Unsupervised & Feature extraction; Feature reduction; Denoising \\ \hline
**DBN** & Feature vectors & Supervised & Feature extraction; Classification \\ \hline
**DNN** & Feature vectors & Supervised & Feature extraction; Classification \\ \hline
**CNN** & Raw data; Feature vectors; Matrices & Supervised & Feature extraction; Classification \\ \hline
**RNN** & Raw data; Feature vectors; Sequence data & Supervised & Feature extraction; Classification \\ \hline
**GAN** & Raw data; Feature vectors & Unsupervised & Data augmentation; Adversarial training \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of various deep learning models  
**Restricted Boltzmann Machine (RBM)**. An RBM is a randomized neural network in which units obey the Boltzmann distribution. An RBM is composed of a visible layer and a hidden layer. The units in the same layer are not connected; however, the units in different layers are fully connected, as shown in Figure 4. where \(v_{i}\) is a visible layer, and \(h_{i}\) is a hidden layer. RBMs do not distinguish between the forward and backward directions; thus, the weights in both directions are the same. RBMs are unsupervised learning models trained by the contrastive divergence algorithm [17], and they are usually applied for feature extraction or denoising. **Deep Brief Network (DBN)**. A DBN consists of several RBM layers and a softmax classification layer, as shown in Figure 5. Training a DBN involves two stages: unsupervised pretraining and supervised fine-tuning [18; 19]. First, each RBM is trained using greedy layer-wise pretraining. Then, the weight of the softmax layer are learned by labeled data. In attack detection, DBNs are used for both feature extraction and classification [20; 21; 22]. **Deep Neural Network (DNN)**. A layer-wise pretraining and fine-tuning strategy makes it possible to construct DNNs with multiple layers, as shown in Figure 6. When training a DNN,  
Figure 4: The structure of the RBM. Figure 5: The structure of the DBN.