# Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey
## 3 Common Machine Learning Algorithms in IDS
### Machine Learning Models
#### 3.1.1 Shallow Models  
The traditional machine learning models (shallow models) for IDS primarily include the artificial neural network (ANN), support vector machine (SVM), K-nearest neighbor (KNN), naive Bayes, logistic regression (LR), decision tree, clustering, and combined and hybrid methods. Some of these methods have been studied for several decades, and their methodology is mature. They focus not only on the detection effect but also on practical problems, e.g., detection efficiency and data management. The pros and cons of various shallow models are shown in Table 3.  
**Artificial Neural Network (ANN)**. The design idea of an ANN is to mimic the way human brains work. An ANN contains an input layer, several hidden layers, and an output layer. The units in adjacent layers are fully connected. An ANN contains a huge number of units and can theoretically approximate arbitrary functions; hence, it has strong fitting ability, especially for nonlinear functions. Due to the complex model structure, training ANNs is time-consuming. It is noteworthy that ANN models are trained by the backpropagation algorithm that cannot be used to train deep networks. Thus, an ANN belongs to shallow models and differs from the deep learning models discussed in Section 3.1.2. **Support Vector Machine (SVM)**. The strategy in SVMs is to find a max-margin separation hyperplane in the n-dimension feature space. SVMs can achieve gratifying results even with small-scale training sets because the separation hyperplane is determined only by a small number of support vectors. However, SVMs are sensitive to noise near the hyperplane. SVMs are able to solve linear \begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Algorithms** & **Advantages** & **Disadvantages** & **Improvement Measures** \\ \hline \multirow{3}{*}{**ANN**} & Able to deal with nonlinear data; Strong fitting ability &
\begin{tabular}{l} Apt to overfitting; prone to become stuck in a local \\ optimum; Model training is \\ time consuming \\ \end{tabular} & Adopted improved optimizers, activation functions, and loss functions \\ \hline \multirow{3}{*}{**SVM**} & Learn useful information from small train set; Strong generation capability & Do not perform well on big data or multiple classification tasks; Sensitive to kernel function parameters & Optimized parameters by particle swarm optimization (PSO)[8] \\ \hline \multirow{3}{*}{**KNN**} & Apply to massive data; Suitable to nonlinear data; Train quickly; Robust to noise & Low accuracy on the minority class; Long test times; Sensitive to the parameter \(K\) & Reduced comparison times by trigonometric inequality; Optimized parameters by particle swarm optimization (PSO) [9]; Balanced datasets using the symbiotic minority oversampling technique (SMOTE) [10] \\ \hline \multirow{3}{*}{**Naïve Bayes**} & Robust to noise; Able to learn incrementally & Do not perform well on attribute-related data & Imported latent variables to relax the independent assumption [11] \\ \hline \multirow{3}{*}{**LR**} & Simple, can be trained rapidly; Automatically scale features & Do not perform well on nonlinear data; Apt to overfitting & Imported regularization to avoid overfitting [12] \\ \hline \multirow{3}{*}{**Decision tree**} & Automatically select features; Strong interpretation & Classification result trends to majority class; Ignore the correlation of data & Balanced datasets with SMOTE; Introduced latent variables \\ \hline \multirow{3}{*}{**K-means**} & Simple, can be trained rapidly; Strong scalability; Can fit to big data & Do not perform well on nonconvex data; Sensitive to initialization; Sensitive to the parameter \(K\) & Improved initialization method [13] \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The pros and cons of various shallow models. problems well. For nonlinear data, kernel functions are usually used. A kernel function maps the original space into a new space so that the original nonlinear data can be separated.