# Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey
## 5 Challenges and Future Directions
However, to reduce harm as much as possible, IDSs need to detect attacks in real time. Thus, a trade-off exists between effect and efficiency. Parallel computing [66; 67] approaches using GPUs [68; 48; 69] are common solutions. From summarizing the recent studies, we can conclude that the major trends of IDS research lie in the following aspects. **(1) Utilizing domain knowledge**. Combining domain knowledge with machine learning can improve the detection effect, especially when the goal is to recognize specific types of attacks in specific application scenarios. * The rule-based detection methods have low false alarm rates but high missed alarm rates include considerable expert knowledge. In contrast, the machine learning methods usually have high false alarm rates and low missed alarm rates. The advantages of both methods are complementary. Combining machine learning methods with rule-based systems, such as Snort [70; 71; 72; 73], can result in IDSs with low false alarm rates and low missed alarm rates. * For specific types of attacks, such as DOS [74; 75; 76; 77; 78; 79], botnet [80], and phishing web [81], proper feature must be extracted according to the attack characteristics that can be abstracted using domain knowledge. * For specific application scenarios, such as cloud computing [82; 83], IoT [84; 85; 86], and smart grids [87; 88], domain knowledge can be used to provide the environmental characteristics that are helpful in data collection and data preprocessing. **(2) Improving machine learning algorithms**. Improvements in machine learning algorithms are the main means to enhance the detection effect. Thus, studies involving deep learning and unsupervised learning methods has an increasing trend. * Compared with shallow models, deep learning methods learn features directly from raw data, and their fitting ability is stronger. Deep learning models with deep structures can be used for classification, feature extraction, feature reduction, data denoising, and data augmentation tasks. Thus, deep learning methods can improve IDSs from many aspects. * Unsupervised learning methods require no labeled data; thus they can be used even when a dataset shortage exists. The usual approach involves dividing data using an unsupervised learning model, manually labeling the clusters, and then training a classification model with supervised learning [89; 90; 91; 92]. **(3) Developing practical models**. Practical IDSs not only need to have high detection accuracy but also high runtime efficiency and interpretability. * In attack detection, the real-time requirement is essential. Thus, one research direction is to improve the efficiency of machine learning models. Reducing the time required for data collection and storage is also of concern. * Interpretability is important for practical IDSs. Many machine learning models, especially deep learning models, are black boxes. These models report only the detection results and have no interpretable basis [93]. However, every cyber security decision should be made cautiously. An output result with no identifiable reason is not convincing. Thus, an IDS with high accuracy, high efficiency and interpretability is more practical.