# A Survey of CNN-Based Network Intrusion Detection
## 8 Empirical Experiment  
As we mentioned above, various datasets are used by researchers to validate their proposed CNN-based IDS. Further, they may use different configurations and hyperparameters. Thus, the comparison of different CNN-based IDSs is not feasible based on their experimental results. To overcome this challenge, we innovatively set up a comprehensive experiment to investigate the use of different CNN schemes on well-known and standard datasets. According to the Analytical Investigation section, two types of datasets, KDD and CIC-IDS 2017, are mainly used in the literature. The KDD dataset consists of KDD Cup99 and NSL-KDD versions, with 14.9% usage and 33.0% usage, respectively, according to Figure 6. Thus, we selected NSL-KDD as the first standard benchmark. CIC-IDS also includes the CIC-IDS 2017 and CIC-IDS 2018 Versions, where Version 2017 is more frequently used than the other. Therefore, we considered CIC-IDS 2017 as the second standard benchmark.  
By investigating the literature, we can conclude that effective CNN schemes for IDSs are mostly constructed from the combination of CNN and other neural network structures, such as dense, LSTM, and autoencoder. Thus, in this experiment, the original CNN and hybrid CNN schemes for IDSs were tested on the two standard benchmarks. Based on the type of layers and topology, five important configurations with nine variations were implemented as follows:  
1D-CNN: Using only 1D convolutional layers followed by dense layers is the basic configuration of the CNN-based IDS [54; 62; 65; 66]. The number of CNN layers in this configuration is a hyperparameter. According to Figure 8, most IDS schemes use one, two, or three layers. Thus, in this experiment, we implemented three 1D-CNN models with one to three CNN layers.  
1D-CNN: Based on the research in [28; 38; 53; 55; 56; 63; 66], this configuration uses 2D convolutional layers followed by dense layers. Like the previous configuration, we implemented three models with one to three CNN layers.  
1 LSTM-1 DCNN: Based on the research in [72; 74; 75; 77; 79; 82], in this scheme, the input is considered to be a time series and processed by LSTM layers; then, the LSTM outputs are delivered to the 1D-CNN layers, and finally, the dense layers are used as the last layers.  
1 LSTM-2 DCNN: Similar to the previous configuration, this one uses LSTM, CNN, and dense layers sequentially, but instead of 1D-CNN, 2D-CNN is used [73; 76; 78; 80; 83; 85].  
1 Autoencoder-CNN: In this configuration, first, an autoencoder network is trained, and then its trained encoder part is employed as a feature extractor. This model consists of an encoder network, CNN, and dense layers sequentially, as proposed in [27; 103].  
As shown in Figure 7, usually, the TensorFlow and Keras libraries are used for IDS implementation. Thus, to make our experiment closer to those in the literature, in this research, we used Keras with the TensorFlow backend in our implementation. Tables 13 and 14 show the experimental results on the CIC-IDS 2017 and NSL-KDD datasets, respectively. In both tables, the best scores are highlighted.  
three CNN layers, with an accuracy of 88.87%. Moreover, in our experiments, the highest accuracy on the CIC-IDS 2017 dataset was found using basic 1D-CNN with two CNN layers, while on NSL-KDD, the highest accuracy was found using hybrid LSTM with two-layer 1D-CNN. In general, our experiment shows that the results on CIC-IDS 2017 are better (higher accuracy and lower false alarm rate) than those on NSL-KDD because deep learning approaches such as CNNs have higher performance when trained with a larger dataset.