# A DNN Architecture Generation Method for DDoS Detection via Genetic Algorithm
## 4 Experiment and Evaluation
### Comparative Experiment  
Additionally, this paper compares the best model generated using the proposed method tp the models of [PERSON], [PERSON], [PERSON], [PERSON] and [PERSON]. This paper also selects some classical machine learning algorithms as the baseline, including decision tree, random forest, SVM, logistic regression and naive Bayes. The evaluation results of each model are obtained through 10-fold cross-validation. To objectively evaluate the model's performance, this paper uses the processed data to train, test and evaluate the model. Table 10 shows that the generated model is superior to the other models in terms of accuracy and F1-score. To evaluate the availability of our proposed generation method, this paper generates models on the CICIDS2017 and CICIDS2018 datasets. Table 11 shows that the proposed method can generate models with good performance in different datasets. A DNN model generated on only one dataset may have over-fitting problems, so this paper conducts model generation experiments on six data sets at the same time to avoid this issue. (The six datasets selected in this paper are the CICDDoS2019 dataset, the CICIDS2017 dataset, the CICIDS2018 dataset, the KDD_CUP99 dataset, the NSL-KDD dataset and the UNSW dataset.) All generation experiments share a GEN LRU, and there is a 10% probability to select individuals generated on different datasets for crossover operation; only individuals within the top 20% in terms of fitness are used for the crossover operation. Table 12 represents the average complexity of the method proposed in this paper and the existing methods. The second column represents the time complexity, and the third column represents the space complexity. It can be seen that the method proposed in this paper has lower complexity in terms of time and space. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Techniques** & **ACC** & **PRE** & **REC** & **F1\_SCORE** \\ \hline Best model generated & **0.9937** & 0.9993 & 0.9881 & **0.9937** \\ [PERSON] et al. (2017) & 0.9913 & 0.9990 & 0.9837 & 0.9913 \\ [PERSON] et al. (2018) & 0.9927 & **0.9999** & 0.9855 & 0.9927 \\ [PERSON] et al. (2019) & 0.9914 & 0.9985 & 0.9846 & 0.9915 \\ [PERSON] et al. (2018) & 0.9866 & 0.9964 & 0.9771 & 0.9867 \\ [PERSON] et al. (2018) & 0.9921 & 0.9987 & 0.9857 & 0.9921 \\ SVM & 0.8024 & 0.9646 & 0.6243 & 0.7580 \\ C4.5 & 0.9817 & 0.9795 & 0.9840 & 0.9817 \\ RF & 0.7862 & 0.9931 & 0.5800 & 0.7323 \\ LR & 0.5071 & 0.5071 & **1.0000** & 0.6729 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: The comparative Results with related works. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Method** & **Flops (M)** & **Params (M)** \\ \hline Model generation using six datasets & 0.2173 & 0.0038 \\ Model generation using one dataset & **0.1105** & **0.0020** \\ [PERSON] et al. (2017) & 0.9507 & 0.0305 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Performce of best model generated. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Dataset** & **ACC** & **PRE** & **REC** & **F1\_SCORE** \\ \hline CICIDS2017 & 0.9906 & 0.9986 & 0.9915 & 0.9906 \\ CICIDS2018 & 0.9993 & 0.9986 & 1.0000 & 0.9993 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Performce of best model generated. This paper uses the best model generated with six datasets, the best model generated with one dataset and the best model generated by [PERSON] et al. to carry out comparative experiments on the CICDDoS2019 dataset. According to Table 13, the best model generated with one dataset has the best performance, and the best model generated with six datasets has a slightly lower performance than best model generated with one dataset. The sixth column in Table 13 shows the time it takes for the generated model processes a single piece of data in the test phase, and the seventh column in the table represents the KS value of the generated model.