# Open DGML: Intrusion Detection Based on Open-Domain Generation Meta-Learning
## 5 Experimental Results
### Performance Comparison in Open Domain  
In the open domain setting, we evaluated the performance of our Open DGML algorithm on one unknown domain and trained it on three source domains. In case of encountering attack types that do not exist in the source domains, such as port scanning in the NDSec-1 dataset, which is absent in the other datasets, we classified them as \"unknown class\". The results, shown in Table 4, indicate that Open DGML outperforms the other algorithms in terms of accuracy and detection rate on most of the datasets. In particular, Open DGML achieves the highest accuracy and detection rate among the evaluated methods on the ISCX2012 dataset, reaching 75.69% and 70.13% respectively, which are the best performances across the four datasets. Notably, Open DGML's detection rate is 14.93% higher than the second-best performer, HAST-IDS. Similarly, on the NDSec-1 dataset, Open DGML exhibits the optimal accuracy and detection rate of 68.50% and 65.99%, significantly outperforming the next-best algorithm, CLAIRE, by 16.64% and 12.39%, respectively. On the CICIDS2017 dataset, Open DGML achieves the highest accuracy of 67.45%, while its detection rate is the second best, only 4.63% lower than the top-performing \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{2}{c}{**ISCX2012**} & \multicolumn{2}{c}{**NDSec-1**} & \multicolumn{2}{c}{**CICIDS2017**} & \multicolumn{2}{c}{**CICIDS2018**} & \multicolumn{2}{c}{**Average**} \\ \cline{2-11}  & **ACC** & **DR** & **ACC** & **DR** & **ACC** & **DR** & **ACC** & **DR** & **ACC** & **DR** \\ \hline HAST-IDS & 99.89\% & 96.96\% & 91.04\% & 90.12\% & 96.88\% & 95.26\% & 96.30\% & 93.37\% & 96.03\% & 93.93\% \\ CLAIRE & 97.95\% & 93.40\% & 87.22\% & 91.60\% & 98.01\% & 95.20\% & 97.58\% & 93.44\% & 95.19\% & 93.41\% \\ FC-Net & 97.87\% & 98.88\% & 89.34\% & 85.17\% & 94.64\% & 99.62\% & 93.22\% & 97.46\% & 93.77 \% & 95.28\% \\ \hline Open DGML & 99.13\% & 98.26\% & 94.20\% & 95.29\% & 97.53\% & 97.30\% & 95.22\% & 97.31\% & 96.52 \% & 97.04 \% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of algorithm performance in non-open domain. FC-Net. Furthermore, on the CICIDS2018 dataset, Open DGML demonstrates the highest accuracy, surpassing the second-best CLAIRE by 4.37%, and also has the best detection rate. When considering the average performance across the four datasets, Open DGML significantly outperforms the other algorithms. Compared to HAST-IDS, Open DGML achieves 19.26% and 16.43% higher accuracy and detection rate, respectively. Similarly, it outperforms CLAIRE by 13.03% and 11.08%, and FC-Net by 11.79% and 11.57%, all improvements exceeding 10%. These results indicate that Open DGML, which integrates Mul-Mixup techniques and meta-learning algorithms, can effectively perform intrusion detection in open-domain settings. The combination of enhanced data sample diversity and improved generalization capabilities when facing unknown data domains contributes to the superior performance. The performance of Open DGML differed significantly from the other algorithms on the NDSec-1 dataset, which contains a previously unseen traffic type, port scanning. Open DGML's design incorporates a confidence threshold method to identify unknown classes, which gives it an edge over the other algorithms. To quantify the model's ability to recognize unknown classes, we evaluated the accuracy of Open DGML and the other algorithms in identifying port scanning as an unknown class in the NDSec-1 dataset. The results, presented in Table 5, show that Open DGML achieved the highest accuracy of 41.70%, which is significantly better than the other algorithms. This is due to the fact that Open DGML was specifically designed to recognize unknown classes, and its overall performance in the target domain was high, which also affects its ability to recognize unknown classes.