# Open DGML: Intrusion Detection Based on Open-Domain Generation Meta-Learning
## 5 Experimental Results
### Validation of Components in Open DGML  
_Ablation Study._ The ablation study dissects the framework into individual modules to assess their impact on the final detection results. The modules include flow image technology, Mul-Mixup data augmentation, and the meta-learning algorithm. The performance comparison in the open-domain setting is measured using accuracy rates, as detailed in Table 6.  
It can be found that for the recognition of the eight attack types, except for the recognition of botnets where the meta-learning-only scheme performs slightly better, the performance ranking in other cases is Open DGML \(>\) DGML \(>\) Mul-Mixup \(>\) Flow Image. This indicates that the improvement in overall model generalization performance brought by meta-learning is more significant than the data augmentation approach of only using Mul-Mixup to improve sample diversity, and the coupled effect of the two is also greater than the performance improvement brought by applying the basic model alone, \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline  & **DoS** & **DDoS** & **Brute Force** & **Infiltration** & **Web Attack** & **Botnet** & **Heartbleed** & **Port Scan** \\ \hline Flow Image & 52.3\% & 43.1\% & 39.8\% & 41.6\% & 35.4\% & 37.7\% & 29.5\% & 17.1\% \\ Mul-Mixup & 60.2\% & 53.9\% & 52.6\% & 57.8\% & 48.9\% & 48.1\% & 42.3\% & 21.2\% \\ DGML & 68.4\% & 67.8\% & 55.1\% & 60.2\% & 47.6\% & 59.4\% & 50.1\% & 32.9\% \\ all & 73.6\% & 69.2\% & 60.7\% & 63.5\% & 52.8\% & 56.8\% & 53.4\% & 41.7\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of ablation experiment. Figure 3: Frecicher distance between source domain and target domain. because the optimization of the two approaches is at different levels. Mul-Mixup is more focused on data augmentation of the original input images to obtain a more general representation of the belonging classes, while the meta-learning algorithm utilizes a large amount of prior knowledge to train the basic model and learns the distribution of samples in the classes through multi-domain knowledge, which is more focused on improving model generalization performance. The combination of flow image, Mul-Mixup, and meta-learning significantly enhances detection accuracy, verifying the effectiveness of these three components in Open DGML. It is worth mentioning that our model has obvious differences in the recognition performance of different categories, and this situation also exists in each module stage. The recognition performance for DoS, DDoS, Brute Force, and Penetration Attacks is better than Web Attacks, Botnets, and Heartbleed, and is much better than Port Scans. The reason is that the first four types of attacks are present in each source domain and target domain, so the training samples for the model are relatively sufficient. However, Web Attacks, Botnets, and Heartbleed are not present in all datasets, so for a particular target domain, the attack category may only exist in one or two source domains, and the sample diversity is somewhat insufficient, resulting in less than ideal final recognition performance. Particularly, the Port Scan attack category only exists in the NDSec-1 dataset, so if this dataset is used as the target domain, there is no training data available in the source domains, and the model's recognition capability for this category is generally poor, especially in the modules without meta-learning, with a recognition accuracy of only 17.1%. After going through meta-learning, the recognition accuracy for unknown categories is directly improved by 15.5%, demonstrating that the meta-learning algorithm provides significant help for the recognition of unknown categories. Comparative Experiments on the Flow Processing Method.We validate the effectiveness of the data preprocessing module proposed in our framework by comparing it with various flow processing methods on the ISCX2012 dataset. To showcase the performance of our flow processing method, we only conduct experiments and testing on the ISCX2012 dataset and do not consider open-domain scenarios.