# MOCA: A Network Intrusion Monitoring and Classification System
## 5 Measurement Results
### Effectiveness of Pre-Trained Binary Classifier  
To further investigate the benefit of using connection features, four Decision Tree-based binary classifiers were designed. Each classifier was trained by one feature (_dst_src_count_, _src_count_, _dst_count_, _src_dport_) and DDoS attacks from the CICIDS2017 dataset. Then, DDoS and Bot attacks were used from ToN-IoT [29], BoT-IoT [12], CICIDS2018 [7], and CICDDOS2019 datasets to evaluate the pre-trained classifiers without retraining. Pre-trained decision trees distinguished DDoS and Botnet attacks from normal traffic. Note that these datasets contain traditional IT and IoT traffic (in BoT-IoT and ToN-IoT datasets) with different types of DDoS attacks. Each pre-trained binary classifier is a lightweight decision tree trained by one feature. The result is shown in Table 4. The table shows that pre-trained classifiers can distinguish most DDoS and Botnet attacks from normal traffic. For example, in the CICDDOS2019 dataset, they had an accuracy of 96%. For the BoT-IoT dataset, a pre-trained classifier detected almost all DDoS attacks in non-IoT and IoT traffic with 99.94% accuracy.  
We noticed that pre-trained classifiers contribute differently to detecting attack classes. For example, _dst_src_count_ was used for the CICDDOS2019 dataset, and _src_dport_ was used for the BoT-IoT dataset. The other two features, _dst_count_ and _src_count_, were used to classify DDoS attacks in CICIDS2018 and ToN-IoT datasets, respectively. The results show that connection features can be selected as the potential features for training a binary classifier instead of searching all features. The results also show that pre-trained binary classifiers can help detect new DDoS and Botnet attacks when training samples are unavailable. In addition, accuracy can be further improved by using the selected connection features and one or two basic features from the new dataset to train the classifier. Therefore, MOCA can minimize training time.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline \multicolumn{6}{c}{**MOCA (Top-20)**} & \multicolumn{4}{c}{**MOCA (Top-7)**} & \multicolumn{4}{c}{**MOCA (Top-3)**} & \multicolumn{4}{c}{**Random Forest**} \\ \hline
**Classes** & **Prec** & **Rec** & **F1** & **Prec** & **Rec** & **F1** & **Prec** & **Rec** & **F1** & **Prec** & **Rec** & **F1** \\ \hline DNS & **1.00** & **1.00** & **1.00** & 0.99 & 0.98 & 0.99 & 0.97 & 0.97 & 0.99 & 0.99 & 0.99 \\ MSSQL & 0.99 & 0.98 & 0.98 & 0.98 & 0.98 & 0.98 & 0.98 & 0.97 & 0.98 & 0.98 & 0.97 & 0.98 \\ NTP & **1.00** & **1.00** & **1.00** & 0.99 & 0.99 & 0.99 & 0.97 & 0.97 & 0.97 & 0.99 & 0.99 & 0.99 \\ SNMP & 0.94 & 0.92 & 0.93 & 0.93 & 0.92 & 0.92 & 0.82 & 0.91 & 0.86 & 0.92 & 0.91 & 0.91 \\ SSDP & 0.85 & 0.73 & 0.79 & 0.84 & 0.70 & 0.76 & 0.83 & 0.70 & 0.76 & 0.78 & 0.67 & 0.72 \\ LDAP & 0.95 & 0.99 & 0.97 & 0.95 & 0.99 & 0.97 & 0.94 & 0.85 & 0.90 & 0.95 & 0.99 & 0.97 \\ NetBIOS & 0.90 & 0.92 & 0.90 & 0.91 & 0.91 & 0.88 & 0.91 & 0.88 & 0.89 & 0.88 & 0.89 & 0.89 \\ Portmap & 0.90 & 0.91 & 0.91 & 0.87 & 0.92 & 0.89 & 0.87 & 0.93 & 0.90 & 0.88 & 0.88 & 0.88 \\ Syn & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** \\ TFTP & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** & **1.00** \\ UDP & 0.85 & 0.95 & 0.90 & 0.85 & 0.96 & 0.90 & 0.85 & 0.96 & 0.90 & 0.82 & 0.93 & 0.87 \\ UDP-lag & 0.84 & 0.82 & 0.83 & 0.84 & 0.82 & 0.83 & 0.83 & 0.82 & 0.83 & 0.78 & 0.75 & 0.77 \\ WebDDos & 0.92 & **1.00** & 0.96 & 0.78 & 0.93 & 0.85 & 0.75 & 0.89 & 0.82 & 0.88 & 0.96 & 0.92 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison of results for detection accuracy of MOCA vs. Random Forest model based on CICDDoS2019 dataset.