## 4 Results
### Case I: Binary Classification and Multiclass Classification (UNSW-NB15)
Model Training and Optimization Phase  
In this step, the experiments were divided into two parts: (1) 42 behavioral patterns of the test samples were identified to conduct the training experiment and (2) ranked features using IGR rate (Table 5) were extracted to derive 23 features from 42 features to examine the RF-based model classification accuracy. Step 2.1. Data bootstrapping  
Bootstrapping was used to divide the training data set into \(m\) subsets to help the data analyzer create DTs with randomly selected data points. Step 2.2. Model training using RF  
In this study, the model was trained on 42 behavioral patterns before extracting 23 features that had the highest IGR rate (Table 5). The performance of the RF-based classification model was then evaluated using Equations (8)-(10) by training component classifiers and aggregate the results of component classifiers from randomly selecting sub-datasets of the training data, and finally trains a meta-classifier with majority voting approach to perform the threat classification of the samples. Using GridSearchCV to set for the model parameters of binary classification on Random Forest. The search parameters of RF are set as Table 6. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Feature** & **Weighting** & **Rank** & **Feature** & **Weighting** & **Rank** \\ \hline Sttl & 0.1543 & 1 & Dmean & 0.0271 & 13 \\ ct\_state\_ttl & 0.0694 & 2 & Simpkt & 0.0267 & 14 \\ Dload & 0.0578 & 3 & dbytes & 0.0250 & 15 \\ Dttl & 0.0527 & 4 & ct\_dst\_src\_ltm & 0.0248 & 16 \\ Tcprtt & 0.0412 & 5 & smean & 0.0246 & 17 \\ Dur & 0.0378 & 6 & state\_INT\({}^{\prime}\) & 0.0219 & 18 \\ Sload & 0.0366 & 7 & ct\_srv\_src & 0.0212 & 19 \\ Ackdat & 0.0351 & 8 & spkts & 0.0165 & 20 \\ Rate & 0.0306 & 9 & djit & 0.0145 & 21 \\ ct\_srv\_dst & 0.0305 & 10 & dloss & 0.0133 & 22 \\ Synack & 0.0303 & 11 & ct\_dst\_sport\_ltm & 0.0125 & 23 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Top 23 features by weight. Step 3. Performance Evaluation Phase  
To obtain the optimal classification accuracy for RF model, it first needs to determine how many trees (component classifiers) to be generated in RF algorithm. Typical value for the number of trees in most cases is 10, 30, or 100. It is very few practical cases more than 300 trees that may increase the cost in computation time for learning these additional trees [31]. In this study, the experiments for deciding the appropriate number of tree on RF were conducted by the different numbers by examining the RF-based model classification accuracy: 25\(\sim\)200 trees (\(n\) = 25\(\sim\)200) of the training samples. The low classification error of the RF algorithm, as presented in Figure 6, was based on data from approximately 100 to 200 trees. From Table 7, it is seen that the there is a better performance of multi-classification (i.e., accuracy precision, recall, F1 Score, and ROC AUC) when \(n\) =100, 150, 200, compared to \(n\) = 50. Normally, more trees are generally better precision for RF algorithm; however, also increase the computational time of model. Notably, training with the training set for 200 trees, it gets about a 0.04% accuracy improvement on RF compared to \(n\) = 100, but increase 10.26 sec of computational time. In other words the improvement decreases as the number of decision trees increases. Therefore, the number of trees for the RF algorithm was set to 100 considering the benefit in prediction performance from learning additional trees in this study. The average accuracy for \(N_{f}\) = 42 was approximately 99.82% (Table 8) and 83.51% for the binary classification results on training and testing data with RF (\(n\) = 100). The accuracies (in %) associated with the optimal parameter \(C\) and \(\gamma\)-values were obtained through a cross-validation scheme for the SVC (Table 8). The binary classification accuracy