# Enhancing Machine Learning Prediction in Cybersecurity Using Dynamic Feature Selector
## 5 Experimental Analysis
### Gradient Boosting  
A benefit of using gradient boosting tree is retrieving importance scores for each attributes in a relatively straight forward manner when the boosted tree is constructed. Importance for every single decision tree is calculated by the amount that each attribute split improves the performance of the model. For both datasets, the cut-off was set by the cumulative feature importance of the features as 99%. The cumulative feature importance is the weighted average of individual feature importance. It is the mean of all the importance values evaluated by the decision trees within the model. Out of the 123 features obtained from one-hot-encoding of the NSL-KDD dataset, the XGBoost classifier suggested 51 very important features. It also assigned 72 features as having zero importance. Out of the 196 features in the UNSW-NB15 dataset after one hot encoding, the XGBoost classifier returned 47 features as important and 149 features as having zero importance. The graphs shown in Figure 2a,b highlight these important features.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{4}{c}{**UNSW NB-15 Dataset**} & \multicolumn{4}{c}{**NSL-KDD Dataset**} \\ \hline
**Feature** & **Corr. Feature** & **Corr.** & **Feature** & **Corr. Feature** & **Corr.** \\ \hline \hline \multicolumn{1}{c}{sbytes} & spkts & 0.964 & num\_root & num\_compromised & 0.998 \\ \hline \hline \multicolumn{1}{c}{dbytes} & dpkts & 0.973 & is\_guest\_login & hot & 0.860 \\ \hline \hline \multicolumn{1}{c}{sloss} & spkts & 0.972 & srv\_server\_rate & serror\_rate & 0.993 \\ \hline \hline \multicolumn{1}{c}{sloss} & sbytes & 0.996 & srv\_error\_rate & reror\_rate & 0.989 \\ \hline \hline \multicolumn{1}{c}{dloss} & dpkts & 0.979 & dst\_host\_same\_srv\_rate & dst\_host\_srv\_count & 0.897 \\ \hline \hline \multicolumn{1}{c}{dloss} & dbytes & 0.997 & dst\_host\_server\_rate & serror\_rate & 0.979 \\ \hline \hline \multicolumn{1}{c}{dwin} & swin & 0.981 & dst\_host\_server\_rate & srv\_server\_rate & 0.978 \\ \hline \hline \multicolumn{1}{c}{synack} & tcprtt & 0.946 & dst\_host\_srv\_server\_rate & serror\_rate & 0.981 \\ \hline \hline \multicolumn{1}{c}{ackdat} & tcprtt & 0.919 & dst\_host\_srv\_server\_rate & srv\_server\_rate & 0.986 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_ltm} & ct\_srv\_src & 0.841 & dst\_host\_srv\_server\_rate & dst\_host\_server\_rate & 0.985 \\ \hline \hline \multicolumn{1}{c}{ct\_src\_dport\_ltm} & ct\_srv\_src & 0.862 & dst\_host\_error\_rate & reror\_rate & 0.927 \\ \hline \hline \multicolumn{1}{c}{ct\_src\_dport\_ltm} & ct\_dst\_ltm & 0.961 & dst\_host\_error\_rate & srv\_error\_rate & 0.918 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_sport\_ltm} & ct\_srv\_src & 0.815 & dst\_host\_srv\_error\_rate & error\_rate & 0.964 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_sport\_ltm} & ct\_dst\_ltm & 0.871 & dst\_host\_srv\_error\_rate & srv\_error\_rate & 0.970 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_sport\_ltm} & ct\_src\_dport\_ltm & 0.908 & dst\_host\_srv\_error\_rate & dst\_host\_error\_rate & 0.925 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_src\_ltm} & ct\_srv\_src & 0.954 & service\_ftp & is\_guest\_login & 0.820 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_src\_ltm} & ct\_dst\_ltm & 0.857 & flag\_REJ & reror\_rate & 0.835 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_src\_ltm} & ct\_src\_dport\_ltm & 0.872 & flag\_REJ & srv\_error\_rate & 0.841 \\ \hline \hline \multicolumn{1}{c}{ct\_dst\_src\_ltm} & ct\_dst\_sport\_ltm & 0.836 & flag\_REJ & dst\_host\_error\_rate & 0.813 \\ \hline \hline \multicolumn{1}{c}{ct\_ftp\_cmd} & is\_ftp\_login & 0.999 & flag\_REJ & dst\_host\_srv\_error\_rate & 0.829 \\ \hline \hline \multicolumn{1}{c}{ct\_src\_ltm} & ct\_dst\_ltm & 0.901 & & & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Pearson Correlation values.