# XGB-RF: A Hybrid Machine Learning Approach for IoT Intrusion Detection
## 2 Materials and Methods
### Classifiers
#### 2.4.2 eXtreme Gradient Boosting (XGBoost)
### Evaluation on Different Train-Test Schemes

As mentioned earlier, in the proposed approach, 75% of data are used for training, and the remaining 25% are used for testing purposes. Data were primarily divided into 75-25% according to our previous experience. We also evaluate the performance using other train-test splitting schemes like 70-30% and 67-33%. These schemes are also used by other N-BaloT studies [35, 36]. The result presented in the Table 4 reveals a very minimal effect on data splitting. ### ROC Curve

It can be seen from Figure 3, that our proposed hybrid XGB-RF accurately classified all the classes, including Gafgyt.udp class. On the other hand, other methods such as RF-RF, RF-RFECV, RF-RFE, RF-SelectKBest and RF-WFS can not classify Gafgyt.udp class well. Therefore, we have generated a receiver operating characteristic curve (ROC) for Gafgyt.udp vs. all other classes in Figure 4. It can be seen that the proposed method \begin{table}
\begin{tabular}{c c c} \hline \hline
**Performance** & **Train-Test (70–30\%)** & **Train-Test (67–33\%)** \\ \hline Accuracy & 99.9564\% & 99.9539\% \\ F1\_Score & 99.9565\% & 99.9539\% \\ Kappa & 99.9518\% & 99.9489\% \\ MCC & 99.9518\% & 99.9489\% \\ Sensitivity & 99.9565\% & 99.9539\% \\ Specificity & 99.9956\% & 99.9953\% \\ Threat\_Score & 99.9220\% & 99.9160\% \\ Balanced Accuracy & 99.9760\% & 99.9746\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance Evaluation on Different Train-Test Schemes. Figure 3: Confusion matrix for (**a**) RF-RF, (**b**) XGB-RF, (**c**) RF-RFECV, (**d**) RF-RFE, (**e**) RF-SelectKBest, (**f**) RF-WFS. Note that X-axis shows the actual label whereas Y-axis represents the predicted label by the classifier. can recognise Gafgyt.udp than other approaches. Note that ROCs for RF-RF, RF-RFECV, RF-RFE, RF-SelectKBest and RF-WFS are overlapped together as they all exhibit the same performance. Additionally, [PERSON] et al. [37] showed that area under ROC (AUROC) is equivalent to statistical Mann-Whitney U-statistic testing and relevant to statistical \(p\) values as we have seen in our paper that ROC of our proposed method is about 1 which is a perfect classifier and equivalently we can say \(p<0.001\). Therefore, we have not performed a statistical test or ANOVA test for statistical significance. ### Performance Comparison with Other Studies

A performance comparison is performed among our proposed XGB-RF model and other contemporary studies concerning some performance indices where five of the other methods used the same N-BaIoT dataset. Therefore, we have used those studies in the comparison. [PERSON] et al. In addition, [PERSON] et al. used RF and Deep Multilayer Perceptron (DMLP) on the same CICIDS2017 dataset and obtained an accuracy of 99.67% and 91%, respectively. Again, [PERSON] et al. employed Support Vector Machine-Decision Tree-Naive Bayes (SVM-DT-NB) classifier on the KDDCup99 dataset, which produced an accuracy of 99.62%. However, [PERSON] et al., [PERSON] et al., [PERSON] et al., [PERSON] et al. In addition, [PERSON] et al. utilized the Naive Bayes-J48 Decision Tree-Artificial Neural Network (NB-J48-ANN)), Classification and Regression Trees (CART), Recurrent Neural Network (RNN), Collective Deep Learning and Local-Global Best Bat Algorithm for Neural Networks (LGBA-NN) approaches on the N-Balot dataset that resulted in an accuracy of 99.10%, 99%, 89.75%, 99.84%, and 90%, respectively. Most of the above-mentioned approaches used state-of-the-art classifiers but still, their performance is less than our proposed XGB-RF approach. XGB-RF is a hybrid machine learning model that selects the prominent feature subset and improves classification accuracy. This is due to the fact that XGB-RF removes the redundant and irrelevant features. Therefore, it provides a better decision boundary that improves the classification performance and reduces the runtime. Here, from Table 5, it is clear that our proposed model outperforms all the previous approaches. Figure 4: ROC Curve for Gafgyt.udp vs. all classes.