# A Novel Intrusion Detection Approach Using Machine Learning Ensemble for IoT Environments
## 4 Experimental Results and Discussion
### Metrics and Method of Model Evaluation and Validation  
The chosen range of the input parameters affected the general performance of the classifiers. The method of random search was used to explore the best optimal input parameters picked from the hyperparameters of the random forest technique and gradient boosting machine method for the CSE-CIC-IDS 2018 dataset. The method of randomized search CVs was used for the hyper-tuning of parameters, implemented in the Python programming environment using the Scikit-learn library. Randomized search CV finds the best optimal parameters settings by performing a method of cross-validation search given by the client. This test uses specific metrics to access the evaluation of the classifiers that are accuracy, specificity, or the rate of true negatives, and sensitivity, or the rate of true positives [36]. The metrics FPR and AUC were mathematically represented through Equations (6) and (7) as follows:  
\[PR=\frac{FP}{(TN+FP)} \tag{6}\]  
\[AUC=\int_{0}^{1}\frac{TP}{(TP+FN)}d\frac{FP}{(FP+TN)} \tag{7}\]In Equations (6) and (7), the term TP represents the true positive, which is the amount of truly classified attack instances and the term TN represents the true negative, which is the amount of true classified specified instances. The amount of incorrectly classified attack instances is defined by the term FP, false positive, and the FN, false negative, is the quantity of incorrectly classified normal instances. Accuracy is the absolute number of examples accurately classified to the total number of cases in the dataset. The sensitivity of the model represents the number of instances of general attacks accurately classified on all specific instances. The FPR represents the number of instances of misclassified attacks over the total number of common instances, and the AUC represents the referenced area under the receiver operating characteristics (ROC) curve, wherein the ROC curve is plotted against the FRP of the defined TRP.  
For the computation process of the model and evaluation of full performance, we directed the experiment using a cyclic hold-out as in the cyclic \(k\)-fold cross-validations (in our case 10-fold) method [37]. As proposed by [PERSON] et al. [38], the cyclic version makes the error estimation stable and reduces the variance of the validation approach. For hold-out validation purposes, we divided the considered dataset into the 7:3 ratio, meaning 70% of the dataset was considered as training instances, while 30% of the dataset was considered as a testing instance. Additionally, for \(k\)-fold cross-validation, \(k\) is assumed to be 10. Cyclic 10f and 100 rounds of hold-out validation are considered as the models of classification which are assumed stable, indicating uniform prediction for the same dataset. To derive accurate information, all performance results reported in this paper were outputs estimated by 10 iterations of each cyclic validation approach and each experiment was repeated using an alternate seed which was also a contributing input to the random number generator.  
The performance evaluation of the machine learning ensemble technique using the random forest model and the gradient boosting machine model with a hyperparameter search space, specifically with the CSE-CIC-IDS2018-V2 dataset, is discussed in this section. The results provide an optimal parameter configuration for both random forest and gradient boosting machine models and statistically analyze the performance of the models. To conduct the hyperparameter search to accomplish the task, Hyperopt was used, a distributed hyperparameter optimization [29; 37; 39; 40] library providing a standardized optimization over awkward search space for serial or parallel searches. Through this experimental study, it was shown that the classifier used in the study is useful for detecting the intrusion of IoT appliances. First, we analyzed the performance results of hyperparameter optimization with the utilization of a random forest classifier.