# An Ensemble Tree-Based Model for Intrusion Detection in Industrial Internet of Things Networks
## 3 Materials and Methods
### Machine Learning Model  
This sub-section discusses the ensemble ML-based models used for detecting attacks in IIoT-based networks.  
1. Extreme Gradient Boosting (XGBoost)  
XGBoost is a modified gradient tree-boosting algorithm that is efficient and scalable. The optimization problem in ensemble algorithms can be solved using the boosting classifiers, where one weaker learner is added in succession to create a new model to lower the classifier loss function and to progressively reduce the mistakes of earlier models [40]. The exemplary features of the algorithm proposed by the authors in [41] are the regularized model, split-seeking algorithm, column block structure, and cache-aware prefetching algorithm. Some current applications of XGBoost include genre classification of Nigerian songs [42], predicting stock price [43], and forecast gene expression value [44].
2. Bagging Classifier  
A group meta-learner is the Bagging classification algorithm. The approach creates a large number of learners by training each unique base learner on a random subset of the actual dataset. The classifier then estimates the final prediction by averaging the results of all the models [45]. This algorithm averages the probability values of base learners for regression tasks and applies the majority voting scheme to classify labels for the classification tasks. This algorithm starts by resampling the training data with replacements. This means that some instances may be selected again and again, while others may not. The strength of this meta-estimator is the reduction in the variance of the base learner by introducing randomness into the ensemble construction and generation method. Concurrent training is conducted on the randomly selected subset of the training set with the base learners using substitution using the initial dataset. Each base classifier's training dataset is distinct from the datasets of the others.
3. Random Forest (RF)  
RF is a group of weak base learners that functions by building various collections of decision trees to enhance the DTs' effectiveness and resilience [46]. This technique combines the bagging approach of instance sampling with the random selection method for features in creating a collection of DTs with a controlled variation. To complete the classification task of an unlabeled instance, each DT in a set acts as a base learner. The algorithm uses majority voting for the classification task and probability averaging of instance values from the regression task. The RF algorithm is immune to noise and over-fitting and has been applied to several domains, including heart disease classification [47] and label ranking [48].
4. Extremely Randomized Trees (Extra Trees)  
Extra Tree is a collection of ML-based models that combine the classifications from several unpruned DTs on different sub-samples of the target to enhance generalization accuracy, being computationally efficient and preventing over-fitting [49]. The entire training instance is used to grow trees, and the nodes at each tree are split by selecting the cut points fully at random. These predictions are made by using a majority voting scheme for classification tasks or averaging prediction values for regression tasks.  
(5) Adaptive Boosting (AdaBoost)  
AdaBoost is an ensemble of ML models adopting the boosting method by joining many weak learners to create a new model using the weighted linear combination method iteratively. To reweight examples of the real train data, it progressively uses a learning algorithm [50]. Firstly, all instances are assigned the same weight. Weights are increased for cases that were incorrectly classified, while they are raised for instances that were correctly classified. This procedure is iterated continually by new weights of the training data on the base model. Finally, a linear combination of all the models generated through the various iterations is used to create the final classification model [51]. This algorithm's weakness is that it is sensitive to anomalies and noisy data.