# An Ensemble-Based Machine Learning Approach for Cyber-Attacks Detection in Wireless Sensor Networks
## 3 Methodology
### Proposed Ensemble Machine Learning Approaches  
We present a lightweight ensemble-based ML approach, WSS, to detect cyber-attacks on WSNs using a set of fundamentally diverse single classifiers in the ensemble. Multiple classifiers have been trained, tested, and evaluated using several evaluation metrics to differentiate the most effective classifier for detecting attacks. This proposed approach was built using a pool of heterogenous supervised ML techniques: SVM, NB, DT, KNN, RF, and LightGBM. We assumed that the individual models used to build a successful WSS were diverse to create a heterogeneous ensemble. Heterogeneous ensembles comprised of different classical individual classifiers were used to increase diversity while maintaining the default settings of the classifiers' parameters during the learning process. We also compared WSS performance to the three classical ensemble techniques: Bagging-based, Boosting-based, and Stacking-based. The Bagging-based model was built using a decision tree algorithm, the Boosting-based model was implemented using an AdaBoost algorithm, and the Stacking-based model was implemented using three different classification algorithms for level 0: KNN, RF, and NB classifiers. Logistic Regression (LR) was used as a level 1 classifier. We aimed to investigate how these ensemble techniques can be applied to detect attacks targeting WSNs. Weak classifiers were fitted iteratively to the training data for the Boosting-based algorithm [10]. The classification was boosted gradually by emphasizing samples that were modeled poorly by the weak classifiers and iteratively updating their weights each round to create a single robust classifier. AdaBoost minimizes exponential loss according to the following equation [32]:  
\[F_{T}(x)=\sum_{t=1}^{T}f_{t}(x)\]  
where \(f_{t}\) represents each weak classifier for an object \(x\)  
\[E_{t}=\sum_{i}E[F_{t-1}(x_{i})+a_{t}h(x_{i})]\]  
where each weak classifier makes its hypothesis \(h(x_{i})\) with a coefficient \(a_{t}\) to minimize the sum of the weighted error rate. The Bagging-based algorithm, or bootstrap aggregation, builds a group of decision trees as base classifiers using multiple versions of the training dataset. Each decision tree is individually classified as a weak learner, then the trees are combined by voting to construct a strong learner as a single model. This process is performed repeatedly for individual trees in the ensemble when a new instance must be classified. Each tree outputs a vote for a class [33]. The class with the maximum number of votes determines the final prediction for each new instance. The Stacking algorithm selects the best classifier based on combining predictions from different individual classifiers to improve stacked generalization. Implementing the stacking classifier involves two types of models: (1) base models (level 0): KNN, RF, and NB classifiers, and (2) meta-model (level 1): LR classifier. The key concept behind the Stacking algorithm lies in using the meta classifier that learns from the level 0 classifiers to predict data samples. The primary difference between WSS and other ensemble classifiers is that WSS final prediction is promoted by one classifier that achieves higher performance compared to others in terms of the considered metrics: there is no integration of the outcomes of the base models. The prediction result is computed by combining the results of all base classifiersusing averaging or majority voting in other ensemble techniques. Figure 3 depicts the general overview of the presented approach: WSS. We used the same dataset during the learning process for the set of base classifiers.