# A Decision-Fusion-Based Ensemble Approach for Malicious Websites Detection
## 3 Materials and Methods
### Methodology of Proposed Approach  
#### 3.2.1 Gradient-Boosting (GB) Classifier Method  
Gradient-boosting (GB) is a supervised learning method. It is used in many applications for regression and classification tasks. GB combines several weak learners to produce a strong predicative model. This process is called an ensemble of weak learners, which are usually decision trees [42; 43]. Conceptually, ensemble learning uses various classifiers to effectively classify the data instances and aggregate their predictions. The boosting technique [44] is a type of ensemble learning that can resolve classification or regression tasks by fusing several models to build an ensemble model. The error of the combined models is minimized by giving extra weight to the training instances that are incorrectly predicted or classified by previous learners. For the final prediction of the ensemble model, a weighted voting rule is used. GB, as one of the boosting methods, aggregates the prediction results of multiple decision trees to boost the performance of a single tree model [45]. Moreover, GB can improve the performance of datasets with class-imbalance issues [46]. Therefore, it is used in our approach to improve the accuracy of malicious websites detection, especially when suffering from a class-imbalance problem. Algorithm 1 generates several decision trees using training dataset examples and builds the GB classifier. ```
Input: A training dataset with \((x_{1},y_{1})\),... \((x_{m},y_{m})\) where \(x_{i}\in X,y_{i}\in Y\); \(T\) is the number of random trees using distribution \(D_{t}\). Output: The final decision tree (GB classifier). Begin
1. For \(t=1\) to \(T\). 1.1.



methods [48]. The XGB method is an improved version of the gradient-boosting concept and is used for both regression and classification applications [49]. It is a scalable learning method of the boosted-trees-based methods, which is commonly and efficiently applied for large-scale features with a notable effect on parallel boosted-tree operations and a flexible manner. The XGB parameters are separated into three groups, which are boosting parameters, general parameters, and learning parameters. To boost the model, it is recommended that the boosting parameters are optimized. Algorithm 2 shows the steps in building an XGB classifier model. ```
Input:\(X=\{(x_{1},\ y_{1}),\ (x_{2},\ y_{2}),\ \ldots,\ (x_{m},\ y_{m})\}\) is a training samples; \(T\) is a maximum number of iterations; max-depth; regularization coefficients, and other parameters. Output: strong learner \(f(x)\); Begin
1. For \(t=1\) to \(T\):
2. Calculate the loss function \((L)\) of the i\(th\) instances \((i=1,\ 2,\ \ldots,\ m)\) in the present iteration based on the max-depth, regularization coefficients, and the other parameters. 3. Calculate the first derivative and second derivative of all instances. 4. Split the decision-tree using the score of the processed node; the value 0 is the default score. The score of the node that is subjected to the splitting process is the sum of the first derivative and second derivative. 5. Split the sub-number using the eigenvalue and division feature based on the maximum value. 5.1. If the maximum value is equal to 0, then establish the present decision tree; 5.1.1. Calculate the weights of all leaf regions to generate the weak learner. 5.1.2. Update the strong learner and enter the next iteration of the weak learner. 5.2. If the maximum value is not equal to 0, continue from step 5 to split the decision tree. End
```
**Algorithm 2** Building XGB classifier model. The proposed approach uses the XGB classifier to classify benign and malicious websites, proving to be an effective classifier in the decision fusion ensemble learning (DFEL) model. #### 3.2.3 Random Forest (RF) Classifier Method

Random Forest is an ensemble-based method. It combines multiple decision trees to improve the accuracy and reduce the over-fitting problem [50]. The algorithm involves two main steps: training and prediction.