# A Decision-Fusion-Based Ensemble Approach for Malicious Websites Detection
## 3 Materials and Methods
### Methodology of Proposed Approach
#### 3.2.1 Gradient-Boosting (GB) Classifier Method
In the training step, a subset of the training data is randomly selected and, for each subset, a set of features is randomly chosen. Then, a decision tree is built using the selected features and the subset of data. After that, the previous steps are repeated a specified number of times (or until a stopping criterion is met). Finally, the decision trees are stored. In the prediction step, the new data points are run through each decision tree in the forest and, for each decision tree, the class of the data point is predicted based on the tree's decision rule [51]. Then, the predictions of all the trees are aggregated. After that, majority vote can be used for classification and the average of the predicted values can be taken for regression. The way in which the RF classifier is mathematically built is given in Algorithm 3. ### Proposed Approach

The proposed approach is based on a decision fusion of three effective ensemble-learning classifiers, namely GB, XGB, and RF classifiers. The final decision of used classifiers is fused using a weighted majority voting rule. The approach consists of three main steps: data pre-processing, model building and training, data classification, and an evaluation step. Figure 2 shows a flowchart of the proposed approach and the following subsections explain the flowchart steps. #### 3.3.1 Data Pre-Processing

This step consists of two main sub-steps: feature cleaning and feature normalization. In feature cleaning, features with null values are replaced with zeros to make them suitable for training the decision-fusion-based ensemble model. In addition, the features with categorical values are encoded into numbers using the label encoding method to ensure

Figure 2: Flowchart of proposed approach. the model is able to learn the numeric values of features. The features that contain unique values are removed because they create noise and decrease the accuracy. During feature normalization, numerical features with different ranges of values might lead to some challenges during model training because the difference between them will increase the distance of the decision boundary. Therefore, it is important to normalize these values in each feature so that the maximum value is one and the minimum value is zero. This providds more homogeneous values for the classifier while maintaining relativity between the values of each attribute. Data normalization will be carried out using the min-max technique. Finally, the correlation between features will be analyzed to check whether the highly correlated features effect the performance of the ensemble classification model. The data pre-processing steps are given in Algorithm 4. ```
Input :\(df_{i=1..n}\) : dataset features; \(n\) : the number of features; Output :\(pdf_{i\in 1..n}\) : processed dataset features; \(\overline{pdf_{i\in 1..n}}\) : processed dataset features without  high correlated features; Begin
1. For \(i=1\) : \(n\) do %This loop is for initialization with zeros
2. if (\(df_{i}\) is null ) then
3. \(df_{i}\gets 0\);
4. endfor
5. For \(i=1\) : \(n\) do %This loop is for removing the unique features
6. if (\(df_{i}\) is unique ) then
7. Remove \(df_{i}\);
8. endfor
9. For \(i=1\) : \(n\) do %Enceo categorical feature values to numbers
10. if (\(df_{i}\) has a categorical value ) then
11. Encode \(df_{i}\) using the label encoding method;
12. endfor
13. for \(i=1\) : \(n\) do %Remove highly correlated features
14. \(pdf_{i}\leftarrow\frac{pdf_{i}-\min\left(pdf_{i}\right)}{\max\left(pdf_{i} \right)-\min\left(pdf_{i}\right)}\);
15. \(\overline{pdf_{i\in 1..n}}\leftarrow\) RemoveHighCorrelatedFeatures\(\left(\ pdf_{i\in 1..n}\right)\)
16. Return\(pdf_{i\in 1..n}\) and \(\overline{pdf_{i\in 1..n}}\)
```

**Algorithm 4** Data pre-processing

#### 3.3.2 Model Building and Training

In this step, the decision-fusion-based model is built by training the three ensemble classifiers described in the previous subsection. The developed model is a meta-model that combines the predictions of GB, XGB, and RF classifiers using a weighted soft-voting rule.