# A Decision-Fusion-Based Ensemble Approach for Malicious Websites Detection
## 3 Materials and Methods
### Methodology of Proposed Approach
#### 3.2.1 Gradient-Boosting (GB) Classifier Method
However, the difference between the accuracy scores for the training and validation of the base models is larger than that of the accuracy scores of the DFEL model for all training sets with more than 600 examples. This means that the performance of the DFEL model is better than its base classifier models. ### Results of Second Experiment

In this experiment, another evaluation experiment is conducted using a 10-fold cross-validation routine to compare the DFEL model with its base models (GB, XGB, and RF). Table 5 demonstrates the average accuracy and F1 score of the ten-fold validation results for malicious and benign classification using GB, XGB, RF, and DFEL models trained on training folds without highly correlated features. As seen in Table 5, the highlighted numbers with bold font are the best evaluation results achieved using a 10-fold cross-validation technique. We can see that the proposed DFEL model outperforms its base models. It can classify the malicious and benign cases with a 97.20% weighted average F1 score and 97.25% average accuracy. In addition, the macro-average F1 scores for each model are less than the micro-average F1 scores because of the class-imbalanced problem. The macro-average F1 score can provide a true evaluation of the classifier model in a class-imbalanced evaluation task. Figure 11: Learning curves of the DFEL model and its base models: (**a**) learning curve of GB classifier, (**b**) learning curve of XGB classifier, (**c**) learning curve of RF classifier, and (**d**) learning curve of DFEL model. To confirm whether highly correlated features affect the DFEL's performance results, we performed 10-fold cross-validation on the training set with highly correlated features. Figure 12 visualizes the 10-fold cross-validation averaged accuracy and F1 score results of the proposed DFEL model trained on the training folds with and without highly correlated features. From Figure 12, we can see that the DFEL model with highly correlated features attains an average of 96.91% and 96% instead of 97.25% and 97.2% in its 10-fold accuracy and 10-fold weighted average F1 scores, respectively. Moreover, there is a significant difference in the average of the 10-fold macro-averaged F1 scores, in which the model improves the result from 90.4% to 93.1%. These outcomes also confirm that removing highly correlated features is necessary for a more accurate classification. ### Comparison of Results with Related Work

In this subsection, we compared the accuracy of the results of this study with the results of methods and techniques in some recent related studies. Table 6 lists the obtained accuracy result of the proposed DFEL model compared with the accuracies of important studies in the literature review. Figure 12: Averaged accuracy and F1 score results of the proposed DFEL model trained on a 10-fold cross-validation with and without highly correlated features. \begin{table}
\begin{tabular}{c c c c c} \hline \multirow{2}{*}{**Model**} & \multicolumn{4}{c}{**Averaged F1 Score**} & **Averaged** \\ \cline{2-5}  & **Micro Avg.** & **Macro Avg.** & **Weighted Avg.** & **Accuracy** \\ \hline GB & 0.955 & 0.888 & 0.954 & 95.45\% \\ XGB & 0.958 & 0.895 & 0.957 & 95.85\% \\ RF & 0.962 & 0.904 & 0.960 & 96.18\% \\ RF + GB & 0.958 & 0.894 & 0.957 & 95.79\% \\ GB + XGB & 0.963 & 0.905 & 0.962 & 96.29\% \\ RF + XGB & 0.967 & 0.914 & 0.966 & 96.74\% \\ Proposed DFEL & 0.973 & 0.931 & 0.972 & 97.25\% \\ \hline \end{tabular}
\end{table} 
Table 5: The 10-fold cross-validation averaged accuracy and F1 score results of the GB, XGB, RF, and DFEL models trained on the training folds without highly correlated features. As shown in Table 6, our DFEL model results demonstrates a substantial improvement (98.50% vs. 96.4% in terms of accuracy) compared to the results of our recent work. Such a notable performance in terms of website classification provides a possible application of the proposed DFEL model to help cyber-security technicians and researchers in the detection of malicious websites.