# A Decision-Fusion-Based Ensemble Approach for Malicious Websites Detection
## 3 Materials and Methods
### Methodology of Proposed Approach
#### 3.2.1 Gradient-Boosting (GB) Classifier Method
This feature can create noise and decrease the accuracy because the difference between them will increase the distance of the model's decision boundary during the training step. The non-numeric values of features, including the 'Type' class label, are also encoded into numbers using label encoding because the classifiers can only learn the numeric values of the features. After that, the correlation between features is analyzed to find highly correlated features. Figure 5 provides a correlation heat map of the dataset features. The correlation heat map is a visual graphical representation of the relationship between features. Figure 5 demonstrates how each feature is correlated with another feature. It measures the strength of the relationship between every two feature variables. Understanding the correlation between features is useful because the value of one feature can be used to predict the other feature value. The correlated features indicate that, as the value of one feature changes, the other feature tends to change in a specific direction. As shown in Figure 5, the 'TCP_CONVERSATION_EXCHANGE', 'APP_PACKETS', 'SOURCE_APP_PACKETS', 'REMOTE_APP_PACKETS', 'APP_BYTES', and 'REMOTE_APP_BYTES' are highly corre

\[\hat{y}=arg\max_{j}\sum_{i=1}^{m}w_{i}f_{j}(x) \tag{1}\]3 Unique and frequency values of features in the dataset. lated features with one correlation score. In the next experiment, we will check whether removing these highly correlated features is necessary for a more accurate classification. ### Results of First Experiment

In first experiment, we randomly select a subset of the dataset to test the models. Two evaluation methods are applied to obtain the results of this experiment. The following subsections explain the evaluation methods, along with their outcomes, in detail. #### 4.1.1 Evaluation Method 1

This evaluation method randomly divides 30% of the dataset to test the models, and the remaining 70% is utilized for training. Figure 6 presents the number of instances of both malicious and benign classes in the training and test sets. As shown in Table 2, the distribution of instances in the training set is imbalanced, as 87.08% of the total classes are malicious and 12.92% are benign. \[\hat{y}=arg\max_{j}\sum_{i=1}^{m}w_{i}f_{j}(x) \tag{1}\]4 Correlation heat map of the dataset features. \[\hat{y}=arg\max_{j}\sum_{i=1}^{m}w_{i}f_{j}(x) \tag{1}\]5 Number of malicious and benign instances in the training and test sets. After building the DFEL model on the training set without the highly correlated features, it is tested on the test set. Figure 7 displays the confusion matrix of the classification results. The true positive (TP) and true negative (TN) instances are colored with a light green color. From Figure 7, we can show that the model can correctly classify 477 malicious instances out of 480 and 50 benign instances out of 55. Based on the corrected classified instances, Table 3 provides the recall, precision, F1-score, and accuracy results for classifying malicious and benign classes. As shown in Table 3, the proposed model achieves notable classification results, with an accuracy of 98.50%. Since the dataset classes are imbalanced, the accuracy metric is not enough for evaluation; hence, the F1-score is taken as another metric. In Table 3, we can see that the weighted avg. F1-score is 98.49%. The accuracy and F1-score results prove the model's capability and effectiveness in alleviating the class-imbalanced problem produced by the dominant malicious class label. To show the effect of highly correlated features, we train the DFEL model on the training set with all features, including the highly correlated features, and test it on the same test set. Figure 8 visualizes the confusion matrix of the classification results. The numbers in the light green color on the confusion matrix are TP and TN instances.