# Detection of Malicious Websites Using Symbolic Classifier
## 3 Results
### Results of GP Symbolic Classifier with Random Hyperparameter Search  
In this subsection, the classification accuracy achieved with GPSC applied to each variation of the dataset is presented. The dataset was split into train and test datasets in the ratio of 70:30 where 70% of the dataset was used to obtain symbolic expression with GPSC and the remaining 30% was used to evaluate each symbolic expression. The random hyperparameter search method was employed to investigate with which combination of GPSC hyperparameters high classification accuracy could be achieved. As already stated in the evaluation methodology the _AUC_ values were obtained for both the train and test datasets and were presented in terms of mean value and standard deviation. The hyperparameters of the GP symbolic classifier that were used to obtain high classification accuracy on each dataset variation are shown in Table 8. The best classification accuracy obtained on each dataset variation is shown in Figure 6. \begin{table}
\begin{tabular}{c c} \hline \hline  & Hyperparameter values \\  & (population\_size, generations, tournament\_size, \\ Dataset Type & init\_depth, p\_crossover, p\_subtree\_mutation, \\  & p\_hoist\_mutation, p\_point\_mutation, stopping\_criteria, \\  & max\_samples, const\_range, parsimony\_coefficient) \\ \hline  & 181, 100, 13, (6, 10), \\ Original dataset & 0.36, 0.12, 0.49, 0.013, \(1.4\times 10^{-5}\), \\  & 0.95, (\(-\)6.73, 9378.93), \(4.1\times 10^{-6}\) \\ \hline  & 129, 172, 23, (7, 10), 0.082, \\ Random undersampling & 0.65, 0.22, 0.032, \(9.7\times 10^{-4}\), \\  & 0.76, (\(-\)992.64, 5971.66), \(6.91\times 10^{-6}\) \\ \hline  & 200, 147, 17, (6, 7), \\ Random oversampling & 0.6, 0.13, 0.17, 0.09, \(5\times 10^{-4}\), \\  & 0.65, (\(-\)670.27, 8900.2), \(9.1\times 10^{-6}\) \\ \hline  & 181, 105, 15, (4, 8), \\ SMOTE & 0.44, 0.36, 0.053, 0.13, \(4.5\times 10^{-4}\), \\  & 0.61, (\(-\)628.93, 1907.64), \(7.69\times 10^{-6}\) \\ \hline  & 132, 202, 10, (7, 12), 0.37, \\ ADASYN & 0.1, 0.17, 0.36, \(1.5\times 10^{-4}\), \\  & 0.6, (\(-\)661.22, 6203.31), \(9.98\times 10^{-6}\) \\ \hline  & 140, 197, 15, (6, 8), 0.017, \\ Borderline SMOTE & 0.72, 0.14, 0.11, \(5\times 10^{-4}\), \\  & 0.99, (\(-\)411.4, 8789.7), \(1.38\times 10^{-6}\) \\ \hline  & 151, 140, 12, (7, 9), 0.17, \\ Kmeans SMOTE & 0.32, 0.058, 0.44, 0.85, \\  & (\(-\)540.69, 7952.57), \(3.81\times 10^{-6}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 8: The GP symbolic classifier hyperparameters used in each dataset variation to obtain high classification accuracy. Figure 6: The mean _ACC_, _AUC_, precision, recall, and F1-score values achieved with GPSC algorithm on each dataset variation (Standard deviation is presented in form of error bars). As seen from Figure 6, the lowest mean _AUC_ value was obtained in the case of the original dataset although results obtained from all datasets are high (0.974-0.999). The mean _AUC_ value in the case of the ADASYN dataset is pretty low when compared to other synthetically generated datasets. The highest standard deviation, i.e., the difference in results obtained on the train and test dataset was in the case of the original dataset followed by random undersampling. It is interesting to notice that using GPSC such a high mean _AUC_ value was obtained using the original dataset due to its large unbalance of dataset classes. The high classification accuracy in the case of the random undersampling dataset can be attributed to the huge reduction of benign websites to match the number of malicious websites. The random undersampling dataset was easier to load and use since it contained 70,630 instances instead of approximately 3 million in datasets obtained using oversampling methods. It should then be noticed that the GPSC algorithm used on the random oversampling dataset achieved a high _AUC_ value although the dataset was obtained by random selection of samples for the minority class. By doing so there are a large number of repeating samples from minority classes (malicious websites).