# Detection of Malicious Websites Using Symbolic Classifier
## 3 Results
### Results of GP Symbolic Classifier with Random Hyperparameter Search and 5-Fold Cross-Validation  
In this case, each dataset variation was initially divided into train and test datasets in the ratio of 70:30. The 70% of the dataset was used for 5-fold cross-validation. Before each cross-validation, the hyperparameters of the GPSC algorithm were randomly selected. If the mean value of _AUC_ after 5-fold cross-validation exceeded 0.98 the GPSC was again trained on 70% of the dataset and tested on 30% of the dataset. In the case where the mean _AUC_ value was below 0.98 the process of 5-fold CV was repeated. The combination of randomly selected hyperparameters for the GP symbolic classifier for each dataset variation is shown in Table 10 while mean and standard deviation _AUC_ values are shown in Figure 7. \begin{table}
\begin{tabular}{c c} \hline \hline \multirow{2}{*}{Dataset type} & Hyperparameter values (population\_size, generations, tournament\_size, init\_depth, p\_crossover, p\_subtree\_mutation, p\_hoist\_mutation, p\_point\_mutation, stopping\_criteria, max\_samples, const\_range, parsimony\_coefficient) \\ \hline \multirow{2}{*}{Original dataset} & 120,100, 20, (5, 10), 0.007, \\  & 0.83, 0.05, 0.1, \(5\times 10^{-4}\), \\  & 0.63, (\(-761.24\), 3968.75), 9.22 \(\times 10^{-6}\) \\ \hline \multirow{2}{*}{Random undersampling} & 187, 159, 39, (6, 8), 0.24, \\  & 0.39, 0.32, 0.04, 0.0007, 0.6, \\  & (\(-80.5\), 5509.26), 6.21 \(\times 10^{-6}\) \\ \hline \multirow{2}{*}{Random undersampling} & 150, 197, 26, (6, 11), 0.14, \\  & 0.66, 0.07, 0.11, \(2.6\times 10^{-4}\), 0.98, \\  & (\(-953.15\), 9148.94), \(5.58\times 10^{-6}\) \\ \hline \multirow{2}{*}{SMOTE} & 108, 180, 17, (7, 9), 0.44, \\  & 0.03, 0.17, 0.35, \(2\times 10^{-4}\), \\  & 0.86, (\(-557.91\), 103.76), 6.95 \(\times 10^{-5}\) \\ \hline \multirow{2}{*}{ADASYN} & 157, 174, 18, (3, 8), 0.31, \\  & 0.33, 0.2, 0.15, \(5.3\times 10^{-5}\), \\  & 0.74, (\(-566.18\), 9234.9), \(4.4\times 10^{-6}\) \\ \hline \multirow{2}{*}{Borderline SMOTE} & 182, 167, 20, (7, 11), 0.17, \\  & 0.74, 0.015, 0.065, \(4\times 10^{-5}\), 0.62, \\  & (\(-316.14\), 14.06), \(2.57\times 10^{-6}\) \\ \hline \multirow{2}{*}{Kmeans SMOTE} & 103, 185, 17, (7, 9), 0.45, \\  & 0.36, 0.12, 0.064, \(6\times 10^{-4}\), 0.77, \\  & (\(-687.65\), 8699.9), \(3.32\times 10^{-6}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 10: GPSC hyperparameters used to obtain best symbolic expressions in terms of high classification accuracy. Figure 7: The mean and standard deviation of evaluation metric values achieved with GPSC, random hyperparameter search, and 5-fold cross-validation on each dataset variation. As seen from Figure 7, the mean _ACC_, _AUC_, _Precision_, _Recall_, and _F1-Score_ values are similar to those obtained in the previous investigation. However, the results obtained for all dataset variations showed a slight improvement in evaluation metric values. The only exception is in the case of symbolic expression obtained for the BorderlineSMOTE dataset which achieved lower evaluation metric values with the application of random hyperparameter search and 5-fold cross-validation than in the case of GPSC with random hyperparameter search method only. The _ACC_, _AUC_, _Precision_, _Recall_, and _F1-Score_ values, in this case, dropped lower than in the case of the original dataset while the standard deviation value increased. In Table 11 the mean and standard deviation values of _ACC_, _AUC_, _Precision_, _Recall_, and _F1-Score_ are shown as well as average CPU time per simulation. The importance of this table was to primarily show how low the standard deviation values of used evaluation metrics are since they are not visible in Figure 7. The average CPU time per simulation, in this case, is much longer than in the previous case. This is because of the 5-fold cross-validation process, i.e., the training is performed on the training dataset 5 times on different folds.