# Detection of Malicious Websites Using Symbolic Classifier
## 3 Results
### Results of GP Symbolic Classifier with Random Hyperparameter Search
The highest mean _AUC_ value was achieved with dataset KmeansSMOTE without overfitting since the difference between train/test scores is almost negligible. In Table 9, the numeric values of mean and standard deviation values of the evaluation metric used in this research are shown as well as the average CPU time per simulation in minutes. The main purpose of this table is two show how low the standard deviation values of evaluation metrics are. The average CPU time per simulation shows how much time was required to generate a symbolic expression. The process is measured from GPSC random hyperparameter selection to finally obtain the symbolic expression. The term average indicates that with GPSC and randomly selected hyperparameters the symbolic expression was not obtained at first and multiple executions of the algorithm were required. There are multiple influences on CPU time required to obtain the symbolic expression using GPSC and these are dataset size, population size, number of generations, and parsimony coefficient. As seen from Table 9 for the datasets that were balanced with oversampling methods, the GPSC with random hyperparameter search requires more time to generate symbolic expression than in the case with random undersampling methods. The population size of GPSC has some influence on the CPU time required to obtain symbolic expression. \begin{table}
\begin{tabular}{c c c c c c c} \hline \multirow{2}{*}{**Dataset Type**} & \(\overline{ACC}\) & \(\overline{AUC}\) & \(\overline{Precision}\) & \(\overline{RecaII}\) & \(\overline{F1 SCORE}\) & **Average CPU Time** \\  & \(\pm SD(ACC)\) & \(\pm SD(AUC)\) & \(\pm SD(Precision)\) & \(\pm SD(Recall)\) & \(\pm SD(F1\)-score) & **per Simulation [min]** \\ \hline Original & 0.974 & 0.9745 & 0.974 & 0.973 & 0.9745 & \\ Dataset & \(\pm 4.99\times 10^{-4}\) & \(\pm 4.99\times 10^{-4}\) & \(\pm 6.6\times 10^{-4}\) & \(\pm 8.16\times 10^{-4}\) & \(\pm 5\times 10^{-4}\) & 45 \\ \hline Random & 0.9978 & 0.9978 & 0.997 & 0.998 & 0.9978 & \\ undersampling & \(\pm 3.37\times 10^{-5}\) & \(\pm 1.9\times 10^{-4}\) & \(\pm 2.82\times 10^{-4}\) & \(\pm 2.91\times 10^{-5}\) & 20 \\ \hline Random & 0.9979 & 0.9978 & 0.998 & 0.997 & 0.9979 & \\ oversampling & \(\pm 1.31\times 10^{-5}\) & \(\pm 4.81\times 10^{-5}\) & \(\pm 1.83\times 10^{-5}\) & \(\pm 7.01\times 10^{-6}\) & \(\pm 1.26\times 10^{-5}\) & 60 \\ \hline ADASYN & 0.9876 & 0.9878 & 0.987 & 0.987 & 0.9876 & \\  & \(\pm 6.24\times 10^{-5}\) & \(\pm 2.742\times 10^{-5}\) & \(\pm 5.72\times 10^{-5}\) & \(\pm 1.35\times 10^{-5}\) & \(\pm 6.28\times 10^{-5}\) & 60 \\ \hline SMOTE & 0.9962 & 0.9965 & 0.997 & 0.995 & 0.9962 & \\  & \(\pm 3.649\times 10^{-5}\) & \(\pm 1.56\times 10^{-5}\) & \(\pm 1.35\times 10^{-5}\) & \(\pm 7.97\times 10^{-5}\) & \(\pm 3.31\times 10^{-5}\) & 60 \\ \hline Borderline & 0.9952 & 0.9955 & 0.995 & 0.994 & 0.9952 & \\ SMOTE & \(\pm 2.24\times 10^{-5}\) & \(\pm 1.619\times 10^{-6}\) & \(\pm 3.07\times 10^{-5}\) & \(\pm 6.57\times 10^{-5}\) & \(\pm 1.91\times 10^{-5}\) & 60 \\ \hline KMeans & 0.9992 & 0.9995 & 0.9995 & 0.999 & 0.9992 & \\ SMOTE & \(2.249\times 10^{-5}\) & \(\pm 9.945\times 10^{-6}\) & \(\pm 1.09\times 10^{-5}\) & \(\pm 5.17\times 10^{-5}\) & \(\pm 5.17\times 10^{-6}\) & 60 \\ \hline \end{tabular}
\end{table} 
Table 9: The mean and standard deviation values of _ACC_, _AUC_, _Precision_, _Recall_, and _F1_-_score_ obtained in case of GPSC with random hyperparameter search on train/test split in ratio 70:30. The larger the population is the more time will be required to finish the GPSC execution process. The number of generations is a hyperparameter that represents the maximum number of generations for which the GPSC population will be evolved. After the maximum number is reached the symbolic expression will be obtained. So the higher the value of this hyperparameter the longer time it takes for GPSC to obtain symbolic expression.