# Using Machine Learning for Web Page Classification in Search Engine Optimization
## 6 Evaluation Results  
The model evaluation was conducted by the hold-out method and cross validation. In the hold-out method, two-thirds of the data set were used for training the model and one-third for its validation. For cross validation, a method of 10-fold nested cross validation was used (with 10 iterations in the inner loop and 10 in the outer loop). For training the classification models, an R tool with the MLR ([[https://github.com/mlr-org/mlr](https://github.com/mlr-org/mlr)]([https://github.com/mlr-org/mlr](https://github.com/mlr-org/mlr)), accessed October 2018) package was used. The training was performed by means of five classification algorithms: decision trees, SVM (Support Vector Machine), Naive Bayes, kNN, and logistic regression ([PERSON] et al., 2017), using the classification accuracy as the evaluation measure. All classifier hyperparameters were tuned with the grid search method. The data were normalized with the min-max method. In cross validation a stratification method was used. Table 7 summarizes the hyperparameter tuning results and accuracy obtained by two validation methods. To analyze the suitability of the data set size, a learning curve was created and plotted for each of the classifiers, as shown in Figure 2. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Variable** & **Correlation** & **Information Gain** & **Chi Square** & **Relief** & **Random Forest** \\ \hline Tlen & 0.257135 & 0.0554402 & 0.3264777 & **2.26 \(\times\) 10\({}^{4}\)** & **21.810107** \\ Tkw & **0.426339** & **0.12293193** & **0.4786585** & \(6.65\times 10^{3}\) & **42.189959** \\ Mlen & **0.388327** & **0.20411246** & **0.5681143** & **3.15 \(\times\) 10\({}^{4}\)** & **51.131839** \\ Mkw & **0.575413** & **0.23275999** & **0.6215979** & **1.70 \(\times\) 10\({}^{4}\)** & **42.067928** \\ h1 & 0.022324 & 0.03913948 & 0.2757038 & \(6.64\times 10^{1}\) & 10.321804 \\ h1 & 0.037721 & 0.03686936 & 0.2678983 & \(-1.22\times 10^{2}\) & 11.734166 \\ h1 & **0.300638** & **0.06749659** & **0.3611576** & **7.46 \(\times\) 10\({}^{3}\)** & 15.475284 \\ h2 & 0.135253 & 0.00000000 & 0.0000000 & \(-8.69\times 10^{3}\) & 15.867193 \\ h2 & 0.099092 & 0.00000000 & 0.0000000 & \(-7.56\times 10^{3}\) & 17.243045 \\ h2 & 0.272581 & 0.05016715 & 0.3036099 & \(1.96\times 10^{3}\) & 13.157693 \\ h3 & 0.077233 & 0.00000000 & 0.0000000 & \(4.78\times 10^{3}\) & 7.044129 \\ h3 & 0.057554 & 0.00000000 & 0.0000000 & \(-4.67\times 10^{3}\) & 7.638001 \\ h3 & 0.182965 & 0.0404664 & 0.27710277 & \(-5.10\times 10^{3}\) & 4.511746 \\ alt & 0.123755 & 0.0000000 & 0.0000000 & \(-3.28\times 10^{3}\) & 8.7917176 \\ altKw & 0.244176 & 0.04367333 & 0.2905980 & \(-2.45\times 10^{2}\) & 5.516950 \\ linkKw & **0.277557** & 0.05469602 & 0.3335990 & \(2.80\times 10^{3}\) & 12.373683 \\ linkOut & 0.082039 & 0.00000000 & 0.0000000 & \(7.93\times 10^{1}\) & 13.193339 \\ urlLen & 0.013055 & 0.00000000 & 0.0000000 & \(6.23\times 10^{3}\) & 1.648894 \\ urlKw & 0.210249 & 0.05061797 & 0.3141724 & \(-7.25\times 10^{3}\) & 11.379830 \\ txtLen & 0.002467 & 0.03307188 & 0.2336873 & \(1.23\times 10^{3}\) & 6.708839 \\ txtKw & 0.268072 & **0.09057668** & **0.4262453** & **9.74 \(\times\) 10\({}^{3}\)** & **31.804311** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Variable importance based on various tests. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Classifier** & **Hyperparameters** & **Accuracy—Holdout** & **Accuracy—10-Folds** \\  & **Optimal Values** & **Method** & **Cross Validation** \\ \hline Decision trees & C = 0.49, M = 16 & 65.67\% & 67.53\% \\ Naive Bayes & - & 58.71\% & 54.69\% \\ KNN & K = 45, \(p\) = 2 & 65.17\% & 69.67\% \\ SVM & C = 7.74 \(\times\) 10\({}^{3}\), \(\sigma\) = 0.000464 & 62.68\% & 66.18\% \\ Logistic regression & C = 2 & 62.19\% & 62.99\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Optimal values of hyperparameters. From Figure 2 one can observe that the accuracy steeply grew, up to about 50% of the training set size, which indicates that the data set size utilized in this research was suitable.