## 5 Experimental Evaluation
### Data Preparation and Evaluation Metrics
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Feature Pattern** & **No. of Features** & **Accuracy** & **Precision** & **Recall** & **F1-Measure** \\ \hline \multirow{4}{*}{**U**} & 24,961 (100\%) & 85.29 & 85.67 & 85.29 & 85.36 \\  & 19,969 (80\%) & 85.46 & 85.85 & 85.46 & 85.52 \\  & 16,225 (65\%) & 85.33 & 85.73 & 85.33 & 85.4 \\  & 12,481 (50\%) & 85.36 & 85.73 & 85.36 & 85.42 \\ \hline \multirow{4}{*}{**B**} & 165,704 (100\%) & 66.21 & 74.62 & 66.21 & 68.05 \\  & 132,563 (80\%) & 66.35 & 74.28 & 66.35 & 68,34 \\  & 107,708 (65\%) & 66.34 & 74.17 & 66.34 & 68.31 \\  & 82,852 (50\%) & 66.39 & 74.11 & 66.39 & 68.26 \\ \hline \multirow{4}{*}{**T**} & 181,514 (100\%) & 42.14 & 73.48 & 42.14 & 45.66 \\  & 145,211 (80\%) & 42.08 & 73.23 & 42.08 & 45.47 \\  & 117,984 (65\%) & 42.73 & 73.62 & 42.73 & 45.96 \\  & 90,757 (50\%) & 43.94 & 76.51 & 43.94 & 47.19 \\ \hline \multirow{4}{*}{**Q**} & 168,482 (100\%) & 30.39 & 72.93 & 30.39 & 30.95 \\  & 134,786 (80\%) & 30.53 & 81.22 & 30.53 & 30.57 \\  & 109,513 (65\%) & 30.26 & 81.8 & 30.26 & 30.25 \\  & 84,241 (50\%) & 29.99 & 82.4 & 29.99 & 30.08 \\ \hline \multirow{4}{*}{**U,B**} & 190,665 (100\%) & 84.98 & 85.28 & 84.98 & 85.02 \\  & 152,532 (80\%) & 85.14 & 85.43 & 85.14 & 85.17 \\  & 123,932 (65\%) & 85.26 & 85.53 & 85.26 & 85.29 \\  & 95,333 (50\%) & 85.3 & 85.57 & 85.3 & 85.34 \\ \hline \multirow{4}{*}{**B,T**} & 347,218 (100\%) & 64.39 & 74.26 & 64.39 & 66.53 \\  & 277,774 (80\%) & 64.82 & 74.37 & 64.82 & 67.18 \\  & 225,692 (65\%) & 64.9 & 74.13 & 64.9 & 67.18 \\  & 173,609 (50\%) & 65.62 & 73.93 & 65.62 & 67.66 \\ \hline \multirow{4}{*}{**T,Q**} & 349,996 (100\%) & 39.3 & 73 & 39.3 & 42.65 \\  & 279,997 (80\%) & 39.68 & 73.01 & 39.68 & 43.04 \\  & 227,497 (65\%) & 41.09 & 73.27 & 41.09 & 44.31 \\  & 174,998 (50\%) & 43.36 & 79.93 & 43.36 & 46.24 \\ \hline \multirow{4}{*}{**U,B,T**} & 372,179 (100\%) & 84.02 & 84.35 & 84.02 & 84.05 \\  & 297,743 (80\%) & 84.45 & 84.74 & 84.45 & 84.47 \\  & 241,916 (65\%) & 84.61 & 84.91 & 84.61 & 84.64 \\  & 186,090 (50\%) & 84.97 & 85.26 & 84.97 & 85.01 \\ \hline \multirow{4}{*}{**U,B,T**} & 540,661 (100\%) & 83.48 & 83.88 & 83.48 & 83.52 \\  & 432,529 (80\%) & 83.39 & 83.68 & 83.39 & 83.39 \\  & 351,430 (65\%) & 84.37 & 84.67 & 84.37 & 84.4 \\  & 270,331 (50\%) & 83.85 & 84.13 & 83.85 & 83.86 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Experimental results for Support Vector Machines (SVM) classifier. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Feature Pattern** & **No. of Features** & **Accuracy** & **Precision** & **Recall** & **F1-Measure** \\ \hline \multirow{4}{*}{**U,B,T**} & 372,179 (100\%) & 78.59 & 79.1 & 78.59 & 78.72 \\  & 297,743 (80\%) & 77.95 & 79.1 & 77.95 & 78.22 \\  & 241,916 (65\%) & 75.25 & 78.31 & 75.25 & 75.83 \\  & 186,090 (50\%) & 75.28 & 78.43 & 75.28 & 75.77 \\ \hline \multirow{4}{*}{**U,B,T**} & 540,661 (100\%) & 78.19 & 78.99 & 78.19 & 78.36 \\  & 432,529 (80\%) & 77.28 & 78.93 & 77.28 & 77.62 \\  & 351,430 (65\%) & 74.23 & 78.02 & 74.23 & 74.91 \\  & 270,331 (50\%) & 74.36 & 77.82 & 74.36 & 74.86 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: _Cont._Figure 4 depicts the results obtained by each classifier for all feature patterns by using the full set of features extracted from the training set. The blue line depicts the results obtained by the Naive Bayes classifier; the red line depicts the results obtained by the kNN classifier; the brown line depicts the results obtained by the SVM classifier, the black line depicts the results obtained by the Random-Forest classifier. We can notice that the Naive Bayes classifier (blue line) always performed as the best classifier, always obtaining the highest accuracy. The SVM classifier (brown line) performed only a little bit worse, but results were comparable. The Random Forest classifier still showed comparable accuracy, even though a little bit less than Naive Bayes and SVM classifiers. In contrast, the inability of the kNN classifier to exploit most of feature patterns was evident.