## 5 Experiment and Results
### Machine Learning Classifiers
The results of the testing performance of different ML classifiers on our dataset are shown in Tables 7-9. Table 7 shows the results of the classification study obtained when using the RF algorithm. According to the experiments conducted in this category, the highest success was obtained when using WordNet ontology and Doc2 Vec together, with an accuracy of 93.01%. This success was achieved with Doc2 Vec using N = 200 as vector size, W = 8 as the window size, 100 workers, and 25 epochs. The results of the experiments using the SVM algorithm are shown in Table 8. In the experiments, the highest success value, an accuracy of 92.25%, was obtained when using WordNet and Doc2 Vec methods together. The parameters used for these methods are given in Tables 6a, 6d and 6f. Finally, in the experiments conducted in the MLP category shown in Table 8, the highest accuracy value of 91.63% was obtained when using the WordNet and Doc2 Vec models together, as shown in Table 9. The parameters of the methods with high results in this category are shown in Tables 6d and 6g. Figure 9 presents the ROC curves for the methods with the highest success among the machine learning classification models. The area values of each class label are shown separately in the ROC curves. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline BoW + RF & 91.29\% & 90.53\% & 90.87\% & 91.35\% \\ TF-IDF + RF & 90.66\% & 90.33\% & 90.43\% & 91.35\% \\ Word2 Vec + RF & 90.57\% & 88.34\% & 89.32\% & 91.14\% \\ Doc2 Vec + RF & 92.67\% & 91.41\% & 91.96\% & 92.39\% \\ WordNet + BoW + RF & 90.40\% & 89.78\% & 90.04\% & 90.73\% \\ WordNet + TF-IDF + RF & 92.19\% & 91.70\% & 91.94\% & 91.90\% \\ WordNet + Word2 Vec + RF & 92.60\% & 90.27\% & 91.33\% & 91.77\% \\ WordNet + Doc2 Vec + RF & 92.55\% & 92.79\% & 92.62\% & 93.01\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Macro averaged scores for RF classification. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline BoW + SVM & 89.36\% & 90.42\% & 89.70\% & 90.45\% \\ TF-IDF + SVM & 89.95\% & 90.09\% & 89.91\% & 91.00\% \\ Word2 Vec + SVM & 86.96\% & 87.36\% & 87.07\% & 88.03\% \\ Doc2 Vec + SVM & 91.46\% & 91.52\% & 91.41\% & 92.18\% \\ WordNet + BoW + SVM & 91.76\% & 91.10\% & 91.38\% & 91.90\% \\ WordNet + TF-IDF + SVM & 90.04\% & 90.94\% & 90.37\% & 90.66\% \\ WordNet + Word2 Vec + SVM & 87.20\% & 86.39\% & 86.68\% & 87.89\% \\ WordNet + Doc2 Vec + SVM & 91.70\% & 92.05\% & 91.85\% & 92.25\% \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Macro averaged scores for SVM classification. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline BoW + MLP & 89.54\% & 89.37\% & 89.43\% & 90.38\% \\ TF-IDF + MLP & 83.72\% & 82.64\% & 82.87\% & 84.99\% \\ Word2 Vec + MLP & 75.91\% & 73.89\% & 74.63\% & 81.32\% \\ Doc2 Vec + MLP & 90.69\% & 89.48\% & 90.00\% & 91.07\% \\ WordNet + + BoW + MLP & 90.35\% & 90.78\% & 90.51\% & 91.07\% \\ WordNet + TF-IDF + MLP & 87.60\% & 87.30\% & 8.41\% & 89.14\% \\ WordNet + Word2 Vec + MLP & 79.41\% & 75.75\% & 77.03\% & 81.88\% \\ WordNet + Doc2 Vec + MLP & 91.84\% & 90.73\% & 91.21\% & 91.63\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Macro averaged scores for MLP classification. As a result, it was seen in the experiments conducted in the RF, SVM and MLP categories that the Doc2 Vec model was effective at increasing the classification success, as well as WordNet. This is because of the semantic preservation of documents achieved by taking the sum and average of all word vectors in the Doc2 Vec method. Another finding is that while high success was achieved with low numbers of epochs for the Doc2 Vec model, the opposite was true for the Word2 Vec model.