## 4 Results
Figure 8: Accuracy for SVM classifier considering cross-validation. Figure 6: Accuracy for KNN classifier using cross-validation. Figure 7: The ROC-AUC value for the KNN model. Table 4 summarises the cross-validated performance of the three classifiers on our multi-class research method prediction task. We report mean accuracy \(\pm\) standard deviation across five stratified folds, along with macro-averaged precision, recall, F1-score, and ROC-AUC. The random forest classifier achieved the highest accuracy (93.8 \(\pm\) 1.9%) and macro-F1 (0.93), indicating robust performance across all seven method categories. Its ROC-AUC of 0.94 further demonstrates strong discriminative ability between classes (Figure 4). In contrast, SVM and KNN attained moderate accuracy (-84.4%) and F1-scores (0.84 and 0.83, respectively), with the SVM's higher ROC-AUC (0.92) reflecting a superior margin-based separation of classes compared to KNN's similarity-based approach (ROC-AUC 0.88) (Figures 7 and 9). Table 5 presents a detailed breakdown of the model's performance across each of the seven research method classes. The random forest classifier exhibits its strongest results on regression analysis, achieving a precision of 0.95, a recall of 0.98, and an F1-score of 0.96, reflecting high accuracy and excellent coverage for this dominant category. Time-series analysis and experimental design also demonstrate robust performance (F\({}_{1}\)-scores of 0.90 and 0.89, respectively). In contrast, qualitative methods such as ethnographic research and grounded theory, yield slightly lower F1-scores (0.82 and 0.81), indicating that minority classes remain more challenging. Overall, the consistency between precision and recall values suggests balanced performance, with no class exhibiting disproportionate bias toward false positives or false negatives. The detailed confusion matrix (Figure 10) reveals that random forest uniformly classified standard methods (e.g., regression analysis, time-series analysis) with F1-scores above 0.95. At the same time, minor classes, such as ethnographic research and grounded theory, saw slightly lower F1 (above 0.80), reflecting fewer training examples. SVM exhibited exceptionally high precision (0.97) for regression analysis--its dominant class--at the expense of recall (0.85), suggesting that its decision boundary favoured conservative predictions for well-represented methods. KNN's performance dipped on minority classes, likely due to the high-dimensional TF-IDF feature space reducing neighbour relevance. To contextualise classifier performance, we evaluated two baselines alongside our primary models (Table 6). A majority class predictor that always selects regression analysis yielded an expected accuracy of 21.8%, underscoring the inherent difficulty of the seven-way classification task. A logistic regression model, trained with balanced class weights, improved accuracy to 72.4%, indicating that linear decision boundaries capture substantial-but not all--feature complexity. In contrast, the random forest ensemble achieved 93.8% accuracy, demonstrating that non-linear, tree-based methods offer markedly superior discriminative power in mapping project descriptions to appropriate research methodologies. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **ROC-AUC** \\ \hline Random Forest & 0.938 & 0.93 & 0.94 & 0.93 & 0.94 \\ \hline Support Vector Machine & 0.844 & 0.85 & 0.84 & 0.84 & 0.92 \\ \hline K-Nearest Neighbours & 0.844 & 0.86 & 0.83 & 0.83 & 0.88 \\ \hline Logistic Regression & 0.724 & 0.70 & 0.72 & 0.71 & 0.81 \\ \hline Majority Class Baseline & 0.218 & — & — & — & — \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Benchmark comparison of classifier performance. Figure 10: Confusion matrix of random forest classification across seven research method categories.