## 4 Results and Discussion
The best accuracy (52%) of SVM was achieved with the following default parameters (random_state = 0, tol = 1 \(\times\) 10\({}^{-5}\)), as shown in Table 6. The TPR and FPR of SVM are 0.53 and 0.49, respectively, as shown in Table 7.  
Finally, we used the XGBoost classifier to predict student specializations by using default parameters (base_score = 0.5, booster = 'gptree', colsample_bylevel = 1, learning_rate = 0.1). XGBoost is a popular boosting technique in ensemble ML, and its performance is good on structured and tabular data. XGBoost is also called GBC. XGBoost uses parallel tree boosting to solve real-life data science problems. We used this technique because its impact has been widely recognized in many machine learning and data mining challenges, where \begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & **Accuracy** \\ \hline Decision Tree Algorithm & 0.5538 \\ \hline Random Forest Algorithm & 0.7538 \\ \hline Extra Trees Classifier & 0.5231 \\ \hline Support Vector Machine & 0.5231 \\ \hline XGBoost & 0.6154 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Experimental results of the machine learning models on the current study dataset. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **TN = 28 (0.80)** & **FP = 7 (0.20)** \\ \hline
**Actually (0)** & FN = 9 (0.30) & TP = 21 (0.70) \\ \hline
**Actually (1)** & **Predicted (0)** & **Predicted (1)** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Confusion matrix of the random forest classifier. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **TN = 18 (0.51)** & **FP = 17 (0.49)** \\ \hline
**Actually (0)** & FN = 14 (0.47) & TP = 16 (0.53) \\ \hline
**Actually (1)** & **Predicted (0)** & **Predicted (1)** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Confusion matrix of the support vector machine (SVM). \begin{table}
\begin{tabular}{c c c} \hline \hline  & **TN = 28 (0.80)** & **FP = 7 (0.20)** \\ \hline
**Actually (0)** & FN = 9 (0.30) & TP = 21 (0.70) \\ \hline
**Actually (1)** & **Predicted (0)** & **Predicted (1)** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Confusion matrix of the random forest classifier. it has become a commonly used and popular tool among Kaggle's competitors and data scientists [87]. XGBoost predicts the student's specialization with an accuracy of 61%, and the TPR and FPR of the XGBoost classifier are 0.57 and 0.35, respectively, as shown in Table 8.  
In the first experiment, the TPR and accuracy of the RF and GBC classifiers were higher than DT, SVM, and ETC, and the FPR was lower than that of DT, SVM, and ETC, as shown in Figure 5, which indicates that the performance of these classifiers in predicting field specializations is good, compared to that of the alternatives. Sometimes, accuracy is misleading when the dataset is imbalanced [94, 95, 96]. In other words, if the ratio of some classes is less than that of others in the dataset, we used the ROC curve and TPR, which is also called recall. We used ROC to further understand the performance of the models. The ROC is an evaluation metric that represents the performance of an ML model in the form graph [97] by plotting the TPR and FPR of the model. Figure 5 shows that the TPR of the RF and GBC classifiers was high, compared to that of the DT, SVM, and ETC. The results also showed that RF and GBC classifiers are appropriate classifiers to build student field recommendation systems because they can handle ordinal, non-ordinal, and categorical data and are also good choices for skewed and multimodal data [98]. Moreover, the RF and GBC classifier ensemble method outperforms simple DT classifiers. The previous study showed that the performance of SVM in small data is good and faster [93]. Furthermore, DT and ETC are unstable and have high sensitivities for overfitting classifiers [15]. **Question 2:** How is a student's undergraduate major path choice associated with that student's academic performance and the job market? We performed a second experiment to investigate the second research question.