## 5 Discussion  
After applying the first experiment, without hyper-tuning on both data split ratios 30:70 and 20:80, the obtained result, as shown in Table 3, demonstrates that our model outperforms the work presented in [3], even without hyper-tuning and using the same testing:training ratio. This is related to the preprocessing performed before training the models, including the normalization and hot encoding, which transformed the data into a format suitable for ML algorithms, leading to higher performance. Moreover, increasing the training dataset size against the testing dataset size (80:20) had a positive impact on the classification rates, especially the RF algorithm, with an increase of 9%. This underlines that the model did not see all the samples when the 70:30 ratio was used, as some characteristics existed on the testing and not the training dataset. This implies the need to use k-fold cross-validation, where the dataset is split into K sets, and the test is applied K times, which can make the models more generalizable and guarantee that the model behavior is optimal. The result is comparable with the results in [3]; however, it can be improved by applying hyper-tuning for the hyperparameters.  
For the next experiment, hyper-tuning was applied on all the ML models. Figure 4a and Table 5 show the confusion matrix for the DT model, which shows the classification rate, where the TPR = 66.67% and FPR = 12% with an accuracy of 79%. This means that the model was able to predict 66.67% the choice of the Marketing and Finance major correctly, while 12.00% of this major was classified erroneously, with the actual choice being Marketing and Human Resources. The SVM, as shown in Figure 4b and Table 5, shows the confusion matrix for the SVM model, with TPR = 75% and FPR = 11.11%. These values show that the SVM model classified 75% of the Marketing and Finance major correctly, while 11.11% was labeled incorrectly with this major. The RF algorithm with the selected parameters was able to outperform both the DT and SVM, with an accuracy of 95% using the five-fold cross-validation. Figure 4c shows the confusion matrix for the RF, with TPR = 94.90% and FPR = 4%. For the prediction of studying Marketing and Finance, 94.90% matched this major, and only 4% was labeled erroneously. Moreover, the constructed ROC in Figure 5  
Figure 6: Input features importance chart.  
implies that the RF has the best performance in implementing our classifier, as its curve is confirmed closest to the upper left corner, which is the optimal point of TPR = 100.00% and FPR = 0.00%; RF had a very close value of TPR = 94.90% and FPR = 4.00%.  
The final experiment was the feature importance test and the dropping of insignificant features. The results in Table 8 illustrate the accuracy after dropping the aforementioned features. It can be seen that the classification rate increases after dropping the low-importance features mentioned above from the dataset.  
In this research, we used the same approach as in [3] as well as the same dataset. However, in our proposed model, we hyper-tuned the ML methods, which led to real improvements in the prediction rates, as shown in Table 9. As part of the evaluation of our machine learning models, a comparison was conducted with this article's outcome. The following table summarizes the accuracy of each model for the related work, our model without/with hyper-tuning using a 30:70 training:testing data ratio and without/with hyper-tuning using a 20:80 training:testing data ratio.  
The results in Table 9 demonstrate that our model outperforms the work presented in [3]; these results confirmed the significance of the correct preprocessing of the data, increasing the training data set size, the use of cross-validation, which helps to make the model more generalizable and perform better on the unseen data, and the hyper-tuning of the machine learning algorithms, which finds the optimal combination of hyperparameters, leading to an improvement in the classification rate and a more accurate model.