# A Practical Model for the Evaluation of High School Student Performance Based on Machine Learning
## 4 Results  
Some attributes of the dataset contain categorical and string values such as parents' education or job attributes. Hence, this data should be encoded to numerical values for the training process. Due to the different range of feature values, data were normalized between 1 and 2 because some models such as SVM are sensitive to these different scales. Data were run on the mentioned models after pre-processing and preparing, and models were evaluated by the k-fold cross-validation technique in which the K variable was considered 10. In Figure 2, the models' learning curves are illustrated. In these curves, the accuracy rates of training and cross-validation models are shown during the addition of more instances (or more epochs). As demonstrated in Figure 2, the models have low bias so that selective models are suitable for our data and have good performance; however, the RF and LR models have more bias than SVM. Nevertheless, they are not overfitted and this bias is negligible. In the case where more training data is available, the models would have more certainty and the value of the bias would decrease. In Figure 3, the accuracy of the models obtained on training data and test data of the original dataset is shown on the box chart. The ANN model has the best performance on the training data. Moreover, this model has more certainty in the accuracy results than other models. In contrast, the RF model has a varied range of results, and the maximum and minimum lines are much different. Besides, RF has the least accuracy (0.82) on training data. On the right-hand side of Figure 3, box charts show the performance of the models on the test data. The accuracies of the models are diverse. The best performance of the models belongs to ANN and then, in order from highest to lowest, SVM, LR and RF models have lower accuracy in terms of the mean value. Notice that, the yellow line inside the boxes shows the median number and the white number inside the boxes shows the mean value in cross-validation results. Figure 2: Learning curve of models on the train set and cross-validation. (**a**) is associated with the learning curve of SVM, (**b**) is associated with RF, (**c**) is associated with LR and (**d**) is associated with ANN. Therefore, there is a difference between mean and median values in the box related to the LR model. If the median value was considered to compare the performance of models, the LR model has the worst performance on test data but the upper side of the LR box is larger than other boxes and this indicates that the model also has high accuracy in some iterations. Figure 4 illustrates the confusion matrix of classifiers that shows the model's ability to distinguish class boundaries. All models have had a good ability to distinguish classes. Although models have been misclassified on the boundary between each class, in class 0 and 1, which represent the A and B classes, all models have had great separation. Figure 4: (**a**) is confusion matrix of SVM, (**b**) is confusion matrix of RF, (**c**) is confusion matrix of LR and (**d**) is confusion matrix of ANN. Figure 3: Accuracy of models on the original dataset. (**a**) demonstrates the accuracy of the models on train data and (**b**) demonstrates the accuracy of the models on test data. The Boruta algorithm was used to obtain the most important feature of the dataset to evaluate the performance of students. This algorithm was implemented on the pre-processed original dataset using ten random forest estimators. The results of the Boruta algorithm are in Table 2. In this table, the importance for each feature, which is called ranking, is specified for each feature. The number of features that the model decides to be maintained can be specified by adjusting the algorithm threshold. As the table indicates, the number of high-quality features which have a number 1 ranking are selected for keeping. Thus, the number of features that are important for the model is 17.