# A Two-Phase Ensemble-Based Method for Predicting Learners' Grade in MOOCs
## 4 Performance Evaluation  
In accordance with the procedure proposed in the previous section, the experiments were configured so that we could use both the HMPC and CNPC datasets. The experiment was mainly implemented using the Python programming language and the Scikit-learn library. To avoid the problems caused by a too-small minority class in the test set, the experiments were processed using five-fold cross-validation, in which 80 percent of the total dataset was designated as the training set and the remaining 20 percent was the testing set. The training and testing process were iterated five times, and the training and testing datasets were different each time. In each iteration of the experiment, the data in each class were divided evenly depending on their respective ratios. To evaluate the performance of a prediction model with an imbalanced dataset, the weighted-precision, weighted-recall, and weighted-average F1 scores are often considered as the standard evaluation measures [34; 35]. Equations (8)-(13) explain how to compute these evaluation scores. \[precision=\sum_{i=0}^{n}\biggl{(}\frac{tp_{i}}{(tp_{i}+fp_{i})}\biggr{)} \tag{8}\]  
\[recall=\sum_{i=0}^{n}\biggl{(}\frac{tp_{i}}{(tp_{i}+fn_{i})}\biggr{)} \tag{9}\]  
\[weighted\ precision=\frac{\sum_{i=0}^{n}(precision_{i}\times N_{i})}{\sum_{i=0}^{n}N_{i}} \tag{10}\]  
\[weighted\ recall=\frac{\sum_{i=0}^{n}(recall_{i}\times N_{i})}{\sum_{i=0}^{n}N_{i}} \tag{11}\]  
\[F1\ score=\biggl{(}2\times\frac{precision\times recall}{precision+recall}\biggr{)} \tag{12}\]  
\[weighted\ average\ F1\ score=\frac{\sum_{i=0}^{n}(F1 score_{i}\times N_{i})}{\sum_{i=0}^{n}N_{i}} \tag{13}\]  
where \(tp_{i}\) is the number of true positive instances for label \(i\). \(tn_{i}\) is the number of true negative instances for label \(i\). \(fp_{i}\) is the number of false positive instances for label \(i\). \(fn_{i}\) is the number of false negative instances for label \(i\). \(n\) is the number of class labels in the dataset. \(N_{i}\) is the number of instances or the support value for label \(i\). The precision value typically represents the proportion of valid positive predictions, whereas the recall value indicates the proportion of positive cases that have been correctly categorized. We also calculated the F1 score, which is a harmonic mean of the precision and recall. To apply these metrics for evaluating the prediction results of an imbalanced dataset, the scores will be divided by the number of true instances of each class. Table 5 represents the results of evaluation metrics that are average values from five-fold cross-validation. To evaluate the performance of the proposed two-phase ensemble-based method, the state-of-the-art machine learning RF method is compared to the improved RF method [17], which is an enhanced RF that was achieved by utilizing clustering SMOTE and a hybrid indicator with a decision mechanism to select the appropriate features. However, to compare it to the proposed method, the resampling data step is left out. The comparison also includes the RF Regressor [12], which is used to predict the learners' grades based on their exam results. Following the same conditions as in Equation (1), the outcomes of the prediction as a floating point score are binned to become a 'grade code'. The deep learning proposed in [13], which implemented a forward-backward algorithm and minimized the cost function by the gradient descent, is also examined. Table 5 illustrates the average weighted precision, weighted recall, and weighted average F1 scores from the five-fold cross-validation configuration of the experiment. To compare the weighted-precision and weighted-recall means, the weighted-average F1 score (weighted F1 score) is utilized. The optimization of the proposed model with the selected distance feature produced the highest weighted F1 score of the two datasets, whereas deep learning produced the lowest score.