## 3 Results
### Supervised Machine Learning Models
#### 3.5.3 Created Models  
We implemented several multi-aspect supervised machine learning models. The aim of these models was to use the widest possible range of variables for the most accurate prediction of the student's GPA. Since our goal was to make the most accurate predictions of the student's GPA, we decided to evaluate the models not from a statistical but from a predictive point of view. As stated before, we therefore implemented the so-called direct validation of our models through error metrics. Hyperparameters and independent model variables were optimized for the best results. The objective of the model optimization was to minimize the error. In the case of linear regression models, the model optimization was performed by the methodology of selection of suitable variables (feature selection). For decision tree and random forest models, cycle was used, which gradually changed the size of the minbucket or nodesize hyperparameters during cross-validation phase. After cross-validation, final validation was performed with such hyper-parameters where the lowest average percentage error was recorded. We created several models of linear regression. We used different approaches for features selection. In all models, the dependent variable was the student's GPA. We modelled the dependent variable using other, independent variables. All mentioned models were created in the R programming language. As stated before, for higher credibility of our results, the linear regression models were also validated through cross-validation. Finally, the model was trained on the training set data and validated on the final validation set data. First, a LIN1 linear regression model was created. This model predicted the dependent variable using such independent variables where a correlation with the dependent variable from correlation data analysis was identified. The LIN2 linear regression model was created almost in the same way; however, multicollinearity between independent variables was removed. The Variance Inflation Factor was used to identify these variables. Subsequently, all variables with a VIF value of more than 5 were removed. In this way, the variables fVideosAlgPS and sAverageSecond were removed. We also used standard feature selection procedure for selecting variables. We used forward regression as well as backward regression. In forward regression, the variables were added to the linear regression model gradually, and then the model, which was the best in terms of the AIC criterion, was selected. During backward regression, the variables were gradually removed from the model. In Table 5 we can see the results of our linear regression models. As seen in Table 5, all models achieved good results. From the results of cross-validation, on average, model LIN4 (implemented through backward regression) was the most accurate on validation sets. Regression decision tree (RDT) was also used to predict the dependent variable. Two regression decision tree models were created. The first regression decision tree (RDT1) used \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**Train Sets**} & \multicolumn{2}{c}{**Test Sets**} \\ \hline
**Model** & **Avg MSE** & **Avg MAPE [in \%]** & **Avg MSE** & **Avg MAPE [in \%]** \\ \hline LIN1 & 0.02209241 & 6.14795342 & 0.11141661 & 14.87208277 \\ LIN2 & 0.03921313 & 8.20007161 & 0.12469694 & 15.53585540 \\ LIN3 & 0.02209241 & 6.14795342 & 0.11146161 & 14.87208277 \\ LIN3v & 0.03921313 & 8.20007161 & 0.12469694 & 15.53585540 \\ LIN4 & 0.02921812 & 7.09180468 & 0.06782395 & 11.91675271 \\ LIN4v & 0.04322373 & 8.65998242 & 0.13475872 & 16.23674155 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Error metrics of linear regression models, average results on 5-fold cross-validation. all variables as inputs. The second model (RDT2) used as inputs only variables in which we found that they statistically significantly affect the student's GPA.