# Optimizing Latent Factors and Collaborative Filtering for Students' Performance Prediction
## 3 Predicting Students' Performance in Online Campuses
### Matrix Factorization and Gradient Descent  
MF is a method used to formulate prediction models for some CF based RS. Consequently, it is also useful to solve the PSP problem. The prediction model is comprised of two matrices, \(W_{1}\) and \(W_{2}\), of sizes (\(S\times K\)) and (\(I\times K\)), respectively. MF approaches \(P\) to the product \(W_{1}W_{2}^{T}\), so that the predicted performance \(\hat{p}_{s,i}\) of user \(s\) and task \(i\) is calculated as in (1), where \(k\) identifies the latent factor from one to \(K\). \[\hat{p}_{s,i}=\sum_{k=1}^{K}{(w1_{s,k}w2_{i,k})}=(W_{1}W_{2}^{T})_{s,i} \tag{1}\]  
A training dataset is used to build this prediction model, whereas a test dataset (usually much smaller) is used to validate the training dataset by means of the Root Mean Squared Error (\(RMSE\)) criterion (2). \[RMSE=\sqrt{\frac{\sum_{s,i\in D^{test}}{(p_{s,i}-\hat{p}_{s,i})^{2}}}{|D^{test}|}} \tag{2}\]  
The prediction methodology follows three steps, according to the model described in [6], similar to the well-known Probabilistic Matrix Factorization (PMF) model [36]. First, we find the best parameters for \(W_{1}\) and \(W_{2}\) in the learning phase, through \(D^{train}\), by measuring the differences between real and predicted values. Next, we check the validity of the model by predicting the values of \(D^{test}\) and calculating the difference from the real values. Last, we calculate the unknown values of \(P\) from the optimal model (that with minimum \(RMSE\)). This process is summarized in Figure 1.  
Learning the model means finding the optimal values for \(W_{1}\) and \(W_{2}\) iteratively. Initially, both matrices contain random values as positive real numbers generated from the normal distribution \(N(0,\sigma^{2})\). Then, we calculate the error (3) on the training dataset, where \(e_{s,i}\) (4) is the error made by the prediction of \(p_{s,i}\), and consequently, \(e_{s,i}^{2}\) is calculated as in (10). \[err=\sum_{(s,i)\in D^{train}}e_{s,i}^{2} \tag{3}\] \begin{table}
\begin{tabular}{l l} \hline \hline
**Term** & **Meaning** \\ \hline \(S\) & Number of students \\ \(I\) & Number of tasks \\ \(K\) & Number of latent factors \\ \(P\) & Performance matrix (\(S\times I\)) \\ \(D^{know}\) & Set of known performances \\ \(D^{unk}\) & Set of unknown performances \\ \(D^{train}\) & Set of training performances, subset of \(D^{know}\) \\ \(D^{test}\) & Set of test performances, subset of \(D^{know}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Notation in the PSP problem. Figure 1: Scheme with the main steps followed by the prediction methodology. \[e_{s,i}=p_{s,i}-\hat{p}_{s,i}=p_{s,i}-\sum_{k=1}^{K}(w_{1_{s,k}}w_{2_{i,k}}) \tag{4}\]  
\[e_{s,i}^{2}=(p_{s,i}-\sum_{k=1}^{K}(w_{1_{s,k}}w_{2_{i,k}}))^{2} \tag{5}\]  
We can minimize the error in (10) by updating \(W_{1}\) and \(W_{2}\) iteratively, considering GD. This algorithm is very useful for solving problems when handling large datasets [37]. In order to apply GD, we must know, for each data point, in which direction to update the values \(w_{1_{s,k}}\) and \(w_{2_{s,k}}\) by calculating the gradients of \(p_{s,i}\). \[\frac{\partial}{\partial w_{1_{s,k}}}e_{s,i}^{2}=-2e_{s,i}w_  
First, we choose large-sized training datasets, comprising all the observed or known performance values. Since the size of the databases from virtual classrooms are not too big, they can be processed with the usual computer resources. Therefore, considering all the known performances as training data remains manageable. Second, we build the test dataset by selecting one performance value by each student (row) for consecutive tasks (columns). Figure 2 shows an example of this strategy. When the selected test performance is an unknown value, we skip to the next column of the same row (Case A in the figure) unless the end of the row has been reached, in which case, we select the first column of the next row (Case B in the figure). It is important to note that test data must be known data.