# Transfer Learning from Deep Neural Networks for Predicting Student Performance
## 4 Research Methodology
### The Proposed Transfer Learning Approach
At the starting point for each transfer learning model (i.e., zero epochs) we used the weights estimated by the previously trained deep network models (on 150 epochs) instead of starting with

\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{\(\{\{C_{1},C_{2}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{1},C_{3}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{1},C_{4}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{1},C_{5}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{2},C_{3}\}\}\)} \\  & \(C_{1}\) & \(C_{2}\) & \(C_{1}\) & \(C_{3}\) & \(C_{1}\) & \(C_{4}\) & \(C_{1}\) & \(C_{5}\) & \(C_{2}\) & \(C_{3}\) \\ \hline Baseline & 0.7627 & 0.6094 & 0.7667 & 0.7047 & 0.7563 & 0.6333 & 0.7424 & 0.5591 & 0.6011 & 0.7144 \\ \hline Epochs & \(C_{2,1}\) & \(C_{1,2}\) & \(C_{3,1}\) & \(C_{1,3}\) & \(C_{4,1}\) & \(C_{1,4}\) & \(C_{5,1}\) & \(C_{1,5}\) & \(C_{3,2}\) & \(C_{2,3}\) \\ \hline
0 & 0.6106 & **0.6172** & 0.7524 & **0.8227** & 0.5675 & **0.6371** & 0.5889 & **0.8644** & **0.6306** & 0.6538 \\
10 & **0.7701** & **0.6414** & **0.7988** & **0.8382** & **0.8128** & **0.6387** & **0.8087** & **0.8561** & 0.5975 & **0.8234** \\
20 & **0.7842** & **0.6299** & **0.7917** & **0.8537** & **0.8093** & 0.5833 & **0.8055** & **0.8553** & **0.6132** & **0.8382** \\
30 & **0.7877** & **0.6132** & **0.8198** & **0.8537** & **0.8126** & 0.6008 & **0.8092** & **0.8644** & **0.6290** & **0.8394** \\
40 & **0.7732** & **0.6234** & **0.8022** & **0.8608** & **0.8236** & 0.6075 & **0.8019** & **0.8553** & **0.6241** & **0.8453** \\
50 & **0.7766** & 0.6076 & **0.8062** & **0.8465** & **0.8166** & 0.6325 & **0.7987** & **0.8386** & **0.6179** & **0.8537** \\
100 & **0.7768** & 0.5968 & **0.7847** & **0.8465** & **0.8061** & **0.6762** & **0.7916** & **0.8114** & **0.6077** & **0.8394** \\
150 & 0.7552 & **0.6185** & **0.7882** & **0.8620** & **0.7738** & 0.6325 & **0.7845** & **0.8371** & **0.6064** & **0.8472** \\ \hline  & \multicolumn{3}{c}{\(\{\{C_{2},C_{4}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{2},C_{5}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{3},C_{4}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{3},C_{5}\}\}\)} & \multicolumn{3}{c}{\(\{\{C_{4},C_{5}\}\}\)} \\  & \(C_{2}\) & \(C_{4}\) & \(C_{2}\) & \(C_{5}\) & \(C_{3}\) & \(C_{4}\) & \(C_{3}\) & \(C_{5}\) & \(C_{4}\) & \(C_{5}\) \\ \hline Baseline & 0.5650 & 0.6263 & 0.5498 & 0.6735 & 0.7096 & 0.6196 & 0.7715 & 0.7955 & 0.6811 & 0.6646 \\ \hline Epochs & \(C_{4,2}\) & \(C_{2,4}\) & \(C_{5,2}\) & \(C_{2,5}\) & \(C_{4,3}\) & \(C_{3,4}\) & \(C_{5,3}\) & \(C_{3,5}\) & \(C_{5,4}\) & \(C_{4,5}\) \\ \hline
0 & **0.6207** & 0.6008 & 0.4112 & **0.8379** & 0.4641 & 0.4988 & **0.8156** & **0.8644** & 0.5375 & **0.6902** \\
10 & **0.5941** & 0.5817 & **0.5916** &



randomly initialized weights. For example, at the starting point of the \(C_{4,2}\) transfer model, we used the weights estimated by the \(C_{4}\) model. Comparing the results of the pretrained weights without further tuning (i.e., zero epochs) to the baseline model, an improvement is noticed in half of the datasets (10 out of 20). The statistical results (t-test) confirm that the difference is not significant when the pre-trained model is not further tuned for the second dataset (target course \(C_{j}\)), since p-value=0.2449>=0.05. However, the transfer model prevails in 16 out of 20 datasets when it is further tuned for only 10 epochs. 2. Will the pre-trained model reduce the training effort for the deep model of the second course? Overall, the increase in the number of epochs improves the performance of the proposed transfer learning model. Moreover, the improvement is significant for every number of epochs, apart from the starting point, as statistically confirmed by the t-test results. It is worth noting that the transfer model prevails in 18 out of 20 datasets after 100 epochs, where the lowest p-value is 0.0002. In addition, we can detect three cases of overfitting, since the accuracy ceases to improve after a certain number of iterations and begins to decrease.