# Transfer Learning from Deep Neural Networks for Predicting Student Performance
## 4 Research Methodology
### The Proposed Transfer Learning Approach  
The present study intends to address the problem of transferring knowledge across different undergraduate courses. Hence, we employed a simple deep neural network architecture, comprised four layers: an input layer, two hidden dense layers and an output one. The input layer consists of input units corresponding to each one of the dataset input features (Table 3). The first hidden layer has 12 hidden units and the second one has eight. Both dense layers use the Relu activation function. Finally, the output layer consists of a single neuron employing the sigmoid activation function and the binary cross entropy loss function for predicting the output class (pass or fail). The experimental procedure was divided into three distinct phases (Figure 2). In the first phase, we constructed all the unique pairs of courses that could be formed (ten pairs of courses in total). Each time, the related datasets were rebuilt to share a common set of features. For each pair of courses, we made use of the following notation:  
\[\left\{C_{i},C_{j}\right\}\ i,j\in\{1,2,3,4,5\},\ i\
eq j. \tag{1}\] \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Learning Resources** & \(C_{1}\) & \(C_{2}\) & \(C_{3}\) & \(C_{4}\) & \(C_{5}\) & **Description** & **Possible Values** \\ \hline Forum & 1 & 1 & 1 & 1 & 1 & & \\ Page & 7 & 6 & 2 & 2 & 0 & & \\ Recourse & 17 & 15 & 4 & 12 & 10 & Total number of times a student accessed the resource & 0 or positive integer \\ Url & 0 & 0 & 1 & 1 & 0 & & \\ Assignments & 8 & 9 & 8 & 8 & 9 & Student Grades & [0, 10] decimal \\ Submitted Assignments & 3 & 9 & 8 & 8 & 9 & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Features extracted from students’ low-level interaction logs. In order to create a common set of features for each pair of courses, we matched features of the first course to related features of the second course one by one. Among the common features were the gender as well as the course total activity and course total views counters. Therefore, the first assignment of the first course was matched with the first assignment of the second course, the second assignment of the first course was matched with the second assignment of the second course and so forth, while the same procedure was followed for all the six types of resources. In cases where a matching feature was not found, a new feature was created, with zero values for each instance. For example, the \(C_{1}\) course contained features related to seven page resources, whereas the \(C_{2}\) course contained features related to six page resources (Table 2). Finally, the new \(\{C_{1},\ C_{2}\}\) pair of datasets contained seven features regarding the page resources, since a new empty feature was created and added in the \(C_{2}\) course dataset, thus matching to the seventh feature of the \(C_{1}\) course (Table 3). The second phase refers to the training process of the two supporting deep networks. The first one was trained on the new source course \(C_{i}\) in order to extract its adjusted weights, while the second one was trained on the new target course \(C_{j}\) in order to calculate the baseline evaluation. In both cases, we calculated the accuracy metric, which corresponds to the percentage of correctly classified instances, while the models were trained for 150 epochs. In addition, the 10-fold cross validation resampling procedure was adopted for evaluating the overall performance of the deep network models. The third phase was the most fundamental, since it implemented the transfer learning strategy. The deep model of the target course was fitted from scratch, but this time the network weights were initialized using the previously calculated weights from the source course (second phase). The pre-trained model was further tuned by running it each time for a certain number of epochs (hereinafter denoted as \(C_{i,j}\)): zero (i.e., the starting point), 10, 20, 30, 40, 50, 100 and 150. Algorithm 1 provides the pseudocode of the proposed transfer learning method.