# Transfer Learning from Deep Neural Networks for Predicting Student Performance
## 2 The Transfer Learning Approach  
The traditional supervised learning methods exploit labeled data to obtain predictive models in the most efficient way. Let us consider the task of predicting whether a student is going to successfully pass or fail the examinations of an undergraduate course \(C_{A}\). In this case, the training and the testing set are both derived from the same domain (course). The training set is used to build a learning model \(h\) by means of a classification algorithm (e.g., a deep network) and subsequently it is applied on the testing set for evaluating its predictive performance (Figure 1a). Some key requirements for achieving high performance models are the quality and sufficiency of the training data which are, unfortunately, not always easy to meet in real world problems. In addition, the direct implementation of model \(h\) for a different course \(C_{B}\) or a new task (e.g., predicting whether a student is going to drop out of the course) seems rather difficult. The existing model does not have the ability to generalize well to data coming from a different distribution, while, at the same time, it is not applicable, since the class labels of the two tasks are different. Contrasting these methods, knowledge transfer or transfer learning intends to improve the performance of learning and provide efficient models in cases where data sources are limited or difficult and expensive to acquire [1; 2], primarily due to their generalization ability to heterogeneous data (i.e., data from different domains, tasks and distributions [19]). Transfer learning might help us to train a predictive model \(h\) based on data derived from course \(C_{A}\) (source course) and apply it on data derived from a different but related course \(C_{B}\) (target course), which are not sufficient to train a model, for predicting the performance of a student. This indeed, is the aim of transfer learning: transfer the knowledge acquired from course \(C_{A}\) to course \(C_{B}\) and improve the predictive performance of model \(h\) (Figure 1b) instead of developing a totally new model, on the basis that both datasets should share some common attributes (i.e., common characteristics of students, such as their academic achievements or interactions within an LMS). More formally, the transfer learning problem is defined as follows [1; 20]:  
A domain \(\mathcal{D}\) is formed by a feature space \(\mathcal{X}\) and a marginal probability distribution \(P(X)\), where \(X=\{x_{1},x_{2},\ldots,x_{n}\}\in\mathcal{X}\). A learning task \(\mathcal{T}\) is formed by a label space \(\Upsilon\) and an objective predicted function \(f(\cdot)\). The function \(f\) can be also written as \(P(y|x)\), representing the conditional probability distribution of label \(y\) given a new instance \(x\). \(P(y|x)\) is learned from the training data \(\{\mathcal{X},\Upsilon\}\). Given a source domain \(\mathcal{D}_{S}=\{\mathcal{X}_{S},P_{S}(X)\}\), its corresponding learning task \(\mathcal{T}_{S}=\{\Upsilon_{S},\ f_{S}(\cdot)\}\), a target domain \(\mathcal{D}_{T}=\{\mathcal{X}_{T},P_{S}(X)\}\) and its corresponding learning task \(\mathcal{T}_{T}=\{\Upsilon_{T},\ f_{T}(\cdot)\}\), the purpose of transfer learning is to obtain an improved target predictive function \(f_{T}(\cdot)\) by using the knowledge in \(\mathcal{D}_{S}\) and \(\mathcal{T}_{S}\), where \(\mathcal{D}_{S}\
eq\mathcal{D}_{T}\) and \(\mathcal{T}_{S}\
eq\mathcal{T}_{T}\). The fact that \(\mathcal{D}_{S}\
eq\mathcal{D}_{T}\) means that either \(\mathcal{X}_{S}\
eq\mathcal{X}_{T}\) or \(P(X_{S})\
eq P(X_{T})\), where \(X_{S_{i}}\in\mathcal{X}_{S}\) and \(X_{T_{i}}\in\mathcal{X}_{T}\). Similarly, the fact that \(\mathcal{T}_{S}\
eq\mathcal{T}_{T}\) means that either \(\Upsilon_{S}\
eq\Upsilon_{T}\) or \(f_{S}(\cdot)\
eq f_{T}(\cdot)\). The inequalities contained in the definition form four different transfer learning settings:  
* \(\mathcal{X}_{S}\
eq\mathcal{X}_{T}\): the feature space of the source and target domain are different.