# Use of Deep Multi-Target Prediction to Identify Learning Styles
## 3 Materials and Methods
\[\begin{array}{l}\text{IF (active\_reflective }<0)\text{ [THEN active\_reflective }=\text{true ELSE false]}\\ \text{IF (sensing\_intuitive }<0)\text{ [}\text{ [}\text{{THEN sensing\_intuitive }}=\text{true }\text{  
active. On the other hand, if a student's processing dimension is \(>\)0, then the student receives FALSE, denoting that it is reflective.  
In addition, we investigate whether the dataset is imbalanced for each target. Imbalanced datasets mean the instances of one class is larger than the instances of another class (for example, more sequential rather than global in understanding dimension), where the majority and minority of class or classes are taken as negative and positive, respectively [11]. Figure 3 shows the distribution of each target.  
As shown in Figure 3, this dataset does not have imbalanced data for any of the targets. For each dataset, the imbalance ratio (IR) is given by the division of the majority class by the minority class [18]. As a result, we obtained active_reflective (1.04), sensing_intuitive (1.22), visual_verbal (1.63), and sequential_global (1.38).  
The algorithm chosen for multi-target prediction was artificial neural network (ANN) for five reasons: (a) there is evidence that this algorithm is better suited to solve learning style classification problems [5]; (b) since many authors use this algorithm, we can compare our results with other published ones [3]; (c) ANN works well with rather small datasets, which is important for this line of research considering that typical datasets are rather small [17]; (d) the problem can be translated to the network structure of an ANN; and (e) ANN allows multiple outputs analyzed at the same time. Moreover, the ANN architecture we used is feedforward multilayer perceptron, which means a neural network with one or more hidden layers [17; 20].  
The hidden layers act as feature detectors; as such, they play an important role in the operation of a multilayer perceptron. As the learning process advances throughout the multilayer perceptron, step by step the hidden neurons start to discover the features that describe the training data. They do so by performing nonlinear processing on the input data and transforming them into a new space, called the feature space. In this new space, the classes of interest in a pattern-classification task, for instance, may be more easily separated from each other than they could in the original input data space. Indeed, it is the creation of this feature space through supervised learning which distinguishes the multilayer perceptron from perceptron. Literature suggests that the number of hidden layers should be between log T (where T is the size of the training set) and 2\(\times\) the number of inputs [17].  
A popular approach for training multilayer perceptron is the back-propagation algorithm, which incorporates the least mean squares (LMS) algorithm as a special case. The training proceeds in two steps. In the first one, referred to as the forward phase, the synaptic weights of the network are updated and the input signal is propagated through the network, layer by layer, until the output. As a consequence, in this phase, adjustments are limited to the activation potentials and outputs of the neurons in the network. (b) In the second one, called the backward phase, an error signal is produced by evaluating the output of the network with an expected response. The resulting error signal is propagated through the network, again layer by layer, but this time the propagation is performed in the backward direction. In this second step, successive updates are made to the synaptic weights of  
Figure 3: The dataset’s target distribution of each dimension. The % of each student’s preferences are represented, such a, sequential 42%, global 58%, visual 38%, verbal 62%, sensing 45%, intuitive 55%, active 49%, and reflective 51%.  
the network. Calculation of the updates for the output layer is straightforward, but it is much more difficult for the hidden layers [17].  
The back-propagation algorithm affords an approximation to the trajectory in weight space computed by the method of the stochastic gradient descent [17]. The smaller the value of the learning rate parameter \(\alpha\), the smaller the changes to the synaptic weights in the network. Consequently, it will be from the first iteration to the next in a smoother fashion during the trajectory in weight space. This improvement, however, is attained at the cost of a slower rate of learning. The learning rates used were between 0.01 and 0.1, in steps of 0.01, leading to the following values: (0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1) [17].  
A training set is one of labeled data (for example, if a student is active or reflective) providing known information, which is used in supervised learning to build a classification or regression model. The training dataset is used to train the model (weights and biases in the case of artificial neural network) and then the model can see and learn from this data.  
The model test is a critical but frequently underestimated part of model building and assessment. After preprocessing the data, they are needed to build a model with the potential to accurately predict further observations. If the built model completely fits the training data, it is probably not reliable after deployment in the real world. This problem is called overfit and needs to be avoided. A common manner on how to address the lack of an independent dataset for model evaluation is to reserve part of the learning data for this purpose. The basis for analyzing classifier performance is a confusion matrix (CF). This matrix describes how well a classifier can predict classes.  
A typical cross validation technique is the k-fold cross validation. This method can be viewed as a recurring holdout method (holdout method divides the original dataset in two subsets, training and testing datasets). The whole data is divided into k equal subsets and each time a subset is assigned as a test set, the others are used for training the model. Thus, each observation gets a chance to be in the test and training sets; therefore, this method reduces the dependence of performance on test-training split and decreases the variance of performance metrics. Further, the extreme case of k-fold cross validation will occur when k is equal to the number of data points. It would mean that the predictive model is trained over all the data points except by one, which takes the role of a test set. This method of leaving one data point as a test set is known as leave-one-out cross validation (LOOCV) [17]. This technique is show in Figure 4.  
As shown in Figure 4, each iteration leaves one observation to test and the others to train. Therefore, the number of iterations is the number of observations in the dataset. The use of the leave-one-out procedure allows the model to be tested with all observations and prevents us from wasting these observations. This method was used to split the original dataset.  
A classifier is evaluated based on performance metrics computed after the training process. In a binary classification problem, a matrix presents the number of instances predicted by each of the four possible outcomes: number of true positives (#TP), number of true negatives (#TN), number of false positives (#FP), and number of false negatives (#FN). Most classifier performance metrics are derived  
Figure 4: Leave-one-out cross validation technique (LOOCV). It is a special case of cross validation where the number of folds equals the number of instances in the dataset [17].  
from the four values [21]. We used the following metrics in order to improve the accuracy of our model (Equations (3)-(14)) [22].  
\[\text{Sensitivity }=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{3}\]  
\[\text{Specificity }=\frac{\text{TN}}{\text{FN}+\text{TN}} \tag{4}\]  
\[\text{Prevalence }=\frac{\text{TP}+\text{FP}}{\text{TP}+\text{FN}+\text{FP}+ \text{TN}} \tag{5}\]  
\[\text{PPV }=\frac{\text{sensitivity }\ast\text{ prevalence}}{(\text{sensitivity }\ast\text{ prevalence})+((1-\text{specificity})\ast(1-\text{ prevalence}))} \tag{6}\]  
\[\text{NPV }=\frac{\text{specificity}\ast(1-\text{prevalence})}{((1-\text{ sensitivity})\ast\text{prevalence})+((1-\text{specificity})\ast(1-\text{prevalence}))} \tag{7}\]  
\[\text{Detection Rate }=\frac{\text{TP}}{\text{TP}+\text{FN}+\text{FP}+ \text{TN}} \tag{8}\]  
\[\text{Detection Prevalence }=\frac{\text{TP}+\text{FN}}{\text{TP}+\text{ FN}+\text{FP}+\text{TN}} \tag{9}\]  
\[\text{Balanced Accuracy }=\frac{\text{sensitivity}+\text{specificity}}{2} \tag{10}\]  
\[{\rm x_{i}}^{\prime}=\frac{{\rm x_{i}}-{\rm min_{i}}[{\rm x_{i}}]}{{\rm max_{ i}}[{\rm x_{i}}]-{\rm min_{i}}[{\rm x_{i}}]} \tag{1}\]0  
\[{\rm x_{i}}^{\prime}=\frac{{\rm x_{i}}-{\rm min_{i}}[{\rm x_{i}}]}{{\rm max_{ i}}[{\rm x_{i}}]-{\rm min_{i}}[{\rm x_{i}}]} \tag{1}\]1  
\[{\rm x_{i}}^{\prime}=\frac{{\rm x_{i}}-{\rm min_{i}}[{\rm x_{i}}]}{{\rm max_{ i}}[{\rm x_{i}}]-{\rm min_{i}}[{\rm x_{i}}]} \tag{1}\]2  
\[{\rm x_{i}}^{\prime}=\frac{{\rm x_{i}}-{\rm min_{i}}[{\rm x_{i}}]}{{\rm max_{ i}}[{\rm x_{i}}]-{\rm min_{i}}[{\rm x_{i}}]} \tag{1}\]3  
For binary problems, the sensitivity, specificity, positive predictive value, and negative predictive value were calculated using the positive argument.