# Student Performance Prediction with Regression Approach and Data Generation
## 3 Experiment and Analysis of Results
### Generation and Use of Synthetic Data
The weights between the input layer and the hidden layer are chosen as the encoder, and the weights between the hidden layer and the output layer are chosen as the decoder. The hidden layer, which is the output layer of the encoder, uses sigmoid as the activation function--this scales the encoding result into 0-1, making it easier for the GANs to learn. The encoder and the decoder are trained on the whole student mathematics performance dataset to best encode and recover the data. Out of eleven trained autoencoders, the one with the best performance (recovering 75.3% of data after encoding) is chosen to embed the data. The imperfection of the auto-encoder also performs as a noise input in this structure since the decoded data are sometimes not the same as the input. The encoder and decoder compress the data dimension into a hyperspace, forming a denser vector, which alleviates the problem caused by highly discrete student data distribution. To train and test the GANs, the encoder first embeds the data from a 59-dimension vector (after one-hot encoding) into a 15-dimension vector. Then, the GANs are trained using the 15-dimension vectors as the collected data. After the GANs are trained, the decoder decodes the output vectors from the generator into one-hot vectors, and then the one-hot vectors are eventually parsed into normal student data format (Figure 1). A GAN system consists of a generator and a discriminator. The discriminator is trained to discriminate data generated by the generator. The generator is trained to fool the discriminator by generating data that are as real as possible. Ideally, when the GANs training is finished, the discriminator reaches the highest loss because it cannot discriminate the generated data, and the generator reaches the lowest loss--0 because all the generated data are recognized as real data by the discriminator. The loss function for the discriminator is shown as follows:  
\[Loss_{D}=-\frac{1}{n}\sum_{i=1}^{n}[lnD(x^{(i)})+ln(1-D(G(z^{(i)})))]=BCE(D(x),1 )+BCE(D(G(z),0)) \tag{1}\] \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Age** & **Medu** & **Fedu** & **Absences** & **G1** & **G2** & **Traveltime** & **Studytime** & **Faliures** & **Famrel** \\ \hline
15 & 1 & 0 & 0 & 0 & 1 & 1 & 1 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Sample data generated by GANs using one-hot encoding. Figure 1: Data generation workflow. The loss function for the generator is shown as follows:  
\[Loss_{D}=-\frac{1}{n}\sum_{i=1}^{n}ln(1-D(G(z^{(i)})))=-BCE(D(G(z),0)) \tag{2}\]  
In Equations (1) and (2), \(x\) represents the collected data, \(z\) represents the input of the generator, \(G(z)\) represents generated data, \(D(a)\) represents the discriminant result of input \(a\), and \(n\) represents the batch size. BCE stands for binary cross entropy loss. During the experiment, multiple GANs are trained, and some GANs do not convert as expected. Table 5 shows some sampled data from an abandoned generator. The sampled data show that this generator generates data with similar feature values. These generated data may match a certain point of the real data distribution so that the discriminator cannot distinguish them. These data pieces are useless because they cannot fill in the gaps between the collected data samples and the real data distribution. In [PERSON]'s paper [21], the improvement of GANs in image generation from 2014 to 2017 is simply demonstrated with an illustration. However, when it comes to student data synthesis, it becomes difficult to tell whether a generated piece of student performance data is good or not. In this research, the generator is chosen based on the following rules:  
* First, the GANs should convert smoothly. As shown in Figure 2, the abandoned GANs convert with a very unusual loss change. It is found that during the experiment, ln(0) is processed as \(-1\), which has a great impact on the training result. Figure 3 shows the converting process of the chosen GANs, which have smoother loss changes.