# Self-Improving Generative Artificial Neural Network for Pseudorehearsal Incremental Class Learning
## 4 Experiments
### Experiment 1: Does the Method Gradually Forget?  
As a first experiment, our model was trained to learn iteratively on splits of these datasets, in order to study if our network can retain previous information while learning new data, and to see how much information is lost when learning a new task using only what the model remembers of the previous task. The dataset was split into four parts, each with an even number of classes. The network was trained using an initial set. After training each set, we evaluated classification accuracy and generated a set of images from each learned class, to append it to the new learning task. The generated images were selected for training if the predicted class from the classifier matched the input class from the generator. In the steps that followed, we retrained the model with the next batch of new classes, plus a defined number of generated data: around 75% of the amount of the original training samples for each class from the previous set. We decided upon this value due to an unbalancing issue with the previously learned classes when training the new set.  
As a base metric, the obtained accuracy for our classifier when evaluating the complete dataset was 87.71% on EMNIST; this result is similar to state of the art on this particular subset (Performance results of different Deep Learning techniques for the MNIST dataset can be found in [[https://benchmarks.ai/mnist](https://benchmarks.ai/mnist)]([https://benchmarks.ai/mnist](https://benchmarks.ai/mnist)), in [46] the authors reports the performance for the EMNIST dataset). While training the generated set, the accuracy of each training iteration lowered in a constant decay per training step of around 9.27 \(\pm\) 0.75%. When trained to add a small subset of the original data to the generated samples, the decay rate in each training step is cut almost in half, as shown in Table 1. This decay can be explained as a gradual interference effect to do with how information is learned within the model, and related to the model's plasticity when learning new features of the newer classes. As a comparison, we trained the same model with generated samples and an additional 10% of randomly sampled original training data, to test performance in pseudorehearsal training.  
This gradual loss of information can be seen in the generated samples from the model in Figure 3. In each training split, the recently learned task starts with a better form style definition, due to how it is encoded on the distribution vector. In following splits, its shape starts to lose form due to the increasing number of classes within the encoding space.  
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Training** & **SIGANN with the** & **Pseudorehearsal** & **Rehearsal SIGANN** \\
**Scheme** & **Complete Data Set** & **SIGANN** & **with 10\% Real Data** \\ \hline Split 1 & 87.71\% & 97.91 \(\pm\) 0.24\% & 97.97 \(\pm\) 0.17\% \\ Split 2 & \(-\) & 89.25 \(\pm\) 0.54\% & 90.34 \(\pm\) 0.42\% \\ Split 3 & \(-\) & 81.84 \(\pm\) 0.98\% & 86.58 \(\pm\) 0.55\% \\ Split 4 & \(-\) & 70.11 \(\pm\) 2.21\% & 81.28 \(\pm\) 0.71\% \\ \hline Decay Rate & \(-\) & 9.27 \(\pm\) 0.75\% & 5.56 \(\pm\) 0.21\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: Accuracy evaluated in the test sets using EMNIST partition in four splits.