# Transfer Incremental Learning Using Data Augmentation
## 5 Conclusions  
In this paper, we have introduced TILDA, a new incremental learning approach inspired by recently proposed methods. TILDA relies on a pre-trained DNN to process data, a projection technique that defines low-dimensional subspaces, NCM inspired classifiers, and data augmentation at both the training and prediction phases. This addresses previous concerns from previous methods, specifically: (a) iCaRL as TILDA reaches a good accuracy when the streamed data contains one class at a time, (b) BRIL as TILDA provides a good accuracy comparable to state-of-art method, (c) NCM as TILDA uses \(k\) anchor vectors instead of one and other methods to increase the accuracy, and (d) Learn++ as TILDA still performs well even if steam data does not contains examples of all classes each time. Experiments on challenging datasets show that: (a) TILDA does not suffer from catastrophic forgetting, (b) TILDA approaches state-of-art accuracy, (c) TILDA uses much less memory usage and gets the same accuracy as nearest neighbour search or even better, (d) TILDA still gives a good accuracy even in the case where we have only one class each time, and (e) finally, to our knowledge, TILDA is the incremental method that reaches the best accuracy. This method is also promising for embedded devices, since it is not necessary to train a DNN or compute extensive operations for learning.  
In future work, we plan to explore further the methods for splitting feature vectors, data augmentation strategies and a weighted majority vote to improve the accuracy. We also plan to propose a hardware architecture of TILDA for incremental learning on chip.  
[PERSON] and [PERSON] conceived and designed the experiments; [PERSON] performed the experiments; [PERSON] and [PERSON] analyzed the data; [PERSON] wrote the original draft, [PERSON], [PERSON] and [PERSON] edited and reviewed the paper, [PERSON] supervised this work.  
The research for this paper was financially supported by Pole de Recherche Avance en Communications (Pracom).  
We would like to thank Nvidia which gave us GPUs we used to compute experiments.  
**Conflicts of Interest:** The authors declare no conflicts of interest.  
\begin{table}
\begin{tabular}{l|l|l|l|l|l|l|l|l} \hline  & \multicolumn{2}{c|}{**Only CI**} & \multicolumn{6}{c|}{**Both CI and EI**} & \multicolumn{1}{c}{**Only EI**} \\ \hline  & **Lear++** & **iCaRL** & **TILDA** & **TILDA-DA** & **NN** & **NCM** & **BRIL** & **Lear++** \\ \hline Acc (CIFAR100) & 34\% & 30\% & **69.6\%** & **65.3\%** & 60.2\% & 58.25\% & 57\% & 34\% \\ \hline M (CIFAR100) & 10.5\% & 8\% & 6\% & 6\% & 100\% & **0.2\%** & 6\% & 6.8\% \\ Acc (CIFAR10) & 79.8\% & 41\% & **88.7\%** & **86.6\%** & 85\% & 83\% & 82\% & 79.5\% \\ M (CIFAR10) & 0.65\% & 2.7\% & 0.6\% & 0.6\% & 100\% & **0.02\%** & 0.6\% & 0.65\% \\ Acc (ImageNet50) & 54.2\% & 64\% & **76\%** & **74.4\%** & 69.7\% & 67.2\% & 67.4\% & 50\% \\ M (ImageNet50) & 4.7\% & 5.6\% & 3.3\% & 3.3\% & 100\% & **0.11\%** & 3.3\% & 3\% \\ \hline \end{tabular}
\end{table}
Table 2: Comparison of accuracy (Acc) and memory usage (M) relative to full dataset (corresponding to 100%) for the different methods. Note that memory usage of Learn++ method represents the size of weak classifiers, and for iCaRL represents the stored feature vectors and the size of the trainable neural network.  
\begin{table}
\begin{tabular}{l l l l l} \hline  & **TILDA** & **TILDA-DA** & **TMLP** & **TSVM** \\ \hline Acc (CIFAR100) & **69.6\%** & 65.16\% & 68.6\% & 67.6\% \\ Acc (CIFAR10) & 88.7\% & 86.6\% & 90\% & 89.2\% \\ Acc (ImageNet50) & **76\%** & 74.4\% & 75.2\% & 75\% \\ \hline \end{tabular}
\end{table}
Table 3: Comparison of TILDA with non-incremental learning methods.