# A Survey of Handwritten Character Recognition with MNIST and EMNIST
## 4 Conclusions  
This paper has provided an exhaustive review of the state of the art for both the MNIST and EMNIST databases. The MNIST database of handwritten digits was introduced almost two decades ago and has been extensively used to validate computer vision algorithms, and more recently, also as a benchmark to test different convolutional neural networks architectures and approaches. To the best of our knowledge, this paper provides the most extensive and updated survey of the MNIST dataset, including papers as recent as 2019 and making a distinction on whether they use preprocessing and/or data augmentation or not. As of today (July 2019), the best result obtained for this dataset involves a test error rate of 0.21%, and uses a convolutional neural network with two convolutional layers, one dense layer, and DropConnect, a regularization approach which generalizes dropout. To achieve such a competitive result, this proposal relies on data augmentation. On the other hand, the best result without any kind of data augmentation corresponds to a test error rate of 0.24% using network-in-network, a technique that shares many properties with convolutional neural networks. After MNIST, we have described the EMNIST database, which was introduced in 2017 and includes a significantly larger number of samples, coming from a difference source than MNIST, and including also handwritten letters; thus constituting a more challenging benchmark. Although the number of works addressing EMNIST is significantly smaller than those testing on MNIST, because of the recency of the former, some works obtaining very competitive results are starting to emerge. In this case, the highest accuracy found for the Letters database is 95.44% when combining Markov random field models and convolutional neural networks. In the case of the Digits database, the best accuracy is achieved using convolutional neural networks. One of the works reporting this top result uses capsule layers after convolution. It is remarkable that MNIST was introduced in 1998 by [PERSON] et al. [2], in the same paper where they presented convolutional neural networks. This technique has evolved over two decades, and with the improvement of graphics hardware they are becoming an industry standard for solving a large number of computer vision problems, as well as in other domains. In particular, the best results reported in this survey are based in convolutional neural networks, sometimes combined with other techniques to further improve performance. In some cases, CNNs are combined with evolutionary \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Technique** & **By\_Class** & **By\_Merge** & **Balanced** & **Letters** & **Digits** \\ \hline DWT-DCT + SVM [71] & – & – & – & 89.51\% & 97.74\% \\ Linear classifier [67] & 51.80\% & 50.51\% & 50.93\% & 55.78\% & 84.70\% \\ OPTUM [67] & 69.71\% & 72.57\% & 78.02\% & 85.15\% & 95.90\% \\ SVMs (one against all + sigmoid) [86] & – & – & – & 98.75\% * \\ Multi-layer perceptron [88] & – & – & – & – & 98.39\% * \\ Hidden Markov model [91] & – & – & – & 90.00\% * & 98.00\% * \\ Record-to-record travel [89] & – & – & – & 93.78\% * & 96.53\% * \\ PSO + fuzzy ARTMAP NNs [87] & – & – & – & – & 96.49\% * \\ Multi-layer perceptron [90] & – & – & 87.79\% * & – \\ \hline CNN (6 conv + 2 dense) [85] & – & – & **90.59** & – & **99.79\%** \\ Markov random field CNN [73] & **87.77\%** & **90.94\%** & 90.29\% & **95.44** & 99.75\% \\ TextCaps [76] & & & 90.46\% & 95.36\% & **99.79\%** \\ CNN (2 conv + 1 dense) [75] & 87.10\% & – & – & – & \\ Conmittees of neuroevolved CNNs [64] & – & – & – & 95.35\% & 99.77\% \\ Deep convolutional ELM [78] & – & – & – & – & 99.775\% \\ Parallelized CNN [74] & – & – & – & – & 99.62\% \\ CNN (flat; 2 conv + 1 dense) [79] & – & – & 87.18\% & 93.63\% & 99.46\% \\ EDEN [80] & – & – & – & 88.30\% & 99.30\% \\ Committee of 7 CNNs [19] & 88.12\% * & – & – & 92.42\% * & 99.19\% * \\ \hline \hline \end{tabular}
\end{table}