# A Survey of Handwritten Character Recognition with MNIST and EMNIST
## 2 MNIST Database
### State of the Art
Similar approaches of evolving a committee of CNNs were presented by [PERSON] et al. [65], achieving a very competitive test error rate of 0.24%; and by [PERSON] et al. [66], where the models comprising the committee were evolved using a genetic algorithm, reporting a test error rate of 0.25%. A summary of all the reviewed works can be found in Tables 1 and 2. Table 1 includes works where some preprocessing or data augmentation took place before classification, whereas Table 2 includes the remaining works. The top of the table shows the performance of classical machine learning, while the lower side shows the results reported using CNNs. In those cases when authors have reported more than one result using a single classifier (e.g., with different parameters), only the best is shown. \begin{table}
\begin{tabular}{l c} \hline \hline
**Technique** & **Test Error Rate** \\ \hline NN 6-layer 5,700 hidden units [12] & 0.35\% \\ MSRV C-SVDDNet [46] & 0.35\% \\ Committee of 25 NN 2-layer 800 hidden units [11] & 0.39\% \\ RNN [48] & 0.45\% \\ K-NN (P2 DHMDM) [6] & 0.52\% \\ COSFIRE [49] & 0.52\% \\ K-NN (IDM) [6] & 0.54\% \\ Task-driven dictionary learning [51] & 0.54\% \\ Virtual SVM, deg-9 poly, 2-pixel jt [8] & 0.56\% \\ RF-C-ELM, 15,000 hidden units [21] & 0.57\% \\ PCANet (LDANet-2) [50] & 0.62\% \\ K-NN (shape context) [5] & 0.63\% \\ Pooling + SVM [52] & 0.64\% \\ Virtual SVM, deg-9 poly, 1-pixel jt [8] & 0.68\% \\ NN 2-layer 800 hidden units, XE loss [9] & 0.70\% \\ SOAE-\(\pi\) with sparse connectivity and activity [53] & 0.75\% \\ SVM, deg-9 poly [4] & 0.80\% \\ Product of stumps on Haar f. [7] & 0.87\% \\ NN 2-layer 800 hidden units, MSE loss [9] & 0.90\% \\ \hline CNN (2 conv, 1 dense, relu) with DropConnect [24] & **0.21\%** \\ Committee of 25 CNNs [20] & 0.23\% \\ CNN with APAC [28] & 0.23\% \\ CNN (2 conv, 1 relu, relu) with dropout [24] & 0.27\% \\ Committee of 7 CNNs [19] & 0.27\% \\ Deep CNN [18] & 0.35\% \\ CNN (2 conv, 1 dense), unsup pretraining [16] & 0.39\% \\ CNN, XE loss [9] & 0.40\% \\ Scattering convolution networks + SVM [41] & 0.43\% \\ Feature Extractor + SVM [14] & 0.54\% \\ CNN Boosted LeNet-4 [4] & 0.70\% \\ CNN LeNet-5 [4] & 0.80\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Side-by-side comparison of the most competitive (error rate \(<\) 1%) results found in the state of the art for the MNIST database with data augmentation or preprocessing. Best achieved performances are boldfaced. \begin{table}
\begin{tabular}{l c} \hline \hline
**Technique** & **Test Error Rate** \\ \hline HOPE+DNN with unsupervised learning features [47] & 0.40\% \\ Deep convex net [10] & 0.83\% \\ CDBN [54] & 0.82\% \\ S-SC + linear SVM [56] & 0.84\% \\
2-layer MP-DBM [58] & 0.88\% \\ DNet-kNN [58] & 0.94\% \\
2-layer Boltzmann machine [57] & 0.95\% \\ \hline Batch-normalized maxout network-in-network [29] & **0.24\%** \\ Committees of evolved CNNs (CEA-CNN) [65] & **0.24\%** \\ Genetically evolved committee of CNNs [66] & 0.25\% \\ Committees of 7 neuroevolved CNNs [64] & 0.28\% \\ CNN with gated pooling function [30] & 0.29\% \\ Inception-Recurrent CNN + LSUV + EVE [60] & 0.29\% \\ Recurrent CNN [31] & 0.31\% \\ CNN with norm. layers and piecewise linear activation units [32] & 0.31\% \\ CNN (5 conv, 3 dense) with full training [45] & 0.32\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Side-by-side comparison of the most competitive (error rate \(<\) 1%) results found in the state of the art for the MNIST database without data augmentation or preprocessing. Best achieved performances are boldfaced.