# Handwritten Digit Recognition: Hyperparameters-Based Analysis
## 2 Previous Work  
There has been much work in the field of classification, especially for MNIST datasets. Many types of neural networks have been used for this purpose, and sometimes a combination of different algorithms has been used. Many new advanced forms of underlying neural networks were also tested on MNIST data.  
[PERSON] et al. [12] used a multilayer perceptron (MLP) for the recognition of handwritten letters to build an optical character recognition (OCR) system. [PERSON] et al. [13] used Siamese neural networks for one-shot image recognition, i.e., only one image of each class is used. [PERSON] et al. [14] built the Independently recurrent neural network (IndRNN), an advanced form of RNN and tested its performance on MNIST datasets. [PERSON] et al. [15] built a probabilistic Siamese network and tested it on MNIST data. [PERSON] used CNN for the image classification of MNIST [16].  
[PERSON] et al. [17] combined CNN with a support vector machine (SVM), a different approach for MNIST data classification. Similar work is done by [PERSON] et al. [18] combined CNN and SVM for image classification. [PERSON] et al. [19] combined SVM and k-nearest neighbors (KNN) for image classification. [PERSON] [20] used Random neural networks with extreme learning machines (ELM) to build a classifier that does not require large training datasets.  
As it is an era that revolves around deep learning, many works have been done in hyperparameter tuning. Mostly, hyperparameters are selected through grid search; but there are many algorithms now that can tune them for better results. [PERSON] et al. [21] proposed covariance matrix adaptation evolution strategy (CMA-ES) for hyperparameter optimization. [PERSON] et al. [22] proposed a disciplined approach for tuning the learning rate, batch size, momentum and weight decay. [PERSON] et al. [23] extrapolated the learning curves to speed up automatic hyperparameter optimization of deep neural networks. [PERSON] et al. [24] merged optimization techniques with surrogate-based ranking for surrogate-based collaborative tuning (SCoT). They verified SCoT in two different experiments that outperformed standard tuning methods, but their approach is still suffering from limited meta-features. [PERSON] et al. [25] studied the comparison between hyperparameters for modeling bioactivity data.  
As seen from the above references, much work exists in the field of classification. No work has been done on a comparison between different neural networks based on changing hyperparameters. There are several givens in this paper:  
* Results for efficiency of each neural network (NN) based on different hyperparameters;
* The best results of a comparison of different NNs based on hyperparameters selected after testing;  
Figure 2: Handwritten digit recognition in MNIST (Modified National Institute of Standards and Technology) database.  
* Best model and hyperparameter values are given.  
With the advent of deep learning, there has been more focus on developing techniques for recognition of handwritten letters and digits, using deep neural networks instead of hand-engineered features. The literature review of methods to classify and test handwritten data, using deep-learning techniques, is summarized in Table 1. This section gives a basic overview of work in the field to support our work, whereas the next section describes the methodology taken to prove our claims.  
The rest of the paper is divided as follows: Section 3 describes the methodology, pipeline and experimental environment used. Section 4 will describe and analyze the results, on the basis of which the optimal hyperparameter values will be selected. In the end, Section 5 will conclude the study with some suggestions for future work.