## 4 Proposed Convolutional Neural Network
The idea behind ReLU is simple, it returns positive values input directly to the output, whereas the negative values are returned as 0, as depicted in Figure 5.  
Figure 5: Rectified linear unit (ReLU) function. The subsequent three layers are the 2D convolutional layers, which are accompanied by one max-pooling layer and a ReLU-activation function. Max pooling is a sample-based discretization process which is used to downsize our input images. It pools the maximum value from each patch of each feature map, thus helping to reduce the dimensionality of the network. Moreover, it reduces the number of parameters by discarding insignificant ones, which decreases the computational burden as well as helping to avoid overfitting. Thus, a \(2\times 2\) max-pooling layer is integrated in each of the convolutional layers except for the first one. The output of the fourth convolutional layer is fed to the flattening layer to convert the input to a 1D string, which is then fed to the fully connected layer, i.e., the dense layer. In the fully connected layer, as the name suggests, all the neurons are linked to the activation units of the following layer. In the proposed model, there are two fully connected layers where all the neurons of the first layer are connected to the activation unit of the second fully connected layer. In the second fully connected layer, all the inputs are passed to the Softmax activation function, which categorizes the features into multiclass as needed. Finally, the determined class of any input image is declared in the output. The proposed model is illustrated in Figure 6 and the resultant parameters of each layer are tabulated in Table 1. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer (Type)** & **Output Shape** & **Param \#** \\ \hline conv\_1 (Conv 2D) & (None, 26, 26, 32) & 320 \\ conv\_2 (Conv 2D) & (None, 26, 26, 64) & 18,496 \\ max\_pooling2D\_18 (MaxPooling2D) & (None, 13, 13, 64) & 0 \\ conv\_3 (Conv 2D) & (None, 13, 13, 128) & 73,856 \\ max\_pooling2D\_19 (MaxPooling2D) & (None, 6, 6, 128) & 0 \\ conv\_4 (Conv 2D) & (None, 6, 6, 256) & 295,168 \\ max\_pooling2D\_20 (MaxPooling2D) & (None, 3, 3, 256) & 0 \\ flatten (Flatten) & (None, 2304) & 0 \\ FC\_1 (Dense) & (None, 64) & 147,520 \\ FC\_2 (Dense) & (None, 10) & 650 \\ \hline Total Params \# & & 536,010 \\ \hline Trainable Params \# & & 536,010 \\ \hline Non-Trainable Params \# & & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Details of the proposed model. Figure 6: Proposed CNN model for character recognition. For generalization, the same proposed model is used to classify both the English alphabets and digits. The only difference is the number of output classes defined in the last fully connected layer, which is the 'fully connected + Softmax' layer, as depicted by Figure 6, and the FC_2 layer, as presented by Table 1. The number of classes is 10 for digit recognition as depicted by the table, and the number of classes is 26 for alphabet recognition. Moreover, for extensive comparative analysis, we also analyzed how the proposed model performs with different optimizers, 'ADAM' and 'RMSprop', which also include the variation of the learning rates (LRs). This analysis helps in understanding how the model performance might vary with the change of optimizers and variation of learning rates which are discussed in detail in Section 5--Results and Analysis. In order to avoid the difficulties posed by the problem of latency in data processing, this project utilizes Colab-pro by Google, which has a 2.20 GHz Intel Xeon Processor, 128 GB RAM, and Tesla P100 16 GB GPU. The model was designed and tested in Colab-pro, keeping in mind the factor of easy reproducibility by the research community, as Colab-pro has built-in support for GPU-enabled TensorFlow and the necessary support for CUDA acceleration.