# Towards Accurate Children's Arabic Handwriting Recognition via Deep Learning
## 3 Material and Methods
### Proposed Deep Learning Models
#### 3.2.1 Convolutional Neural Network Model  
Our proposed CNN deep learning model was comprised of seven layers as presented in Figure 2. The first four layers were convolutional layers, while the last three were fully connected. Furthermore, there were pooling and activation layers between the convolutional and fully connected layers. The hidden layers used the nonlinearity function ReLU [24] to speed up the training phase, defined as shown in Equation (1). \[ReLU(x)=max(0,x) \tag{1}\]  
The Hijja dataset was divided into 70% for training the model, 10% for validation, and 10% for testing. Then, we conducted two experiments on handwriting recognition using the same model, the first for 29 classes and the second for 108 classes for the variations in the handwritten Arabic letters. **Convolutional layers 1 and 2:** The input images were \(32\times 32\times 1\) grayscale images. At least one layer in a CNN is used for convolution. In a convolutional layer, there are several filters (kernels) to produce a feature map. Each filter functions as a feature extractor, identifying features such as vertices, edges, and endpoints. Therefore, the first convolutional layer was the first step of feature extraction. In this layer, the input image of  
Figure 1: Sample of the Hijja dataset [2]. Figure 2: Our CNN architecture. size \(32\times 32\) was processed with 64 filters with a kernel size of \(3\times 3\). In addition, the stride size was 1, where the stride refers to the movement across the image, meaning the filter moves one pixel at a time. We used zero padding, which prevented information loss around the perimeter of the image. We also set the initial random weights of the layers using the HeNormal initializer. HeNormal draws the weights from a normal distribution \(X\sim\mathcal{N}(\mu,\,\sigma^{2})\) with the mean \(\mu\) and standard deviation \(\sigma\) defined as in (2). \[\mu =0,\] \[\sigma^{2} =\sqrt{\frac{2}{fan_{in}}}, \tag{2}\]  
where \(fan_{in}\) is the number of input units in the weight tensor. In addition, the ReLU function (1) was used as the activation function in the hidden layers. **Pooling Layer:** To minimize the size of the feature maps, a pooling layer is typically placed after the convolution layer. Pooling is applied to each feature map and can be performed in several ways, such as max-pooling, average-pooling, or min-pooling with a \(q\times q\) window, where \(q\) is the filter size [1]. We used max-pooling to construct a two-dimensional pooling layer. Figure 2 shows that MP1 was the first pooling layer. This layer managed each feature map individually, resulting in an output of 64 feature maps, each with a size of \(16\times 16\). The layer selected the max-pooling layer over \(2\times 2\) area; then, a batch normalization layer was used to normalize the output. **Convolutional Layer 3:** Convolutional Layer 3 was also used to extract features. This layer and its specifications were very similar to those of the previous convolutional layers. The input data, consisting of feature maps received from the first convolutional layer, were processed with 128 convolutional kernels, each with a size of \(3\times 3\). A stride of one pixel and zero padding were used. Thus, we extracted 128 feature maps with a size of \(16\times 16\). We also employed the ReLU activation function, followed by a max-pooling layer of the same size as the previous max-pooling layer, resulting in an \(8\times 8\) layer. An additional layer of batch normalization was used after this layer. **Convolutional Layer 4:** Convolutional Layer 4 performed similar tasks to the previous convolutional layers with a filter size of 128 to extract additional features. We also employed the activation function ReLU followed by a max-pooling layer with the same size as the previous max-pooling layer, to create a \(4\times 4\) layer followed by a batch normalization layer. Finally, the dropout technique was used in each convolutional layer except the first one. The probability of the dropout rate in the network was 20%.