# Improved Arabic Alphabet Characters Classification Using Convolutional Neural Networks (CNN)

Authors: Nesrine Wagaa; Hichem Kallel; Nédra Mellouli
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

Handwritten characters recognition is a challenging research topic. A lot of works have been present to recognize letters of different languages. The availability of Arabic handwritten characters databases is limited. Motivated by this topic of research, we propose a convolution neural network for the classification of Arabic handwritten letters. Also, seven optimization algorithms are performed, and the best algorithm is reported. Faced with few available Arabic handwritten datasets, various data augmentation techniques are implemented to improve the robustness needed for the convolution neural network model. The proposed model is improved by using the dropout regularization method to avoid data overfitting problems. Moreover, suitable change is presented in the choice of optimization algorithms and data augmentation approaches to achieve a good performance. The model has been trained on two Arabic handwritten characters datasets AHCD and Hijja. The proposed algorithm achieved high recognition accuracy of 98.48% and 91.24% on AHCD and Hijja, respectively, outperforming other state-of-the-art models.


---

## Body

## 1. Introduction

Approximately a quarter of a billion people around the world speak and write the Arabic language [1]. There are a lot of historical books and documents that represent a crucial data set for most Arabic countries written in the Arabic language [1, 2].Recently, the area of Arabic handwritten characters recognition (AHCR) has received increased research attention [3–5]. It is a challenging topic of computer vision and pattern recognition [1]. This is due to the following:(i)
The difference between handwriting patterns [3].(ii)
The form similarity between Arabic alphabets [1, 3].(iii)
The diacritics of Arabic characters [6].(iv)
As shown in Figure1, in the Arabic language the shape of each handwritten character depends on its position in the world. For example, here in the word “أمراء” the character “Alif” is written in two different forms “أ” and “ا”, where, in the Arabic language, each character has between two and four shapes. Table 1 shows the different shapes of the twenty-eight Arabic alphabets.Figure 1
One possible connected component box for the word “أمراء”.Table 1
Twenty-eight shapes of Arabic alphabets.
No.NameIsolatedBeginningMiddleEnd1Alifااـإـأ2Baaببــبــب3Teaتتــتــت4Theaثثــثــث5Jamججــجــج6Haaححــحــح7Khaaخخــخــخ8Daalددـدـد9Thaalذذـذـذ10Raaررـرـر11Zaayززـزـز12Seenسســســس13Sheenششــشــش14Sadصصــصــص15Dhadضضــضــض16Tahططــطــط17Dhaظظــظــظ18Ainععــعــع19Ghenغغــغــغ20Faففــفــف21Qafققــقــق22Kafككــكــك23Lamللــلــل24Meemممــمــم25Noonننــنــن26Haههــهــه27Wawووـوـو28YaaييــيــيWith the development of deep learning (DL), convolution neural networks (CNNs) have shown a significant capability to recognize handwritten characters of different languages [3, 7, 8]: Latin [9, 10], Chine [11], Devanagari [12], Malayalam [11], etc.Most researchers improved the CNN architecture to achieve good handwritten characters recognition performance [6, 13]. However, a neural network with excellent performance usually requires a good tuning of CNN hyperparameters and a good choice of applied optimization algorithms [14–16]. Also, a large amount of training dataset [17, 18] is required to achieve outstanding performance.The main contributions of this research can be summarized as follows:(i)
Suggesting a CNN model for recognizing Arabic handwritten characters.(ii)
Tuning of different hyperparameters to improve the model performance.(iii)
Applying different optimization algorithms. Reporting the effectiveness of the best ones.(iv)
Presenting different data augmentation techniques. Reporting the influence of each method on the improvement of Arabic handwritten characters recognition.(v)
Mixing two different Arabic handwritten characters datasets for shape varying. Testing the impact of the presented data augmentation approaches on the mixed dataset.The rest of this paper is organized as follows. In Section2, we expose the related works in Arabic handwritten character classification. In Sections 3 and 4, we describe the convolution neural network architecture and the model tuning hyperparameters. In Section 5, we make a detailed description of various used optimization algorithms. In Section 6, we describe the different utilized data augmentation techniques chosen in this study. In Section 7, we provide an overview of the experimental results showing the CNN distinguished performance. Section 8 is conclusion and possible future research directions.


## 2. Related Work

In recent years, many studies have addressed the classification and recognition of letters, including Arabic handwritten characters. On the other hand, there are a smaller number of proposed approaches for recognizing individual characters in the Arabic language. As a result, Arabic handwritten character recognition is less common compared to English, French, Chinese, Devanagari, Hangul, Malayalam, etc.Impressive results were achieved in the classification of handwritten characters from different languages, using deep learning models and in particular the CNN.El-Sawy et al. [6] gathered their own Arabic Handwritten Character dataset (AHCD) from 60 participants. AHCD consists of 16.800 characters. They have achieved a classification accuracy of 88% by using a CNN model consisting of 2 convolutional layers. To improve the CNN performance, regularization and different optimization techniques have been implemented to the model. The testing accuracy was improved to 94.93%.Altwaijry and Turaiki [13] presented a new Arabic handwritten letters dataset (named “Hijja”). It comprised 47.434 characters written by 591 participants. Their proposed CNN model was able to achieve 88% and 97% testing accuracy, using the Hijja and AHCD datasets, respectively.Younis [19] designed a CNN model to recognize Arabic handwritten characters. The CNN consisted of three convolutional layers followed by one final fully connected layer. The model achieved an accuracy of 94.7% for the AHCD database and 94.8% for the AIA9K (Arabic alphabet’s dataset).Latif et al. [20] designed a CNN to recognize a mix of handwriting of multiple languages: Persian, Devanagari, Eastern Arabic, Urdu, and Western Arabic. The input image is of size (28 × 28) pixels, followed by two convolutional layers, and then a max-pooling operation is applied to both convolution layers. The overall accuracy of the combined multilanguage database was 99.26%. The average accuracy is around 99% for each individual language.Alrobah and Albahl [21] analyzed the Hijja dataset and found irregularities, such as some distorted letters, blurred symbols, and some blurry characters. They used the CNN model to extract the important features and SVM model for data classification. They achieved a testing accuracy of 96.3%.Mudhsh et al. [22] designed the VGG net architecture for recognizing Arabic handwritten characters and digits. The model consists of 13 convolutional layers, 2 max-pooling layers, and 3 fully connected layers. Data augmentation and dropout methods were used to avoid the overfitting problem. The model was trained and evaluated by using two different datasets: the ADBase for the Arabic handwritten digits classification topic and HACDB for the Arabic handwritten characters classification task. The model achieved an accuracy of 99.66% and 97.32% for ADBase and HACDB, respectively.Boufenar et al. [23] used the popular CNN architecture Alexnet. It consists of 5 convolutional layers, 3 max-pooling layers, and 3 fully connected layers. Experiments were conducted on two different databases, OIHACDB-40 and AHCD. Based on the good tuning of the CNN hyperparameters and by using dropout and minibatch techniques, a CNN accuracy of 100% and 99.98% for OIHACDB-40 and AHCD was achieved.Mustapha et al. [24] proposed a Conditional Deep Convolutional Generative Adversarial Network (CDCGAN) for a guided generation of isolated handwritten Arabic characters. The CDCGAN was trained on the AHCD dataset. They achieved a 10% performance gap between real and generated handwritten Arabic characters.Table2 summarizes the literature reviewed for recognizing Arabic handwriting characters using the CNN models. From the previous literature, we notice that most CNN architectures have been trained by using adult Arabic handwriting letters “AHCD”. In addition, we observe that most researchers try to improve the performance through the good tuning of the CNN model hyperparameters.Table 2
Summary of Arabic handwritten characters recognition using CNN model.
ReferencesYearDatasetType (size)MethodOptimizationAccuracy (%)Loss (%)El-Sawy et al. [6]2017AHCDChars (16,800)CNN(i) Minibatch94.935.1Mudhsh et al. [22]2017ADBaseDigits (6.600)CNN (based on VGG net)(ii) Dropout99.6—HACDBChars (70.000)(iii) Data augmentation97.32—Boufenar et al. [23]2017OIHACDBChars (6.600)CNN (based on Alexnet)(i) Dropout100—AHCD(ii) Minibatch99.98Younis [19]2018AHCDChars (8.737)CNN—97.7—AIA9K94.8—Latif et al. [20]2018Mix of handwriting of multiple languagesCharsCNN—99.260.02Altwaijry and Turaiki [13]2020HijjaChars (47,434)CNN—88—AHCD97—Alrobah &Albahl [21]2021HijjaChars (47,434)CNN + SVM—96.3—Mustapha et al. [24]2021AHCDCDCGAN———


## 3. The Proposed Arabic Handwritten Characters Recognition System

As shown in Figure2, the model that we proposed in this study is composed of three principal components: CNN proposed architecture, optimization algorithms, and data augmentation techniques.Figure 2
Proposed recognition schema for Arabic handwritten characters datasets.In this paper, the proposed CNN model contains four convolution layers, two max-pooling operations, and an ANN model with three fully hidden layers used for the classification. To avoid the overfitting problems and improve the model performance, various optimization techniques were used such as dropout, minipatch, choice of the activation function, etc.Figure3 describes the proposed CNN model. Also, in this work, the recognition performance of Arabic handwritten letters was improved through the good choice of the optimization algorithm and by using different data augmentation techniques “geometric transformations, feature space augmentation, noise injection, and mixing images.”Figure 3
Proposed CNN architecture.


## 4. Convolution Neural Network Architecture

A CNN model [25–34] is a series of convolution layers followed by fully connected layers. Convolution layers allow the extraction of important features from the input data. Fully connected layers are used for the classification of data. The CNN input is the image to be classified; the output corresponds to the predicted class of the Arabic handwritten character.


### 4.1. Input Data

The input data is an imageI of size m×m×s. m×m Defines the width and the height of the image and s denotes the space or number of channels. The value of s is 1 for a grayscale image and equals 3 for a RGB color image.


### 4.2. Convolution Layer

The convolution layer consists of a convolution operation followed by a pooling operation.


#### 4.2.1. Convolution Operation

The basic concept of the classical convolution operation between an input imageI of dimension m×m and a filter F of size n×n is defined as follows (see Figure 4):(1)C=I⊗F.Figure 4
Example of 2D convolution operation [13, 31].Here, ⊗ denotes the convolution operation.C is the convolution map of size (a × a), where a=m−n+2p/sL+1. sL is the stride and denotes the number of pixels by which Fissliding over I. p is the padding; often it is necessary to add a bounding of zeros around I to preserve complete image information. Figure 4 is an example of the convolution operation between an input image of dimension (8 × 8) and a filter F of size (3 × 3). Here, the convolution map C is of size (6 × 6) with a stride sL=1 and a padding p=0.Generally, a nonlinear activation function is applied on the convolution map C. The commonly used activation functions are Sigmoid [34–36], Hyperbolic Tangent “Tanh” [35, 37], and Rectified Linear Unit “ReLU” [37, 38] where(2)Ca=fC,here, Ca is the convolution map after applying the nonlinear activation function f. Figure 5 shows the Ca map when the ReLU activation function is applied on C.Figure 5
Convolution map after applying ReLU activation function [13, 25, 31].


#### 4.2.2. Pooling Operation

The pooling operation is used to reduce the dimension ofCa thus reducing the computational complexity of the network. During the pooling operation, a kernel K of size sp×sp is sliding over Ca. sp denotes the number of patches by which Kis sliding over Ca. In our analysis sp is set to 2. The pooling operation is expressed as(3)P=poolCa,where P is the pooling map and pool is the pooling operation. The commonly used pooling operations are average-pooling, max-pooling, and min-pooling. Figure 6 describes the concept of average-pooling and max-pooling operations using a kernel of size 2×2 and a stride of 2.Figure 6
Average-pooling, and max-pooling with filter2×2 and stride 2 [13, 31].


### 4.3. Concatenation Operation

The concatenation operation maps the set of the convoluted images into a vector called the concatenation vectorY.(4)Y=P1cP2c⋮Pnc,here, Pic is the output of the ith convolution layer. n denotes the number of filters applied on the convoluted images Pi−1c−1.


### 4.4. Fully Connected Layer

The CNN classification operation is performed through the fully connected layer [39]. Its input is the concatenation vector Y; the predicted class y is the output of the CNN classifier. The classification operation is performed through a series of t fully connected hidden layers. Each fully connected hidden layer is a parallel collection of artificial neurons. Like synapses in the biological brain, the artificial neurons are connected through weights W. The model output of the ith fully connected hidden layer is (5)Yi=fHi,where the weight sum vector Hi is(6)Hi=WiYi−1+Bi,here, f is a nonlinear activation function (sigmoid, Tanh, ReLU, etc.). The bias value Bi defines the activation level of the artificial neurons.


## 4.1. Input Data

The input data is an imageI of size m×m×s. m×m Defines the width and the height of the image and s denotes the space or number of channels. The value of s is 1 for a grayscale image and equals 3 for a RGB color image.


## 4.2. Convolution Layer

The convolution layer consists of a convolution operation followed by a pooling operation.


### 4.2.1. Convolution Operation

The basic concept of the classical convolution operation between an input imageI of dimension m×m and a filter F of size n×n is defined as follows (see Figure 4):(1)C=I⊗F.Figure 4
Example of 2D convolution operation [13, 31].Here, ⊗ denotes the convolution operation.C is the convolution map of size (a × a), where a=m−n+2p/sL+1. sL is the stride and denotes the number of pixels by which Fissliding over I. p is the padding; often it is necessary to add a bounding of zeros around I to preserve complete image information. Figure 4 is an example of the convolution operation between an input image of dimension (8 × 8) and a filter F of size (3 × 3). Here, the convolution map C is of size (6 × 6) with a stride sL=1 and a padding p=0.Generally, a nonlinear activation function is applied on the convolution map C. The commonly used activation functions are Sigmoid [34–36], Hyperbolic Tangent “Tanh” [35, 37], and Rectified Linear Unit “ReLU” [37, 38] where(2)Ca=fC,here, Ca is the convolution map after applying the nonlinear activation function f. Figure 5 shows the Ca map when the ReLU activation function is applied on C.Figure 5
Convolution map after applying ReLU activation function [13, 25, 31].


### 4.2.2. Pooling Operation

The pooling operation is used to reduce the dimension ofCa thus reducing the computational complexity of the network. During the pooling operation, a kernel K of size sp×sp is sliding over Ca. sp denotes the number of patches by which Kis sliding over Ca. In our analysis sp is set to 2. The pooling operation is expressed as(3)P=poolCa,where P is the pooling map and pool is the pooling operation. The commonly used pooling operations are average-pooling, max-pooling, and min-pooling. Figure 6 describes the concept of average-pooling and max-pooling operations using a kernel of size 2×2 and a stride of 2.Figure 6
Average-pooling, and max-pooling with filter2×2 and stride 2 [13, 31].


## 4.2.1. Convolution Operation

The basic concept of the classical convolution operation between an input imageI of dimension m×m and a filter F of size n×n is defined as follows (see Figure 4):(1)C=I⊗F.Figure 4
Example of 2D convolution operation [13, 31].Here, ⊗ denotes the convolution operation.C is the convolution map of size (a × a), where a=m−n+2p/sL+1. sL is the stride and denotes the number of pixels by which Fissliding over I. p is the padding; often it is necessary to add a bounding of zeros around I to preserve complete image information. Figure 4 is an example of the convolution operation between an input image of dimension (8 × 8) and a filter F of size (3 × 3). Here, the convolution map C is of size (6 × 6) with a stride sL=1 and a padding p=0.Generally, a nonlinear activation function is applied on the convolution map C. The commonly used activation functions are Sigmoid [34–36], Hyperbolic Tangent “Tanh” [35, 37], and Rectified Linear Unit “ReLU” [37, 38] where(2)Ca=fC,here, Ca is the convolution map after applying the nonlinear activation function f. Figure 5 shows the Ca map when the ReLU activation function is applied on C.Figure 5
Convolution map after applying ReLU activation function [13, 25, 31].


## 4.2.2. Pooling Operation

The pooling operation is used to reduce the dimension ofCa thus reducing the computational complexity of the network. During the pooling operation, a kernel K of size sp×sp is sliding over Ca. sp denotes the number of patches by which Kis sliding over Ca. In our analysis sp is set to 2. The pooling operation is expressed as(3)P=poolCa,where P is the pooling map and pool is the pooling operation. The commonly used pooling operations are average-pooling, max-pooling, and min-pooling. Figure 6 describes the concept of average-pooling and max-pooling operations using a kernel of size 2×2 and a stride of 2.Figure 6
Average-pooling, and max-pooling with filter2×2 and stride 2 [13, 31].


## 4.3. Concatenation Operation

The concatenation operation maps the set of the convoluted images into a vector called the concatenation vectorY.(4)Y=P1cP2c⋮Pnc,here, Pic is the output of the ith convolution layer. n denotes the number of filters applied on the convoluted images Pi−1c−1.


## 4.4. Fully Connected Layer

The CNN classification operation is performed through the fully connected layer [39]. Its input is the concatenation vector Y; the predicted class y is the output of the CNN classifier. The classification operation is performed through a series of t fully connected hidden layers. Each fully connected hidden layer is a parallel collection of artificial neurons. Like synapses in the biological brain, the artificial neurons are connected through weights W. The model output of the ith fully connected hidden layer is (5)Yi=fHi,where the weight sum vector Hi is(6)Hi=WiYi−1+Bi,here, f is a nonlinear activation function (sigmoid, Tanh, ReLU, etc.). The bias value Bi defines the activation level of the artificial neurons.


## 5. CNN Learning Process

A trained CNN is a system capable of determining the exact class of a given input data. The training is achieved through an update of the layer’s parameters (filters, weights, and biases) based on the error between the CNN predicted class and the class label. The CNN learning process is an iterative process based on the feedforward propagation and backpropagation operations.


### 5.1. Feedforward Propagation

For the CNN model, the feedforward equations can be derived from (1)–(5) and (6). The Softmax activation [40, 41] function is applied in the final layer to generate the predicted value of the class of the input image I. For a multiclass model, the Softmax is expressed as follows:(7)yi=exphi∑jcexphj,where c denotes the number of classes, yi is the ith coordinate of the output vector y, and the artificial neural output hi=∑j=1nhiwij.


### 5.2. Backpropagation

To update the CNN parameters and perform the learning process, a backpropagation optimization algorithm is developed to minimize a selected cost functionE. In this analysis, the cross-entropy (CE) cost function [40] is used.(8)E=−∑j=1pyiˇlogyi+1+yiˇlog1−yi,here, yiˇ is the desired output (data label).The most used optimization algorithm to solve classification problems is the gradient descent (GD). Various optimizers for the GD algorithm such as momentum, AdaGrad, RMSprop, Adam, AdaMax, and Nadam were used to improve the CNN performance.


#### 5.2.1. Gradient Descent [40, 42]

GD is the simplest form of optimization gradient descent algorithms. It is easy to implement and gives significant classification accuracy. The general update equation of the CNN parameters using the GD algorithm is(9)φt+1=φt−α∂E∂φt,where φ represents the update of the filters F, the weights W, and the biases B. ∂E/∂φ is the gradient with respect to the parameter φ.α is the model learning rate. A too-large value of α may lead to the divergence of the GD algorithm and may cause the oscillation of the model performance. A too-small α stops the learning process.


#### 5.2.2. Gradient Descent with Momentum [43]

The momentum hyperparameterm defines the velocity by which the learning rate α must be increased when the model approaches to the minimal of the cost function E. The update equations using the momentum GD algorithm are expressed as follows:(10)vt=mvt−1+∂E∂φ,φt+1=φt−αvt,where vt is the moment gained at tth iteration.


#### 5.2.3. AdaGrad [44]

In this algorithm, the learning rate is a function of the gradient∂E/∂φ. It is defined as follows:(11)αt=βGt+ɛ,where(12)β=α0ɛ,Gt=∑i=1t∂E∂φt2,where ϵ is a small smoothing value used to avoid the division by 0 and Gt is the sum of the squares of the gradients ∂E/∂φt.With a small magnitude of∂E/∂φ, the value of α is increasing. If ∂E/∂φ is very large, the value of α is a constant. AdaGrad optimization algorithm changes the learning rate for each parameter  at a given time t with considering the previous gradient update. The parameter update equation using AdaGrad is expressed as follows:(13)φt+1=φt−αt∂E∂φt.


#### 5.2.4. AdaDelta [45]

The issue of AdaGrad is that with much iteration the learning rate becomes very small which leads to a slow convergence. To fix this problem, AdaDelta algorithm proposed to take an exponentially decaying average as a solution, where(14)EG2t=γEG2t−1+1−γG2t,Δθt=1EG2t+εGt,φt+1=φt−αΔθt,where EG2t is the decaying average over past squared gradients and γ is a set usually around 0.9.


#### 5.2.5. RMSprop [45, 46]

In reality, RMSprop is identical to AdaDelta’s initial update vector, which we derived above:(15)EG2t=0.9EG2t−1+0.1G2t,φt+1=φt−αΔθt.


#### 5.2.6. ADAM [17, 45, 46]

This gradient descent optimizer algorithm computes the learning rateα based on two vectors:(16)rt=β1rt−1+1−β1∂E∂φt,vt=β2vt−1+1−β2∂E∂φt2,where rt and vt are the 1st and the 2nd order moments vectors. β1 and β2 are the decay rates. rt−1 and vt−1 represent the mean and the variance of the previous gradient.Whenrt and vt are very small, a large step size is needed for parameters update. To avoid this issue, a bias correction value is added to rt and vt.(17)r^t=rt1−β1t,v^t=vt1−β2t,where β1t is β1 power t and β2t is β2 power t.The Adam update equation is expressed as follows:(18)φt+1=φt−αr^tv^t+ɛ.


#### 5.2.7. AdaMax [45, 47]

The factorvt in the Adam algorithm adjusts the gradient inversely proportionate to the ℓ2 norm of previous gradients (via the vt−1) and current gradient t∂E/∂φt:(19)vt=β2vt−1+1−β2∂E∂φt2.The generalization of this update to theℓp norm is as follows:(20)vt=β2pvt−1+1−β2p∂E∂φtp.To avoid being numerically unstable, ℓ1 and ℓ2 norms are most common in practice. However, in general ℓ∞ also shows stable behavior. As a result, the authors propose AdaMax and demonstrate thatvt with ℓ∞ converges to the more stable value. Here,(21)ut=β2∞vt−1+1−β2∞∂E∂φt∞=maxβ2.vt−1,∂E∂φt,φt+1=φt−αr^tut.


#### 5.2.8. Nadam [43]

It is a combination of Adam and NAG, where the parameters update equation using NAG is defined as follows:(22)vt=γvt−1+α∂E∂φt−γvt−1,φt+1=φt−αvt.The update equation using Nadam is expressed as follows:(23)r=β1r^t+1−β11−βt1∂E∂φt,φt+1=φt−αv^t+ɛr.


## 5.1. Feedforward Propagation

For the CNN model, the feedforward equations can be derived from (1)–(5) and (6). The Softmax activation [40, 41] function is applied in the final layer to generate the predicted value of the class of the input image I. For a multiclass model, the Softmax is expressed as follows:(7)yi=exphi∑jcexphj,where c denotes the number of classes, yi is the ith coordinate of the output vector y, and the artificial neural output hi=∑j=1nhiwij.


## 5.2. Backpropagation

To update the CNN parameters and perform the learning process, a backpropagation optimization algorithm is developed to minimize a selected cost functionE. In this analysis, the cross-entropy (CE) cost function [40] is used.(8)E=−∑j=1pyiˇlogyi+1+yiˇlog1−yi,here, yiˇ is the desired output (data label).The most used optimization algorithm to solve classification problems is the gradient descent (GD). Various optimizers for the GD algorithm such as momentum, AdaGrad, RMSprop, Adam, AdaMax, and Nadam were used to improve the CNN performance.


### 5.2.1. Gradient Descent [40, 42]

GD is the simplest form of optimization gradient descent algorithms. It is easy to implement and gives significant classification accuracy. The general update equation of the CNN parameters using the GD algorithm is(9)φt+1=φt−α∂E∂φt,where φ represents the update of the filters F, the weights W, and the biases B. ∂E/∂φ is the gradient with respect to the parameter φ.α is the model learning rate. A too-large value of α may lead to the divergence of the GD algorithm and may cause the oscillation of the model performance. A too-small α stops the learning process.


### 5.2.2. Gradient Descent with Momentum [43]

The momentum hyperparameterm defines the velocity by which the learning rate α must be increased when the model approaches to the minimal of the cost function E. The update equations using the momentum GD algorithm are expressed as follows:(10)vt=mvt−1+∂E∂φ,φt+1=φt−αvt,where vt is the moment gained at tth iteration.


### 5.2.3. AdaGrad [44]

In this algorithm, the learning rate is a function of the gradient∂E/∂φ. It is defined as follows:(11)αt=βGt+ɛ,where(12)β=α0ɛ,Gt=∑i=1t∂E∂φt2,where ϵ is a small smoothing value used to avoid the division by 0 and Gt is the sum of the squares of the gradients ∂E/∂φt.With a small magnitude of∂E/∂φ, the value of α is increasing. If ∂E/∂φ is very large, the value of α is a constant. AdaGrad optimization algorithm changes the learning rate for each parameter  at a given time t with considering the previous gradient update. The parameter update equation using AdaGrad is expressed as follows:(13)φt+1=φt−αt∂E∂φt.


### 5.2.4. AdaDelta [45]

The issue of AdaGrad is that with much iteration the learning rate becomes very small which leads to a slow convergence. To fix this problem, AdaDelta algorithm proposed to take an exponentially decaying average as a solution, where(14)EG2t=γEG2t−1+1−γG2t,Δθt=1EG2t+εGt,φt+1=φt−αΔθt,where EG2t is the decaying average over past squared gradients and γ is a set usually around 0.9.


### 5.2.5. RMSprop [45, 46]

In reality, RMSprop is identical to AdaDelta’s initial update vector, which we derived above:(15)EG2t=0.9EG2t−1+0.1G2t,φt+1=φt−αΔθt.


### 5.2.6. ADAM [17, 45, 46]

This gradient descent optimizer algorithm computes the learning rateα based on two vectors:(16)rt=β1rt−1+1−β1∂E∂φt,vt=β2vt−1+1−β2∂E∂φt2,where rt and vt are the 1st and the 2nd order moments vectors. β1 and β2 are the decay rates. rt−1 and vt−1 represent the mean and the variance of the previous gradient.Whenrt and vt are very small, a large step size is needed for parameters update. To avoid this issue, a bias correction value is added to rt and vt.(17)r^t=rt1−β1t,v^t=vt1−β2t,where β1t is β1 power t and β2t is β2 power t.The Adam update equation is expressed as follows:(18)φt+1=φt−αr^tv^t+ɛ.


### 5.2.7. AdaMax [45, 47]

The factorvt in the Adam algorithm adjusts the gradient inversely proportionate to the ℓ2 norm of previous gradients (via the vt−1) and current gradient t∂E/∂φt:(19)vt=β2vt−1+1−β2∂E∂φt2.The generalization of this update to theℓp norm is as follows:(20)vt=β2pvt−1+1−β2p∂E∂φtp.To avoid being numerically unstable, ℓ1 and ℓ2 norms are most common in practice. However, in general ℓ∞ also shows stable behavior. As a result, the authors propose AdaMax and demonstrate thatvt with ℓ∞ converges to the more stable value. Here,(21)ut=β2∞vt−1+1−β2∞∂E∂φt∞=maxβ2.vt−1,∂E∂φt,φt+1=φt−αr^tut.


### 5.2.8. Nadam [43]

It is a combination of Adam and NAG, where the parameters update equation using NAG is defined as follows:(22)vt=γvt−1+α∂E∂φt−γvt−1,φt+1=φt−αvt.The update equation using Nadam is expressed as follows:(23)r=β1r^t+1−β11−βt1∂E∂φt,φt+1=φt−αv^t+ɛr.


## 5.2.1. Gradient Descent [40, 42]

GD is the simplest form of optimization gradient descent algorithms. It is easy to implement and gives significant classification accuracy. The general update equation of the CNN parameters using the GD algorithm is(9)φt+1=φt−α∂E∂φt,where φ represents the update of the filters F, the weights W, and the biases B. ∂E/∂φ is the gradient with respect to the parameter φ.α is the model learning rate. A too-large value of α may lead to the divergence of the GD algorithm and may cause the oscillation of the model performance. A too-small α stops the learning process.


## 5.2.2. Gradient Descent with Momentum [43]

The momentum hyperparameterm defines the velocity by which the learning rate α must be increased when the model approaches to the minimal of the cost function E. The update equations using the momentum GD algorithm are expressed as follows:(10)vt=mvt−1+∂E∂φ,φt+1=φt−αvt,where vt is the moment gained at tth iteration.


## 5.2.3. AdaGrad [44]

In this algorithm, the learning rate is a function of the gradient∂E/∂φ. It is defined as follows:(11)αt=βGt+ɛ,where(12)β=α0ɛ,Gt=∑i=1t∂E∂φt2,where ϵ is a small smoothing value used to avoid the division by 0 and Gt is the sum of the squares of the gradients ∂E/∂φt.With a small magnitude of∂E/∂φ, the value of α is increasing. If ∂E/∂φ is very large, the value of α is a constant. AdaGrad optimization algorithm changes the learning rate for each parameter  at a given time t with considering the previous gradient update. The parameter update equation using AdaGrad is expressed as follows:(13)φt+1=φt−αt∂E∂φt.


## 5.2.4. AdaDelta [45]

The issue of AdaGrad is that with much iteration the learning rate becomes very small which leads to a slow convergence. To fix this problem, AdaDelta algorithm proposed to take an exponentially decaying average as a solution, where(14)EG2t=γEG2t−1+1−γG2t,Δθt=1EG2t+εGt,φt+1=φt−αΔθt,where EG2t is the decaying average over past squared gradients and γ is a set usually around 0.9.


## 5.2.5. RMSprop [45, 46]

In reality, RMSprop is identical to AdaDelta’s initial update vector, which we derived above:(15)EG2t=0.9EG2t−1+0.1G2t,φt+1=φt−αΔθt.


## 5.2.6. ADAM [17, 45, 46]

This gradient descent optimizer algorithm computes the learning rateα based on two vectors:(16)rt=β1rt−1+1−β1∂E∂φt,vt=β2vt−1+1−β2∂E∂φt2,where rt and vt are the 1st and the 2nd order moments vectors. β1 and β2 are the decay rates. rt−1 and vt−1 represent the mean and the variance of the previous gradient.Whenrt and vt are very small, a large step size is needed for parameters update. To avoid this issue, a bias correction value is added to rt and vt.(17)r^t=rt1−β1t,v^t=vt1−β2t,where β1t is β1 power t and β2t is β2 power t.The Adam update equation is expressed as follows:(18)φt+1=φt−αr^tv^t+ɛ.


## 5.2.7. AdaMax [45, 47]

The factorvt in the Adam algorithm adjusts the gradient inversely proportionate to the ℓ2 norm of previous gradients (via the vt−1) and current gradient t∂E/∂φt:(19)vt=β2vt−1+1−β2∂E∂φt2.The generalization of this update to theℓp norm is as follows:(20)vt=β2pvt−1+1−β2p∂E∂φtp.To avoid being numerically unstable, ℓ1 and ℓ2 norms are most common in practice. However, in general ℓ∞ also shows stable behavior. As a result, the authors propose AdaMax and demonstrate thatvt with ℓ∞ converges to the more stable value. Here,(21)ut=β2∞vt−1+1−β2∞∂E∂φt∞=maxβ2.vt−1,∂E∂φt,φt+1=φt−αr^tut.


## 5.2.8. Nadam [43]

It is a combination of Adam and NAG, where the parameters update equation using NAG is defined as follows:(22)vt=γvt−1+α∂E∂φt−γvt−1,φt+1=φt−αvt.The update equation using Nadam is expressed as follows:(23)r=β1r^t+1−β11−βt1∂E∂φt,φt+1=φt−αv^t+ɛr.


## 6. Data Augmentation Techniques

Deep convolutional neural networks are heavily reliant on big data to achieve excellent performance and avoid the overfitting problem.To solve the problem of insufficient data for Arabic handwritten characters, we present some basic data augmentation techniques that enhance the size and quality of training datasets.The image augmentation approaches used in this study include geometric transformations, feature space augmentation, noise injection, and mixing images.Data augmentation based on geometric transformations and feature space augmentation [17, 48] is often related to the application of rotation, flipping, shifting, and zooming.


### 6.1. Rotation

The input data is rotated right or left on an axis between 1° and 359°. The rotation degree parameter has a significant impact on the safety of the dataset. For example, on digit identification tasks like MNIST, slight rotations like 1 to 20 or −1 to −20 could be useful, but when the rotation degree increases, properly the CNN network cannot accurately distinguish between some digits.


### 6.2. Flipping

The input image is flipped horizontally or vertically. This augmentation is one of the simplest to implement and has proven useful on some datasets such as ImageNet and CIFAR-10.


### 6.3. Shifting

The input image is shifting right, left, up, or down. This transformation is a highly effective adjustment to prevent positional bias. Figure7 shows an example of shifting data augmentation technique using Arabic alphabet characters.Figure 7
Data augmentation through shifting a single data in four directions by two pixels.


### 6.4. Zooming

The input image is zooming, either by adding some pixels around the image or by applying random zooms to the image. The amount of zooming has an influence on the quality of the image; for example, if we apply a lot of zooming, we can lose some image pixels.


### 6.5. Noise Injection

As it could be seen on Arabic handwritten characters, natural noises are presented in images. Noises make recognition more difficult and for this reason, noises are reduced by image preprocessing techniques. The cos of noise reduction is to perform a high classification, but it causes the alteration of the character shape. The main datasets in this research topic are considered with denoising images. The question which we answer here is how the method could be robust to any noise.Adding noise [48, 49] to a convolution neural network during training helps the model learn more robust features, resulting in better performance and faster learning. We can add several types of noise when recognizing images, such as the following.(i)
Gaussian noise: injecting a matrix of random values drawn from a Gaussian distribution(ii)
Salt-and-pepper noise: changing randomly a certain amount of the pixels to completely white or completely black(iii)
Speckle noise: only adding black pixels “pepper” or white pixels “salt”Adding noise to the input data is the most commonly used approach, but during training, we can add random noise to other parts of the CNN model. Some examples include the following:(i)
Adding noise to the outputs of each layer(ii)
Adding noise to the gradients to update the model parameters(iii)
Adding noise to the target variables


### 6.6. Mixing Image’s Databases

In this study, we augment the training dataset by mixing two different Arabic handwritten characters datasets, AHCD and Hijja, respectively. AHCD is a clean database, but Hijja is a dataset with very low-resolution images. It comprises many distorted alphabets images.Then, we evaluate the influence of different mentioned data augmentation techniques (geometric transformations, feature space augmentation, and noise injection) on the recognition performance of the new mixing dataset.


## 6.1. Rotation

The input data is rotated right or left on an axis between 1° and 359°. The rotation degree parameter has a significant impact on the safety of the dataset. For example, on digit identification tasks like MNIST, slight rotations like 1 to 20 or −1 to −20 could be useful, but when the rotation degree increases, properly the CNN network cannot accurately distinguish between some digits.


## 6.2. Flipping

The input image is flipped horizontally or vertically. This augmentation is one of the simplest to implement and has proven useful on some datasets such as ImageNet and CIFAR-10.


## 6.3. Shifting

The input image is shifting right, left, up, or down. This transformation is a highly effective adjustment to prevent positional bias. Figure7 shows an example of shifting data augmentation technique using Arabic alphabet characters.Figure 7
Data augmentation through shifting a single data in four directions by two pixels.


## 6.4. Zooming

The input image is zooming, either by adding some pixels around the image or by applying random zooms to the image. The amount of zooming has an influence on the quality of the image; for example, if we apply a lot of zooming, we can lose some image pixels.


## 6.5. Noise Injection

As it could be seen on Arabic handwritten characters, natural noises are presented in images. Noises make recognition more difficult and for this reason, noises are reduced by image preprocessing techniques. The cos of noise reduction is to perform a high classification, but it causes the alteration of the character shape. The main datasets in this research topic are considered with denoising images. The question which we answer here is how the method could be robust to any noise.Adding noise [48, 49] to a convolution neural network during training helps the model learn more robust features, resulting in better performance and faster learning. We can add several types of noise when recognizing images, such as the following.(i)
Gaussian noise: injecting a matrix of random values drawn from a Gaussian distribution(ii)
Salt-and-pepper noise: changing randomly a certain amount of the pixels to completely white or completely black(iii)
Speckle noise: only adding black pixels “pepper” or white pixels “salt”Adding noise to the input data is the most commonly used approach, but during training, we can add random noise to other parts of the CNN model. Some examples include the following:(i)
Adding noise to the outputs of each layer(ii)
Adding noise to the gradients to update the model parameters(iii)
Adding noise to the target variables


## 6.6. Mixing Image’s Databases

In this study, we augment the training dataset by mixing two different Arabic handwritten characters datasets, AHCD and Hijja, respectively. AHCD is a clean database, but Hijja is a dataset with very low-resolution images. It comprises many distorted alphabets images.Then, we evaluate the influence of different mentioned data augmentation techniques (geometric transformations, feature space augmentation, and noise injection) on the recognition performance of the new mixing dataset.


## 7. Experimental Results and Discussion

### 7.1. Datasets

In this study, two datasets of Arabic handwritten characters were used: Arabic handwritten characters dataset “AHCD” and Hijja dataset.AHCD [6] comprises 16.800 handwritten characters of size (32 × 32 × 1) pixels. It was written by 60 participants between the ages of 19 and 40 years and most of the participants are right handed. Each participant wrote the Arabic alphabet from “alef” to “yeh” 10 times. The dataset has 28 classes. It is divided into a training set of 13.440 characters and a testing set of 3.360 characters.Hijja dataset [13] consists of 4.434 Arabic characters of size (32 × 32 × 1) pixels. It was written by 591 school children ranging in age between 7 to 12 years. Collecting data from children is a very hard task. Malformed characters are characteristic of children’s handwriting; therefore the dataset comprises repeated letters, missing letters, and many distorted or unclear characters. The dataset has 29 classes. It is divided into a training set of 37.933 characters and a testing set of 9.501 characters (80% for training and 20% for test).Figure8 shows a sample of AHCD and Hijja Arabic handwritten letters datasets.Figure 8
Arabic handwritten letters dataset. (a) AHCD dataset. (b) Hijja dataset.
(a)(b)


### 7.2. Experimental Environment and Performance Evaluation

In this study the implementation and the evaluation of the CNN model are done out in Keras deep learning environment with TensorFlow backend on Google Colab using GPU accelerator.We evaluate the performance of our proposed model via the following measures:Accuracy (A) is a measure for how many correct predictions your model made for the complete test dataset: (24)A=TP+TNTP+TN+FN+FP.Recall (R) is the fraction of images that are correctly classified over the total number of images that belong to class:(25)R=TPTP+FN.Precision (P) is the fraction of images that are correctly classified over the total number of images classified:(26)P=TPTP+FP.F1 measure is a combination of Recall and Precision measures:(27)F1=2∗P∗RP+R.Here, TP = true positive (is the total number of images that can be correctly labeled as belonging to a class x), FP = false positive (represents the total number of images that have been incorrectly labeled as belonging to a class x), FN = false negative (represents the total number of images that have been incorrectly labeled as not belonging to a class x),TN = true negative (represents the total number of images that have been correctly labeled as not belonging to a class x).Also we draw the area under the ROC curve (AUC), where we have the following.An ROC curve (receiver operating characteristic curve) is a graph showing the performance of all classification thresholds. This curve plots two parameters:(i)
True-positive rate(ii)
False-positive rateAUC stands for “area under the ROC curve.” That is, AUC measures the entire two-dimensional area underneath the entire ROC curve from (0.0) to (1.1).


### 7.3. Tuning of CNN Hyperparameters

The objective is to choose the best model that fits the AHCD and Hijja datasets well. Many try-and-error trials in the network configuration tuning mechanism were performed.The best performance was achieved when the CNN model was constructed of four convolution layers followed by three fully connected hidden layers. The model starts with two convolution layers with 16 filters of size (3 × 3), then the remaining 2 convolution layers are with 32 filters of size (3 × 3), and each two convolution layers are followed by max-pooling layers with (2 × 2) kernel dimension. Finally, three fully connected layers (dense layers) with Softmax activation function to perform prediction. ELU, a nonlinear activation function, was used to remove negative values by converting them into 0.001. The values of weights and bias are updated by a backward propagation process to minimize the loss function.To reduce the overfitting problem a dropout of 0.6 rate is added to a model between the dense layers and applies to outputs of the prior layer that are fed to the subsequent layer. The optimized parameters used to improve the CNN performance were as follows: Optimizer algorithm is Adam, the loss function is the cross-entropy, learning rate = 0.001, batch size = 16, and epochs = 40.We compare our model to CNN-for-AHCD over both the Hijja dataset and the AHCD dataset. The code for CNN-for-AHCD is available online [31], which allows comparison of its performance over various datasets.On the Hijja dataset, which has 29 classes, our model achieved an average overall test set accuracy of 88.46%, precision of 87.98%, recall of 88.46%, and an F1 score of 88.47%, while CNN-for-AHCD achieved an average overall test set accuracy of 80%, precision of 80.79%, recall of 80.47%, and an F1 score of 80.4%.On the AHCD dataset, which has 28 classes, our model achieved an average overall test set accuracy of 96.66%, precision of 96.75%, recall of 96.67%, and an F1 score of 96.67%, while CNN-for-AHCD achieved an average overall test set accuracy of 93.84%, precision of 93.99%, recall of 93.84%, and an F1 score of 93.84%.The detailed metrics are reported per character in Table3.Table 3
Experimental results on the Hijja and AHCD datasets via CNN-for-AHCD model.
CharactersHijja datasetAHCD datasetCNN-for-AHCDOur modelCNN-for-AHCDOur modelPRF1PRF1PRF1PRF11. Alif0.930.970.950.970.980.980.960.990.980.961.000.982. Baa0.820.910.860.930.960.940.970.970.970.990.970.983. Tea0.660.880.750.840.920.880.870.950.910.960.950.954. Thea0.760.810.780.940.930.930.950.880.920.970.950.965. Jam0.790.850.820.800.900.840.950.960.950.990.980.996. Haa0.830.600.700.850.890.870.930.930.930.990.960.977. Khaa0.760.770.770.820.730.770.940.930.930.990.960.978. Daal0.650.690.670.790.680.730.910.940.930.950.880.929. Thaal0.700.680.690.830.920.870.960.910.930.880.950.9210. Raa0.860.870.870.810.910.860.890.980.940.940.990.9611. Zaay0.870.890.880.960.900.930.940.880.910.960.920.9412. Seen0.840.920.880.950.920.940.950.910.931.000.970.9913. Sheen0.860.820.840.900.880.890.920.980.950.991.001.0014. Sad0.750.810.780.900.860.880.840.960.900.960.970.9715. Dhad0.800.760.780.910.930.921.000.890.940.970.950.9616. Tah0.900.830.870.960.890.920.960.940.950.940.980.9617. Dha0.830.870.850.840.820.830.970.940.950.970.950.9618. Ain0.740.700.710.860.860.860.950.900.920.980.980.9819. Ghen0.830.710.770.750.860.800.890.970.930.980.990.9920. Fa0.770.650.710.920.870.890.920.840.880.940.980.9621. Qaf0.810.800.810.870.890.880.870.910.890.990.950.9722. Kaf0.860.780.820.930.890.910.980.960.970.980.930.9623. Lam0.900.870.880.910.890.900.980.970.980.990.980.9924. Meem0.830.850.840.820.800.810.980.980.980.980.980.9825. Noon0.700.770.730.880.830.860.920.920.920.860.970.9126. Ha0.810.760.780.890.940.910.970.960.960.970.960.9727. Waw0.9300.820.870.940.920.930.960.940.950.970.940.9528. Yaa0.820.810.820.870.820.840.970.970.970.990.980.9929. Hamza0.740.730.74nanananananaAcc. (train)0.880.980.911.00Acc. (test)0.800.880.940.96Macro avg0.810.800.800.880.880.880.940.940.940.970.960.96Weighted avg0.810.800.800.890.880.880.940.940.940.970.960.96We note that our model outperforms CNN-for-AHCD by a large margin on all metrics.Figure9 shows the testing result AUC of AHCD and Hijja dataset.Figure 9
AUC curve of Arabic handwritten characters dataset. (a) AUC of AHCD testing dataset. (b) AUC of Hijja testing dataset.
(a)(b)


### 7.4. Optimizer Algorithms

The objective is to choose the best optimizers algorithms that fit the AHCD and Hijja best performance. In this context, we tested the influence of the following algorithms on the classification of handwritten Arabic characters:(i)
Adam(ii)
SGD(iii)
RMSprop(iv)
AdaGrad(v)
Nadam(vi)
Momentum(vii)
AdaMaxBy using Nadam optimization algorithm, on the Hijja dataset, our model achieved an average overall test set accuracy of 88.57%, precision of 87.86%, recall of 87.98%, and an F1 score of 87.95%.On the AHCD dataset, our model achieved an average overall test set accuracy of 96.73%, precision of 96.80%, recall of 96.73%, and an F1 score of 96.72%.The detailed results of different optimizations algorithms are mentioned in Table4.Table 4
Experimental results on the Hijja and AHCD through different optimizers algorithms.
AlgorithmAccuracy (%)Precision (%)Recall (%)F1 score (%)AHCDHijjaAHCDHijjaAHCDHijjaAHCDHijjaTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingAdam99.7696.6697.6088.4699.7696.7597.4087.9899.7696.6797.6088.4699.7696.6797.6088.47SGD99.4495.6894.1687.8599.4595.8494.2887.9399.4595.6894.1687.8599.4595.7094.1787.88RMSprop99.7296.2296.5587.9899.7396.3094.2887.8699.7296.2294.1687.9899.7296.2394.1787.95AdaGrad88.6283.5463.9060.3788.7383.7887.8664.9588.6283.5487.9863.9188.6083.5287.9563.77Nadam99.7796.7397.7688.5799.7796.8097.5787.8699.7796.7397.7687.9899.7796.7297.7687.95Momentum99.1695.5694.1687.7699.1895.7294.1187.7399.1795.5794.1787.7799.1795.5894.1587.74AdaMax99.8996.1997.1987.5799.9096.2597.0187.8699.9096.1997.1987.9899.9096.1997.2087.95


### 7.5. Results of Data Augmentation Techniques

Generally, the neural network performance is improved through the good tuning of the model hyperparameters. Such improvement in the CNN accuracy is linked to the availability of training dataset. However, the networks are heavily reliant on big data to avoid overfitting problem and perform well.Data augmentation is the solution to the problem of limited data. The image augmentation techniques used and discussed in this study include geometric transformations and feature space augmentation (rotation, shifting, flipping, and zooming), noise injection, and mixing images from two different datasets.For the geometric transformations and feature space augmentation, we try to well choose the percentage of rotation, shifting, flipping, and zooming for the model attending a good performance. For example, if we rotate the Latin handwritten number database (MNIST) by 180°, the network will not be able to accurately distinguish between the handwritten digits “6” and “9”. Likewise, on the AHCD and Hijja datasets, if rotating or flipping techniques are used the network will be unable to distinguish between some handwritten Arabic characters. For example, as shown in Figure10, with a rotation of 180°, the character Daal isolated (د) will be the same as the character Noon isolated (ن).Figure 10
Example of Arabic handwritten characters that cannot be properly distinguished when applying the rotation and flipping data augmentation techniques. (a) Confusion caused by rotation. (b) Confusion caused by flipping.
(a)(b)The detailed results of rotation, shifting, flipping, and zooming data augmentation techniques are mentioned in Table5.Table 5
Experimental results of data augmentation techniques on Hijja and AHCD datasets.
DatasetData augmentation techniques with using Nadam algorithmRotationShiftingRotation and shiftingFlippingZoomingTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyAHCD99.8297.6499.5698.0999.4198.4899.6997.2999.8598.00Hijja97.1889.8692.7290.2891.7391.2495.3988.9499.1688.66As shown in Table5 and Figure 11, by using rotation and shifting augmentation approaches, our model achieved a testing accuracy of 98.48% and 91.24% on AHCD dataset and Hijja dataset, respectively. We achieved this accuracy through rotating the input image by 10° and shifting it just by one pixel.Figure 11
Rotation and shifting data augmentation results of Arabic handwritten characters datasets. (a) Rotation and shifting results of AHCD dataset. (b) Rotation and shifting results of Hijja dataset.
(a)(b)Adding noise is a technique used to augment the training input data. Also in most of the cases, this is bound to increase the robustness of our network.In this work we used the three types of noise to augment our data:(i)
Gaussian noise(ii)
Salt-and-pepper noise(iii)
Speckle noiseThe detailed results of different types of noise injection are mentioned in Table6. As shown by adding different types of noise, the model accuracy is improved, which demonstrate the robustness of our proposed architecture. We achieved good results when adding noise to the outputs of each layer.Table 6
Experimental results on the Hijja and AHCD through noise injection.
Type of noise injection with using Nadam algorithmGaussian noiseSalt-and-pepper noiseSpeckle noiseTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyAHCD99.1796.7899.1697.1599.1496.82Hijja97.1889.8592.9690.1091.3289.73The proposed idea in this study is to augment the number of training databases by mixing the two datasets AHCD and Hijja, and then we apply the previously mentioned data augmentation methods on the new mixed dataset. Our purpose to use malformed handwritten characters as it proposes the Hijja dataset is to improve the accuracy of our method with noised data.The detailed results of data augmentation techniques on the mixed database are mentioned in Table7. As shown, the model performance depends on the rate of using Arabic handwriting “Hijja” database. The children had trouble following the reference paper, which results in very low-resolution images comprising many unclear characters. Therefore mixing the datasets would certainly reduce performance.Table 7
Experimental results on mixing of Hijja and AHCD through different data augmentation techniques.
Mixed datasetData augmentation techniques with using Nadam algorithmRotationShiftingRotation and shiftingFlippingGaussian noiseTrainingTestingTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracy80(%): AHCD20(%): Hijja20(%): AHCD 10(%): Hijja99.6297.4299.3698.0299.3898.3299.7797.0899.1596.7480(%): Hijja20(%): AHCD20(%): Hijja10(%): AHCD97.3688.4793.6689.0794.9190.5495.2188.2196.6888.4980(%): AHCD80(%): Hijja20(%): AHCD20(%): Hijja97.2774.5397.1675.1398.1378.1398.1674.2296.9874.02


## 7.1. Datasets

In this study, two datasets of Arabic handwritten characters were used: Arabic handwritten characters dataset “AHCD” and Hijja dataset.AHCD [6] comprises 16.800 handwritten characters of size (32 × 32 × 1) pixels. It was written by 60 participants between the ages of 19 and 40 years and most of the participants are right handed. Each participant wrote the Arabic alphabet from “alef” to “yeh” 10 times. The dataset has 28 classes. It is divided into a training set of 13.440 characters and a testing set of 3.360 characters.Hijja dataset [13] consists of 4.434 Arabic characters of size (32 × 32 × 1) pixels. It was written by 591 school children ranging in age between 7 to 12 years. Collecting data from children is a very hard task. Malformed characters are characteristic of children’s handwriting; therefore the dataset comprises repeated letters, missing letters, and many distorted or unclear characters. The dataset has 29 classes. It is divided into a training set of 37.933 characters and a testing set of 9.501 characters (80% for training and 20% for test).Figure8 shows a sample of AHCD and Hijja Arabic handwritten letters datasets.Figure 8
Arabic handwritten letters dataset. (a) AHCD dataset. (b) Hijja dataset.
(a)(b)


## 7.2. Experimental Environment and Performance Evaluation

In this study the implementation and the evaluation of the CNN model are done out in Keras deep learning environment with TensorFlow backend on Google Colab using GPU accelerator.We evaluate the performance of our proposed model via the following measures:Accuracy (A) is a measure for how many correct predictions your model made for the complete test dataset: (24)A=TP+TNTP+TN+FN+FP.Recall (R) is the fraction of images that are correctly classified over the total number of images that belong to class:(25)R=TPTP+FN.Precision (P) is the fraction of images that are correctly classified over the total number of images classified:(26)P=TPTP+FP.F1 measure is a combination of Recall and Precision measures:(27)F1=2∗P∗RP+R.Here, TP = true positive (is the total number of images that can be correctly labeled as belonging to a class x), FP = false positive (represents the total number of images that have been incorrectly labeled as belonging to a class x), FN = false negative (represents the total number of images that have been incorrectly labeled as not belonging to a class x),TN = true negative (represents the total number of images that have been correctly labeled as not belonging to a class x).Also we draw the area under the ROC curve (AUC), where we have the following.An ROC curve (receiver operating characteristic curve) is a graph showing the performance of all classification thresholds. This curve plots two parameters:(i)
True-positive rate(ii)
False-positive rateAUC stands for “area under the ROC curve.” That is, AUC measures the entire two-dimensional area underneath the entire ROC curve from (0.0) to (1.1).


## 7.3. Tuning of CNN Hyperparameters

The objective is to choose the best model that fits the AHCD and Hijja datasets well. Many try-and-error trials in the network configuration tuning mechanism were performed.The best performance was achieved when the CNN model was constructed of four convolution layers followed by three fully connected hidden layers. The model starts with two convolution layers with 16 filters of size (3 × 3), then the remaining 2 convolution layers are with 32 filters of size (3 × 3), and each two convolution layers are followed by max-pooling layers with (2 × 2) kernel dimension. Finally, three fully connected layers (dense layers) with Softmax activation function to perform prediction. ELU, a nonlinear activation function, was used to remove negative values by converting them into 0.001. The values of weights and bias are updated by a backward propagation process to minimize the loss function.To reduce the overfitting problem a dropout of 0.6 rate is added to a model between the dense layers and applies to outputs of the prior layer that are fed to the subsequent layer. The optimized parameters used to improve the CNN performance were as follows: Optimizer algorithm is Adam, the loss function is the cross-entropy, learning rate = 0.001, batch size = 16, and epochs = 40.We compare our model to CNN-for-AHCD over both the Hijja dataset and the AHCD dataset. The code for CNN-for-AHCD is available online [31], which allows comparison of its performance over various datasets.On the Hijja dataset, which has 29 classes, our model achieved an average overall test set accuracy of 88.46%, precision of 87.98%, recall of 88.46%, and an F1 score of 88.47%, while CNN-for-AHCD achieved an average overall test set accuracy of 80%, precision of 80.79%, recall of 80.47%, and an F1 score of 80.4%.On the AHCD dataset, which has 28 classes, our model achieved an average overall test set accuracy of 96.66%, precision of 96.75%, recall of 96.67%, and an F1 score of 96.67%, while CNN-for-AHCD achieved an average overall test set accuracy of 93.84%, precision of 93.99%, recall of 93.84%, and an F1 score of 93.84%.The detailed metrics are reported per character in Table3.Table 3
Experimental results on the Hijja and AHCD datasets via CNN-for-AHCD model.
CharactersHijja datasetAHCD datasetCNN-for-AHCDOur modelCNN-for-AHCDOur modelPRF1PRF1PRF1PRF11. Alif0.930.970.950.970.980.980.960.990.980.961.000.982. Baa0.820.910.860.930.960.940.970.970.970.990.970.983. Tea0.660.880.750.840.920.880.870.950.910.960.950.954. Thea0.760.810.780.940.930.930.950.880.920.970.950.965. Jam0.790.850.820.800.900.840.950.960.950.990.980.996. Haa0.830.600.700.850.890.870.930.930.930.990.960.977. Khaa0.760.770.770.820.730.770.940.930.930.990.960.978. Daal0.650.690.670.790.680.730.910.940.930.950.880.929. Thaal0.700.680.690.830.920.870.960.910.930.880.950.9210. Raa0.860.870.870.810.910.860.890.980.940.940.990.9611. Zaay0.870.890.880.960.900.930.940.880.910.960.920.9412. Seen0.840.920.880.950.920.940.950.910.931.000.970.9913. Sheen0.860.820.840.900.880.890.920.980.950.991.001.0014. Sad0.750.810.780.900.860.880.840.960.900.960.970.9715. Dhad0.800.760.780.910.930.921.000.890.940.970.950.9616. Tah0.900.830.870.960.890.920.960.940.950.940.980.9617. Dha0.830.870.850.840.820.830.970.940.950.970.950.9618. Ain0.740.700.710.860.860.860.950.900.920.980.980.9819. Ghen0.830.710.770.750.860.800.890.970.930.980.990.9920. Fa0.770.650.710.920.870.890.920.840.880.940.980.9621. Qaf0.810.800.810.870.890.880.870.910.890.990.950.9722. Kaf0.860.780.820.930.890.910.980.960.970.980.930.9623. Lam0.900.870.880.910.890.900.980.970.980.990.980.9924. Meem0.830.850.840.820.800.810.980.980.980.980.980.9825. Noon0.700.770.730.880.830.860.920.920.920.860.970.9126. Ha0.810.760.780.890.940.910.970.960.960.970.960.9727. Waw0.9300.820.870.940.920.930.960.940.950.970.940.9528. Yaa0.820.810.820.870.820.840.970.970.970.990.980.9929. Hamza0.740.730.74nanananananaAcc. (train)0.880.980.911.00Acc. (test)0.800.880.940.96Macro avg0.810.800.800.880.880.880.940.940.940.970.960.96Weighted avg0.810.800.800.890.880.880.940.940.940.970.960.96We note that our model outperforms CNN-for-AHCD by a large margin on all metrics.Figure9 shows the testing result AUC of AHCD and Hijja dataset.Figure 9
AUC curve of Arabic handwritten characters dataset. (a) AUC of AHCD testing dataset. (b) AUC of Hijja testing dataset.
(a)(b)


## 7.4. Optimizer Algorithms

The objective is to choose the best optimizers algorithms that fit the AHCD and Hijja best performance. In this context, we tested the influence of the following algorithms on the classification of handwritten Arabic characters:(i)
Adam(ii)
SGD(iii)
RMSprop(iv)
AdaGrad(v)
Nadam(vi)
Momentum(vii)
AdaMaxBy using Nadam optimization algorithm, on the Hijja dataset, our model achieved an average overall test set accuracy of 88.57%, precision of 87.86%, recall of 87.98%, and an F1 score of 87.95%.On the AHCD dataset, our model achieved an average overall test set accuracy of 96.73%, precision of 96.80%, recall of 96.73%, and an F1 score of 96.72%.The detailed results of different optimizations algorithms are mentioned in Table4.Table 4
Experimental results on the Hijja and AHCD through different optimizers algorithms.
AlgorithmAccuracy (%)Precision (%)Recall (%)F1 score (%)AHCDHijjaAHCDHijjaAHCDHijjaAHCDHijjaTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingTrainingTestingAdam99.7696.6697.6088.4699.7696.7597.4087.9899.7696.6797.6088.4699.7696.6797.6088.47SGD99.4495.6894.1687.8599.4595.8494.2887.9399.4595.6894.1687.8599.4595.7094.1787.88RMSprop99.7296.2296.5587.9899.7396.3094.2887.8699.7296.2294.1687.9899.7296.2394.1787.95AdaGrad88.6283.5463.9060.3788.7383.7887.8664.9588.6283.5487.9863.9188.6083.5287.9563.77Nadam99.7796.7397.7688.5799.7796.8097.5787.8699.7796.7397.7687.9899.7796.7297.7687.95Momentum99.1695.5694.1687.7699.1895.7294.1187.7399.1795.5794.1787.7799.1795.5894.1587.74AdaMax99.8996.1997.1987.5799.9096.2597.0187.8699.9096.1997.1987.9899.9096.1997.2087.95


## 7.5. Results of Data Augmentation Techniques

Generally, the neural network performance is improved through the good tuning of the model hyperparameters. Such improvement in the CNN accuracy is linked to the availability of training dataset. However, the networks are heavily reliant on big data to avoid overfitting problem and perform well.Data augmentation is the solution to the problem of limited data. The image augmentation techniques used and discussed in this study include geometric transformations and feature space augmentation (rotation, shifting, flipping, and zooming), noise injection, and mixing images from two different datasets.For the geometric transformations and feature space augmentation, we try to well choose the percentage of rotation, shifting, flipping, and zooming for the model attending a good performance. For example, if we rotate the Latin handwritten number database (MNIST) by 180°, the network will not be able to accurately distinguish between the handwritten digits “6” and “9”. Likewise, on the AHCD and Hijja datasets, if rotating or flipping techniques are used the network will be unable to distinguish between some handwritten Arabic characters. For example, as shown in Figure10, with a rotation of 180°, the character Daal isolated (د) will be the same as the character Noon isolated (ن).Figure 10
Example of Arabic handwritten characters that cannot be properly distinguished when applying the rotation and flipping data augmentation techniques. (a) Confusion caused by rotation. (b) Confusion caused by flipping.
(a)(b)The detailed results of rotation, shifting, flipping, and zooming data augmentation techniques are mentioned in Table5.Table 5
Experimental results of data augmentation techniques on Hijja and AHCD datasets.
DatasetData augmentation techniques with using Nadam algorithmRotationShiftingRotation and shiftingFlippingZoomingTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyAHCD99.8297.6499.5698.0999.4198.4899.6997.2999.8598.00Hijja97.1889.8692.7290.2891.7391.2495.3988.9499.1688.66As shown in Table5 and Figure 11, by using rotation and shifting augmentation approaches, our model achieved a testing accuracy of 98.48% and 91.24% on AHCD dataset and Hijja dataset, respectively. We achieved this accuracy through rotating the input image by 10° and shifting it just by one pixel.Figure 11
Rotation and shifting data augmentation results of Arabic handwritten characters datasets. (a) Rotation and shifting results of AHCD dataset. (b) Rotation and shifting results of Hijja dataset.
(a)(b)Adding noise is a technique used to augment the training input data. Also in most of the cases, this is bound to increase the robustness of our network.In this work we used the three types of noise to augment our data:(i)
Gaussian noise(ii)
Salt-and-pepper noise(iii)
Speckle noiseThe detailed results of different types of noise injection are mentioned in Table6. As shown by adding different types of noise, the model accuracy is improved, which demonstrate the robustness of our proposed architecture. We achieved good results when adding noise to the outputs of each layer.Table 6
Experimental results on the Hijja and AHCD through noise injection.
Type of noise injection with using Nadam algorithmGaussian noiseSalt-and-pepper noiseSpeckle noiseTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyAHCD99.1796.7899.1697.1599.1496.82Hijja97.1889.8592.9690.1091.3289.73The proposed idea in this study is to augment the number of training databases by mixing the two datasets AHCD and Hijja, and then we apply the previously mentioned data augmentation methods on the new mixed dataset. Our purpose to use malformed handwritten characters as it proposes the Hijja dataset is to improve the accuracy of our method with noised data.The detailed results of data augmentation techniques on the mixed database are mentioned in Table7. As shown, the model performance depends on the rate of using Arabic handwriting “Hijja” database. The children had trouble following the reference paper, which results in very low-resolution images comprising many unclear characters. Therefore mixing the datasets would certainly reduce performance.Table 7
Experimental results on mixing of Hijja and AHCD through different data augmentation techniques.
Mixed datasetData augmentation techniques with using Nadam algorithmRotationShiftingRotation and shiftingFlippingGaussian noiseTrainingTestingTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracyTraining accuracyTesting accuracy80(%): AHCD20(%): Hijja20(%): AHCD 10(%): Hijja99.6297.4299.3698.0299.3898.3299.7797.0899.1596.7480(%): Hijja20(%): AHCD20(%): Hijja10(%): AHCD97.3688.4793.6689.0794.9190.5495.2188.2196.6888.4980(%): AHCD80(%): Hijja20(%): AHCD20(%): Hijja97.2774.5397.1675.1398.1378.1398.1674.2296.9874.02


## 8. Conclusions and Possible Future Research Directions

In this paper, we proposed a convolution neural network (CNN) to recognize Arabic handwritten characters dataset. We have trained the model on two Arabic datasets AHCD and Hijja. By the good tuning of the network hyperparameters, we achieved an accuracy of 96.73% and 88.57% on AHCD and Hijja.To improve the model performance, we have implemented different optimization algorithms. For both databases, we achieved an excellent performance by using Nadam optimizer.To solve the problem of insufficient Arabic handwritten datasets, we have applied different data augmentation techniques. The augmentation approaches are based on geometric transformation, feature space augmentation, noise injection, and mixing of datasets.By using rotation and shifting techniques, we achieved a good accuracy equal to 98.48% and 91.24% on AHCD and Hijja.To improve the robustness of the CNN model and increase the number of training datasets, we added three types of noise (Gaussian noise, Salt-and-pepper, and Speckle noise).Also in this work we first augmented the database by mixing two Arabic handwritten characters datasets; then we tested the results of the previously mentioned data augmentation techniques on the new mixed dataset, where the first database “AHCD” comprises clear images with a very good resolution, but the second database “Hijja” has many distorted characters. Experimentally show that the geometric transformations (rotation, shifting, and flipping), feature space augmentation, and noise injection always improve the network performance, but the rate of using the unclean database “Hijja” harms the model accuracy.An interesting future direction is the cleaning and processing of Hijja dataset to eliminate the problem of low-resolution and unclear images and then the implementation of the proposed CNN network and data augmentation techniques on the new mixed and cleaned database.In addition, we are interested in evaluating the result of other augmentation approaches, like adversarial training, neural style transfer, and generative adversarial networks on the recognition of Arabic handwritten characters dataset. We plan to incorporate our work into an application for children that teaches Arabic spelling.



---

*Source: 9965426-2022-01-11.xml*