# Intelligent Arabic Handwriting Recognition Using Different Standalone and Hybrid CNN Architectures
## 3 Materials and Methods
### Architecture of the Applied CNN Models
* Logistic regression uses L1, L2, and ElasticNet as regularization techniques and then calculates the probability of a particular set of data points belonging to either of those classes. For this experiment, the log was used as the cost function. * A support vector machine (SVM) is a supervised machine-learning algorithm for classification. It takes data points as inputs and outputs a hyperplane that separates the classes with the aim of achieving a hyperplane that maximizes the margin between the classes. This is the best hyperplane. For this experiment, two kernels were tested, the RBF and the linear kernel. * A feed-forward neural network is an artificial neural network wherein connections between the nodes do not form a cycle. This means that information moves only in the forward direction, from the input nodes, through the hidden nodes, to the output nodes [48]. Four optimization methods were experimented on, including Adam Optimizer, RMSprop, Adagrad, and stochastic gradient descent. Table 3 shows the feed-forward network architecture and all the parameters. * All these proposed algorithms were used for the recognition of handwritten Arabic characters, where statistical type features, shape-based features (curvatures), and categorical features with indexes were used. * A convolutional neural network is a deep-learning algorithm that can take in an input image, assign importance (learnable weights and biases) to various aspects/objects in the image, and differentiate one from another. The convolutional layer is first applied to create a feature map with the right stride and padding. Next, pooling is performed to lessen the dimensionality and properly adjust the quality of parameters used for the training, thereby reducing preparation time and battling overfitting. This experiment used max pooling, which takes the maximum incentive in the pooling window. Transfer learning was also carried out. This focuses on storing knowledge gained while solving one problem and applying it to a different, related problem; it is particularly useful in our case, since there is a limited number of data. A sequential model was used while using different optimization algorithms, including RMSProp, Adagrad, stochastic gradient descent, and Adam Optimizer. Table 4 shows the architecture of the convolutional neural network, with all the layers, the shapes expected, and the number of parameters. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Feed-Forward Network** & \\ \hline
**Architecture Design** & \\ \hline
**Layer (type)** & Output Shape & Param \# \\ \hline
**Dense\_30 (Dense)** & (None, 512) & 401,920 \\ \hline
**Dense\_31 (Dense)** & (None, 512) & 262,656 \\ \hline
**Dense\_32 (Dense)** & (None, 10) & 5130 \\ \hline
**Total params: 669,706** & \\
**Trainable params: 669,706** & \\
**Non-trainable params: 0** & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The feed-forward architecture design. CNN parameters were selected, and 129,162 parameters were chosen during the model training. Trainable parameters were set at 128,714, whereas the non-trainable parameters numbered 448 during the model training. The number of epochs was 20, the batch size was 32, and there were 3600 training samples and 2400 validation samples.