# Intelligent Arabic Handwriting Recognition Using Different Standalone and Hybrid CNN Architectures
## 3 Materials and Methods
### Architecture of the Applied CNN Models  
This section outlines the machine-learning and deep-learning algorithms used in this experiment. All artificial intelligence experiments follow the same procedure. First, relevant data are collected and preprocessed to ensure that the raw data are suitable for training and testing. Second, certain features are extracted from the data and used to train and test the models. Finally, the extracted features are used for prediction, depending on the purpose of the research. Our research is a classification problem, so the extracted data will be used to classify the handwritten Arabic characters when we get to the final stage. From the literature review, it can be seen that machine-learning models have not been used as extensively as deep-learning models. Most of the papers examined were generally focused on feature ANN and CNN approaches. This research explores advanced ensemble methods of classification which have not previously been used in related experiments. The two datasets used for all the models were split into training and test sets at a ratio of 70% to 30%. The first dataset used was the Arabic MNIST dataset, which contains 10 classes for 0-9 numerical digits; the other dataset contains handwritten Arabic characters and has 28 classes resembling the 20 Arabic characters. Two different strategies were used to conduct a series of experiments. The first strategy involved standalone models, while the second strategy utilized hybrid models, which were designed by combining two models. The two strategies are discussed below:  
Standalone Models
* XGBoost stands for extreme gradient boosting. It is an ensemble machine that uses trees for boosting. It makes use of gradient boosting for the decision tree, thereby increasing speed and performance. These trees are built sequentially to reduce errors from the preceding tree, and each new tree learns from the previous one. Hence, as new trees grow, more knowledge is passed on. To enhance accuracy, the model, after the iterations, tries to minimize the following objective function, which comprises a loss function and regularization. There are three main forms of boosting: 1. A gradient-boosting algorithm uses a gradient algorithm and the learning rate; 2. A stochastic gradient-boosting algorithm uses sampling at the row and column per split levels; 3. A regularized gradient-boosting algorithm uses L1 and L2 regularization. * Random forest is a meta-estimator that fits several different decision trees on various subsamples, and the output is averaged so as to control overfitting. Random forest was used for this experiment because the error generated is always lesser than the decision tree due to out-of-bag error. Decision trees are a popular method for machine-learning tasks. Tree learning derives from the shell method for data mining because of its invariant behavior when it comes to scaling and other transformation methods for feature values, which are robust given the inclusion of feature values. The error generated by a random forest is always lesser than that generated by a decision tree because of the out-of-bag error, which is also called the out-of-bag estimate. This error-estimation technique is a method of estimating the prediction errors of random forests, which involves boosted decision trees that utilize bootstrap aggregation to  
Figure 3: A typical convolutional neural network. subsample data samples required for training. The parameters used for training were max_depth = 12, random_state = 0, and n_estimators = 100. * CatBoost stands for category boosting, which was developed by Yandex [47]. It uses the gradient-boosting technique and does not require conversion of the dataset to a specific format, unlike other machine-learning algorithms, making it more reliable and easier to implement. The parameters used for this experiment were iterations = 100, depth = 4, and learning_rate = 0.1.