# An Approach for Pronunciation Classification of Classical Arabic Phonemes Using Deep Learning
## 4 CNN Model for Arabic Short Vowels Classification
### Fine-Tuning the CNN Architecture  
We started with a sequential CNN architecture with eight processing layers to develop our baseline model. The architecture is made up of convolutional layers such that each layer applies a set of convolution filters to an input followed by a non-linear activation function [40]. The convolutional layers are defined with a kernel of size 3 and the number of filters set to 32, 64, and 128 in different layers. A resizing layer is used to downsample each input to speed up the model training process. A normalization layer normalizes each pixel in the image using mean and standard deviation values. The other important layer of the CNN architecture is the pooling layer [41], and its function is to progressively reduce the input to decrease the number of parameters and increase the network performance. There are different types of pooling operations: max pooling, average pooling, global max pooling, and global average pooling used for downsampling of the input samples. The CNN can also have additional layers for optimization and improved performance. The batch normalization layer [42] allows normalizing the input of each layer, as the problem of internal covariate shift occurs due to constantly changing the distribution of activation, and each layer needs to learn to adapt to a new distribution. Similarly, the dropout layer [43] is used to reduce the model overfitting by randomly dropping out some percentage of the layer output. The flatten layer is used to convert the two-dimensional data into one-dimension for final classification by the dense layer.  
Each layer has its associated activation function [44], whose task is to define how the weighted sum of input is transformed to the output from the nodes in a layer of the network. There are various activation functions, and the most popular ones are the ReLU (rectified linear unit), Sigmoid, and SoftMax. The ReLU function does not allow the activation of all the neurons simultaneously; when the output of the linear transformation is zero, the output neurons get deactivated. SoftMax function restricts the output values in the range 0 to 1, which are treated as probabilities of a particular class and usually used in the  
Figure 2: The results of different data augmentation techniques on an audio file: (**a**) the original audio data, (**b**) the audio signal after adding noise to original file, (**c**) the audio file after time-shifting, and (**d**) the audio file after increasing the speed.  
last layer of the neural network. Mathematically, the SoftMax function \(\sigma\) is applied to a vector of inputs, \(\overrightarrow{z}\), where each component of \(z\) is converted into corresponding probability according to its weight, \(e^{z_{i}}\) is standard exponential function for input vector, \(k\) stands for the number of classes and \(e^{z_{j}}\) presents standard exponential function for output vector shown in Equation (1) as follows:  
\[\sigma(\overrightarrow{z})_{i}=\frac{e^{z_{i}}}{\sum_{j=1}^{k}e^{z_{j}}} \tag{1}\]  
The dataset was divided into a ratio of 80% for the training set and 10% for validation and testing sets each. The batch size was set to 32. With the CNN architecture having eight layers, we achieved a testing accuracy of 80%. An improvement strategy [35] was to update the baseline sequential model by adding more convolutional, max pooling, and dense layers and updating the network to 12 layers. To find the optimal CNN model, we ran our model by applying a random search for the best parameters, which allows finding the optimal values of filter, kernel, and learning rate to achieve maximum accuracy. We found that the accuracy can be increased by increasing the filter size. We tested many CNN model modifications in a trial-and-error [35,45] manner and kept increasing the testing and validation accuracy in small steps. That allowed us to improve the testing accuracy to 90.0%.