The greatest challenge of machine learning problem s is to select suitable techniques and resources such as tools and datasets. Despite the e xistence of millions of speakers around the globe and the rich literary history of more than a thousa nd years, it is expensive to find the computational linguistic work related to Punjabi Shahmukhi script , a member of the Perso-Arabic context- specific script low-resource language family. This paper presents a deep insight into the related work with summary statistics, advocating the popula rity and success of artificial neural networks and related techniques. The paper includes support from recent trends from the authentic sources based on the top-level researchers' feedback includ ing the machine learning frameworks. A comprehensive comparison of the most popular deep l earning techniques convolutional neural network and the recursive neural network has been p resented for the cursive context-specific scripts of Perso-Arabic nature. The overview of the available benchmark datasets for machine learning problems, especially for the Perso-Arabic group, is added. This paper incorporates essential knowledge contents for the researchers in machine learning and natural language processing disciplines on the selection of algorith ms, architectures, and resources. 2 Keywords: Machine Learning, Pattern Analysis, Natural Languag e Processing, Artificial multilayered Neural Networks, Optical Character Rec ognition system, Benchmark dataset. Introduction Background To preserve history, knowledge, culture, facts, an d figures for the present and future generations, written text is a medium of human comm unication for language expression with symbols. The written text is a durable and authenti c alternative to human speech for centuries in various forms. The field of automated script recogn ition has achieved significant real-world success for daily life applications, including writ er’s identification, recognition of home address, written messages, signatures, legal documents and b ank cheques, signature verification, to name a few. Alphabet (strokes and ligatures) recognition i s the foundation of a written text identification task approaching two or more letter words’ identifi cation. The advanced approaches are to develop systems for reading complete statements in context, completing incomplete words and sentences, spelling and grammar correction, and translation to other languages. Almost every language research group around the world is engaged in the d evelopment of machine learning systems for the native language. The scholarly work related to western languages is at its pinnacle however, Asian languages are in the way of development. Punjabi language with ov er 125 million speakers around the world can be written mainly in two scripts Shahmukhi and Gurm ukhi. Gurmukhi is an Indo-Aryan language script, used in the Indian side of Punjab, written in Landa script, is a derivative of the Devanagari style. Shahmukhi, a Perso-Arabic script, used to wr ite the Punjabi language, at the Pakistan side of Punjab, officially written in Nasta’leeq font, b orrowed from Urdu. Fig 1. shows the word Punjabi written in two scripts. The language has a n older history, more historic and scholarly work; however, a small development in machine under standing, with humble quality and lack of standardization. 3 Objective To develop a quality machine learning system capab le of recognizing the printed or handwritten Perso-Arabic scripts, it is desirable t o know the characteristics of the systems suitable for cursive context-specific scripts. This paper pr esents a detailed insight into the classification and recognition methodologies specific to the Perso -Arabic family of scripts using Machine learning techniques for optical character recogniti on (OCR), a procedure applied to printed or handwritten texts for the representation of compute r-generated file version of it. A general OCR algorithm includes data acquisition (text scanning), local segmentation (extraction of text from the images), preprocessing , text segmentation (ligatures and strokes), feature extraction, classification, and post-proces sing. Its general components are shown in Fig 2. Fig 1. Two scripts of the Punjabi language (left Gu rmukhi, right Shahmukhi in Nasta'liq) Fig 2. General components of the OCR 4 This paper is divided into the following sections: Introduction, Related work, Machine learning techniques suitable for cursive context-sp ecific scripts, variants of CNN and RNN architectures, and machine learning datasets. Related Work Most Asian languages are considered low-resourced languages. In the Perso-Arabic group Arabic and Persian languages have some growing tren ds towards computational linguistic research, but Urdu with millions of speakers worldw ide, has caught less attention of the researchers comparatively, and the case becomes worst when it c omes to the ‘Shahmukhi’ script, compared with ‘Gurmukhi’ [ 1]. The previous work addresses the Shahmukhi script recognition and classification is rare. Therefore, a referral appro ach has been adapted in which the language of the same group (Perso-Arabic) with similar scripts has been included in the literature review. The first category is the computational linguistic (CL) work of the Punjabi Shahmukhi script. The second category describes the computational linguistic wor k on other languages of the Perso-Arabic group, such as Arabic, and Urdu. The third category describes the database design-related work. The research questions include objective, language /script, methodology, dataset, framework, or customized program code available, the accuracy of the system. Research questions for the third category include dataset type Printed (synthetic, o r scanned), handwritten (forms; images; scanned), language/script: Perso-Arabic (Shahmukhi ; Urdu; Arabic; others), Lattin (English), database type: customized dataset, available (modif ication: vital changes or additions), data collection methods, and characteristics (database n ame/title, size, type, etc.), image formats, data file types, image size, resolution, preprocessed or raw, tested, tagged and percentage of success. In [ 2], the authors presented a Shahmukhi corpus using b uilt-in frameworks (Word2Vec and Gensim Lib) for the named entity recognition (NER) of proper nouns using supervised learning techniques (CRF, HMM, and RNN) for the validation o f their corpus with a success rate of 85% (for RNN). In [ 1], the authors presented similar work for the devel opment of Punjabi morphology, corpus, and lexicon using the built-in (GF) framewo rk. In [ 3]–[5], authors have used rule-based approaches to construct transliteration systems bet ween two scripts of the Punjabi language 5 Shahmukhi and Gurmukhi. Stemming of Punjabi words is carried out in [ 6] and [ 7] using the rule- based approach and Porter’s algorithms respectively with good accuracy. Punjabi stop words extraction is carried out in [ 8] using Gurmukhi. Shahmukhi mapping (lookup) method . A word extraction from sentences has been carried out in [ 9] using a rule-based approach. Using their space-based segmentation technique, this work has d eveloped a dataset from newspapers and online resources of three million words. A two-mill ion-word tagged corpus of semantically related words of Shahmukhi has been developed by [ 10 ]. In [ 11 ], the authors presented a survey of machine translation techniques between Punjabi and Urdu languages. In the Perso-Arabic category, the classification o f more than 3000 ligatures, printed in Urdu font ‘Noori Nasta’leeq’ from custom-built CLE-datas et has been presented in [ 12 ] using DCT sliding window and HMM (HMM Toolkit). A non-holisti c approach of Arabic script recognition has been carried out in [ 13 ] using convolutional neural networks (CNN) impleme nted in Keras- LeNet-5 architecture with Nvidia Tesla GPU, using s ingle-font images from the Arabic printed text image multi-font (APTID-MF) dataset. An MDLSTM classifier (works on extracted features by CovNet), has been trained on the MNIST dataset f irst and then applied to Urdu handwritten text dataset UNHD, exploiting the transfer learning technique in [ 14 ]. Statistics of The Related Work The field study has been performed over six months , between October 2020 to April 2021. From 2006 to 2021 the related work has been sifted from reputable sources such as IEEE, Springer, Elsevier, and others. With 70% journal and 30% conf erence papers, 6% were survey papers, whereas 94% were from applied research, with 85% of classification and OCR-based work. Among these Neural networks, HMM, and SVM has been found most popular.Fig 3 , shows the contribution of machine learning techniques related to OCR and classification. Their average percentage values of success are illustrated in Fig 4., showing that neural networks are the most suitable technique for the family of cursive contex t-specific scripts. 6 Fig 3. Contribution of Machine learning techniques in applied research work Fig 4. Success average percentage of machine learni ng techniques he family of cursive context-specific scripts Machine Learning Techniques for The Cursive Context-Specific Scripts A better way to reduce the experimentation efforts is to study the best-suited characteristics of the algorithms and architecture from the availab le work. Table 1 summarizes the survey work [15 ] of text classification techniques, describing mac hine learning techniques are superior based on their usage popularity and accuracy compared to the rule-based techniques. Table 2 summarizes the performance of deep learning, naïve Bayes, and SVM. [ 15 ]. 7 Table 1. Comparison of text classification approach es Approach Usage Working Accuracy Computational level Rule-based  Handcrafted learning (linguistic) rules Moderate; poorly compared with ML Variable; mostly easy Machine learning  Features based on data High High; easier due to available frameworks Hybrid ~ Combination High Variable Table 2. Comparison of Text Classification Approach es Approach Usage Data size required Computational power/resources required Accuracy Naïve Bayes Popular Small Low Good SVM Popular Small High; medium for multiclass problems Good Deep learning Popular Large High Best Available Neural Network Architectures A list of available artificial neural network (ANN ) architectures is given in Table 3 showing popular architectures used in appli cations like image classification, OCR, and NLP. Related research works are also mentioned wher e CNN, RNN, and their variants are popular architectures. CNNs are multi-layered Neural networ ks, to recognize visual patterns straight from image pixels with minimal preprocessing requirement s. RNNs due to ‘direct-graph-temporal- sequence’ architecture are best suited for the clas sification of the variable-length input sequences, e.g., ligatures. 8 Table 3. ANN architectures NN Architecture Published Perceptron (P) and multilayered perceptron 1958 Deep CNN, [ 13 ], [ 17 ]–[22 ] 1980s Hopfield network, Kohonen Net 1982 Boltzmann machine; restricted BM 1985 Recurrent Neural Network/ MDRNN, [ 2], [ 14 ], [ 19 ], [ 20 ], [ 23 ], [ 24 ] 1986 Autoencoder/sparse ae 1986 Radial Bias Networks 1988 Long short-term memory, [ 17 ], [ 19 ], [ 20 ], [ 24 ] 1997 Capsule Net (CAPSNET) 2000 Extreme Learning Machine 2001 Deep Belief Network 2006 Deconvolutional Neural Net (DNN) 2010 Gated Recurrent Unit, Generative Adversarial Net (G AN), Neural Turing Machine (NTM) 2014 Deep Conv. Inverse Graphic Net (DC-IGN) 2015 Markov Chain Nnet 2018 Proposed Architectures CNN and RNN the subclasses of artificial neural ne tworks (ANNs) are layered architecture comprising of three main layers: Input, output, and middle (hidden) layers. With many hidden layers, they are called deep ANNs. These layers are composed of cells (neurons) that work together to perform pattern recognition tasks. ANNs can be f urther categorized as cyclic and acyclic ANNs; cyclic have feedback connections; whereas, acyclic have not, and also called feedforward networks (FFNs) [ 16 ]. 9 CNN is a unique class of multi-layer feed-forward n eural networks with the superb ability of typed and handwritten text recognition. It is prim arily centered around on-the-spot learning. It can learn variable, complex, non-linear trends f rom the input images. Generally, CNN comprises two stages: feature extraction (convoluti on layers, pooling, and an activation function) and the classification stage (the fully c onnected and the classification layers). Researchers have used different CNN architectures f or the text and character recognition tasks for a range of Perso-Arabic scripts. Fig 5 shows the basic and general architecture outline o f the CNN. Fig 5. Outline of CNN Architecture [https://medium. com/techiepedia/binary-image- classifier-cnn-using-tensorflow-a3f5d6746697] This architecture shows distinct features for word and character recognition of cursive context-specific scripts and fonts: CNNs do not require hand-crafted features. To a cer tain limit, they can learn noise impermeable, shift, translation, and shape distorti ons invariant, silent features from the training data. [ 13 ], [ 19 ]. This feature outperforms many other traditional techniques with a considerable error rate reduction [ 18 ]. Low chances of overfitting and reduced computationa l complexities due to CNN weight sharing and local connectivity properties, thus bet ter learning capacity [ 19 ]. Due to convolution operation, CNN keeps the spatial features of the in put image intact, thus used for temporal information (visual data) classification and featur e extraction. Due to large data size requirements and expensive computational cost, the use of CNN may be limited to feature 10 extraction or to use a transfer approach of using a pre-trained CNN [ 19 ], such as (AlexNet, VGG16, VGG19, GoogleNet, InceptionV3, and ResNet101 ) [18 ]. The presence of the pooling stage in CNN reduces th e dimensionality of the feature maps. In the training phase, the filter weights’ adjustment to loss function provides customized optimal filter layers for a specific machine learning probl em. The RNN works with a minimum of one feedback elemen t thus activation circulates in a loop. The recurring path between hidden cells causes a ge nerative and sequential learning modeling characteristic [ 24 ], useful in speech recognition, machine translati on, handwriting recognition, and generative modeling (e.g., simulated datasets). The basic architecture is shown in Fig 6 . Fig 6. The basic architecture of RNN [pydeeeplearni ng.weebly.com] Some of the characteristics are given: When previous information is needed to comprehend the recent (or delayed) stage, RNN is the best choice. It can handle variable lengths of input sequences thus suitable for machine learning tasks of NLP and the network gets unaffect ed in terms of its model size. Its popular applications are sentiment analysis in text and spe ech, name entity recognition, and machine translation. RNNs are suitable for temporal information classif ication as CNNs; however, owing to their feedback property, they can handle sequences like video frames in a better way comparatively. RNNs are computationally efficient; however, generally harder to train compared with CNN, thus requiring pre-extracted fea tures or a CNN stage before the classification stage [ 20 ]. 11 During backpropagation, an RNN might suffer the va nishing gradient problem where the gradient becomes very small, inappropriate to modif y the weights during the training phase. The solution is long-short-term-memory RNN (LSTM- R NN). Fig 7 shows the simple architecture of the LSTM network. LSTM-RNN can filt er the required information as per the requirements of the task during the training phase using its memory units [ 20 ]. These gates are responsible to control the activation of the intern al unit. The RNN-LSTM is best suited with the CTC layer. A transfer learning approach (Tl-Dee p-MDLSTM) can produce promising results [ 14 ]. Fig 7. LSTM architecture [Wikipedia.com: LSTM] Bidirectional RNN (BRNNs) are two independent RNN c onnected in forward and reverse directions. Thus, a BRNN gets trained twice to the input sequence. Preservation ability of temporal and contextual characteristics of the even ts, RNN and its variants BD-LSTM and BLSTM are very popular for cursive writing recognit ion [ 24 ]. Integration of BDRNN and BLSTM results in MDLSTM architecture, shown in Fig 8. The MDLSTM is recommended for cursive context-specific scripts as it can handle a lmost all complexities of this family. [ 16 ]. An output CTC layer supplies a tremendous result. MDRNN can handle multidimensional data due to mult iple recurrent connections. The variants of RNN perform better than RNN at high com putational costs. 12 Architectures of CNN and RNN Some of the popular architectures have been present ed here, used in cursive context-specific script recognition problems. Fig 8. MD-LSTM architecture [https://analyticsindi amag.com/complete-guide-to- bidirectional-lstm-with-python-codes/] CNN Architectures LeNet-5: Invented in 1989, principally a classic CN N architecture and simplest one. With two convolutional (and pooling) layers and three fully connected layers (total 5), with around 60,000 parameters. In [ 13 ] authors have used this architecture for Arabic te xt segmentation. AlexNet: CNN architecture was invented in 2012, wit h around 60 million parameters, five convolutional (and pooling), and three fully connec ted, eight in total layers [ 25 ]. The authors (K. Alex, S. Ilya, and Hinton G.) first introduced rectified linear unit (ReLU) as an activation function. [ 18 ] used AlexNet as a pre-trained network for their s ystem of identification of Urdu text ligatures from video frame captions. VGG16: 138 parameters, 13 convolutional and 3 fully connected layers, five max-pooling layers, and one SoftMax layer, from visual geometry group in 2014 for the large-scale image recognition, the VGG16 was a deeper network compare d to its predecessors [ 25 ]. There are other versions of VGG, such as VGG11, VGG16, and VG G19 with different layers. In [ 18 ], the authors used VGG16 and VGG19 (sixteen convoluti onal layers, five pooling, and three fully-connected layers) as pre-trained networks for their system of identification of Urdu text 13 ligatures from video frame captions. ResNet-50: Residual network, a 50-layered deep CNN, is capable of handling millions of images, is based on many computer vision algorithms . The recent versions of ResNet have more than 100 layers. [ 18 ], used this architecture for translational learnin g in Urdu text recognition. RNN Architectures RNNs are especially suitable for NLP, and time seri es applications as they model sequences. The specific architecture of RNN contro ls the information among various cells (neurons), therefore a selection of specific archit ecture is very important for the success of a task. ERNN: Elman recurrent neural network, is the basic and simplest architecture of RNNs. It can be used to learn grammar from a dataset without ann otation, to predict potential words. The input and output layers are feedforward layers, whe reas the hidden layers are recurrent. GRU: Gated recurrent unit, a gated architecture com bines the input and forget gates into an update gate and was introduced by K. Cho et al in 2 014. It is a variation of LSTM architecture with forget gate and no output gate. Its usage and performance are like LSTM with a lesser number of parameters. Machine Learning Datasets There are various datasets available for different machine learning problems free of cost, paid, and free upon request. Some of the popular da tasets have been given below along with the basic information required to get fundamental g uidelines. These datasets are available online for non-commercial use. Benchmark Datasets MNIST dataset: A modified National Institute of Sta ndards and Technology database (created in 1998) used for the training and testing of sever al machine learning algorithms. It contains images of single digits from 0-9. It is good for le arning as now it is considered a resolved dataset for advanced machine learning algorithms. 14 CIFAR10: Canadian Institute for Advanced Research, an image classification dataset for object recognition, comprising of 10 classes such as image s of airplanes, automobiles, birds, cats, dogs, deer, frogs, horses, ships, and trucks. CIFAR100: A modified version of CIFAR10, contains 1 00 classes of objects including subcategories within a class such as types of trees . ImageNet: A large-scale object recognition dataset with 1000 classes, 1,281,167 training images 50k validation images, and 100k test images of variable sizes. The dataset is designed following WordNet (a lexical database of English ma intained by Princeton University). Omniglot: A large dataset of handwritten 1623 categ ories and 20 versions of each category (character). Characters are 50 (real and imaginary) alphabets from different scripts along with different strokes and position coordinates. Languag es are Malay, Hebrew, Korean, Latin, Greek, and many others. Table 4 summarizes the important characteristics of the discussed datasets and Fig 9 , Fig 10 , Fig 11 , Fig 12 , and Fig 13 show samples of these datasets. These are benchmark datasets and are useful to (a) train the algorithm for transfer learning, (b) study the dataset design, and (c) do practice if it is in tended to work with a customized dataset. Fig 9. MNIST dataset sample Fig 10. CIFAR10 dataset sample 15 Fig 11. CIFAR100 dataset sample Fig 12. ImageNet dataset sample Fig 13. Omniglot dataset sample 16 Table 4. Image classification dataset - summary Title Classes Type Size MNIST 10 Labeled 28 x 28 Grayscale image 50k: Training, 10k: Test CIFAR10 10 Labeled 32 x 32 RGB image 50k: Training, 10k: Test, 6k/class CIFAR100 100 Labeled 32 x 32 RGB image 50k: Training (500/class)10k: Test (100/class) 20 (5/class) superclasses ImageNet 1000 Labeled Variable 1.3mil: Training, 50k: Validation (50/class), 100k: Test (100/class) Omniglot 1623 categories (50 alphabets) - 20 images/category Perso-Arabic Datasets There are a limited number of benchmark datasets fo r Perso-Arabic scripts. Some of them have been presented here: UPTI: Urdu Printed Text Image dataset, used by [ 19 ], [ 20 ], [ 24 ], [ 26 ]–[29 ], comprised of 10000 Urdu text synthetic lines in Nasta’liq font coverin g a variety of topics. CLE: A linguistic resource by College of Language E ngineering, University of Engineering and Technology Lahore, Pakistan. This comprehensive dat abase is available online for purchase. It is comprised of image, speech, and text corpora in dif ferent font sizes. CENPARMI: A dataset developed by Centre for Pattern Recognition and Machine Intelligence, Concordia University, Canada comprising of digits, numeral strings, special symbols, and isolated characters, and Urdu words and Urdu dates in differ ent styles [ 30 ]. Although the first dataset of its kind till that time, this handwritten offline datas et has a limited number of data samples, 44 individual characters, and 57 Urdu words, focusing on one field mostly, the financial terms. 17 Although the dataset has been tested to achieve a s uccess rate of 98.61%, the limited number of words from one field is a compromise over the gener alization. Further, only one method has been used as a data classifier (SVM), instead of using m ore methods to validate the results and performance, whereas no standard error analysis met hod has been described. To the best of the author’s knowledge, most of the discussed datasets are not available to researchers at all or are available online at a con siderable cost. Conclusion The paper has described the detailed review of the related work in the field of Perso-Arabic text recognition mainly in three categories: Shahmu khi computational-linguistic-based research available work, Shahmukhi (and other Perso-Arabic s cripts) recognition systems, and benchmark database development work related to Perso-Arabic s cript recognition. The research findings include review categories, work type classification , distribution of techniques, machine learning techniques, the success rate for machine learning t echniques, and research article category. The paper also presented summaries of comparisons of co mmon text classification approaches and machine learning-based text classification techniqu es. Recent trends showing the popularity of Convolutional neural networks and Recurrent neural networks in methods and algorithms; and dominant Python-based tools for machine learning fr ameworks have also been presented. A review of neural network architectures for optical charact er recognition, natural language processing, and image processing has been described. Deep learning architectures such as CNN and RNN and their variants have been described with their important c haracteristics useful for the selection of a machine learning algorithm. Available benchmark dat asets and low-resource language datasets have been described. 18