Novel approaches for basic unit modeling in online handwriting recognition

An unconstrained, writer-independent, large vocabulary online handwritten word recognition system can be developed by modeling the basic recognition unit of the script. The basic recognition unit or the basic unit for brevity, represents the set of patterns (characters/strokes) for which the handwriting models are created. This thesis focuses on developing novel methods for basic unit modeling in online handwriting recognition. Three different directions of work are presented in this thesis for basic unit modeling. In the first part, we propose two novel strategies towards improving the classification ability of an existing basic unit recognition system by performing a reevaluation of the decision of a classifier. To begin with, we propose a strategy to reevaluate the decision of an HMM-based basic unit recognition system. The choice of the HMM classifier is owing to its success in several prior works of online handwriting recognition. In a conventional system, a test sample is recognized by first computing the log-likelihood scores from each of the class-specific HMMs. The class corresponding to the highest score is assigned to the sample. We demonstrated that, at times, the sole dependence of log-likelihood score of the HMM states may not be effective in capturing the finer nuances of the online trace that discriminates similar basic units / patterns. To alleviate this issue, we propose to analyze the HMM states corresponding to the top-2 confusion classes with the objective of identifying a subset of states (referred to as ‘discriminative states’) that can help provide cues to discriminate them better. Accordingly, we compare the log-likelihood scores for discriminative states of the two confused classes for the final decision. We also extend our proposal to develop a large vocabulary word recognition system by employing the HMM framework. In the second reevaluation technique, we propose a novel strategy to detect parts of the trace that present fine structural differences in similar looking basic units, which are referred to as ‘Discriminative Region’ in this thesis. Subsequent to their extraction, features TH-2066_136102002 are extracted with the hope of classifying the confused classes better. To identify the discriminative region, the proposed technique first splits the trace of the handwritten samples of a class into several segments and represents the distribution of each of them with the parameters of the k-means clustering. Thereafter, we apply the Dynamic Time Warping (DTW) algorithm to match the statistical characterizations associated with the basic units that form a confusion pair. In addition, we also propose a single-stage classification framework that takes into consideration the discriminative regions extracted between the basic units. In particular, we employ an ensemble of ( C 2 ) classifiers corresponding to all possible pairs of classes of basic units, wherein each of the classifiers are trained by utilizing the features from the discriminative region. At the time of recognition, a majority voting scheme is applied to the ensemble for assignment of the identity to the test basic unit pattern. Further, the discriminative region analysis is extended for the case of large vocabulary word recognition task. In the second part of the thesis, we focus on proposing two new feature representations for the basic units, mainly from a probabilistic viewpoint. Firstly, we propose a set of probabilistic features that are derived from a set of Gaussian mixture models (GMMs). The GMMs, being a generative model are intended to capture the class dependent characteristics. We show that the so derived posterior features aid in minimizing intra-class variability in the feature space while at the same time improving the separability between classes. The second feature representation is extracted directly from the trace of online handwriting by adopting a convolutional neural network (CNN). To the best of our knowledge, this is the first work of its kind that applies CNN directly on the (x, y) coordinates of the online handwriting data. The efficacy of the proposed GMM and CNN features are demonstrated for basic unit and large vocabulary word recognition tasks. In the third part of the thesis, we explore a novel classification approach to model the basic units. Our proposal is based on the utility of a hybrid deep neural network hidden Markov model (DNN-HMM) framework, that is quite prominent in the research of speech recognition. In a conventional HMM-based system, the Gaussian mixture model (GMM) TH-2066_136102002 is used to compute the value of the probability distribution of a given observation assigned to a HMM state. Different to it, in this work, a deep neural network (DNN) is trained to output the posterior probabilities of the observations. The posteriors are then converted into quasi-likelihoods by dividing them by the prior of the states. These quasi-likelihoods are then used with the HMM during the time of testing. The key feature of the proposed system is that the DNNs with many hidden layers are capable of learning very complex and highly non-linear relationships between the input and output. They can also incorporate high-dimensional feature vectors (obtained by considering contextual information) which help the system to make a better prediction of the input data. In the experimental analysis, we demonstrate the effect of several hidden layers in developing the DNN-HMM framework for online handwritten basic unit and word recognition systems. Furthermore, we combine the different explorations made in the thesis and evaluate the integrated system on online handwritten word samples. In particular, the features derived from the CNN are employed to characterize the data. Thereafter the DNN-HMM is used to model the basic unit model, with the reevaluation of the output class being performed by utilizing the discriminant region-based strategy. We perform experiments for both basic unit and large vocabulary word recognition tasks on databases of two scripts namely • the locally collected Assamese character and word datasets • the publicly available English UNIPEN character and UNIPEN-ICROW-03 word datasets We empirically show that the results obtained are promising with regard to the prior works reported in the literature.