Arabic and Latin handwritting recognition using multi-stream hidden Markov models
In this paper, we present a multi-stream approach for off-line handwritten word recognition. The multi-stream formalism presents many advantages: it can combine several kinds of independent features. The combination can be adaptive: some sources of information can be weighted, or even rejected if they are not reliable. The topology of the HMM can be adapted to each source of information. It also allows asynchronous modelling of streams. The proposed approach combines low level feature streams namely, density based features extracted from 2 different sliding windows with different widths, and contour based features extracted from upper and lower contours. Significant experiments have been carried out on two publicly available word databases: IFN/ENIT benchmark database (Arabic script) and IRONOFF database (Latin script). In order to model the Latin characters, we built 26 uppercase character models and 26 lowercase character models). In the case of Arabic characters, we built up to 159 character models. An Arabic character may actually have different shapes according to its position within the word (beginning, middle, end word position). Other models are specified with additional marks such as "shadda". In both Latin and Arabic script, each character model is composed of 4 emitting states. The observation probabilities are modelled with Gaussian Mixtures (3 per state). Embedded training is used where all character models are trained in parallel using Baum-Welch algorithm applied on word examples. The system builds a word HMM by concatenation of the character HMM corresponding to the word transcription of the training sample. The recognition step is doing allowing the HMM-recombination algorithm that consists in building the product HMM and using a classical Viterbi decoding algorithm. We investigate the extension of 2-stream approach to N streams (N=2,...,4) and analyze the improvement in the recognition performance. The computational cost of this extension is discussed. The developed system has been tested on two publicly available databases. For both scripts the results show significant improvement while using a multi-stream approach. The comparison of the multi-stream performances to the classical combination strategies namely, fusion of features and fusion of decisions shows the superiority of the multi-stream approach. Moreover, the proposed recognition system provides significant results comparable to the best results reported in the literature on both databases.
multi-stream hidden markov models arabic script latin script off-line handwriting recognition information combination