Handwritten Text Segmentation Using U -Net and Shuffled Frog-Leaping Algorithm with Scale Space Technique (2023)

Abstract: The paper introduces a new method for segmenting words from handwritten Bangla documents. We found that the available handwritten character recognition (HCR) systems do not provide the desired accuracy in recognizing the text written by school students. Recognizing students’ handwritten text becomes challenging due to certain factors, including a non-uniform gap between lines and words, and ambiguous, overlapping characters. The performance may be improved if the words in the text are segmented correctly before recognition. For the segmentation, we propose a combination of U-Net and a modified Scale Space method enhanced by the Shuffled Frog-Leaping Algorithm (SFLA). We employ the U-Net model for line segmentation; it effectively handles the variable spacing and skewed lines. After line segmentation, for segmenting the words, we use SFLA with Scale Space, allowing adaptive scaling and optimized parameter tuning. The proposed technique has been tested on two datasets: the openly available BN-HTR dataset and an in-house dataset prepared by collecting Bengali handwritten answer books from schools. In our experiments, we found that the proposed technique achieved promising performance on both datasets.

## introduction the process of converting handwritten text into machine-readable form is known as handwritten character recognition (hcr). when the input to the hcr system is an image of a whole handwritten page containing several lines of text, it is essential to extract the individual words for better processing. handwritten word segmentation is the process of extracting the words from a handwritten text image. although segmentation primarily focuses on separating words, it also refers to separating lines or characters in some domains. in the case of printed documents, the space between any two consecutive words, characters, and lines almost always remains consistent. however, in handwritten documents, the space varies since each person's writing style is unique. when addressing handwritten word segmentation, intricacies arise due to the presence of slant and skew at the word level and the non-uniform interword, intra-word, and inter-line gaps. also, noise and other language-specific features like matra, zone, and compound characters make the task more challenging. the task's difficulty increases when the skewness, non-uniformity in gaps, and noise level increase. for instance, we observed that word segmentation from bengali handwritten school answer books is more challenging than many other domains. in our study, we have developed a hybrid method for segmenting handwritten words. first, we employ the u-net model @xcite for segmenting lines from handwritten documents. the u-net model is based on the convolutional network architecture renowned for its success in biomedical image segmentation. the u-net model is particularly effective in this task due to its convolutional layers that adeptly learn handwriting characteristics such as stroke width and line curvature. its ability to process images at multiple resolutions allows it to adeptly segment text lines that are skewed or densely packed, overcoming typical challenges in handwriting recognition. following line segmentation, we utilize the shuffled frog-leaping algorithm (sfla) @xcite to enhance the performance of the traditional scale space method @xcite , previously applied to handwritten word segmentation in english and other languages. the efficacy of the scale space method is highly dependent on the precise selection of scaling parameters. the integration of sfla automates this parameter tuning, aiming for an optimal scaling that improves segmentation. consequently, this enhances the segmentation accuracy and overall performance of the scale space method. to compare the performance of the proposed technique, we have also considered three other widely used methods: the vanilla scale space method @xcite , the connected component analysis method @xcite , and the stroke width analysis method @xcite . all four methods are implemented and tested with the same dataset to measure their relative performance. the test dataset contains images from two different sources: an open online bengali hcr dataset -bn-htr @xcite , and our in-house dataset that contains images of handwritten answer books of grade six bengali medium students. in our experiments, we found that the proposed technique performs better than others. by integrating the shuffled frog-leaping algorithm with the scale space method (sfla-ss), we attained an accuracy of 93.64% on the bn-htr dataset and 56.84% on our in-house dataset. the subsequent application of the u-net model in conjunction with sfla-ss resulted in an enhanced accuracy of 96.18% on the bn-htr dataset and 69.29% on the in-house dataset. bengali, also known as bangla, is one of the major languages in the indian subcontinent. numerous research works have already been published on handwritten bengali character recognition. it's the seventh most spoken native language in the world, with nearly 230 million total speakers. the bengali script, also known as bangla, is an abugida, which means each character represents a consonant with an inherent vowel, and other vowels are represented with diacritics. it also includes a number of compound characters and conjuncts that combine two or more characters. this complexity makes it difficult to distinguish where one word ends and another begins. many research works are investigated for handwriting text recognition and handwritten document segmentation of different indian scripts @xcite ; @xcite ; @xcite . some of the popular text segmentation methods are the projection profile method, connected component analysis (cca), scale space method, stroke width analysis, and machine learning and deep learning-based methods. there are some studies in the literature that focus solely on segmenting handwritten bengali lines and words. @xcite applied the projection profile method for text line segmentation from bengali handwritten documents. @xcite proposed a segmentation technique for extracting individual text-lines from handwritten document images using morphological operation and run-length smearing algorithm (rlsa). a word in bengali script can be divided into three horizontally adjacent zones called the bottom zone, middle zone, and upper zone. these zones were utilized in this article @xcite to identify words that were contained within a single line. pramanik et al. @xcite proposes a method for recognizing handwritten bengali and devanagari words. it detects and corrects skew within words, estimates the headline, and segments words into meaningful pseudo characters. @xcite used the scale space approach for word segmentation from handwritten historical documents. @xcite proposed a tri-level handwritten text segmentation technique where they have used the scale space method for word-level segmentation of handwritten documents. to our knowledge, this is the only work that recognizes bengali words from images. all these methods have some drawbacks in the case of oblique texts and critical overlapping situations. also, most of these methods were applied on datasets where the space between lines and words was clear enough and noticeable. after observing the efficacy of all existing methods, we discerned that there is still significant scope for the enhancement of the word segmentation process from handwritten document images. ## existing algorithms for segmentation as we found in the literature, several techniques have been applied to segment handwritten words. these can be broadly grouped into learning-based and non-learning-based techniques. learningbased techniques refer to those using a machine learning or deep learning technique applied to training data to generate the model. on the other hand, non-learning-based techniques use an algorithm to analyze the pixels, their gaps, and other properties to perform the segmentation. as the training data is unavailable in our domain of interest (bengali handwritten answer books from schools), we consider the non-learning-based techniques in this study. as per our study of our domain of interest and theoretical understanding, we have chosen three methods that we implemented first. these are discussed below. ## result and discussion to assess the effectiveness of the proposed methods, we implemented some existing techniques and tested them using both datasets. the existing techniques are connected component analysis, stroke width analysis, and scale space method. then, we implemented sfla-ss for both line and word segmentation. the segmentation process was executed on the test set images, followed by a detailed evaluation based on the factors outlined earlier. ## conclusion in this study, we began by employing a nonlearning-based approach for word segmentation from handwritten documents, utilizing the shuffled frog-leaping algorithm (sfla) in conjunction with the standard scale space technique. this initial method proved sensitive to skewed lines and inconsistent spacing between lines. to address these issues, we subsequently incorporated a learning-based semantic segmentation approach, using the u-net model to segment the text lines from the handwritten documents first. we then applied the sfla-ss technique to segment the indi-vidual words from those lines. this combined technique outperformed the initial baseline method, resulting in a higher number of accurately segmented words. although our results have improved by adapting a traditional baseline segmentation algorithm and integrating it with the learning-based u-net model, there is potential for further refinement. in spite of all our efforts, our final accuracy reached up to 70% only on the school dataset. however, any real application demands higher accuracy. we observed when the line is not properly maintained by the students, which is quite common in school students; for example, a phrase is written inside the gap between two lines, words are written like a superscript or subscript, a portion of the text is written with footnote, words are too close, and the overlaped words/letters the proposed method doesn't work perfectly. we need to explore the literature for a better model in such a scenario. also, the dataset we use for training is not sufficient; we need to annotate a larger training data in order to obtain better performance of the u-net model.