HICMA : The Handwriting Identification for Calligraphy and Manuscripts in A rabic Dataset (2023)

Abstract: Arabic is one of the most globally spoken languages with more than 313 million speakers worldwide. Arabic handwriting is known for its cursive nature and the variety of writing styles used. Despite the increase in effort to digitize artistic and historical elements, no public dataset was released to deal with Arabic text recognition for realistic manuscripts and calligraphic text. We present the Handwriting Identification of Manuscripts and Calligraphy in Arabic (HICMA) dataset as the first publicly available dataset with real-world and diverse samples of Arabic handwritten text in manuscripts and calligraphy. With more than 5,000 images across five different styles, the HICMA dataset includes image-text pairs and style labels for all images. We further present a comparison of the current state-of-the-art optical character recognition models in Arabic and benchmark their performance on the HICMA dataset, which serves as a baseline for future works. Both the HICMA dataset and its benchmarking tool are made available to the public under the CC BY-NC 4.0 license in the hope that the presented work opens the door to further enhancements of complex Arabic text recognition.

## introduction handwriting is a method used by humans to convey information in a written medium. every person possesses a unique style when drawing characters. this leads to a wide variation in the expression of written characters and texts. arabic text is of particular interest as arabic is one of the most globally spoken languages with more than 313 million speakers worldwide. in the arabic language, the complexity of written text increases since each character inherently has different forms depending on its position in the word, that is, whether it is in the beginning, middle, or end of the word. historical arabic text is abundant with more than ten centuries of rich arabic history and is often in need of being digitized. arabic historical manuscripts typically encompass handwritten texts, often of a significant age, characterized by cursive script, varying styles, and various artistic intricacies surrounding the written text. arabic calligraphy is a special form of arabic handwriting often used in manuscripts and as a prominent tool for ornating architecture. the arabic language relies on a variety of styles in manuscripts and calligraphy, each providing a different level of aesthetic artistic views and possessing its own rules. the most popular styles of handwriting in arabic manuscripts and calligraphy are diwani, thuluth, kufic, farsi, naskh, and ruqaa. arabic calligraphy is usually hand-drawn by experienced artists with complex drawing techniques that include heavy use of diacritics and decorative symbols. consequently, non-expert readers struggle to understand the calligraphic text. handwriting recognition is the task involved in converting handwritten text, which is typically captured as images, into machine-readable text. the complexity of this task is in accurately recognizing variations in the different styles of writing. moreover, the complexity becomes more apparent in historical arabic handwritten text due to its nature. to address the challenges in handwritten arabic and enhance the accessibility of arabic calligraphic content, the development of models capable of accurately recognizing this intricate handwritten text becomes essential. this, in turn, necessitates the availability of large datasets for the training and validation of such models. many works focused on creating datasets for the task of style classification of arabic calligraphy, such as the work of @xcite , while others focused on creating datasets for single character recognition @xcite , alrehali et al.'s (2020) foot_0 or single-digit recognition @xcite . the calliar dataset @xcite is the only existing dataset today that is tailored for arabic calligraphy recognition, on the character, word, sentence, and stroke levels. this dataset, however, contains calligraphic text drawn using digital pens on a plain white background, eliminating the realistic calligraphy style found in real-world arabic scripts. despite the plethora of datasets available in the arabic handwriting recognition space, very few represent a realistic and rich variety of styles for both historic manuscripts and calligraphy, target full-sentence handwriting recognition from unprocessed images, and are publicly accessible. we present the first publicly available dataset for arabic handwritten text in both manuscripts and calligraphy forms called the handwriting identification for calligraphy and manuscripts in arabic (hicma) dataset. with more than 5,000 images across five different arabic writing styles, the hicma dataset includes image-text pairs and style labels for all images. in this manuscript, we describe the collection, labeling, and processing steps of the novel hicma dataset and present a benchmark evaluation of the latest optical character recognition (ocr) models for the arabic language on hicma. the contributions of our work are three-fold: we make the hicma dataset foot_2 and the benchmarking tool foot_3 presented in this manuscript publicly accessible to the research community. ## related work several studies have dealt with collecting various types of datasets for different formats of arabic handwriting. for regular arabic handwriting, there are many datasets present in literature such as @xcite , consisting of 1,000 handwritten forms collected across 1,000 different writers from different countries. it was then extended to the online-khatt (mahmoud et al.) dataset consisting of 10,040 lines of handwritten text by 623 different writers. adab @xcite is another dataset that consists of 32,492 arabic words handwritten by more than 1,000 writers. there are also multilingual datasets that combine arabic and english like @xcite , which consists of 67,825 samples written by 355 writers. the mayastroun dataset consists of varying script types including words, characters, digits, mathematical expressions, and signatures. in contrast to regular arabic handwriting datasets, few studies in the literature have dealt with arabic manuscript and calligraphy text. one important dataset for arabic calligraphy is the calliar dataset @xcite which records digitized versions of images as strokes and drawings using digital pens. calliar is annotated for stroke, character, word, and sentence-level prediction. it also consists of 45,572 strokes, 7,556 words, and 2,500 sentences. however, the resulting dataset overlooks the contextual details present in real-world calligraphy such as the texture of the paper, surrounding artistic styles, noise, and interactions with other elements in the artwork. this as a result impacts an optical character recognition (ocr) model's ability to recognize calligraphy in diverse and authentic settings. other datasets in literature targeted calligraphy style classification by focusing on the style classification alone such as the dataset by @xcite . @xcite collected 1,685 images and classified them into 9 different calligraphic styles including thuluth, naskh, and diwani. each calligraphy style consists of around 180 to 195 images. moreover, allaf and al-hmouz (2016) developed a dataset and designed a system for classifying calligraphy images with artistic arabic calligraphy types, mainly thuluth, reqaa, and kufi. their dataset consists of 267 images divided evenly across the three calligraphy types. salamah and king (2018) also approached the challenge of calligraphy style classification and collected 1,000 calligraphy images scraped from public websites in various calligraphy styles. other sophisticated datasets, such as kertas @xcite , studied images of historical manuscripts. for producing kertas, 2,000 images were taken from various handwritten arabic scripts dating back to the fourteenth century and were manually annotated and segmented to extract images of the characters in the text. furthermore, khayyat and elrefaei (2020) collected 8,638 images of historical arabic manuscripts. their dataset is categorized into fourteen classes with six handwriting styles. @xcite collected 330 images of isolated arabic letters that were extracted from ancient manuscripts. this dataset consists of ruqaa, diwani, kufi, naskh, and farsi styles and has been used to classify arabic script styles based on segmented letters. the aforementioned calligraphy works can be classified into two categories, (a) datasets that simplified calligraphy for recognition tasks and (b) datasets that focused only on style classification with authentic calligraphy text. the simplified calligraphy datasets removed the contextual details commonly seen in real-world calligraphy. the remaining datasets that preserved the calligraphy in its true form were focused only on style classification, making them not directly useful for handwriting recognition. to the best of our knowledge, there is no dataset in the literature that deals with arabic handwriting recognition in both manuscript and calligraphy images. furthermore, many of the aforementioned datasets were either not publicly available or did not allow tampering with their dataset content. this makes the majority of the datasets in the literature not readily accessible for research purposes. in ## limitations as we present the hicma arabic dataset and the methodologies employed in this research, it is essential to acknowledge a few limitations that remain open for enhancement in future work. by addressing these limitations, future research will lead to advancements in arabic ocr technology. ## conclusion in this work, we presented hicma as the largest and most diverse public dataset to date for handwriting identification of calligraphy and manuscripts in arabic. the introduced dataset includes more than 5,000 images across five diverse arabic text styles along with image-text sentence pairs and style labels for all images. this dataset fills the existing literature gap for arabic manuscript and calligraphy text recognition. in this work, we detailed the data collection, labeling, and pre-processing steps of the created hicma dataset. we further presented statistics about the dataset styles and label size diversity. we finally conducted a benchmark evaluation of the top three current state-of-the-art ocr models for arabic and reported their performance on the hicma dataset, serving as a baseline for future works. upon analysis of the benchmark results, we highlight remaining open challenges in the hicma dataset and the existing ocr models that support arabic as a language. the hicma dataset and the accompanied benchmarking tool are made publicly available for the research community. we believe our work is the first among many making more inclusive arabic handwriting recognition for manuscripts and calligraphy possible. kattoura, and all other volunteers for their meticulousness and attention to detail which significantly enhanced the dataset's quality. their collective efforts exemplify collaboration, curiosity, and innovation, and without them, this project would not have been possible.