Text Extraction and Script Completion in Images of A rabic Script-Based Calligraphy: A Thesis Proposal (2025)

Abstract: Arabic calligraphy carries rich historical information and meaning. However, the complexity of its artistic elements and the absence of a consistent baseline make text extraction from such works highly challenging. In this paper, we provide an in-depth analysis of the unique obstacles in processing and interpreting these images, including the variability in calligraphic styles, the influence of artistic distortions, and the challenges posed by missing or damaged text elements. We explore potential solutions by leveraging state-of-the-art architectures and deep learning models, including visual language models, to improve text extraction and script completion.

## introduction arabic calligraphy initially emerged as a way to create a visually appealing script, evolving into a highly respected art form. this development was especially prominent during the ottoman empire, where calligraphy adorned buildings, mosque foundations, and a variety of other structures. from the reign of mehmed ii onwards, distinct schools of calligraphy began to emerge, gaining further momentum when sheikh hamdullah, the founder of the ottoman calligraphy school, arrived in istanbul during the rule of bayezid ii. under turkish craftsmen, arabic calligraphy was refined and achieved its most perfect forms @xcite . this art has experienced significant development in several countries, including iran, egypt, saudi arabia, and morocco. however, istanbul is particularly notable for its diverse and well-established tradition in calligraphy, largely shaped by ottoman influence. while most of these calligraphic works are in the arabic language, calligraphy using arabic letters also appears in other languages such as urdu, persian, and ottoman turkish. arabic serves as the common language in these artworks, as it is central to islamic texts, with many works featuring quranic verses, hadith, or prayers (duas) @xcite . while the artistic dimension of calligraphy is often the main focus, these works also encode significant linguistic, cultural, and historical information. calligraphy not only conveys islamic thoughts and architectural aesthetics but also serves as a valuable record of historical and cultural contexts. however, the complexity of calligraphic styles-where sentences vary dramatically in layout and form-often makes them challenging to read, even for those fluent in arabic. traditional optical character recognition (ocr) techniques fall short in interpreting such intricate designs, as they are not suited for the overlapping, stylized, or highly curved forms that define calligraphy. understanding and analyzing these works is crucial for preserving and studying historical and cultural heritage. the phrase consists of five words with twelve distinct letters, but the calligraphic styles vary in how these letters are connected, arranged, and stylized through different ligatures and artistic compositions. as can be seen in the figure, same sentence in various calligraphic styles and arrangements is challenging to recognize-even identifying the beginning of the sentence can be difficult due to the non-standard layouts and artistic variations. in this thesis proposal, we outline a specialized methodology designed to analyze and interpret documents and images featuring arabic script-based calligraphy with high accuracy. this approach aims to bridge the gap between visual artistry and textual extraction, enabling both aesthetic appreciation and understanding of these culturally significant texts. thesis contributions will be as follows: ## background this research explores arabic script-based calligraphy analysis, emphasizing its intricate and artistic nature. to develop a comprehensive approach, we reviewed related studies in arabic script analysis, calligraphic works, and non-arabic calligraphy. arabic character recognition is challenging due to its cursive and context-sensitive script. traditional methods like hog with svm achieved over 99% recognition on a dataset of 43,000 handwritten words @xcite , while random forests, knn, and mlp attained 100% accuracy on a dataset of 600 images across 28 classes @xcite . though designed for handwriting, these approaches could aid in labeling calligraphy datasets. deep learning methods have further improved recognition. cnns with lrelu achieved 99% accuracy by mitigating overfitting @xcite , and foundation models like qalam have advanced arabic ocr and handwriting recognition capabilities @xcite . qalam, combining a swinv2 encoder and a roberta decoder, excels in arabic script recognition. trained on over 4.5 million manuscript images and 60,000 synthetic pairs, it achieves a wer of 0.80% in handwriting recognition and 1.18% in ocr tasks. its support for diacritics and high-resolution inputs addresses limitations of many ocr systems. however, while useful for initial insights, these methods are less suited to the artistic variability and baseline inconsistencies in arabic calligraphy images. arabic calligraphy has been the focus of several research efforts, each contributing to the field with unique datasets, methodologies, and findings. the calliar dataset @xcite is a comprehensive resource, featuring 2,500 sentences and over 40,000 strokes. it covers multiple levels-stroke, character, word, and sentence-and includes styles like diwani, thuluth, and farisi, enabling tasks such as style classification, character recognition, and calligraphy generation. the salamah dataset (alsalamah and king, 2018) contains 3,467 images across 32 categories of arabic calligraphic letters, representing diverse styles. @xcite developed feature descriptors tailored to specific calligraphy styles, achieving superior performance on a dataset of 1,685 images across nine styles compared to existing methods, including deep learning approaches. a complementary study (gürer and gökbay, 2024) analyzed two datasets for classification tasks, achieving f1 scores of 90% for style classification and 79% for letter classification using transfer learning techniques. efforts in content recognition remain limited despite valuable datasets and classification studies. a study @xcite on a dataset of 388 images achieved 75% accuracy in mapping calligraphic images to their corresponding quotations, highlighting the challenges in recognizing content from artistic calligraphy and underscoring the need for more advanced methodologies. generative adversarial networks (gans) have been used to synthesize calligraphic styles like nastaliq, blending traditional and contemporary elements @xcite . similar methods in non-arabic calligraphy, such as chinese and japanese scripts, have applied cnns and transformers for style recognition and glyph generation @xcite @xcite @xcite . while these methods offer valuable insights, their direct use for arabic calligraphy content recognition is limited due to the unique features of arabic script. however, the strategies in these studies can guide the development of tailored approaches for arabic calligraphy. ## research goals and questions the main goal of this thesis is to analyze arabic calligraphy images to accurately extract the script contained within them. our main research question is: rq: what are the optimal methods for accurately extracting and reconstructing text from arabic calligraphy images, considering the unique artistic and structural challenges? we examine the challenges arising from the artistic elements and absence of consistent baselines, seeking methods to extract letters, phrases or complete sentences with high accuracy. to visually represent the methodology for tackling these challenges, figure 2 illustrates the step-by-step process for analyzing and extracting text from arabic calligraphy images. we outline three objectives to answer our research question: (i) data collection -gathering an extensive dataset of arabic calligraphy images. (ii) text extraction -developing methods to accurately extract textual data from images. (iii) script completion -reconstructing incomplete scripts when necessary. in the following sections, we define sub-questions for these objectives. ## conclusion this proposal outlines steps to extract textual data from arabic calligraphy images, addressing challenges from the language's unique features and the art's complexity. it focuses on reconstructing incomplete or damaged calligraphy using arabicspecific language models to enhance recognition accuracy. the research aims to develop tools for arabic calligraphy text recognition, benefiting areas such as historical document preservation and cultural heritage digitization. while primarily focused on text, it also acknowledges the importance of calligraphic styles and structures. it lays the groundwork for scalable, precise models capable of handling diverse calligraphic styles. to respect the cultural and historical significance of arabic calligraphy, the research will involve consultations with domain experts in art history and cultural heritage. this will ensure the accuracy and sensitivity of interpretations, while following best practices in digitization and preserving the integrity of these valuable artifacts.