R ound T rip OCR : A Data Generation Technique for Enhancing Post- OCR Error Correction in Low-Resource D evanagari Languages (2024)

Abstract: Optical Character Recognition (OCR) technology has revolutionized the digitization of printed text, enabling efficient data extraction and analysis across various domains. Just like Machine Translation systems, OCR systems are prone to errors. In this work, we address the challenge of data generation and post-OCR error correction, specifically for low-resource languages. We propose an approach for synthetic data generation for Devanagari languages, RoundTripOCR, that tackles the scarcity of the post-OCR Error Correction datasets for low-resource languages. We release post-OCR text correction datasets for Hindi, Marathi, Bodo, Nepali, Konkani and Sanskrit. We also present a novel approach for OCR error correction by leveraging techniques from machine translation. Our method involves translating erroneous OCR output into a corrected form by treating the OCR errors as mistranslations in a parallel text corpus, employing pre-trained transformer models to learn the mapping from erroneous to correct text pairs, effectively correcting OCR errors.

## introduction the devanagari script is the most extensively used writing system in the indian subcontinent. it was the principal script for sanskrit, the ancient literary language of indian civilization. sanskrit was used to write a wide range of texts covering various domains, including literature, philosophy, science, art, architecture, and mathematics. this includes the vedas, upanishads, and epics like mahabharata and ramayana. devanagari script originated from ancient brahmi script through various transformations @xcite . apart from vowels, modifiers and consonants (figure 1 ), it has a rich set of conjunct consonants, known as ligatures, where multiple characters combine to form new glyphs. these are difficult to segment and recognize because they don't correspond directly to individual letters. devanagari characters often have vowel signs (matras) and other diacritical marks that appear above, below, or beside the base character. these modifiers can be challenging to detect, segment, and associate correctly with the base character. devanagari script includes a horizontal line (called the shirorekha) that connects the characters in each word. unlike in latin scripts, where spaces clearly divide words, in devanagari, words often connect via the headline, making word segmentation harder for ocr systems. proper segmentation of devanagari words, characters, and sub-components (such as vowels and consonants) is difficult because components often overlap, connect through ligatures, or blend with the shirorekha line. this is less common in simpler scripts like latin, where individual letters are often spaced apart and stand independently. many devanagari characters look quite similar, especially in certain fonts or degraded images, leading to higher chances of ocr errors. ocr technology has revolutionized the digitization and processing of written or printed text by en-abling machines to automatically convert scanned documents or handwritten texts into editable and searchable text formats. however, despite significant advancements over the years, the accurate recognition of text from scanned documents remains a challenging task due to inherent complexities in document layouts, font variations, noise, and other distortions. traditional ocr systems typically follow a pipeline approach comprising image preprocessing, feature extraction, character segmentation, and recognition stages. while these systems have achieved remarkable success in many applications, they are susceptible to errors, especially when dealing with degraded or low-quality document images. ocr errors can manifest in various forms, including misrecognitions, substitutions, omissions, and insertions, leading to inaccuracies in the recognized text output. these errors not only impede the reliability of ocr systems but also pose significant challenges for downstream tasks such as information extraction, text mining, and machine translation @xcite @xcite . addressing ocr errors requires robust error detection and correction mechanisms that can effectively handle a wide range of error patterns and variations. ## roundtripocr the creation of artificial ocr data involves a systematic process aimed at simulating real-world scenarios while taking into consideration the common ocr error types and generating diverse datasets for training and evaluation purposes. to introduce variability into the dataset, 50 different devanagari font combinations were selected from google fonts 2 . each font style offered unique characteristics, such as varying # of sent. ## sequence to sequence models we conducted a series of experiments employing sequence-to-sequence models: mbart, mt5 and indicbart. these are powerful models designed for multilingual tasks, particularly in low-resource languages. mbart (multilingual bart) is a sequence-tosequence denoising autoencoder that pre-trains on a variety of languages by corrupting and reconstructing text, making it highly effective for tasks like machine translation and text generation across different languages @xcite . it has been extensively used for tasks involving noisy inputs, such as post-ocr error correction, due to its ability to learn contextual representations and perform cross-lingual transfer @xcite . we used mbart-large-50 version of mbart. indicbart is a variant of mbart that is specifically tailored for indic languages like hindi, bengali, marathi, and others @xcite . it adapts the pre-training and fine-tuning processes to better handle the linguistic and scriptural characteristics of these languages, which are often underrepresented in large-scale language models. in-dicbart has proven to be highly effective for tasks such as machine translation in indic scripts. ## experiments and results the pre-trained models were sourced from hugging face foot_4 and finetuned using nvidia a100 gpu for 2 to 3 epochs. a learning rate of 5e-4 was applied, managed by a polynomial learning rate scheduler. the training was conducted with 32-bit floating-point precision, and the best-performing model from each run was saved for evaluation. to facilitate effective model training and evaluation, we partitioned the dataset into training, testing, and validation sets. the testing set and validation set contained 10,000 pairs each. we further curated the second dataset exclusively featuring a single font style; in particular, we chose the sumana font as it shows a close to average cer when compared with all the fonts used in the creation of the dataset as shown in figure 4 . this bifurcation allowed us to explore the potential advantages conferred by employing data with ## conclusion and future work we introduced a novel approach for ocr error correction data generation and created a vast dataset comprising 3.1 million sentences in hindi, 1.58 million sentences in marathi, 2.54 million sentences in bodo, 2.97 million sentences in nepali, 1.95 million sentences in konkani, and 4.07 million sentences in sanskrit. our proposed methodology is versatile and can be extended to other low-resource languages that follow the devanagari script. by leveraging monolingual corpora, our approach enables the generation of ocr correction datasets, thus addressing the scarcity of data in such languages. the findings from our experimentation underscore the efficacy of approaches from machine translation for the task of ocr error output correction, specifically state-of-the-art models like mbart, trained on diverse datasets to substantially enhance ocr accuracy. our research contributes to making textual content more accessible and usable, thereby facilitating broader access to information and knowledge in multilingual so-cieties. our findings also confirm that models trained on a diverse range of fonts perform more robustly than those trained solely on a single font. this observation underscores the importance of font diversity in enhancing ocr error correction models' performance and resilience. our findings motivate the exploration of data augmentation techniques utilizing synthetically generated images as future work. by incorporating these images with controlled variations in font styles, noise levels, and image degradations using a synthetic data generator tool foot_5 for text recognition, we can investigate the impact on model generalization and robustness towards real-world document image complexities. we propose the experimental findings in this work as a baseline, based on which future work can focus on novel and sophisticated techniques for the task of ocr error correction and detection, including improvements to the architecture.