Advancing Post- OCR Correction: A Comparative Study of Synthetic Data (2024)

Abstract: This paper explores the application of synthetic data in the post-OCR domain on multiple fronts by conducting experiments to assess the impact of data volume, augmentation, and synthetic data generation methods on model performance. Furthermore, we introduce a novel algorithm that leverages computer vision feature detection algorithms to calculate glyph similarity for constructing post-OCR synthetic data. Through experiments conducted across a variety of languages, including several low-resource ones, we demonstrate that models like ByT5 can significantly reduce Character Error Rates (CER) without the need for manually annotated data, and our proposed synthetic data generation method shows advantages over traditional methods, particularly in low-resource languages.

## introduction digital libraries, like the internet archive, offer a vast collection of historical and culturally important books in image formats, including works written in low-resource and endangered languages. however, their image-only format limits content accessibility, hindering the use of these essential resources. therefore, optical character recognition (ocr) technologies are evidently useful in this context. however, ocr outputs frequently contain errors, particularly when working with texts featuring complex styles, archaic fonts, or unconventional layouts. these errors may include character recognition mistakes, formatting issues, and hyphenation problems, which are particularly prominent when dealing with low-resource languages @xcite . poor quality ocr can reduce the usefulness of these digital texts, adversely affecting downstream tasks @xcite . post-ocr correction is crucial for multiple reasons, including cultural heritage preservation @xcite , expanding the accessibility of knowledge and information @xcite , and supporting further downstream tasks in cultural analytics @xcite . additionally, accurate historical text data is extremely important for training large language models (llms) @xcite , which require high-quality data to improve their ability to handle complex queries, especially those involving history and culture. crowdsourcing has been the primary approach to acquire post-ocr training data in this domain @xcite @xcite . while this can provide highly accurate training data, the process is often time consuming and expensive. with the advent of transformer architecture and attention mechanisms @xcite , deep learning models have emerged as the standard approach for post-ocr tasks. these models require large amounts of data for training. thus, synthetic data has been increasingly adopted in this context. @xcite @xcite . however, most existing literature on generating synthetic data relies on additional existing data for generation, and no comprehensive comparison has been made between different methods to understand how the synthetic data generated in various ways affects post-ocr performance. to address these issues, this paper explores the impact of data volume and data augmentation methods on the performance of post-ocr models. we examine several common methods for creating synthetic data in the post-ocr domain and propose a novel method based on feature detection algorithms from computer vision to calculate glyph similarity for synthetic data construction. we conduct experiments on eight languages, including several low-resource languages, achieving significant cer reductions ranging from 12.41% to 48.18%. ## related work popular ocr systems include the google vision api ocr system @xcite and the tesseract ocr engine @xcite . @xcite performed statistical analysis on the types of errors commonly produced by ocr systems. post-ocr correction, while often overlooked, is an important nlp task. lexical approaches to postcorrection concentrate on character and word level inaccuracies, primarily employing dictionaries and heuristic rules. bassil and alwani (2012) exploited google's search suggestions for context-based corrections, circumventing the need for exhaustive dictionaries. strategies specific to certain domains, such as those proposed by @xcite , @xcite , and kettunen (2016), highlight the necessity of tailored dictionaries for texts that possess unique features, like historical typefaces. @xcite focused on aligning and merging outputs from various scans, including those from different editions, to rectify errors. recent studies have framed post-ocr tasks as seq2seq tasks, with researchers applying both statistical machine translation (smt) and neural machine translation (nmt) models @xcite . @xcite and @xcite employed bert @xcite and bart @xcite ) models, respectively. @xcite conducted a comparison between pre-trained models and traditional seq2seq models, with their findings indicating that pre-trained models like byt5 @xcite outperform the conventional models. @xcite segmented documents into character n-grams, before aggregating their corrections into the final output via majority voting @xcite , essentially acting as an ensemble of sequence models. data is fundamental to the success of deep learning, as an increasing amount of research is directed towards leveraging data-driven strategies. these strategies aim to significantly improve model performance by optimizing data usage, rather than modifying the underlying model structure @xcite . these efforts often involve generating large quantities of synthetic data @xcite , involving data manipulation measures like filtering @xcite , data augmentation @xcite , and noise injection @xcite . such synthetic data has been widely used in various nlp tasks, such as grammatical error correction @xcite , language identification @xcite , question answering @xcite , and named entity recognition @xcite . for the task of text denoising, the primary method for constructing synthetic data is noise injection @xcite . this technique involves inserting errors into clean text to generate training data pairs. d'hondt et al. (2017) added artificial ocr errors into sentences using a random process, while @xcite focused on the lowresource icelandic language for post-ocr tasks. the authors extracted ocr errors from real digitised documents and inserting them into clean text in similar proportions. @xcite analyzed real data to create replacement sets for each word. davydkin et al. (2023) adopted a different approach, developing a system to generate handwritten image data which were then processed with ocr. the resulting ocr outputs and the original texts were subsequently used to train a t5 model @xcite for correction purposes. @xcite synthesized text image datasets by manipulating parameters, like font spacing and image saturation, then compare the original text with the ocr output text, to evaluate ocr systems' performance on various low-resource languages, and they enhanced machine translation (mt) through backtranslation @xcite . some studies have explored improving post-ocr text correction by analyzing the visual forms of characters, known as glyphs. @xcite attempting to use the charbert model @xcite for post-ocr. their method consists of two parallel cnn encoders and a transformer decoder, taking charbert and glyph embedding as inputs. amrhein and clematide (2018 )'s experiment included nmt models and glyph embedding, but it did not enhance model performance. other research has focused on the detection of homoglyphs, with @xcite employing a grid method to assess the similarity of glyphs by counting the number of overlapping grids between characters. the similarity of glyphs is actually based on visual features. in the field of computer vision, feature detection and matching algorithms, such as sift @xcite , orb @xcite , and akaze (alcantarilla and solutions, 2011) can extract feature points from an image and match them with those appearing in other images. ## methods we now describe three common methods for generating synthetic data in the post-ocr domain, before introducing a new method based on glyph similarity in section 3.4. each method makes use of a clean corpus a. as a common preprocessing step, we segment a into multiple chunks by initially dividing sentences based on punctuation marks and then concatenating these sentences, ensuring that the total length of each chunk does not exceed a fixed limit of 230 characters, with the exception of russian and telugu, where the limits are 140 and 90 characters, respectively. ## datasets in this paper, we undertake post-ocr experiments involving english, frisian, german, icelandic, irish, russian, spanish, and telugu texts. clean corpora a for these languages were obtained as follows: data for icelandic, irish, and frisian were sourced from the cc-100 corpus @xcite . telugu foot_1 and russian foot_2 datasets were obtained from kaggle, while english, spanish, and german were sourced from project gutenberg. summary statistics for the corpora are given in ## models in our experiments, we compare the performance of various models which have been previously adopted for post-ocr tasks, including models that operate at the subword-level, character-level, and byte-level tokenizers. additionally, we evaluate current sota models which is based on n-gram and majority voting. this comparison encompasses both pre-trained models and those trained from scratch. training parameters are in appendix a. ## experiments the primary objective of our empirical analysis in this paper is to explore the key factors affecting the performance of models in post-ocr tasks, aiming to identify the most appropriate settings for such tasks. we conduct three different experiments: ## conclusion our experiments demonstrated that augmenting data by replicating clean text and constructing synthetic data based on glyph similarity significantly improves model performance for post-ocr correction. furthermore, we proposed a novel approach to constructing post-ocr synthetic data based on glyph similarity, which outperforms traditional noise injection methods, especially in scenarios with limited data volume, as it does not require additional external data. in our experiments, pretrained models demonstrated superior performance compared to those without pre-training, with the byte-level byt5 model achieving the best performance. for rich-resource languages, this approach can reduce the cer by approximately 45%, while for low-resource languages the reduction varies between 12.41% and 31.07%.