Neural OCR Post-Hoc Correction of Historical Corpora (2021)

Abstract: Optical character recognition (OCR) is crucial for a deeper access to historical collections. OCR needs to account for orthographic variations, typefaces, or language evolution (i.e., new letters, word spellings), as the main source of character, word, or word segmentation transcription errors. For digital corpora of historical prints, the errors are further exacerbated due to low scan quality and lack of language standardization. For the task of OCR post-hoc correction, we propose a neural approach based on a combination of recurrent (RNN) and deep convolutional network (ConvNet) to correct OCR transcription errors. At character level we flexibly capture errors, and decode the corrected output based on a novel attention mechanism. Accounting for the input and output similarity, we propose a new loss function that rewards the model’s correcting behavior. Evaluation on a historical book corpus in German language shows that our models are robust in capturing diverse OCR transcription errors and reduce the word error rate of 32.3% by more than 89%.

## introduction ocr is at the forefront of digitization projects for cultural heritage preservation. the main task is to identify characters from their visual form into their textual representation. scan quality, book layout, visual character similarity are some of the factors that impact the output quality of ocr systems. this problem is severe for historical corpora, which is the case in this work. we deal with historical books in german language from the 16th-18th century, where characters are added or removed (e.g., long s -), word spellings change (e.g., @xcite that often lead to word and character transcription errors. figure 1 shows examples pages conveying the complexity of this task. there are several strategies to correct ocr transcription errors. post-hoc correction is the most common setup @xcite . the input is an ocr transcribed text, and the output is its corrected version according to the error-free ground-truth transcription. for instance, @xcite use a multi-input attention to leverage redundancy among textual snippets for correction. alternatively, domain specific ocr engines can be trained @xcite , by using manually aligned line image segments and line text @xcite . however, manually acquiring such ground-truth is highly expensive, and furthermore, typically, historical corpora do not contain redundant information. moreover, each book has its own characteristics-typeface styles, regional and publisher's use of language, and so forth. in this work, we propose a post-hoc approach to correct ocr transcription errors, and apply it to a historical collection of books in german language. as input we have only the ocr transcription of book from their scans, for which we output the corrected text, that we assess with respect to the ground-truth transcription carried out by human annotators without any spelling change, language normalization, or any other form of interpretation. by considering only the textual modality for our approach, we provide greater flexibility of applying our approach to historical collections where the image scans are not available. however, note that since orthography was not standardized, there can be parallel spellings of the ''same @xcite within the same book, which may pose challenges for approaches that use the text modality only. our approach, cr, consists of an encoderdecoder architecture. it encodes the erroneous input text at character level, and outputs the corrected text during the decoding phase. representation at character level is necessary given that ocr transcription errors at the most basic level are at character level. the input is encoded through a combination of rnn and deep convnet (lecun et al., 1995) networks. our encoder architecture allows us to flexibly encode the erroneous input for post-hoc correction. rnns capture the global input context, whereas convnets construct local sub-word and word compound structures. during decoding the errors are corrected through an rnn decoder, which at each step through an attention mechanism combines the rnn and convnet representations and outputs the corrected text. finally, since the input and output snippets are highly similar, loss functions like cross-entropy lean heavily towards rewarding copying behavior. we propose a custom loss function that rewards the model's ability to correct transcription errors. in this work, we make the following contributions: rule based correction. rule based approaches compute the edit cost between two text snippets based on weighted finite state machines (wfsm) @xcite @xcite @xcite . wfsm require predefined rules (insertion, deletion, etc., of characters) and a lexicon, which is used to assess the transformations. the rewrite rules require the mapping to be done at the word and character level @xcite . this process is expensive and prohibits learning rules at scale. furthermore, lexicons are severely affected by out-of-vocabulary (oov) problems, especially for historical corpora. a similar strategy is followed by @xcite , who uses a spell checker to detect ocr errors and generate correction candidates by computing the edit distance. ocr transcription errors are highly contextual and there are no one-to-one mappings of misrecognized characters that can be addressed by rules (cf. figure 6 ). ## data collection & ground-truth in this section, we describe our data collection efforts and the ground-truth construction process. currently, there is no large-scale historical corpus in german language that can be used for post-hoc correction of ocr transcribed texts. the collected corpus and constructed ground-truth of more than 854k pairs of ocr transcribed textual snippets and their corresponding manual transcriptions, together with the source code are available. foot_1 ## data analysis based on a manual analysis of a random sample of 100 snippet pairs taken from each book from our ground-truth, we analyze the various ocr transcription error types. this is a crucial step towards developing post-hoc correction models in a systematic manner. ocr errors are highly contextual and are dependent on several factors, and as such there are no one-to-one rules that can be used to correct ocr errors. furthermore, these errors are increased when dealing with historical corpora, as fontfaces, book layouts, and language use are highly unstandardized. we differentiate between the following errors: (i) over-segmentation is an error when multiple words are merged into one, (ii) undersegmentation when a word is split into two, and (iii) word error, typically caused by misrecognized characters, converting it into an invalid word or changing its meaning to a different valid word. ## neural ocr post-hoc correction figure 7 shows an overview of our encoderdecoder architecture for post-hoc ocr correction. at its core, the encoder combines rnn and deep convnets for representation of the erroneous ocr transcribed snippets at the character level. during the decoder phase an rnn model corrects the errors one character at a time by using an attention mechanism that combines the encoder representations, a process repeated until an end of a sentence is encountered. ## experimental setup in this section, we introduce the experimental setup and the competing methods for the task of post-hoc ocr correction. ## evaluation in this section, we provide a detailed evaluation discussion and discuss limitations. ## conclusion in this work we assessed several approaches towards post-hoc correction. we find out that ocr transcription errors are contextual, and a large set are due word-segmentation, followed by word-errors. models like transformers have limited utility in this task, as pre-training is difficult to undertake, given the scarcity of historical corpora. we proposed a ocr post-hoc correction approach for historical corpora, which provides flexible means to capturing various ocr transcription errors that are subject to language evolution, typeface and book layout issues. through our approach cr we achieve great wer reduction rates with 82% and 89% for eval-1 and eval-2 scenarios, respectively. furthermore, ablation studies show that all the introduced components in cr yield consistent improvement over the competitors. apart from post-hoc correction performance at snippet level, cr proved to be robust at page level too, where the undertaken correction steps are highly accurate. finally, we construct a release a new dataset for post-hoc correction of historical corpora in german language, consisting of more than 850k parallel textual snippets, which can help facilitate research for historical and low-resource corpora.