A Two-Step Approach for Automatic OCR Post-Correction (2020)

Abstract: The quality of Optical Character Recognition (OCR) is a key factor in the digitisation of historical documents. OCR errors are a major obstacle for downstream tasks and have hindered advances in the usage of the digitised documents. In this paper we present a two-step approach to automatic OCR post-correction. The first component is responsible for detecting erroneous sequences in a set of OCRed texts, while the second is designed for correcting OCR errors in them. We show that applying the preceding detection model reduces both the character error rate (CER) compared to a simple one-step correction model and the amount of falsely changed correct characters.

## introduction the digitisation of historical documents is a central objective for libraries, archives and museums. using ocr, the digitised documents can be converted to electronic full text. important advances have been achieved in ocr over recent years, mainly through the use of neural networks @xcite . however, in the case of historical documents there still remains a substantial amount of ocr errors that hinders the usage of the digitised documents and requires efficient methods for ocr post-correction. by defining ocr post-correction as a translation problem, current approaches are often based on neural networks @xcite , especially sequence-to-sequence models that are inspired by successes achieved in neural machine translation (nmt) @xcite . while these models perform well for data sets with a high amount of ocr errors, post-correction becomes more difficult when the cer is less severe @xcite . in this paper we present an alternative approach to ocr post-correction. instead of solving the task in one step, i.e. by training a single translation model 1 , we propose a two-step approach that avails itself of two separate models. we chose a long short term memory (lstm) architecture @xcite for the actual sequence-to-sequence translation step (henceforth called translator). in addition, we inserted a model before the translator that functions as a filter by separating erroneous sequences foot_0 from correct ones (henceforth called detector). the benefits of this approach are two-fold: 1) by decreasing the proportion of correct ocred data, the cer of the data fed into the translator is artificially increased. this is assumed to improve the translation results. foot_1 2) by excluding the majority of correct sequences from the translation step, we can avoid to mistakenly insert errors in them. this paper is structured as follows: section 2 briefly summarizes the relevant previous work. section 3 presents an initial ocr post-correction approach using a standard sequence-to-sequence model. it also describes the used data and the applied preprocessing steps. afterwards, section 4 presents our alternative two-step approach to ocr post-correction. both components are analysed independently using different test sets. in section 5 we show first results of the full pipeline applied to one data set. finally, in section 6 we discuss our results and give a short outlook. ## related work traditionally, the problem of ocr post-correction has been approached by the application of statistical language modelling techniques, frequently in combination with lexical resources or n-grams @xcite . string distance metrics are frequently used to generate correction candidates for erroneous tokens, aided by a probabilistic scoring mechanism or noisy channel model @xcite . however, these methods are less successful for historical documents that exhibit large amounts of ocr errors, historical spelling variation and where fitting language resources are scarce @xcite . accordingly, reffle and ringlstetter (2013) compute a two-channel profile for historical language variation and ocr errors to improve the correction rate while minimizing the injection of new errors. this method was further improved by @xcite with the addition of an adaptive feedback mechanism and the treatment of non-interpretable tokens as conjectured errors. an alternative technique is anagram hashing @xcite , but more recent evaluations indicate that this approach requires fine-tuning @xcite . since different ocr engines (or models) also produce different ocr error patterns, another strategy is merging multiple ocr results, supported by heuristics and lexical resources @xcite . ocr post-correction has also been studied as a problem of statistical @xcite or neural machine translation @xcite . an overview of the recent state-of-the-art in ocr post-correction is provided by the icdar competition on post-ocr text correction. the competition comprises of two tasks, error detection and error correction, with evaluation sets for english and french language. the 2017 edition saw weighted finite-state transducers (wfst) obtain best results in the error detection task, with an smt/nmt system achieving the highest reduction of errors @xcite . in the 2019 edition, a system based on bert @xcite came out best for both tasks @xcite . for comparison, we refer to @xcite as our baseline (see section 6). ## the standard sequence-to-sequence approach we first approached the ocr post-correction task by implementing a standard lstm-based sequence-tosequence translation model using the pytorch framework @xcite . the architecture consists of an encoder model responsible for calculating a matrix representation of the ocred input sequence. this is used by the subsequent decoder model for creating a correct output sequence. ## the two-step approach in order to improve the translator model's results, we decided on building a two-step pipeline. this pipeline consists of an initial ocr error detection model and a subsequent error translation (i.e. correction) model. 7 in this architecture, only sequences that the detector considers erroneous would be forwarded to the translator model. this procedure has the added benefit of artificially increasing the proportion of ocr errors which are fed into the translator. it also has the potential of decreasing the amount of erroneously changed correct characters. ## the full ocr post-correction pipeline as both components have been analysed independently in section 4, we now present results yielded by applying the full two-step pipeline on a combination of both test sets (83,600). as the detector's task is the reduction of correct sequences in the data that is to be fed into the translator, we did not remove any correct sequences manually. ## discussion and outlook in this paper, we presented an alternative approach to automatic ocr post-correction using a two-step architecture consisting of a detector and a translator model. while the detector model identifies erroneous sequences in an ocr data set, the translator model is used to correct these sequences. this approach has two major benefits. first, removing correct sequences from the to-be-corrected data set increases the proportion of ocr errors fed into the translator model. this is supposed to improve the translator's results. second, a detector model with a high precision limits the amount of correct sequences processed by the translator, thereby reducing the likelihood of correct characters being erroneously changed. as a successful ocr post-correction model is supposed to leave the correct data unaltered, the importance of the second aspect should not be underestimated. comparisons of both approaches' results show that by inserting the detector model the translator's results can be substantially improved. whereas the simple one-step translator model actually increases the cer from 1.1% to 2.1%, we were able to reduce the error rate to 0.9% by applying the two-step architecture. this is a relative improvement of 18.2%. while this cer improvement is smaller than the score achieved for the english monographs (pre-correction cer: 1.8%, rel. improvement: 31.3%) in @xcite , their improvement achieved for the french monographs (pre-correction cer: 1.6%, rel. improvement: 18.3%) is similar to our result. 9 we are aware that it is difficult to directly compare our results with @xcite given the differences between the two studies (german vs english/french data, differences in the model hyperparameters, notable differences in precorrection cer). however, we consider the partially similar results as a promising indicator that our two-step model can actually compete with previously proposed methods. while the cer reduction is noteworthy in itself, a special focus should be laid on the rates of erroneously changed correct characters. the two-step approach yielded a promising rate of changed correct characters of 0.3%. however, correcting the data set with the one-step model resulted in a high rate of 6%, which indicates that the correct sub-parts of the data have been severely modified. given that we did not use different model parameters for training both sequence-to-sequence translators, we can conclude that this reduction is associated with the insertion of the detector. in other words, our results can be interpreted as evidence for the assumption that a low cer poses a notable problem for a one-step translation model. 10 as the detector component already strongly improves the ocr post-correction results, our future work will primarily focus on the further development of the translator component. on the one hand, we plan to improve the attention mechanism implemented in the translator's decoder model. as ocr errors are characterized by their local context, using a local attention approach @xcite may have benefiting effects on ocr post-correction. furthermore, we will investigate alternatives to the lstm architecture. in particular, we consider the application of generative adversarial networks @xcite as a fruitful path of future research.