Post- OCR Correction of Historical G erman Periodicals using LLM s (2025)

Abstract: Optical Character Recognition (OCR) is critical for accurate access to historical corpora, providing a foundation for processing pipelines and the reliable interpretation of historical texts. Despite advances, the quality of OCR in historical documents remains limited, often requiring post-OCR correction to address residual errors. Building on recent progress with instruction-tuned Llama 2 models applied to English historical newspapers, we examine the potential of German Llama 2 and Mistral models for post-OCR correction of German medical historical periodicals. We perform instruction tuning using two configurations of training data, augmenting our small annotated dataset with two German datasets from the same time period. The results demonstrate that German Mistral enhances the raw OCR output, achieving a lower average word error rate (WER). However, the average character error rate (CER) either decreases or remains unchanged across all models considered. We perform an analysis of performance within the error groups and provide an interpretation of the results.

## introduction the effectiveness of transcription methods, such as optical character recognition (ocr), in processing historical documents critically influences the accuracy of search and analysis in text processing pipelines @xcite . despite advances in ocr technology, library and archive collection transcriptions often contain significant errors and noise due to factors such as scan quality, language, layout complexity, and character similarity. inaccuracies in ocr transcriptions can propagate through multistep historical text processing pipelines, hinder performance on downstream natural language processing (nlp) tasks, and create a risk of distorted interpretations @xcite . post-ocr correction plays an important role in mitigating these errors and improving transcription quality. we focus on the post-correction of a dataset from an ongoing project foot_1 on the modern history of medicine, which explores ten european patient organizations. in this paper, we consider the periodical of the german diabetes association "der @xcite . the materials predate the german spelling and punctuation reform of 1996, when new rules were implemented regarding the double s (ß), consonants, capitalization, hyphenation, and loanwords, making the dataset different from modern texts. the quality of raw ocr output varies significantly, with simpler layouts achieving higher accuracy, while complex multicolumn layouts containing advertisements and rare fonts often result in numerous errors. in this paper, we address the following research questions: this paper unfolds as follows. section 2 describes prior research. in section 3, we present our annotated dataset alongside the augmentation datasets. section 4 lays out the experimental setup. finally, section 5 discusses our findings and section 6 concludes the paper. ## related work post-ocr correction of historical documents has become a central theme at the international conference on document analysis and recognition (icdar). the conference hosted two competitions in 2017 and 2019 dedicated to post-ocr correction, introducing two key tasks: error detection and error correction. sequence-to-sequence neural machine translation emerged as the dominant methodology among the most successful approaches showcased at this conference @xcite . the authors of the competition emphasize that historical newspapers and periodicals continue to pose a substantial challenge to ocr systems, mainly due to their intricate layouts and typographic diversity @xcite . following the conclusion of these competitions, the benchmarks were further utilized to advance the state of the art in post-ocr correction of newspapers with pre-trained models, specifically by finetuning bart @xcite . @xcite are the first to explore the instruction tuning of generative models for post-ocr correction of an english dataset of 19th century newspapers. llama 2 models @xcite are reported to considerably outperform bart. the authors emphasize the adaptability of models like llama 2 to downstream tasks with limited instruction-tuning data @xcite in contrast to machine translation models like bart that typically depend on large volumes of parallel data for optimal performance @xcite . this paper addresses a real-world scenario involving a very limited annotated dataset of german historical medical periodicals, characterized by varying quality in the initial ocr. the dataset includes layouts and fonts that are easily recognized by models, as well as more complex layouts with distorted reading order, images, and advertisements featuring rare fonts and skewed text. given the small size of this dataset, which precludes instruction tuning, we augment it with a german dataset from the icdar 2019 competition, which includes a similar time period and source -newspapers. additionally, we explore augmentation using another icdar 2019 dataset, which represents a different source -cultural and religious materials from the german national library. this study does not explore augmentation with synthetic data. while artificially inserted errors can enhance model performance, they may fail to capture the complexity and diversity of real-world ocr errors, limiting the models' generalization ability @xcite . this is particularly relevant for our dataset, where typical error insertion is insufficient due to the intricate challenges posed by complex layouts, such as those with advertisements. we leave the exploration of error generation approaches for our specific context to future work. our experiments contribute to post-ocr correction for german historical documents by comparing the performance of a finetuned german bart model with instruction-tuned german generative models, such as llama 2 13b and mistral 7b @xcite . beyond evaluating average performance metrics, we focus on error categories to better understand how the models handle specific types of errors and whether they degrade the quality in areas where ocr is already accurate. ## experimental setup in this study, we evaluate german bart and generative models, llama 2 13b and mistral 7b, comparing their wer and cer metrics foot_5 against raw ocr outputs. at the core of the training process lies a base dataset consisting of der diabetiker pages, a small annotated collection, combined with nzz, a comparable newspaper source. to evaluate the impact of dataset composition, we train the models with and without augmentation using gnl, which adds greater diversity to the data. the training data is structured in two configurations: one that retains misaligned spans and another that excludes them. to deepen our understanding of models' performance, we analyze their handling of diverse ocr errors across three distinct error categories. our primary focus is the correction of errors in the der diabetiker test data. the approach identified as successful will be further refined and expanded for application in post-ocr correction of the entire german segment of our project's dataset of patient organizations' periodicals. ## conclusion this paper compares the performance of large language models, specifically bart as an encoderdecoder, and llama 2 13b and mistral as generative models, for post-ocr correction of the german historical periodical der diabetiker, published by the german diabetes association. we examine the impact of different dataset configurations and the effect of dataset augmentation with data from a distant source. the results suggest that bart detects fewer errors compared to the generative models. however, since bart does not correct these errors, it also avoids introducing larger changes -an issue that we observe in the generative models. also, bart tends to correct more spans that were already accurate in the first place, leading to unnecessary modifications. this behavior aligns with the average cer and wer scores in among the evaluated models, mistral 7b stands out as the most promising in terms of performance on historical data from patient organization periodicals. it achieves a significant 22% improvement in average wer and retains the highest proportion of correctly ocr-ed paragraphs compared to other models. despite these strengths, mistral maintains the average cer without improvement, and further investigation is needed to understand how it handles major errors. specifically, more research is required to manage the model's creativity in generating corrections, ensuring that it