Historical Ink: 19th Century L atin A merican S panish Newspaper Corpus with LLM OCR Correction (2024)

Abstract: This paper presents two significant contributions: First, it introduces a novel dataset of 19th-century Latin American newspaper texts, addressing a critical gap in specialized corpora for historical and linguistic analysis in this region. Second, it develops a flexible framework that utilizes a Large Language Model for OCR error correction and linguistic surface form detection in digitized corpora. This semi-automated framework is adaptable to various contexts and datasets and is applied to the newly created dataset.

## introduction the computational processing of historical newspaper texts is crucial due to the valuable information these texts contain about political, economic, and cultural history. over the past three decades, digital humanities has driven extensive digitization efforts, resulting in numerous curated digital collections @xcite . however, converting these images into machine-readable texts remains challenging, particularly in achieving accurate transcription. a primary challenge is the accuracy of ocr technology, especially with the extremely diverse newspaper layouts, materially degraded documents, and nonstandardized fonts typical of historical texts. traditional ocr methods often produce errors that complicate subsequent analysis. to address these challenges, we employed gpt-4o-mini (openai, 2024), a large language model (llm), within a pipeline for ocr error correction. while the llm is capable of fixing ocr-related errors that traditional systems often miss @xcite , our pipeline also detects and classifies potential hallucinations to avoid further issues and streamline the process. additionally, it contributes by identifying surface forms-specific word occurrences-within the dataset. ## sourcing the dataset was initially compiled from colombian digital newspaper archives. the primary focus was on publications that included cartoons or illustrations, which were intended for subsequent multimodal modeling. this review also extended to the physical collections on-site, as only approximately 50% of the physical collection had been digitized. through this process, 64 newspaper titles were identified, representing 7% of the total 1,655 publications in the collections. this first iteration resulted in a dataset consisting of 4,032 pages of scanned pages of newspapers, primarily from nueva granada-a former country encompassing colombia, panama, venezuela, and ecuador-. a second iteration completed the revision of 3,038 digitized newspapers of 58 digital collections across mexico, argentina, colombia, peru, chile, panama, venezuela, uruguay, bolivia, cuba, and ecuador as shown in table c1 . some countries, such as bolivia, cuba, and venezuela have very limited or no web collections, resulting in their underrepresentation or absence from the final dataset. additionally, some newspapers were printed in europe due to lower costs; in some cases, printing outsourcing was utilized. the final dataset comprises 197 newspaper titles and 23,522 pages of scanned images, primarily from mexico city (mexico is the only country that has digitized its entire collection), but also includes publications from other latin american cities, such as buenos aires, lima, bogota, and santiago de chile. an example of a newspaper image can be observed in figure 1 . originally, the latin american 19-century newspaper dataset consists of scanned images. these images were processed using a layout model, followed by an ocr service. the layout model was specifically trained using data from annotated newspapers available in roboflow ocr (2022); alpha (2023); rscoe (2023); grabadosxix (2023). these datasets were merged into a single dataset (cd) consisting of 1368 images of newspapers annotated for binary layout classification: images and texts. the cd dataset includes 10% of images from our newspapers dataset, labeled by hand, and it was enriched with data augmentation for shear and rotation. these techniques help to increase the model's performance in images with scanning errors. the cd dataset was used to train an image recognition model from azure cognitive services foot_1 , which can extract the images in the newspaper page and extract the text through the ocr. the model's performance scored map@75 of 87.0%, resulting in a collection of annotations and coordinates for both text and images. these coordinates were used to crop the original image, and then process it with the ocr model. once the ocr results were obtained, we merged the processed text with the images, creating a dataset that contains the newspaper images and their associated text. from a sample of 2,500 transcribed texts, each containing 1,000 characters, manual supervision revealed that 8.5% were unreadable. the remaining texts contained multiple transcription errors, primarily due to the artisanal printing techniques and the grammatical and lexical variations of the era. these errors significantly impacted readability, introducing bias when using the texts as input for nlp-llm models. ## processing the dataset includes samples of newspapers that were either handwritten or produced using early carving machines. over time, these machines would wear out, leading to text features that were easily confused with backward accent marks, unwanted punctuations, or misplaced characters between words. such misreadings disrupted the continuity of the text without adding any semantic meaning. detecting these errors automatically poses a challenge due to the linguistic shifts between modern and 19th-century spanish. ocr models trained on such historical texts are lacking, especially considering the semantic and orthographic changes over time. for instance, what might appear as an ocr error could instead be a historical surface form of a word; for example, the conjunction "y" (and) was often written as "i". additionally, some texts were completely unintelligible for ocr, and challenging for humans to interpret, due to the fonts used in certain newspapers. the varied layouts of these newspapers also resulted in texts filled with scores or numbers, or in some cases, samples containing only chapter titles or numbering (e.g., "iii iv v"), which added noise to the dataset. a general overview of the pipeline from the source until the final post-processed, is observed in figure 2 . ## results following the outlined steps, we produced the latamxix dataset, as shown in old spanish surface forms are particularly useful for semantic change detection, capturing meaning variations of specific words and aiding comparisons of their historical evolution across different periods and spanish-speaking regions. in terms of llm post-ocr corrections, the system generated 830,951 corrections. however, a no moreover, due to azure openai's api content policy for the chosen llm (gpt-4o-mini), 2,899 rows (4.52%) were excluded from processing because they contained content flagged as harmful, violent, or sexual. this limitation underscores the challenges content moderation policies pose when applying llms to historical texts. the percentage of flagged content provides insight into the prevalence of such material in 19th-century spanish, offering valuable perspectives for comparative analysis with modern spanish foot_2 . ## future work while the ocr correction using llms has progressed towards a more automated pipeline, a substantial portion of rule definition within the presented framework still requires manual professional input. to advance this process, future work should aim to enhance the automation of the ruledefining procedures. by reducing the reliance on human expertise, we can improve both the efficiency and accuracy of the ocr correction framework. ## limitations a significant limitation of this work is the reliance on manual evaluations for assessing ocr accuracy, as most evaluations and rule definitions were performed by experts. this manual process introduces subjectivity and limits scalability. the absence of a comprehensive automated evaluation method prevents more consistent accuracy assessments and restricts the ability to refine the framework based on objective metrics like character error rate (cer). elizabeth soper, stanley fujimoto, and yen-yun yu. 2021a. bart for post-correction of ocr newspaper text. in proceedings of the seventh workshop on noisy user-generated text (w-nut 2021), pages 284-290, online. association for computational linguistics. elizabeth soper, stanley fujimoto, and yen-yun yu. 2021b. bart for post-correction of ocr newspaper text. in proceedings of the seventh workshop on noisy user-generated text (w-nut 2021), pages 284-290, online. association for computational linguistics. alan thomas, robert gaizauskas, and haiping lu. 2024. leveraging llms for post-ocr correction of historical newspapers. in proceedings of the third workshop on language technologies for historical and ancient languages (lt4hala) @ lrec-coling-2024, pages 116-121, torino, italia. elra and iccl. m.e.b. veninga. 2024. llms for ocr post-correction.