Cheap Character Noise for OCR -Robust Multilingual Embeddings (2025)

Abstract: The large amount of text collections digitized by imperfect OCR systems requires semantic search models that perform robustly on noisy input. Such collections are highly heterogeneous, with varying degrees of OCR quality, spelling conventions and other inconsistencies —all phenomena that are underrepresented in the training data of standard embedding models, with ramifications for their generalization. In our paper, we show that this problem can be alleviated with a simple and inexpensive method that does not require supervision or in-domain training. Specifically, we fine-tune existing multilingual models using noisy texts and a contrastive loss. We show that these models show considerable improvements across different noise conditions. Control experiments indicate minimal, and occasionally positive, impact on standard similarity tasks. These findings suggest that embedding models can be inexpensively adapted for cross-lingual semantic search in heterogeneous, digitized corpora. We publicly release our code, datasets, and models at https://github.com/impresso/ocr-robust-multilingual-embeddings.

## introduction optical character recognition (ocr) technology plays a central role in the digitization of historical documents, rendering large volumes of textual data accessible to natural language processing (nlp) tools. to enable effective retrieval from such collections, semantic search based on text embeddings presents an appealing approach @xcite . however, ocr output is inherently imperfect and often introduces errors due to factors such as poor image quality, complex layouts and fonts, low contrast and degradation of the source material @xcite . consequentially, the noisy ocr output can severely degrade the performance of nlp systems by disrupting the syntactic and semantic structure of the text, possibly further exacerbated in multilingual contexts, where models show different performance degradations for different input languages @xcite . a particular problem is that this noise disrupts tokenization processes, resulting in fragmented or incorrect sub-word units @xcite . figure 1 showcases an example of ramifications for text embedding models: a multi-lingual embedding model assigns vastly different similarities to two parallel texts, just depending on their font before undergoing the digitization process. in this work, we empirically analyze how ocrinduced noise affects multilingual embedding models and propose a method based on contrastive finetuning with randomly noised text. our main contributions are: ## background and related work semantic search refers to a class of tasks aimed at retrieving texts that are semantically similar, or relevant to a given query. today's dominant paradigm to address this tasks is embedding texts as vectors with pre-trained encoders fine-tuned on contrastive tasks @xcite . in many tasks, this technique has been proven efficient and effective. however, these tasks typically involve contemporary and more normative text. hence, it is unclear whether the models maintain their performance in noisy and highly heterogeneous text collections, as we would encounter in large digitized collections of text, e.g., within institutions such as libraries. typo robustness. a related line of research investigates the robustness of dense retrieval and embedding models to typos and character-level perturbations. while ocr errors and human typos differ in origin, both introduce noise that disrupts tokenization and embedding similarity. @xcite propose a robust dense retrieval training method that aligns representations of pristine and misspelled queries while enforcing contrast to distinguish unrelated inputs. their approach combines query augmentation with a dual self-teaching loss, significantly improving performance on both synthetic and real-world misspelled queries without sacrificing accuracy on clean inputs. similarly, @xcite examine the vulnerability of dual encoder architectures to misspellings, finding that even minor character-level changes can substantially degrade retrieval performance. they propose contrastive learning-based strategies to mitigate this sensitivity. training models on character-level noise has been shown to be effective for machine translation @xcite @xcite and cross-lingual transfer between closely related languages @xcite . we demonstrate that multilingual embedding models have worse semantic search capabilities within digitized collections due to the presence of ocr errors. we then evaluate versions of further training approaches that aim to bring closer the representations of randomly noised text with its original version and show increased capabilities of semantic search in digitized collections even for languages pairs not included in our noised fine-tuning. ## notation and preliminaries throughout this work, we make use of multilingual parallel datasets that may contain noise of different degree. in particular, the term parallel here does not necessarily only mean that there are pairs with texts from two different languages, but it can also mean, e.g., that the dataset is monolingual but contains pairs of texts where one part contains ocr noise. to denote such a dataset we use dataset @xmath0. in this notation, ns (nt) are variables that denote the type of observed noise, whereas ls (lt) refer to the source (target) language within the dataset (if they differ). we formalize forms of ocr noise within the following categories: the languages that we mention in our paper are german (de), french (fr), english (en), luxembourgish (lb), turkish (tr) and arabic (ar). for example, in a parallel dataset, we consider a case where french source sentences in standard clean text are paired with german text ocred from blackletter font. we denote this dataset as dataset ∅→bl @xmath1. a full description of the generation process for our multilingual noised training data and a separate description of generating the realistic ocr evaluation data follows in §5. ## method overview the core idea of our method is straightforward: we generate a training set by inducing random character-level noise into a given dataset. this enables us to fine-tune a pre-trained embedding model to better align clean and character-noised versions of the same text using a contrastive loss objective. our study focuses on lightweight adaptations of multilingual models, requiring only 20,000 positive pairs and modest computational resources (approximately 20 minutes on a single tesla t4 16gb gpu). because the method is fully unsupervised and language-agnostic, it can be easily applied to adapt multilingual embedding models for improved robustness to ocr-induced noise. ## experimental results we evaluate our approach in two scenarios: (1) @xmath2, representing a typical user-query setting where clean input is used to search over noisy, digitized content; and (2) @xmath3, representing semantic search conducted entirely within ocr-processed corpora. ## discussion generalization issues in off-the-shelf models: is it the tokenizer's fault? our adaptation strategies reduce the performance drop that standard multilingual embedding models exhibit when processing ocr-noisy text. a central question, however, is what causes these models to generalize poorly to such noisy input. we hypothesize that the tokenizer plays a crucial role in this degradation. modern embedding models rely on subword tokenization, and ocrinduced character perturbations often result in token boundary mismatches. these mismatches propagate through the network layers, distorting semantic representations. to investigate this hypothesis, we examine the xlm-roberta tokenizer @xcite used by both m-e5b and m-gte. this tokenizer is built on the sentencepiece unigram algorithm @xcite . we start by analyzing words and short phrases from original noisy text pairs in the clsd evaluation set (see appendix to quantify the effect of the subtokenization changes, we compare the tokenizations of clean and ocred text by measuring their vocabulary overlap. specifically, we compute precision, re- method bl/sd snp histlux base 78.1 81.6 83.8 random noise 82.2 84.0 87.9 bl/sd noise 83.2 86.0 84.3 snp noise 82.8 86.1 82.8 ## conclusions we propose an inexpensive and effective strategy for adapting multilingual embedding models to be more robust to heterogeneous digitized text. our adaptation approach improves performance across different noise conditions and even over historical digitized text in related languages. through control experiments, we show that our method has minimal impact on the overall embedding quality. we find that ocr noise increases token fragmentation, which may explain the observed performance degradation. we believe that our work is an important step towards building and evaluating reliable semantic search systems for large and diverse digitized text collections.