Title: Teaching LLMs to Admit Uncertainty in OCR
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Optical Character Recognition, Visually Degraded Document, Uncertainty, LLM

Abstract: Large language models (LLMs) with visual capabilities are increasingly replacing traditional OCR pipelines, but on visually degraded documents they often hallucinate, producing fluent yet incorrect text without signaling uncertainty. This occurs because current post-training emphasizes accuracy, which encourages models to guess even when uncertain. The problem persists in state-of-the-art systems and severely impacts OCR reliability. To improve the trustworthiness of OCR on degraded documents, we propose uncertainty-aware OCR. Rather than suppressing guesses, our model transcribes while explicitly bracketing spans it deems unreliable with uncertainty tags. To train our model, we use Group Relative Policy Optimization (GRPO). We define the usage rules for uncertainty tags and an evaluation protocol. We introduce a pseudo-labeled cold start and a multi-objective reward that balances transcription accuracy and uncertainty coverage while preventing reward hacking. 
We explore different combinations of cold start and reward granularity and verify the effect of reward parameters in preventing reward hacking and improving the corresponding metrics. We also introduce Blur-OCR, the benchmark for uncertainty-aware OCR on degraded documents. In detailed experiments, our model maintains strong transcription accuracy while achieving uncertainty tag F1 of 0.685, and it substantially outperforms both open- and closed-source baselines.

TL;DR: We propose uncertainty-aware OCR, our model transcribes while explicitly bracketing spans it deems unreliable with uncertainty tags.