Teaching VLMs to Admit Uncertainty in OCR from Lossy Visual Inputs
Authors: Shuhao Guan
Moule Lin
Cheng Xu
Jinman Zhao
Derek Greene
Venue: ICLR 2026 Poster (2026)
Keywords: Optical Character Recognition
Visually Degraded Document
Uncertainty
LLM
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose uncertainty-aware OCR, our model transcribes while explicitly bracketing spans it deems unreliable with uncertainty tags.

Abstract: Vision-language models (VLMs) are increasingly replacing traditional OCR pipelines. However, they often hallucinate on lossy visual inputs, such as visually degraded document images, producing fluent yet incorrect text without signaling uncertainty. This occurs because current post-training emphasizes accuracy, which encourages models to guess even when uncertain. The problem persists in state-of-the-art systems and severely impacts OCR reliability. To improve the trustworthiness of OCR on degraded documents, we propose uncertainty-aware OCR. Rather than suppressing guesses, our model transcribes while explicitly bracketing spans it deems unreliable with uncertainty tags. To train our model, we use Group Relative Policy Optimization (GRPO). We define usage rules for uncertainty tags and an evaluation protocol, introducing a pseudo-labeled cold start and a multi-objective reward that balances transcription accuracy and uncertainty coverage while preventing reward hacking. We explore different combinations of cold-start and reward granularity. We also assess the effect of reward parameters in preventing reward hacking and improving the corresponding metrics. Furthermore, we introduce Blur-OCR, a challenging benchmark for uncertainty-aware OCR on degraded document images under lossy visual conditions. In extensive experiments, our model maintains transcription accuracy while achieving an uncertainty tag F1 score of 0.685.

Published as a conference paper at ICLR 2026
TEACHING VLMs TO ADMIT UNCERTAINTY IN OCR FROM LOSSY VISUAL INPUTS
Shuhao Guan
Moule Lin Cheng Xu'
Jinman Zhao
Derek Greene'
1 School of Computer Science University College Dublin,
2 Trinity College Dublin and Lero," University of Toronto
ABSTRACT
Vision-language models (VLMs) are increasingly replacing traditional OCR pipelines. However, they often hallucinate on lossy visual inputs, such visually degraded document images, producing fluent yet incorrect text without signal- ing uncertainty. This occurs because current post-training emphasizes accuracy, which encourages models to guess even when uncertain. The problem persists in state-of-the-ee systems and severely impacts OCR reliability. To improve the trustworthiness of OCR on degraded documents, we propose uncertainty aware OCR. Rather than suppressing guesses, our model transcribes while explicitly bracketing spans it deems unreliable with uncertainty tags. To train our model, we use Group Relative Policy Optimization (GRPO). We define usage rules for uncertainty tags and evaluation protocol, introducing pseudo-labeled cold start and a multi-objective reward that balances transcription accuracy and un- certainty coverage while preventing reward hacking. We explore different com- binations of cold-start and reward granularity. We also assess the effect of re- ward parameters in preventing reward hacking and improving the correspond- ing metrics. Furthermore, we introduce Blur-OCR, a challenging benchmark for uncertainty-aware OCR degraded document images under lossy visual condi- tions. In extensive experiments, our model maintains transcription accuracy while achieving an uncertainty tag Fl score of 0.685. Data and code are available com htt.a///....oo/lllooaaaa/ttttaat
INTRODUCTION
Transformer-based vision-language models (VLMs) are now central to modern OCR systems (Vaswani et al., 2017 Blecher et al. 2023 Wei et al. 2024 Feng et al,, 2025). However, these models frequently hallucinate when faced with lossy visual imputs, such as degraded historical doc- uments, generating plausible text that is unsupported by the visual evidence and does not indicate uncertainty (Kalai al., 2025 Kaltchenko, 2025 including in unreadable regions. Such inputs arise from blur, low resolution, compression artifacts, occlusion, or severe degradation, where vi- sual evidence is insufficient for faithful transcription (Guan & Greene 2024). In contrast, classical OCR models tend to emit garbled or ill-formed strings in unreadable areas, which are easier to lo- calize and thus less likely to be incorrectly used widely propagated (Jatowt al. 2019). VLM OCR errors, by comparison, often appear plausible (Yang et al., 2024), making error localization and correction difficult without external knowledge, which in turn increases the risk of error propagation and can severely degrade downstream analytics, model training, and cultural-heritae digitization (Hamdi et al., 2022; Zhang et af.) 2024, Oeischlager] 2024 Kang et al. (0066). Despite these issues, VLMs substantially outperform traditional OCR models many benchmarks (Liu et al. 2024 Wei ctt al., 2024; Guan et al. 2024), motivating the need for Transformer based OCR methods that can identify and explicitly mark localized uncertainty while maintaining strong transcription accuracy.
In this paper, we present an uncertainty-awar OCR paradigm in which the model transcribes text while also enclosing spans it deems uncertain potentially erroneous within paired delimiters: <C> ... </C> uncertainty (UNC) tags. This approach does not suppress guesses but instead makes them transparent by marking self-identified uncertainty. We formalize tag semantics and member-

---

Published as a conference paper at ICLR 2026
ship rules at both character and word granularities, together with an evaluation protocol reporting precision/recal//FI for uncertainty tagging alongside transcription standard accuracy metrics.
To train model with this behavior, we employ pscudo-labeled cold start followed by Group Relative Policy Optimization (GRPO) (Shao et al., 2024). together with a multi-objective reward that balances transcription accuracy and uncertainty coverage while preventing reward hacking. We study tag granularity at both character and word levels, uncovering a counterintuitive yet effective configuration: a character-level cold start paired with word-level reward yields the best calibration. We also introduce Blur-OCR, a challenging OCR benchmark of document images with highly di- verse synthetic degradations, which includes 107,520 training images and 2,048 evaluation images, and is designed to test our uncertainty-awant OCR on degraded documents.
This paper is structured as follows. Dectionn3cfine the usage and membership rules for UNC tags. S introduces evaluation metrics for uncertainty-awan OCR. Section presents our pseudo- labeled cold-start procedure and the multi-objective reward that prevents reward hacking. Section 6 describes the new Blur-OCR benchmark dataset, and Section 7 periments. We demonstrate the importance of appropriate cold-start design for final performance, verify the benefits of GRPO, and analyze the impact of reward parameters. We also evaluate mul- tiple open- and closed-source models on Blur-OCR and compare several uncertainty-estimatie ap- proaches. Our model shows strong transcription accuracy while achieving uncertainty- precision of 0.839 and recall of 0.620, substantially outperforming all baselines.
reports results from extensive ex-
RELATED WORK
Early neural OCR methods commonly followed a two-stage "detect - recognize" paradigm. For instance, CTPN (Tian et al. (2166) and EAST (Zhou et al., 2017) proposed text regions, while DB- Net (Liao et al., 2020) introduced differentiable binarization to improve recall on irregular shapes. For recognition, CRNN popularized CTC-based sequence modeling (Shi et al. 2016), and later attention/seq2seq systems such as SAR (Li et al. 2019) and ASTER (Shi et al, 2018) improved robustness to perspective and curved text via 2D attention and rectification modules. Benchmarks like ICDAR-MLT (Nayef et al. 2019) tracked rapid progress in multilingual settings. Transformer- based, end-to-end recognizers then displaced RNNs: TrOCR (Li et al., (023) framed OCR as ViT-encodeer/decoder-- LM sequence-ter se-uueeece problem; Donut (Kim et al. (022) took an "OCR- free" path that directly generated structured outputs from page images; and Nougat (Blecher al.) 2023) specialized to scientific documents, producing LaTeX-likee markup. In parallel, document- aware large language models (L..Ms) such as Pix2Struct (Lee et al., 2023) and PaI.I (Chen et al., 2022) have increasingly integrated OCR capabilities, though community evaluations (OCRBench
OCRBench v2) show persistent challenges on noisy, degraded, or fine- mne-grained text (Liu et al. 2024
Fu et al., 2024. Yu et al. 2025). "OCR-2.0" efforts pursue unified, end-to-end systems: GOT Wei et al. 2024) unifies text, formulas, tables, and other character- signals within a single model and supports interactive region/mullipage use: Dolphin (Feng al.) (2225) and MinerU2.5 (Niu al.
2025) follow an analyze-then-parse paradigm. Meanwhile, general LLMs such as ChatGPT and Gemini, continue to advance on OCR (Hurst et al. 2024 Team et al. 2024 Yang et al.) 2024).
Classical OCR methods exposed character/word confidences that enabled triage, human-in-the-loop review, and downstream re- re-weighting (Mor & Wolf 2018). Empirical studies have shown that these scores correlate with errors and, after per-engine calibration, can prioritize low-quality pages for re-OCR (Cuper et al. 2023). In scene text recognition, some work calibrated predictions at the word/sequence level (Slossberg et al., 2022), while other work treated confidence- based error detection as text classification by feeding classical OCR outputs (with engine confidences) into a BERT detector (Hemmer et al. 2024). For large models, uncertainty signals have been investigated via entropy or output disagreement (Kaltchenko) 2025 Ajayi et al. 2025 Zhang et al. (002)). /Zhan
al. (2025) use OCR output disagreement to detect and correct OCR errors. He et al. (2025) targeted to
Visual Question Answering from degraded documents with GRPO and refusal/sell
limit hallucinations in answers. There is also increasing interest in uncertainty for VLMs (Kostumov
et al. 2024| Fang et al. 2025 Yang et al. 2025 Lin et al. 2025).
In LLM post-training, RL has enabled optimization of sequence-level, non-differentiable objec- tives beyond token likelihood. Preference-based RL (RLHF/RLAIF) often uses PPO (Schulman et al. 2017), while preference matching without leamed critic (e.g., DPO) is alternative option
2

---

Published as a conference paper at ICLR 2026
RRaaillvv et al., 2023). In our work, adopt GRPO (Shao et al. 2024 which replaces value model with group-relative advantages, making it convenient to optimize a composite reward that combines transcription accuracy with cocerra terms. This fits naturally into the Rein- forcement Learning with Verifiable Rewards (RIVR) paradigm (Lambert et al. 2024), since both components of the reward can be computed from the ground-truth transcripts.
3 UNCERTAINTY TAGGING
In this paper, span uncertain if the model judges that transcription may not match the ground- truth text given the image evidence. A UNC tag is defined as the paired delimiter <C> </C>>>>>>>>////// which the model emits to enclose spans it considers uncertain or possibly erroneous. At inference time, the model expresses this belief by bracketing such spans with paired UNC tags. To avoid ambiguity in evaluation and for the purpose of describing a coverage-baser reward later Section 5.2 we first formalize the semanties of UNC tags and the rules for what counts inside outside. We evaluate at two granularities L E {char, word). In the former, we operate at the character level; in the latter, at the word level with whitespace tokenization.
Tag validity and alignment. Since UNC tags function as brackets and models may emit invalid structures, we first drop all invalid tags (nested, overlapping, or unclosed) and keep only well-formed pairs. We then align the prediction the ground truth (GT) at the chosen granularity determine, for each character word, whether it falls inside or outside UNC.
Character level. In model prediction y. character is explicitly inside UNC if it lies between paired <C> and </C>. A contiguous GT segment that is present GT but missing in ÿ is treated as implicitly inside UNC if, on the alignment path, the gap lies directly adjacent to the boundary of an explicit UNC span (i.e., there are no aligned characters in between). All other positions are outside. Thus, the total number of characters counted as inside UNC the sum of explicitly tagged characters and implicitly attributed missing characters.
Word level. A word is considered explicitly inside UNC if at least one its characters is explicitly inside at the character level, with the span expanded to cover full word boundaries. For GT missing segments, mirror the character-level rule the word granularity. Thus, any word-level gap that is adjacent to an explicit UNC span (allowing intervening whitespace) is implicitly counted as inside.
METRICS
3 we now present evaluation metrics aligned with the two core goals of uncertainty-aware OCR: (i) transcription accuracy, and (ii) correctness of UNC tag usage. We compute both at each granularity L. We report both metrics macro averages.
Building on the definitions in Section
Transcription accuracy. Before calculating transcription accuracy, apply minimal normaliza- tion to remove newline characters, and collapse consecutive spaces into a single space. We then strip all UNC tags and compute standard error rates. Let FDr(yy, ū) be the Levenshtein edit distance level I., and ly|r. l. the length of the reference in units of L. We then define error rate and accuracy as:
ED,
eL
Accuracy, = EL
1/1
UNC precision, recall and Fi. To assess tag usage, we treat "error" as the positive class and cast the inside/outside decision as binary classifier over units at level L. Let Emrlng be erroneous units inside UNC (true positives). Correctin; be correct units inside UNC (false positives), and EroQtt, be erroneous units outside UNC (false negatives). Then Precision PL, recall R,, and Fi, are:
Eriin
Erring
2PLR1 FILL
Pt =
R = EnrOut, Errin;"
CorrectIn; + Errln,,'
Pr. + R
All counts are computed under the UNC membership rule above (explicit spans plus the implicit coverage of adjacent gaps) at the chosen granularity L. By construction, the total number of erro- neous units equals the edit distance at level L, so R, measures the fraction of all errors covered by UNC, while Pú, measures the purity of what UNC encloses.
3

---

Published as a conference paper at ICLR 2026
GRPO age
① inputt
magg
base model
3 cold start SFT
UNC model
A
CS e-da-e-mage
GRPO
predict
Taagiing
GRPO adage
0.8
CS damage
predict
glign
GRPO .p. ca
0.944
Cs image
GRPO .p- ime
Figure 1: Uncertainty- aware OCR pipeline: Cold-start (left) and GRPO (right). ① Run the frozen base model on degraded CS images to obtain transcripts. @ Align each prediction with the ground truth (GT) and insert UNC tags (character-level shown). @ Cold-start SFT on (image, OCR-with tags) to obtain a tag-aware UNC model. @ GRPO on a disjoint RL set: sample multiple completions, compute the composite reward, convert to group-relative advantages A, and update the model.
UNC error rate gap. We also report gap metric that compares the enoor rate inside UNC spans to that outside. Let I L and lylom, be the GT lengths assigned to the inside and outside regions under the UNC membership rule (Sec. 3). We define
Erring
ErrOutz
L
Cout,
Gapz
Ein,
Eout, L.
|y out.
If Iy,.. = 0 or I L 0, we report the corresponding rate and Gap, n Since insertions increase Errin, or Erout, without increasing the corresponding GT length, Lm Cou, and thus Gapz can exceed 1.
5 METHODS
We frame uncertainty-aware OCR as a sequence decision task. As defined in Section 3 UNC tags are paired brackets that can span long text, so their placement a global choice that depends on the entire transcript. Token-level SFT is therefore misaligned: it optimizes local next-token likelihood, whereas our evaluation targets span-level uncertainty quality together with transcription accuracy. Local likelihood can over-penalize globally better transcripts (with near-correct spans) and under- penalize globally worse ones (e.g., invalid or truncated tags). This section presents a two-phase training pipeline (Fig. D. cold start (CS) followed by GRPO-which optimizes multi-objective reward balancing transcription accuracy and uncertainty coverage while mitigating reward hacking.
5.1 PSEUDO-LABELER COLD START
Reinforcement learning alone is insufficient to teach VLM: without prior exposure, the policy rarely explores valid tag structures, convergence slows, and calibration is poor (Stiennon et al., 2020 Ouyang et al. 2022). We therefore initialize training with novel pseudo-labeled cold start.
The cold-start supervision we seek consists of degraded images paired with transcripts that may contain errors, with these errors wrapped in UNC tags. Our goal is not train the model to tag all text arising from visually degraded regions, but rather to tag spans where its own predictions are unreliable. Since VLMs can rely on both visual evidence and textual context, model may confidently infer the correct content from context even in visually unreadable regions. In such cases, we do not want the model to add UNC tag, because the resulting transcription is still correct and useful to downstream users. Concretely, images, align each prediction to the ground-truth transcript to localize substitutions, insertions, and deletions, and then insert UNC tags into the model's transcript around the detected errors.
pretrained base model set of degraded
We prepare labels at two granularities: (i) character level, where tags precisely enclose erroneous characters from substitutions insertions, and (ii) word level, where any word containing at least one error is entirely wrapped in tags. For both granularities, missing characters or words in the prediction have no corresponding text in the transcript and therefore cannot be tagged. To address
4

---

Published as a conference paper at ICLR 2026
this, assign the nearest word to carry the uncertainty and wrap that word with UNC tags. We then perform SFT on base model using pairs of (image, OCR with-tags) for CS. Examples of CS labels provided in Appendix X signals for both accuracy and tagging, which stabilizes training and accelerates convergence.
Starting GRPO from this tag-awary initialization provides informative
We tag the model's output, not the GT. At inference the model only has its own transcript, the GT; we tagged the GT, the model would learn to place tags at GT positions it never sees at test time, and insertions/deletions would break the mapping. Tagging its own output keeps training and inference consistent. Note that SFT on model outputs carries risk of reducing transcription accuracy, so during SFT we also mix in subset of (image, GT) pairs to prevent large drop in accuracy; accuracy can be recovered with following GRPO, we discuss this effect in Section 7.1
5.2
UNCERTAINTY-AWA. OCR REWARD
We seek to reinforce two capabilities established during the cold-start process: (i) accurate tran- scription and (ii) tagging uncertain potentially erroneous spans with UNC. We do not penalize tag-format mistakes in the reward, because the policy may emit tags multiple times and malformed tags are common; penalizing them would discourage tag usage. Instead, we ignore invalid tags during evaluation and reward computation, as we discussed in Section Therefore, we combine accuracy and tagging quality into single reward function:
R1 v) = max 0, (1 - eL) + AneLFoL
Transcription term. This is simply (1-ez). Since e, can exceed on difficult examples, (1-eL) may become negative; the outer (0, clamps the total reward to be nonnegative for stability.
Tagging term. The tagging score is Fa.r. the parameter / controls the trade-off: 3 biases precision, 3 > biases recall which is equivalent to the B-weighted generalization of the Fj measure
(2+ PL R
F8.L
33Pr + R
Length-mismatch damping 7)- When the hypothesis length diverges substantially from the ref- erence, a tag placed near boundary can, under our implicit-coverage rule, "cover" large adjacent gaps and thus inflate the tagging score despite poor transcription. To neutralize this failure mode, we downweight the tagging term only under extreme length mismatch. Specifically, we define
max (lu1., 1lr.)
= 2-1ppr]]]
PL = min (0l lvlE.)
where [[-] is the indicator (true 1, false 0). Thus 7 2/2 only when PL T (extreme length mismatch), and = otherwise; we set 7 = 1.3 in all experiments. This damping is necessary because, under extreme mismatch, boundary UNC span |ilı < lv| or a large surplus when jūا > |ylz- In both cases many tokens are counted as "correctly tagged" despite poor transcription, which inflates the tagging term. The factor 7) halves the tagging contribution only when PL > preventing reward hacking via truncation or over- generation. We vary 77 in Exp.
cover large adjacent gap when
No incentive to manufacture errors. For notational convenience, we omit below; the state- ments hold at both granularities. Ignoring the outer max(-, 0} for an apper-boung argument, since < Fg S
and 17 1,
R(D.v) (1- (1-e) +1 = (1 A)e Rmax(e).
I < A < 1, Rmax (e) strictly decreases because
- < 0. Discretely, for one d'Rmaxx de.
extra unit error Ae M that is perfectly flagged,
AR < Ae + AAc 1)-1) < O.
so increasing errors always reduces the reward, even under perfect tagging. So set 0 < X <.
5

---

Published as a conference paper at ICLR 2026
5.3 GROUP RELATIVE POLICY OPTIMIZATION
Using the sequence-level alignment-based reward described above, we train the policy with Group Relative Policy Optimization (GRPO) (Shao al. 2024). GRPO samples multiple completions for the same input, ranks them by reward, and converts the relative differences into normalized advantages, which removes the need for learned critic and reduces sensitivity to the absolute reward scale. These properties suit our incertainty-awan OCR setting, where rewards can be noisy and difficult to calibrate.
For each input % with reference yb, we sample group of G completions (J6.1...., /b.c) } from the behavior policy Vota (from the previous iteration). Each completion receives scalar sequence-lever reward (ib.g 2h), we then standardize rewards within the group to form relative advantages Ab.g-
The policy xp updated by maximizing a clipped GRPO objective with a KI. regularizer to refer- ence model Tref
B
G
(o) B
Σ g=1
min Pb.g Ab.g: clip(Pb.g, 1-c, 1+c) Ab.s) - .KL DKL(TO Tref))
G
b=1
where B is the number of inputs per batch, e is the clipping range for the likelihood ratio, and 3Kr. weights the KL penalty toward the reference. We use the sequence-level likelihood ratio ph.g
no (3o. of go) The divergence D Tret] is computed per token and averaged over the sequence.
aad Do d 9))
6 BLUR-OCR BENCHMARK
To support the experiments in Section we construct document images that are deliberately chal- lenging for OCR and prone VLM hallucination. Obtaining large-scale, diverse set of real de- graded documents with GT annotations is difficult. As an alternative, prior work has shown that adding synthetic degradations to document images effectively mimics real degradation (Blecher) it al. 2023; Guan et al., 2025 Yu et al.] 2025; He et al. (025). We therefore adopt a synthetic- degradation pipeline to produce large set of degraded document images. We collect source ma- terials from Project Gutenberg, covering diverse genres, fonts, and publication eras, with layouts varying in line length, line spacing, and character spacing. Following the Prep-OCR corruption method (Guan et al., ( we apply sequence of controlled corruptions in random order and with randomized strengths: additive noise, resolution reduction, Gaussian blur, background textures, stain overlays, and thin black/white lines that mimic scratches folds. We also introduce random opaque or transparent patches of varying sizes and apply morphological operations to simulate bro- ken strokes and ink bleed. Finally, after these global perturbations, we select several random regions on each page for strong local degradation. The full set of parameter settings for each degradation operation listed in Appendix C We create an evaluation benchmark of 2,045 images with paired GT transcripts, referred to as Blur-OCR. The training split contains 107,520 images but with disjoint -aware aaare OCR transcription on degraded documents. Figure reports the per-page text-length distribution and shows sample pages.
source data. To the best of our knowledge, this is the first benchmark dataset for uncertainty
In the Blur-OCR benchmark, we do not explicitly annotate which regions are "truly unreadable." The notion of "unreadable" is inherently model- and observer-dependent: different VL.Ms or human readers may disagree on which regions are still decipherable. Some datasets (e.g.. KIE-HVQA) use explicit region-level labels tied to synthetic occlusions that equate unreadability with specific visual artifacts (He et al. 2025); we instead define uncertainty at the transcript level to prevent overfitting to specific visual patterns. This approach focuses on the correctness and uncertainty of the output text, rather than on pixel-level readability of the image.
7 EXPERIMENTS
7.1 EXP1: SFT vs. GRPO AND THE EFFECT OF UNCERTAINTY TAGS ON OCR ACCURACY
Setup. In this experiment, we want to study (i) whether GRPO improves OCR accuracy beyond SFT and (ii) whether learning to emit UNC tags affects accuracy while enabling reliable tagging.

---

Published as a conference paper at ICLR 2026
Distriutionn Blun-CR Text Lengths
60
40
2500
Figure 2: Per-par Per-page text-length distribution and example pages from the Blur-OCR benchmark.
We train four variants of the same backbone (Qwen2.5-VLL-77) and evaluate on Blur-OCR. Unless noted otherwise, cold-start auto-tagging and the GRPO reward operate at the character level; GRPO uses X=0.9 and $=1.0.
supervised fine-tuning on 48k image- GT pairs without tags (plain transcripts), producing an OCR-strong baseline. M2 (w/ UNC, SFT->GRPO); con- tinue from MI and run GRPO on 107,520 images with the transcription-only objective (A=0 in the reward). M3 (w/ UNC, SFT): apply the cold-start procedure (Sec. 5.1 to obtain 48k character-level UNC-tagged pairs, then SFT Dwen2.5-VLL-7B on these (image, tCh-wwith-ags)
Model variants. M1 (w/o UNC, SFT):
pairs. M4 (w/ UNC, GRPO:) continue from M3 and run GRPO on 107,520 images using the character-level reward. We use LLaMA Factory (Zheng et al.) 2024) for SFT and VERL (Sheng al. 2024) for GRPO; training hyperparameters are listed in the Appendix B
Table 1: Results for Expl on Blur-OCR, with character- cold start and character-level reward.
Fi (t)
Accuracy (t)
Model
char
word
char
word
base: Qwen2.5.VL.s
0,858
0.776
MI: w/o UNC, SFT
0.868
0.752
M2: w/o UNC, SFT-+GRPO
0.892
0.817
M3: w/ UNC, SFT
0.243
0.286
0.830
0.710
M4: w/ UNC, SFT-
0.289
0.363
0.889
0.829
Results and analysis. From Tab. I we observe that the untuned base Qwen2.5-VL-7B starts at 0.858/0.776 (char/word). Adding UNC tags only during SFT reduces accuracy relative to SFT without tags (M3 vs. M1), while GRPO without tags (M2) improves over SFT. Notably, optimizing tagging with GRPO in M4 restores accuracy while also improving tagging quality, with word Fi 0.363, up from 0.286 with SFT-only tags. In summary, GRPO removes the accuracy penalty caused by SFT with tags and yields more reliable uncertainty spans, outperforming the baselines.
7.2 EXP2: COLD START AND REWARD GRANULARITY
Setup. Next, we investigate two questions: (i) whether GRPO can learn UNC tagging without cold start and whether a random-tag cold start is sufficient, and (ii) how matching versus mismatching the granularity of the cold start and the GRPO reward affects performance. For cold starts, SFT uses 48,000 data pairs; GRPO uses 107,520 images. "Random-tag" cold start injects UNC tags at random alphanumeric positions. See Appendix [ for details. "No cold start" starts GRPO directly from the base model with instruction on how use tags, prompt in Appendix D the reward for this experiment.
We set X 3=1 in
Results and analysis. From Tab. 2 observe that although GRPO improves transcription across the board, a cold start is essential for meaningful UNC usage. Without a cold start the model pro- duces almost no tags (F, ss 0.01), and a random start teaches only the format, with poor placement
7

---

Published as a conference paper at ICLR 2026
Table 2: Exp2 Blur-OCR: impact of cold-start (CS) supervision and reward granularity. Metrics follow Sec. and "2" denotes none for the corresponding column (i.c., no cold start or GRPO).
Precision (1)
Recall (1)
F1 (1)
Accuracy (1)
Gap (t)
CS
Reward
char
word
char
word
char
word
char
word
char
word
Char
0.013
0.012
0.007
0.005
0.010
0.009
0.870
0.735
0.000
0.001
Char
0.207
0.310
0.226
0.319
0.210
0.298
0.884
0.752
0.190
0.203
Random
Word
0.120
0.248
0.236
0.285
0.159
0.269
0.879
0.820
0.239
0.258
0.198
0.435
0.266
0.220
0.200
0.258
0.811
0.671
0.175
0.308
Word
Char
0.231
0,434
0.299
0.231
0.240
0.300
0.895
0.830
0.187
0.317
Word
0.205
0,488
0,404
0.366
0.225
0.378
0.879
0.805
0.162
0.381
0.335
0.524
0.247
0.233
0.243
0.286
0.830
0.710
0.268
0.396
Char
Char
0.342
0.523
0.306
0.312
0.289
0.363
0.889
0.829
0.310
0.437
Word
0.445
0.683
0,513
0.549
0.422
0.574
0.885
0.839
0.433
0.590
(4) Recall
Precictonn
L3) Accuracy
Lengttt Gop
3.50
0.1
053
3.70
0.5
-50
0.5
-000
035
620
3.50
035
169 137
140
541 50 200
80
300
220
346
300
300
330
22
130 120
340 100 150 200
100 500 200 200
Figure 3: Exp3: (a-b) 8: controls the precision- recall trade-off, 0 gives the fastest precision gains with stable recall, while larger 3 hurts precision and very small 8 slightly lowers recall. (c) A: runs with ><1 converge to higher accuracy than \=1, which tends to manufacture errors. (d) ry: the red curve 1,=1, no damping) steadily shortens outputs, whereas the bundle of gray curves (damping on, 2 - L.3] across various A)) stay near zero length- -ga and thus remain stable. (a-c) Word-level. (d) Char-level.
(word F, 0.27-0.29) After applying structured CS followed by GRPO, the accuracies of different configurations are broadly similar, so the main differences lie in UNC quality. The best UNC per- formance is achieved by Char-CS paired with a Word-reward, yielding strong uncertainty metrics (word F, R 0.57, Gap 2 0.59) while maintaining high accuracy. Our interpretation is that Char-CS teaches precise, tight boundaries so the model can place tags with higher precision, while the word- level reward then provides denser and more forgiving credit, the model can accumulate effective rewards and improve steadily. In contrast, a Char-reward is too strict and provides sparse credit, which makes stable improvement more difficult. Therefore, in subsequent experiments a character-level cold start with a word-level reward to train models.
7.3 EXP3: SENSITIVITY TO B, X, AND 17
Setup. Building on Expl and Exp2, we fix the training recipe Char-CS Word-reward and vary only the reward hyperparameters. We investigate the parameter ranges:
(i B E {0.3, 0.5, 0.8, 1.0, 1.2]. which controls the precision-recal balance in Fe.L-
(ii) X E (0.0, 0.5, 0.9, 1.0) to probe both the tagging weight and the slope/monotonicity of the upper bound Rmxxx( (e) = 1 11-0111 (01-)ee Je: X = 0 removes the tagging term; for 0 < À < 1 the bound decreases with error with slope (1 A);
makes it flat.
(iii) v E 2 [2-1o2>13)] 1.0 compare our versus no damping.
damping (active only when pr. > 1.3)
Results and analysis. The results in Tab. and Fig. 3 show some key trends at the word-level.
(i) 8. Large/ 3 drives aggressive tagging, inflating UNC spans and collapsing precision (see Fig. 3
b). In contrast, 3 E [0.5, accelerates precision without overly harming recall; the recall curves change little across 8, while 3=1.2 still yields poor precision with only minor recall gains. Overall,
8

---

Published as a conference paper at ICLR 2026
Table Exp3 on Blur-OCR with Char-CS and Word-reward, with ablation over 3, X, and 77. Overall, 3 s 0.5 gives the best precision- recall balance, ><1 maintains higher accuracy than \=1, and
turning on 1y damping stabilizes length and improves accuracy.
Precision (1)
Recall (t)
Fy (t)
Accuracy (1)
Gap (t)
Ablation
Setting
char
word
word
word
char
word
chur
word
0=0.3
0.630
0.828
0,510
0.533
0,529
0.617
0.885
0.831
0.693
0.714
0=0.5
0.626
0.839
0.599
0.620
0.572
0.685
0.881
0.839
0.694
0.739
0=2 11.3113
0=0.8
0.485
0.760
0,532
0.588
0.486
0.624
0,883
0.832
0.478
0.630
6=1.0
0,445
0.683
0,513
0,549
0,442
0.574
0.885
0.839
0.433
0,590
1=1
0,341
0.552
0.521
0,566
0.320
0.558
0.875
0.814
0,380
0.526
A=0.0
0.289
0.451
0,189
0.192
0,230
0,250
0,880
0.838
0.241
0,352
$=0.5, 122 2-2.0,,2--
A=0.5
0,534
0.628
0.498
0,514
0,482
0,581
0,882
0.832
0.572
0.625
A=0.9
0.626
0.839
0.599
0.620
0.572
0.685
0
0.839
0.694
0.739
A=1.0
0.555
0.941
0,748
0.747
0.591
0.811
0,814
0.767
0.585
0.835
0.626
0.839
0.599
0.620
0.572
0.685
0.881
0.839
0.694
0.739
$=0.5, X=0.9
1=1.0
0.600
0.816
0.510
0.511
0.486
0.577
0.792
0.762
0.523
0.684
3=0.5 gives the best balance with precision (+% 0.839) and recall (Rx 0.620), and also there is large error rate gap (<s 0.739), so we adopt 3=0.5 thereafter.
(ii) X. In Fig. 3c, A=1 raises accuracy early but then stalls and converges below the A<1 runs. The reason the flat upper bound Rmax (e) (1-1-1106000111 at A=1: once errors are perfectly wrapped by UNC, adding more errors no longer reduces reward, so the policy could tend to manufacture and tag errors rather than only tag necessary spans. Using a larger weight below (we use x=0.9) restores a negative slope e, allowing accuracy to improve while still providing strong tagging signal.
(iii) n.. With no damping (1=1), the output length keeps decreasing during training (Fig. Bi), which lowers accuracy, while tagging scores rise. Since our UNC rule assigns each missing segment to the nearest UNC span, the model can shorten its output and place a few tags near boundaries. Thus, many of the missing tokens count as "inside" UNC, artificially boosting precision/recal//Fi without real transcription. Notably, under A=0.9 with no damping, a degenerate "tag-everything, say-little" strategy can still earn high reward; enabling the damping 1 2 2-1pp>>>] counteracts this behavior. Based on these results, we use 3=0.5, X=0.9, and the damping rule for ry in subsequent experiments, as this setting maintains strong transcription while producing precise, well-calibrated UNC tags.
7.4 EXP4: TROSS-MODER COMPARISON ON Blur-OCR
Setup. We compare general VLMs and OCR OCe-specid models on the Blur-OCR dataset. General models receive an instruction to transcribe and wrap uncertain spans with UNC tags (see prompt in Appendix D and decoded with temperature 0.2. OCR baselines are run with their recommended inference settings, they cannot emit tags, so the entries for tagging metrics and Gap are marked "_", In addition, we train uncertainty-awan 8=0.5 and A=0.9 using our method and apply greedy decoding at test time. We also train a plain OCR model and derive uncertainty from token-level Shannon entropy (Manakul et al. 2023), where a single threshold is selected on validation set to maximize Fi. Tokens whose entropy exceeds the threshold marked uncertain. Finally, we include a multi-model voting baseline, Ensembles: we run five general VLMS for transcription and take Gemini-2.55 2.5--nn as the anchor. If at least two of the other four models disagree with Gemini on region, then we mark that region as uncertain. We use these two conventional uncertainty quantification (UQ) methods as baselines for comparison.
models based on Qwen-2.5-VL and Intern 52.5 with
Results and analysis. The results are in Tab. 4 General VLMs achieve similar word-level accuracy, from 0.733 to 0.826. Within this group. models that expose stepwise reasoning (Gemini-2.55 P Claude-Opus-4, Intern VL-33.5-241B 2-1--A28BB obtain higher word-level Fj for tagging (0. (5..500))) than non-reasoning models such as GPT-40 and Qwen-2.5-VV VL- (0.039 and 0.013), although the absolute numbers remain low. The Ensembles approach achieves reasonable recall but suffers from low precision, while the entropy baseline provides partial localization with 0.410 word-level Fi. Meanwhile, speciiaeea systems perform poorly on Alur-OCR, partly due to smaller parameter counts and limited exposure heavily degraded pages; their word-level accuracy ranges from 0.219 to 0.732, and GOT exhibits repetition artifacts. In contrast, our uncertainty- awwre Qwen-2.5-VL. VL-BB model reaches 0.685 word-level F, with 0.839 word-level accuracy, outperforming all baselines on

---

Published as a conference paper at ICLR 2026
Table Exp4 on cross- comparison.
Precision (1)
Recall (?)
F; (1)
Accurucy (1)
Gap (?)
Category
Model
char
word
char
word
cbar
word
char
word
char
word
GPT-4o-2024-11-20
0.111
0.151
0.021
0.026
0.032
0.039
0.839
0.734
0.070
-0.003
Qwen-2.5-VL-722.
0.079
0.117
0,009
0.008
0.014
0.013
0.873
0.801
0.053
-0.079
General VL.Ss
Gemini-2.5-Pro
0.185
0.348
0.156
0.138
143
0.163
0.880
0.826
0.158
0,249
Chmo6-opus-4-20255514
0.305
0.421
0.176
0.155
0.187
0.205
0.840
0.733
0.275
0.277
Intmm\l.35-241B-2288
0.279
0.377
0.120
0.138
0.131
0.156
0.853
0.752
0.162
0.201
MinerU2.5 Niu al. 2025
0.801
0,732
OCR-specialzee
PaddioOCR 325 CCl
2025
0.702
0.308
Dolphin Feng ct al. 4725
0.632
0.549
Gon at 2024
0,239
0.219
Ensembles
0.389
0.703
0.458
0.491
0,880
0.358
0.488
UQ baseline
Qwcn-2.5-VL-7B (entropy)
0.327
0.406
0.339
0.440
0.313
0.410
0.888
0.840
0.304
0.350
laiemVL2..58B (UNC)
0.669
0.535
0.550
Ours
Qwen-2.5-VL-33 (UNC)
0.548
0.786
0.520
0.613
0.504
0.638
0.872
0.798
0.623
0.636
Qwea-2.3-VL-7B- (UNC)
0.626
0.839
0.599
0.620
0,572
0.685
0.881
0.839
0.694
0.739
uncertainty localization while preserving strong transcription. Our uncertainty-aware Intern VL.2.5
model learns similar ability to bracket uncertain spans, indicating that the approach transfers across architectures. Detailed qualitative examples are provided in Appendix
8 CONCLUSION AND FUTURE WORK
This work introduced uncertainty-aware OCR approach that trains VLM-based OCR systems to bracket self-identified uncertain spans with uncertainty tags on degraded documents. Using pseudo-labeled cold start and a GRPO-driven multi-objective reward, our method preserves tran- scription quality while accurately localizing uncertainty. We also introduced Blur-OCR, a chal- lenging OCR benchmark consisting of document images with highly diverse synthetic degradations, comprising 107,520 training images and 2,048 evaluation images. On the Blur-OCR benchmark, our best 7B model with Char-CS + Word-reward achieves a word-level Fl of 0.685, outperforming both open- and closed-soure source baselines for uncertainty tagging without loss of accuracy. These results demonstrate that making uncertainty explicit offers a practical path to more trustworthy degraded document understanding.
While our experiments instantiate this framework in degraded document OCR, the underlying prob- lem is more general. Our goal is to train VLMs to explicitly express uncertainty when their inputs are difficult or ambiguous, for example when the visual evidence lossy or compressed, or when the underlying content is intrinsically hard recognize, rather than guessing from incomplete in- formation. Similar challenges arise in many other settings. For example, in speech recognition from noisy low-quality audio, an ASR-LLM system (Bai al. 2024) could mark acoustically ambigu- ous regions with UNC tags. Recent work such as DeepSeek-OGe OCR (Wei et al., 2025) compresses long visual contexts into visual tokens, which act as low- sow-iidelity sketch of the original document. This compression is analogous to our degraded images, and UNC tags could similarly flag where the sketch loses critical detail. QA systems that must answer under partial evidence could also benefit from localized uncertainty marking rather than overconfident generation. Another direction for fu- ture work to train on more diverse document distributions to build general-purpose purpose OCR system. We also proposed methods starting point for broader uncertainty-awe aware generation from lossy inputs, generalizing beyond OCR.
REPRODUCIBILITY STATEMENT
The paper specifies the tagging semantics and evaluation protocol in Sections and the pseudo-
labeled cold start in Section 5.1 and the GRPO objective in Section (.2 Experimental settings
detailed in Section / training hardware and key hyperparameters are listed Appendix B prompts provided in Appendix D. and UNC tagging examples given in Appendix
10

---

Published as a conference paper at ICLR 2026
Our training pipeline builds open source frameworks. We release code and resources at https: 7/github. com/ NikoGuan woeoouuuunnuneetatnnnt oottoooootittttoRtRataoR including implementations for generating pseudo-label UNC tags from goodel-ground-tee alignments and computing the GRPO reward. We also provide the Blur-OCR benchmark dataset.
ACKNOWLEDGMENTS
This publication is part of project that has received funding from (i) the European Research Council (ERC) under the Horizon 2020 research and innovation programme (Grant agreement No. 884951); (ii) Research Ireland to the Insight Centre for Data Analytics under grant No. 12/RC/2289 _P.
This publication is also based on research funded by European Union's 's Horizon Europe 2021-2027 framework programme. Marie Sklodowska-Cure Actions, Grant Agreement No. 101072456; Taighde Eireann Research Ireland under grant No. 13/RC/2094.2 Lero the Research Ireland Centre for Software.
REFERENCES
Kehinde Ajayi, Yi He, and Jian Wu. Uncertainty-aware complex scientific table data extraction. arXiv preprint arXiv:2507.02009, 2025.
Ye Bai, Jingping Chen, Jitong Chen, Wei Chen, Zhuo Chen, Chuang Ding, Linhao Dong, Qianqian Dong, Yujiao Du Kepan Gao, al. Seed-asr: Understanding diverse speech and contexts with Ilm-based speech recognition. arXiv preprint urXiv:2407.046677 2024.
Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. arXiv preprint arXiv:2308. 13418, 2023.
Xi Chen, Xiao Wang. Soravit Changpinyo, Anthony Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, al. Pali: jointly-saaled multilingual language-image model. arXiv preprint arXiv:2209.0667 2022.
Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang. Zelun Zhang, Changda Zhou, Hongen Liu, Yue Zhang, Wenyu Lx, Kui Huang, Yichao Zhang, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. PaddleOCR 3.0 technical report. arXiv preprint wxXvv2507.00595. 2025.
Mirjam Cuper, Corine van Dongen, and Tineke Koster. Unraveling confidence: examining confi- dence scores as proxy for OCR quality. In International Conference on Document Analysis and Recognition, pp. 104 120. Springer, 2023.
Yixiong Fang, Ziran Yang. Zhaorun Chen, Zhuokai Zhao, and Jiawei Zhou. Enhancing vision- language model reliability with uncertainty-guided dropout decoding, In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Hao Feng, Shu Wei, Xiang Fei, Wei Shi, Yingdong Han, Lei Liao, Jinghui Lu, Binghong Wu, Qi Liu, Chunhui Lin, al. Dolphin: Document image parsing via heterogeneous anchor prompting. arXiv preprint arXiv:2505. 1405599955....9,, 2025.
Ling Fu, Zhebin Kuang, Jiajun Song, Mingxin Huang, Biao Yang, Yuzhe Li, Linghao Zhu, Qidi Luo, Xinyu Wang, Hao Lu, al. OCRBench v2: An improved benchmark for evaluating large multimodal models on visual text localization and reasoning. arXiv preprint arXiv:2501.000. 000,00 2024.
Shuhao Guan and Derek Greene. Advancing post-OCR correction: A comparative study of syn- thetic data. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Findings of the Asso- ciation for Computational Linguistics: ACL 2024, pp. 6036 6036-6047, Bangkok, Thailand, August www.244..nnnng-al....m URL
2024. Association for Computational Linguistics. doi: 10. (..18653/v1/22244 https: //acl antho ogy ogw.org//022.....m htdei//2.....iiiseeeeii .3611

---

Published as a conference paper at ICLR 2026
Shuhao Guan, Cheng Xu, Moule Lin, and Derek Greene. Effective synthetic data and test-time adaptation for OCR correction. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp- 15412-1 1525,, Miami, Florida, USA, November 2024. Association for Computational Linguis- tics. doi: 1
URL / /acl anthellogy.org/2000
emr.p imaii.... 8627
Shuhao Guan, Moule Lin, Cheng Xu, Xinyi Liu, Jimman Zhao, Jiexin Fan, Qi Xu, and Derek Greene. PreP-OCR: A Complete Pipeline for Document Image Restoration and Enhanced OCR Accuracy. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 15413-15425, ACL, 2025.
Ahmed Hamdi, Elvys Linhares Pontes, Nicolas Sidère, Mickaël Coustaty, and Antoine Doucet. In- depth analysis of the impact of OCR errors named entity recognition and linking. Journal of Natural Language Processing, pp. 24, 03 2022.
Zhentao He, Can Zhang, Ziheng Wu, Zhenghao Chen, Yufei Zhan, Yifan Li. Zhao Zhang, Xian Wang, and Minghui Qiu. Seeing is believing? Mitigating OCR hallucinations in multimodal large language models. arXiv preprint (XXi::2506.2066. 2025.
Arthur Hemmer, Mickaël Coustaty, Nicola Bartolo, and Jean-Mare Ogier. Confidence-aware docu- ment OCR error detection. In Document Analysis Systems, pp. 213-228. Springer Nature Switzer- land, 2024. ISBN 978- 978-3-031-70442-0.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.212777 2024.
Adam Jatowt, Mickael Coustaty, Nhu-Van Nguyen, Antoine Doucet, et al. Deep statistical analysis of OCR errors for effective post-OCR processing. In 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL), pp. 29-38. IEEE, 2019.
Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. Why language models hallucinate. arXiv preprint 2509. 040644 2025.
Alexei Kaltchenko. Assessing GPT model uncertainty in mathematical OCR tasks via entropy anal- ysis. International Journal of Information Technology, pp. 1-10, 2025.
Zhaolu Kang, Junhao Gong, Jiaxu Yan, Wanke Xia, Yian Wang. Zhuo Cheng. Wenhao Cao, Ziwen Wang, Zhi Feng, Huaxuan Ding, Siqi He, Shannan Yan, Xiaomin He, Junzhe Chen, Chaoya Jiang. Wei Ye, Kaidong Yu, and Xuelong Li. HSSBench: Benchmarking humanities and social sciences ability for multimodal large language models. In The Fourteenth International Confer- ence on Learning Representations, 2026. URL iQsKotob31
ew net. / forum?id
Geewook Kim, Teakgyu Hong. Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinycong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. OCR-free free document understanding transformer. In European Conference on Computer Vision, pp. 498-517. Springer, 2022.
Vasily Kostumov, Bulat Nutfullin, Oleg Pilipenko, and Eugene Ilyushin. Uncertainty-aware evalua- tion for vision-language models, arXiv preprint arXiv:2402.14418, 2024.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brah- man, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, al. Tulu 3: Pushing frontiers in open language model post-training, arXiv preprint arXiv:2411.15122 2024.
Kenton Lee, Mandar Joshi, Iulia Raluca Ture, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wes Wei Chang, and Kristina Toutanova. Pix2struct: Screen- shot parsing as pretraining for visual language understanding. In International Conference on Machine Learning, pp. 18893-18912. PMLR, 2023.
12

---

Published as a conference paper at ICLR 2026
Hui Li, Peng Wang. Chunhua Shen, and Guyu Zhang. Show, attend and read: A simple and strong baseline for irregular text recognition. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 33, pp. 8610-860 1110--667,40 2019.
Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhou- jun Li, and Furu Wei. Troer: Transformer-basee optical character recognition with pre-trained models. In Proceedings of the AAA! Conference on Artificial Intelligence, volume 37, pp. 13094- 13102, 2023.
Minghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen, and Xiang Bai. Real-time scene text detection with differentiable binarization. In Proceedings of the AAAI Conference on artificial intelligence, volume 34, pp. 11474-114 11481, 2020.
Moule Lin, Andrea Patane, Weipeng Jing, Shuhao Guan, and Goetz Botterweck. Flow-induced diagonal gaussian processes, 2025. URL lattps:t:://ar org/ abs//209..7152
Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang. Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. OCRBench: On the hidden mystery of OCR in large multimodal models. Science China Information Sciences, 67(12):22010,, 2024.
Potsawee Manakul, Adian Liusie, and Mark JF Gales. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.088 088009896, 2023.
Noam Mor and Lior Wolf. Confidence prediction for lexicon-free OCR. 2018 IEEE Winter Conference Applications of Computer Vision (WACV), pp. 218-225 IEEE, 2018.
Nibal Nayef, Yash Patel, Michal Busta, Pinaki Nath Chowdhury, Dimosthenis Karatzas, Wafa Khlif, Jiri Matas, Umapada Pal, Jean- -hrritohrttppe Burie. Cheng-lin lin Liu, et al. ICDAR2019 Robust Read- ing Challenge on Multi-lingual Scene Text Detection and Recognition RRC-MM.T-2019, In 2019 International Conference Document Analysis and Recognition (ICDAR), pp- 1582-1587, 2019.
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang. Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Yuanhong Zheng, Dongsheng Ma, Zirui Tang, Boyu Niu, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Jingzhou Chen, Fangdong Wang. Xi- aomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, Ruiliang Xu, Yuanyuan Cao. Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Keming Wang, Dechen Lin, Guanlin Shen, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang. Xiaoyi Dong. Jiaqi Wang. Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang. Zhongying Tu, Chao Xu, Kai Chen, Yu Qiao, Bowen Zhou, Dahua Lin, Wentao Zhang, and Conghui He. Mineru2.5: A decoupled vision-language model for efficient high-resolution document parsing. arXiv preprint 25Xiv:2509.221 22, 2025.
Richard Willhelm Oelschlager. Evaluating the impact of hallucinations user trust and satisfaction in LLM-based systems. Bachelor's 's thesis, Linnacus University, 2024.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems,
2023.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint a0Xiv:2402.03300, 2024.
13

---

Published as a conference paper at ICLR 2026
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. HybridFlow: flexible and efficient RLHF framework. arXiv preprint arXiv: 2409.192566 2024.
Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-end trainable neural network for image- based sequence recognition and its application to seene text recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2)242222-3334 2016.
Baoguang Shi, Mingkun Yang, Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai. Aster: An attentional scene text recognizer with flexible rectification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2035-20444 2018
Ron Slossberg, Oron Anschel, Amir Markovitz, Ron Litman, Aviad Aberdam, Shahar Tsiper, Shai Mazor, Jon Wu, and Manmatha. On calibration of scene-text recognition models. European Conference Computer Vision, pp. 263-279. Springer, 2022.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3021, 2020.
Gemini Team, Petko Georgiev, Ving lan Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context. arXiv preprint urXiv:2403. 003.005500 2024.
Zhi Tian, Weilin Huang, Tong He, Pan He, and Yu Qiao. Detecting text in natural image with connectionist text proposal network. In European Conference on Computer Vision, pp. 56-72. Springer, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances Neural Informa- tion Processing Systems, 30, 2017.
Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, al. General OCR theory: Towards OCR-2.0 via unified end-to-end model. arXiv preprint arXi::2409.01744 2024.
Haoran Wei, Yaofeng Sun, and Yukun Li. DeepSeek-OCR: Contexts Optical Compression. arXiv preprint arXiv:2510.1825:1 2025.
Zhibo Yang, Jun Tang, Zhaohai Li, Pengfei Wang, Jianqiang Wan, Humen Zhong, Xuejing Liu, Mingkun Yang, Peng Wang, Shuai Bai, et al. CC-OCR: A comprehensive and challenging OCR benchmark for evaluating large multimodal models in literacy. arXiv preprint arXiv:2412.02210., 2024.
Ziran Yang, Shibo Hao, Hao Sun, Lai Jiang, Qiyue Gao, Yian Ma, and Zhiting Hu. Understand- ing the sources of uncertainty for large language and multimodal models. In ICLR Workshop: Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable Al, 2025.
Haiyang Yu, Yuchuan Wu, Fan Shi, Lei Liao, Jinghui Lu, Xiaodong Ge, Han Wang, Minghan Zhuo, Xuecheng Wu, Xiang Fei, et al. Benchmarking vision-language models on Chinese ancient doc- uments: From OCR knowledge reasoning. arXiv preprint arXiv:2509.091 09731, 2025.
Junyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang, Zichen Wen, Ying Li, Ka-Ho Chow, Conghui He, and Wentao Zhang. OCR hinders RAG: Evaluating the cascading impact OCR on retrieval-augmented generation. arXiv preprint arXiv:2412.02552,, 2024.
Yulong Zhang, Tianyi Liang, Xinyue Huang, Erfei Cui, Xu Guo, Pei Chu, Chenhui Li, Ru Zhang, Wenhai Wang. and Gongshen Liu. Consensus entropy: Harnessing multi-Vin VLM agreement for self-verifying and self-improving OCR. arXiv preprint 2rxiv:2504. 1101, 2025.
14

---

Published as a conference paper at ICLR 2026
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhanghhi Feng, and Yongqiang Ma. LlamaFactory: Unified efficient fine-tumin tone-tuning 100+ language models. In Pro- ceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). ACI,, 2024.
Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. East: efficient and accurate scene detector. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2017.
A
USE LLMS
We used LLMs general- purposs assistive tools for literature search and language polishing. We also trained and evaluated LLMs on the Blur-OCR benchmark.
B TRAINING HYPERPARAMETERS
The following describes our training configuration. All runs were on 8x NVIDIA H200 (141 GB) with 6f16. For SFT (LLaMA Factory). used per-device batch size 1 with gradient accumulation =4 (effective batch size = 32 on 8 GPUs), AdamW, learning rate 2.5x10 10% warmup, and epochs; other SFT options were left library defaults. For GRPO (VERL), trained I epoch with global train batch size 512, actor learning rate 4.0x10 and a cosine warmup (warmup ratio 0.03, min-LR ratio 0.10, cycles 0.5). entropy coeff 0.001, and we set ÅKL 0. Rollouts used 6 on vLLM backend and sampled completions per input. All other engine/FS- cwfoffload/ccchigg//ga
cosine schedule with
flags followed the framework defaults. The 2. 2.160 SFT steps took around hour. GRPO 210 steps took around 20 hours.
C UNC TAGGING EXAMPLES
Here we illustrate how pscudo-labels are constructed for our proposed cold start strategy. For each degraded image, we run the base model obtain transcript, align it with the ground truth, and insert UNC tags into the model's transcript around localized errors. Below we show both character- and word-level variants used for SFT.
GT
The quick brown fox jumps quick brownnn fox jumaps <C>T<</C> quick brown<C>nn</C>>>> fex jum<C><</C>pp
OCR
Char
Word
<<>Te<</>> quick <>>brownn</C>>>>> fex <C>jumaps</C>>
GT
The quick brown fox jumps the gaick brown fox jumps
OCR
Char
<C>t</C>he q<C>a</Cxick brown fax jumps
Word
<C>the qaick</C> brown fox jumps
hello world
OCR
hello wirld 123
Char
hello <<c>1</c>r1de> 123</C>
Word
hello <C>wlrld 123</C>
D PROMPT
use the following instruction general VLMSs Exp4:
You intelligent OCR system.
Perform OCR to transcribe all text from the image. When you encounter characters or words you not confident about, wrap them with </>...</C>,
You may use <C>...</C> multiple times. Do not nest or overlap <C> regions.
Use the last span that covers the uncertainty. Preserve natural reading order and basig line breaks. Keep the
15

---

Published as a conference paper at ICLR 2026
aiae pidh
tomple Aunge Boloeng
the shous
tea, All
oad, sill Bost leforl ma dock Haaluned Nua bold of culing on staslled
daesha ln del she ault
with
hase bud the somple
The aouds
hoanded pelides
Rerois the bosnin was
walt. thes the aleat the sraand.
the sollihtt bughtes
a88200
301010
120 30002008
$20
Tet Lnwtiraattr
Outede ooin sd wat bvill gie had ogar early and forper
shivering and
with
led, dted up
be drie Dolly Mic jane Sutiing her de thoiihh a was dey old
The idea talding repprrable hetm with munkly
"Wiaat did you emp
and pooundd gettr Am eat would...
levulyy
and tiees des ld, Shank' non
equaed
50- gor before roell rgaln.
105
Match
UNC
Error
+V UNC Error
Figure 4: Examples of outputs from our uncertainty-aware OCR method on the Blur-OCR bench- mark. For each page we show (top-bottoo): the degraded input image; the model transcription with UNC tags and the ground truth (right, "Output/GT"); and an alignment map. Colors: green match, blue = text inside UNC, red = error outside UNC, red in blue box error inside UNC. The panel at lower right reports word-level counts (Correct/Erroo in/out of UNC) and summary metrics (precision/recall/F, for UNC and word accuracy).
original language, punctuation, and casing. Output only the final text (no explanations, no code blocks, no quotes).
Input: <image>
E RANDOM-TAG COLD START (FOR EXP2)
This baseline ignores the image and inserts UNC tags at random. For each clean sentence, we split it into
S E {1,....88 8) segments. For each segment, we sample an uncertainty rate from mixture: 10% with r 0; 40% with r [0, 0.1]; 10% with r U[0.2, 0.5]: 25% with ~ U 0.05 0.8); and 15% with r ~ U' ('00,8, 1.0.. We then mark [r- len(segment)] characters in that segment as uncertain and wrap contiguous selections with <C>..</C>, This yields non-uniform, segment- wise UNC distribution similar to real text.
F SAMPLE OUTPUT
This appendix provides additional qualitative examples from the Blur-OCR benchmark, following the visual- ization format in Fig. [ For given page, show: (1) the degraded input image; (2) the model transcription with UNC tags alongside the ground truth (Outuut/TT); and (3) alignment map.
16

---

Published as a conference paper at ICLR 2026
Table 5: List of document degradation parameters by noise level.
Parameter
Level-1
Level-2
Level-3
Level-4
Noise Factor
[0,10]
[0,30]
[0,50]
[0,50]
Scale Factor
[0.2,1]
[0.2.1]
[0.2,1]
[0.2,1]
Gaussian Blur (px)
[0.1]
[0.1]
[0,2]
[0,2]
Background Intensity
[0.0.1]
[0,0.3]
[0.0.6]
[0,0.6]
Stain Transparency
[0.0.3]
[0,0.6]
[0,0.8]
[0,0.8]
Max Stains
[0.1]
[0.3]
[0,5]
[0,5]
Contrast Factor
[0.6.1]
[0.6.1]
[0.6.1]
[0.3,1]
Black Spot Size (px)
1x1
1xl
Ix1
1x1
Black Spots per Page
[0,HW/3000]
[0.HW/2000]
[0,HW/1000]
[0.HW/1000]
White Patch Size (px)
[0,3]x[0.3]
[0,5]x[0,5]
[0,5)x(0,5]
[0,5]x[0,5]
White Patchcs per Page
[0,HW/500]
[0,HW/300]
[0,HW/200]
[0,HW/100]
Line Artifacts
[0,4]
[0,6]
[0,8]
[0,10]
Dilation Iterations
[0.2]
[0.2]
[0,2]
[0,2]
Erosion Iterations
[0.2]
[0.2]
[0.2]
[0,2]
G DEGRADATION OPERATIONS AND PARAMETERS
To add further diversity to the degradation process. we implement four progressive degradation levels, with the corresponding parameter ranges shown in Table 5 Each level consists of series of degradation operations applied in a random order, that different operation sequences can produce substantially different effects. Higher levels introduce more aggressive distortions.
17


Reviews:
[{"review_id": "t8HKmO2kml", "reviewer": "Reviewer_29Ew", "summary": "This paper focuses on a task: when performing OCR, a VLM should mark uncertain and hard-to-recognize spans by surrounding them with a custom UNC tag. To enable this, the authors build a training set of about 100K samples and a 2K-sample benchmark. For training, they use two phases — SFT for warm-start, followed by RL.", "questions": "is it possible to do semi-supervised learning on this task since uncertainty can also be generated by LLM itself (although it may not be so well-calibrated)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The writing is clear, and the presentation of different settings is easy to follow.\n\n2. Introducing uncertainty-aware generation in OCR to mark unclear spans has practical value.\n\n3. The experiments include extensive ablation study, which helps clarify the effectiveness of the method, and I appreciate that.", "weaknesses": "1. Minor: Although admitting uncertainty in OCR has some practical value, on the other hand, the broader significance is also limited since the work focuses on a specific application setting.\n\n2. The backbone choice is quite limited.\n\n3. The benchmark is constructed by the authors, while appreciated, i also want to know how well the method (and models) generalizes to more OOD scenarios.\n\n4. Minor: I think the paper should use “VLM” instead of “LLM.”\n\n5. While I appreciate the detailed metric section and the following ablation studies, it feels somewhat over-emphasized. The data component should likely deserve more effort than defining the metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fEOgVROtRk", "reviewer": "Reviewer_MjWd", "summary": "This work tackles the problem of hallucinations for OCR for vision-language models. The models hallucinate when they are provided with blurry documents. This work proposed to use Reinforcement learning - GRPO to tackle this problem where the model is made to answer uncertainty tags along with transcription. They construct a multi-objective reward that balances accuracy with uncertainty and also mitigates reward hacking. They also provide a benchmark to measure uncertainty aware OCR performance on degraded documents and show that their method outperforms baselines.", "questions": "- The authors discuss tag validity and alignment around lines 113. I don’t fully understand how the GT segments which are not inside y^hat work. It would be nice to clarify with an example.\n- In lines 370, the authors discuss the tradeoff of using character level vs word level granularity for tags. Could the authors provide an example along with the text? This would make it easier to follow. What do the authors mean by character level rewards?\n- Can the authors compare their method to some baselines used in the uncertainty literature like softmax probabilities, entropy etc?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Uncertainty estimation has been a long studies problem but this work studies it in the context of OCR which is an important problem and the technique of using RL to is interesting. \n- The work discusses the importance of cold-start SFT, describes in detail their reward formulation, character vs word level tagging, different hyperparameters and provide comprehensive experiments.", "weaknesses": "- The benchmark that they introduce has synthetic degradations. It is unclear how much the results transfer to actual degradations found in practice.\n-  The paper has nice set of experiments but lacks some intuitions examples as detailed below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Jk3dGxkOJZ", "reviewer": "Reviewer_8tLK", "summary": "The paper presents an uncertainty-aware fine-tuning framework for OCR-capable large language models (LLMs).\nInstead of producing overconfident transcriptions for visually degraded documents, the model is trained to explicitly mark uncertain spans with specific tags.\nThe approach combines two stages: (1) a cold-start supervised fine-tuning (SFT) phase using pseudo uncertainty labels automatically derived from model errors, and (2) Group Relative Policy Optimization (GRPO), a reinforcement learning algorithm that jointly optimizes transcription accuracy and uncertainty tagging quality using a reward function balancing edit distance and F-beta-based span precision–recall.\nExperiments on the new Blur-OCR benchmark demonstrate that GRPO improves both transcription correctness and uncertainty calibration compared to the cold-start baseline, while avoiding degenerate behaviors such as excessive tagging.", "questions": "1. Did the authors try training with other values of λ — how sensitive are results to this hyperparameter?\n2. How would the approach behave in non-OCR settings, such as code transcription or speech recognition?\n3. Did you observe any tendency for the model to under-tag uncertainty after GRPO fine-tuning?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The problem is clearly motivated — OCR hallucination is a realistic and underexplored setting for uncertainty estimation.\n2. The proposed explicit uncertainty tagging paradigm is conceptually simple yet effective.\n3. The GRPO objective is well designed, balancing transcription accuracy and tagging F1, and preventing pathological behaviors via the reward-damping term.\n4. The evaluation setup is thorough: they report both accuracy and uncertainty metrics and analyze different training stages.\n5. The paper is very clearly written and easy to follow, with transparent motivation and mathematical detail.", "weaknesses": "1. Experiments are limited to a single OCR model family. It remains unclear whether the proposed method generalizes to other setups (e.g., multi-modal vision-language models such as Donut or TrOCR).\n2. Comparison baselines are relatively narrow — no direct comparison with alternative uncertainty modeling approaches such as entropy-based rejection or calibration.\n3. There is limited qualitative analysis of false-positive tags (over-tagging). Some visual examples or error breakdowns could strengthen interpretability claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SJjGbrxrVZ", "reviewer": "Reviewer_ixBu", "summary": "This paper introduces uncertainty-aware OCR, where vision-language models transcribe degraded documents while bracketing uncertain spans with explicit uncertainty tags (`<C>...</C>`). The authors employ a pseudo-labeled cold start followed by Group Relative Policy Optimization (GRPO) with a multi-objective reward that balances transcription accuracy and uncertainty coverage. They introduce Blur-OCR, a benchmark of 2,048 synthetically degraded images from Project Gutenberg. The best model (Qwen2.5-VL-7B) achieves word-level F1 of 0.685 for uncertainty tagging and 0.839 accuracy, outperforming several baseline models including GPT-4o and Claude-Opus4.\n\nHowever, the limited evaluation scope, missing key baselines, and conceptual concerns about the cold start procedure prevent a stronger recommendation. With revisions addressing the generalization questions and including comparisons with MinerU systems and uncertainty quantification methods, this could become a solid contribution.", "questions": "1. **Generalization to real degradations**: Can the authors evaluate on real-world degraded documents (e.g., historical document datasets) to demonstrate that the approach generalizes beyond the specific synthetic degradation pipeline?\n\n2. **Comparison with MinerU systems**: MinerU and MinerU2.5 [2,3] represent recent advances in document parsing. How does the proposed method compare against these systems on Blur-OCR? If these systems cannot produce uncertainty estimates, can they be combined with the proposed tagging approach?\n\n3. **OmniDocBench evaluation**: OmniDocBench [1] provides diverse document types. Can the authors evaluate whether uncertainty tagging helps on this more realistic benchmark?\n\n4. **Quantitative uncertainty measures**: How does explicit binary tagging compare with continuous uncertainty scores (e.g., consensus entropy [4], token-level entropy, or multi-model voting confidence)? Can the authors provide experiments showing when discrete tags are preferable to probabilistic scores?\n\n5. **Hallucination on clean images**: Does the trained model successfully tag hallucinations that occur on non-degraded, high-quality document images? This would demonstrate that the model learns genuine uncertainty rather than merely memorizing the training degradation distribution.\n\n6. **Cold start validation**: Can the authors provide analysis showing that the pseudo-labels in the cold start actually correspond to visually degraded regions? For example, human annotations on a sample of cold-start labels, or correlation analysis between degradation strength and tagging frequency.\n\n7. **Breakdown by error type**: What types of errors are well-covered by UNC tags vs. those that escape detection? Are character-level substitutions easier to catch than word-level hallucinations?\n\n\n## References\n\n[1] Linke Ouyang, Yuan Qu, Hongbin Zhou, et al. OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations. arXiv:2412.07626, 2024. (CVPR’25)\n\n[2] Bin Wang, Chao Xu, Xiaomeng Zhao, et al. MinerU: An Open-Source Solution for Precise Document Content Extraction. arXiv:2409.18839, 2024.\n\n[3] Junbo Niu, Zheng Liu, Zhuangcheng Gu, et al. MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing. arXiv:2509.22186, 2025.\n\n[4] Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR\n\n[5] Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. Why language models hallucinate. arXiv:2509.04664, 2025.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "**Clear problem formulation**: The paper addresses a real problem—VLM-based OCR systems hallucinate on degraded documents without signaling uncertainty, which is worse than classical OCR systems that produce obviously garbled output. The motivation is well-articulated.\n\n**Systematic methodology**: The two-stage training approach (pseudo-labeled cold start + GRPO) is reasonable and well-described. The multi-objective reward design with safeguards against reward hacking (especially the length-mismatch damping factor η) demonstrates careful engineering.", "weaknesses": "### 1. Limited Benchmark Coverage and Missing Baselines\n\nThe evaluation is restricted to a single synthetic benchmark (Blur-OCR). The paper does not evaluate on:\n- Established document understanding benchmarks like OmniDocBench [1], which provides diverse real-world PDF documents with comprehensive annotations\n- More general OCR benchmarks beyond the two mentioned in Related Work (OCRBench/OCRBench v2)\n- Recent document parsing systems like MinerU [2] or MinerU2.5 [3], which represent state-of-the-art in document content extraction\n\n### 2. Incomplete Related Work on Uncertainty Quantification\n\nThe paper misses critical recent work on OCR uncertainty estimation. Notably, it does not cite or compare with methods that provide quantitative uncertainty measures. For instance, recent work on consensus entropy for multi-VLM agreement [4] provides token-level uncertainty scores that can be directly compared with the proposed tagging approach. While the paper mentions entropy-based baselines briefly in Exp4 (Section 7.4), it lacks:\n- Proper contextualization within the broader uncertainty quantification or calibration literature\n- Discussion of how the proposed explicit tagging approach differs from or improves upon probabilistic uncertainty measures\n\n### 3. Conceptual Issues with the Cold Start Procedure\n\nThe pseudo-labeling strategy has a fundamental problem: it tags the **model's own errors** on degraded images, not necessarily the **visually unreadable regions**. This conflates two distinct phenomena:\n\na) Text that is visually degraded/unreadable in the image  \nb) Text where the model happened to make a mistake\n\nThe paper claims (Section 5.1) that \"When the image is unreadable, models tend to guess and are often wrong,\" but this assumption is not validated. Two problematic cases arise:\n\n- **False positive tags**: The model may correctly transcribe text from a degraded-but-readable region, yet the cold start labels it as uncertain simply because it differs from GT due to actual degradation differences\n- **False negative tags**: The model may confidently hallucinate on clear, undegraded regions (a known VLM failure mode [5]), which would not receive uncertainty tags\n\n\n### 4. Limited Analysis of Generalization Beyond Synthetic Degradations\n\nVLMs are known to hallucinate even on clean, high-quality document images [5]. The paper does not:\n- Test whether the uncertainty-aware model can identify hallucinations on non-degraded documents\n- Evaluate on real-world degraded documents (e.g., historical documents, which are mentioned in the motivation but never tested)\n- Compare performance on different types of errors: character substitutions vs. hallucinated words/phrases\n\n### 5. Benchmark Construction Concerns\n\nThe Blur-OCR benchmark applies random combinations of degradations to clean text, but:\n- No analysis is provided on whether the degradations are realistic compared to actual historical documents or real-world low-quality scans\n- The paper does not discuss the distribution of degradation severity or provide statistics on what fraction of text becomes truly unreadable\n- Figure 2 shows sample pages, but there is no quantitative analysis of degradation characteristics\n\nThis makes it difficult to assess whether Blur-OCR represents realistic use cases or is primarily useful for evaluating this specific training paradigm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)